feat(compressor): redesign compressor with depth-aware ratios and grace period

The old compressor selected candidates greedily and merged them at a fixed 2:1
ratio until the history block fit the budget. On large sessions this could
swallow 80%+ of all compartments in one pass (selecting ~292 out of 352
compartments into one LLM call), causing ordinal drift in the output and
pathological over-compression of recent work that did not need it yet.

Selection rewrite:
- Bound each pass to at most `max_compartments_per_pass` (default 15) so the
  LLM never handles more context than it can track reliably.
- Always exclude the newest `grace_compartments` compartments (default 10) so
  freshly published historian output has time to be used before compression
  touches it. Count-based instead of time-based so long autonomous runs that
  publish dozens of compartments overnight stay protected.
- Pick the oldest contiguous band of same-rounded-depth compartments,
  respecting the count floor and merge-depth cap. Mixed-depth bands are
  skipped so compression progresses from depth 0 upward uniformly.
- Replace the fixed 2:1 ratio with per-depth ratios: 4:3 at depth 1, 3:2 at
  depth 2, 2:1 at depths 3 and 4, title-only collapse at depth 5. Depth 1
  keeps ~75% of tokens instead of halving them, protecting narrative content.

Ordinal-snap fix:
- `compartment-runner-compressor.ts` previously required exact messageId
  matches when mapping LLM output back to DB rows. Any off-by-one drift
  failed the whole pass. Now off-by-one LLM output snaps to the enclosing
  input compartment's canonical boundary. The snap count is logged for
  observability.

New config surfaces under top-level `compressor`: `enabled`,
`min_compartment_ratio`, `max_merge_depth`, `cooldown_ms`,
`max_compartments_per_pass`, `grace_compartments`. Schema regenerated,
documented in CONFIGURATION.md, and threaded through hook, transform, and
runner dependencies.

Author: ualtinok

CONFIGURATION.md

@@ -74,6 +74,7 @@ Higher-tier models with longer cache windows benefit from a longer TTL. Setting
 | `history_budget_percentage` | `number` (0.05–0.5) | `0.15` | Fraction of usable context (`context_limit × execute_threshold`) reserved for the history block. Triggers compression when exceeded. |
 | `compaction_markers` | `boolean` | `true` | Inject compaction boundaries into OpenCode's DB after historian publishes. Reduces transform input size for long sessions. |
 | `commit_cluster_trigger` | `object` | See below | Controls the commit-cluster historian trigger. |
+| `compressor` | `object` | See below | Controls the background compressor that merges older compartments when the history block exceeds its budget. |
 
 ### `commit_cluster_trigger`
 
@@ -88,6 +89,56 @@ A **commit cluster** is a distinct work phase where the agent made one or more g
 }
 ```
 
+### `compressor`
+
+Compressor is a background pass that runs when the rendered `<session-history>` block exceeds its budget. It merges older compartments using progressively aggressive **caveman-style** compression at each depth level, enforcing style consistency via a deterministic post-process after the historian LLM call. Each compartment range can be compressed at most `max_merge_depth` times.
+
+**Depth tiers** (applied progressively as compartments are re-compressed):
+
+| Depth | Style | What happens |
+|---|---|---|
+| 1 | **Merge only** | Preserve narrative and all U: lines. Drop only duplicates spanning compartments. |
+| 2 | **Lite caveman** | Drop filler words (just, really, basically) and hedging. Keep grammar. |
+| 3 | **Full caveman** | Drop articles (the, a, an), weak auxiliaries. Fragments OK. Single paragraph per compartment. |
+| 4 | **Ultra caveman** | Telegraphic. Symbol connectives (`→`, `+`, `//`, `\|`). Pattern: `[thing] [action] [reason]`. |
+| 5 | **Title-only collapse** | Content cleared (no LLM call). Raw messages recoverable via `ctx_expand`. |
+
+Inspired by the [caveman Claude Code skill](https://github.com/JuliusBrussee/caveman) which validated telegraph-style compression as LLM-friendly (and saves tokens without tokenizer fallback issues that character-dropping causes).
+
+```jsonc
+{
+  "compressor": {
+    "enabled": true,                  // default: true
+    "min_compartment_ratio": 1000,     // default: 1000 (floor = ceil(total_raw_messages / ratio))
+    "max_merge_depth": 5,             // default: 5 (1-5, deeper = more aggressive)
+    "cooldown_ms": 600000,            // default: 600000 (10 min between background runs)
+    "max_compartments_per_pass": 15,  // default: 15 (LLM batch cap)
+    "grace_compartments": 10          // default: 10 (newest N compartments never compressed)
+  }
+}
+```
+
+**Merge ratios per depth** (applied per LLM pass — small ratios preserve more narrative):
+
+| Depth transition | Ratio | Shape |
+|---|---|---|
+| 0 → 1 | 1.33× (4:3) | Narrative merge; preserve all `U:` lines |
+| 1 → 2 | 1.5× (3:2) | Drop filler, keep grammar (caveman-lite) |
+| 2 → 3 | 2× (2:1) | Paragraph, fragments OK (caveman-full) |
+| 3 → 4 | 2× (2:1) | Telegraph + symbol connectives (caveman-ultra) |
+| 4 → 5 | — | Title-only collapse (no LLM, recoverable via `ctx_expand`) |
+
+**Selection strategy:** The compressor picks the oldest contiguous run of compartments that share the SAME rounded compression depth (up to `max_compartments_per_pass`). This progresses naturally: depth-0 bands get compressed first → depth-1 bands compressed next → and so on. Each run goes through one LLM call.
+
+**Floor protection:** The compressor never reduces your session's compartment count below `ceil(total_raw_messages / min_compartment_ratio)`. For a 20K-message session with the default ratio, that's a floor of 20 compartments.
+
+**Grace period:** The newest `grace_compartments` compartments are always excluded from compression. This protects freshly-published historian output from being re-compressed before it has been used. Default is 10, which works well even for long autonomous runs that publish many compartments per hour.
+
+**Ordinal snap:** When the LLM drifts by ±1-2 ordinals on merged boundaries (e.g. outputs `start=8161` when the actual input boundary is `8160`), the runtime snaps those values to the enclosing input compartment's canonical boundary rather than rejecting the whole pass. Snaps are logged for observability.
+
+**Disable entirely:** Set `compressor.enabled: false` to skip all background compression. Older sessions will simply carry a larger history footprint.
+
+
 | Field | Type | Default | Description |
 |-------|------|---------|-------------|
 | `enabled` | `boolean` | `true` | Enable commit-cluster based historian triggering. |
@@ -401,6 +452,12 @@ When enabled, dreamer analyzes which files each session's agent reads most frequ
   "drop_tool_structure": true,
   "history_budget_percentage": 0.15,
   "compaction_markers": true,
+  "compressor": {
+    "enabled": true,
+    "min_compartment_ratio": 1000,
+    "max_merge_depth": 5,
+    "cooldown_ms": 600000
+  },
 
   "historian": {
     "model": "github-copilot/gpt-5.4",

assets/magic-context.schema.json

@@ -166,6 +166,60 @@
       "default": true,
       "description": "Inject compaction boundaries into OpenCode's DB after historian publishes. Reduces transform input size for long sessions by letting OpenCode's filterCompacted skip older messages."
     },
+    "compressor": {
+      "type": "object",
+      "properties": {
+        "enabled": {
+          "type": "boolean",
+          "default": true,
+          "description": "Enable background compressor. When false, history block compression never runs and older sessions may carry a larger history footprint."
+        },
+        "min_compartment_ratio": {
+          "type": "number",
+          "minimum": 100,
+          "maximum": 10000,
+          "default": 1000,
+          "description": "Floor = ceil(total_raw_messages / min_compartment_ratio). Compressor never reduces compartment count below this floor."
+        },
+        "max_merge_depth": {
+          "type": "number",
+          "minimum": 1,
+          "maximum": 5,
+          "default": 5,
+          "description": "Maximum compression depth a compartment range can reach. Depth 5 collapses to title-only (recoverable via ctx_expand). Depths 1-4 apply caveman lite/full/ultra compression."
+        },
+        "cooldown_ms": {
+          "type": "number",
+          "minimum": 60000,
+          "default": 600000,
+          "description": "Minimum milliseconds between background compressor runs for a session."
+        },
+        "max_compartments_per_pass": {
+          "type": "number",
+          "minimum": 3,
+          "maximum": 50,
+          "default": 15,
+          "description": "Cap on compartments sent to the LLM in one pass. Smaller batches avoid ordinal drift and dedup mistakes on large inputs."
+        },
+        "grace_compartments": {
+          "type": "number",
+          "minimum": 0,
+          "maximum": 100,
+          "default": 10,
+          "description": "Number of newest compartments always excluded from compression. Protects freshly published historian output from being re-compressed before it has been used."
+        }
+      },
+      "additionalProperties": false,
+      "default": {
+        "enabled": true,
+        "min_compartment_ratio": 1000,
+        "max_merge_depth": 5,
+        "cooldown_ms": 600000,
+        "max_compartments_per_pass": 15,
+        "grace_compartments": 10
+      },
+      "description": "Background compressor configuration — merges older compartments with caveman-style compression when the history block exceeds its budget."
+    },
     "experimental": {
       "type": "object",
       "properties": {

packages/plugin/scripts/build-schema.ts

@@ -221,6 +221,68 @@ function buildSchema(): Record<string, unknown> {
                     "Inject compaction boundaries into OpenCode's DB after historian publishes. Reduces transform input size for long sessions by letting OpenCode's filterCompacted skip older messages.",
             },
 
+            compressor: {
+                type: "object",
+                properties: {
+                    enabled: {
+                        type: "boolean",
+                        default: true,
+                        description:
+                            "Enable background compressor. When false, history block compression never runs and older sessions may carry a larger history footprint.",
+                    },
+                    min_compartment_ratio: {
+                        type: "number",
+                        minimum: 100,
+                        maximum: 10000,
+                        default: 1000,
+                        description:
+                            "Floor = ceil(total_raw_messages / min_compartment_ratio). Compressor never reduces compartment count below this floor.",
+                    },
+                    max_merge_depth: {
+                        type: "number",
+                        minimum: 1,
+                        maximum: 5,
+                        default: 5,
+                        description:
+                            "Maximum compression depth a compartment range can reach. Depth 5 collapses to title-only (recoverable via ctx_expand). Depths 1-4 apply caveman lite/full/ultra compression.",
+                    },
+                    cooldown_ms: {
+                        type: "number",
+                        minimum: 60000,
+                        default: 600000,
+                        description:
+                            "Minimum milliseconds between background compressor runs for a session.",
+                    },
+                    max_compartments_per_pass: {
+                        type: "number",
+                        minimum: 3,
+                        maximum: 50,
+                        default: 15,
+                        description:
+                            "Cap on compartments sent to the LLM in one pass. Smaller batches avoid ordinal drift and dedup mistakes on large inputs.",
+                    },
+                    grace_compartments: {
+                        type: "number",
+                        minimum: 0,
+                        maximum: 100,
+                        default: 10,
+                        description:
+                            "Number of newest compartments always excluded from compression. Protects freshly published historian output from being re-compressed before it has been used.",
+                    },
+                },
+                additionalProperties: false,
+                default: {
+                    enabled: true,
+                    min_compartment_ratio: 1000,
+                    max_merge_depth: 5,
+                    cooldown_ms: 600000,
+                    max_compartments_per_pass: 15,
+                    grace_compartments: 10,
+                },
+                description:
+                    "Background compressor configuration — merges older compartments with caveman-style compression when the history block exceeds its budget.",
+            },
+
             experimental: {
                 type: "object",
                 properties: {

packages/plugin/src/config/schema/magic-context.test.ts

@@ -62,6 +62,14 @@ describe("MagicContextConfigSchema", () => {
                     min_clusters: 3,
                 },
                 compaction_markers: true,
+                compressor: {
+                    enabled: true,
+                    min_compartment_ratio: 1000,
+                    max_merge_depth: 5,
+                    cooldown_ms: 600_000,
+                    max_compartments_per_pass: 15,
+                    grace_compartments: 10,
+                },
                 experimental: {
                     user_memories: {
                         enabled: false,

packages/plugin/src/config/schema/magic-context.ts

@@ -8,6 +8,26 @@ export const DEFAULT_EXECUTE_THRESHOLD_PERCENTAGE = 65;
 export const DEFAULT_HISTORIAN_TIMEOUT_MS = 300_000;
 export const DEFAULT_HISTORY_BUDGET_PERCENTAGE = 0.15;
 export const DEFAULT_LOCAL_EMBEDDING_MODEL = "Xenova/all-MiniLM-L6-v2";
+/** Compressor defaults — see CompressorConfigSchema below for details. */
+export const DEFAULT_COMPRESSOR_MIN_COMPARTMENT_RATIO = 1000;
+export const DEFAULT_COMPRESSOR_MAX_MERGE_DEPTH = 5;
+export const DEFAULT_COMPRESSOR_COOLDOWN_MS = 600_000;
+/** Max compartments merged in one LLM pass. LLM quality degrades with larger inputs,
+ *  and smaller batches reduce ordinal-drift risk when the model outputs merged boundaries. */
+export const DEFAULT_COMPRESSOR_MAX_COMPARTMENTS_PER_PASS = 15;
+/** Number of newest compartments always excluded from compression.
+ *  Protects freshly-published historian compartments from immediate re-compression,
+ *  which would lose narrative quality before the agent has even used the compartment. */
+export const DEFAULT_COMPRESSOR_GRACE_COMPARTMENTS = 10;
+/** Output count at each depth = ceil(input / ratio).
+ *  Lower ratios = gentler compression. Depth 5 is title-only (no LLM call). */
+export const COMPRESSOR_MERGE_RATIO_BY_DEPTH: Record<number, number> = {
+    1: 1.33, // 4:3 — merge only, preserve narrative + U: lines
+    2: 1.5, // 3:2 — lite caveman
+    3: 2.0, // 2:1 — full caveman
+    4: 2.0, // 2:1 — ultra caveman (caveman post-process does heavy lifting)
+    5: 0, // title-only collapse (no merge, no LLM)
+};
 
 export const DREAMER_TASKS = [
     "consolidate",
@@ -141,6 +161,14 @@ export interface MagicContextConfig {
         min_clusters: number;
     };
     compaction_markers: boolean;
+    compressor: {
+        enabled: boolean;
+        min_compartment_ratio: number;
+        max_merge_depth: number;
+        cooldown_ms: number;
+        max_compartments_per_pass: number;
+        grace_compartments: number;
+    };
     experimental: {
         user_memories: {
             enabled: boolean;
@@ -231,6 +259,62 @@ export const MagicContextConfigSchema = z
          *  After historian publishes compartments, a compaction boundary is written into
          *  OpenCode's message/part tables so older messages are skipped at load time. Default: true. */
         compaction_markers: z.boolean().default(true),
+        /** Compressor configuration — controls second-pass compression of older
+         *  compartments when the rendered history block exceeds its budget.
+         *  The compressor merges adjacent compartments and applies progressively
+         *  aggressive caveman-style compression at deeper depths. */
+        compressor: z
+            .object({
+                /** Enable background compressor. When false, history block compression
+                 *  never runs, so very old sessions may carry a larger history footprint.
+                 *  (default: true) */
+                enabled: z.boolean().default(true),
+                /** Floor = ceil(total_raw_messages / min_compartment_ratio).
+                 *  Compressor will never reduce session compartment count below this
+                 *  floor. Lower ratio = more compartments preserved. Prevents runaway
+                 *  merging into a single mega-compartment. (min: 100, max: 10000, default: 1000) */
+                min_compartment_ratio: z
+                    .number()
+                    .min(100)
+                    .max(10000)
+                    .default(DEFAULT_COMPRESSOR_MIN_COMPARTMENT_RATIO),
+                /** Maximum compression depth for any compartment range. At depth 5,
+                 *  compartments are collapsed to title-only (content recoverable via
+                 *  ctx_expand). Depths 1-4 apply caveman-lite/full/ultra compression.
+                 *  (min: 1, max: 5, default: 5) */
+                max_merge_depth: z
+                    .number()
+                    .min(1)
+                    .max(5)
+                    .default(DEFAULT_COMPRESSOR_MAX_MERGE_DEPTH),
+                /** Minimum milliseconds between background compressor runs for a session.
+                 *  (min: 60000, default: 600000 = 10 min) */
+                cooldown_ms: z.number().min(60_000).default(DEFAULT_COMPRESSOR_COOLDOWN_MS),
+                /** Max compartments the compressor will send to one LLM call in a single pass.
+                 *  Keeping this low avoids LLM ordinal drift and dedup mistakes on large inputs.
+                 *  (min: 3, max: 50, default: 15) */
+                max_compartments_per_pass: z
+                    .number()
+                    .min(3)
+                    .max(50)
+                    .default(DEFAULT_COMPRESSOR_MAX_COMPARTMENTS_PER_PASS),
+                /** Number of newest compartments always excluded from compression.
+                 *  Protects freshly published historian output from being re-compressed before it
+                 *  has been used. (min: 0, max: 100, default: 10) */
+                grace_compartments: z
+                    .number()
+                    .min(0)
+                    .max(100)
+                    .default(DEFAULT_COMPRESSOR_GRACE_COMPARTMENTS),
+            })
+            .default({
+                enabled: true,
+                min_compartment_ratio: DEFAULT_COMPRESSOR_MIN_COMPARTMENT_RATIO,
+                max_merge_depth: DEFAULT_COMPRESSOR_MAX_MERGE_DEPTH,
+                cooldown_ms: DEFAULT_COMPRESSOR_COOLDOWN_MS,
+                max_compartments_per_pass: DEFAULT_COMPRESSOR_MAX_COMPARTMENTS_PER_PASS,
+                grace_compartments: DEFAULT_COMPRESSOR_GRACE_COMPARTMENTS,
+            }),
         /** Embedding provider configuration */
         embedding: EmbeddingConfigSchema.default({
             provider: "local",