Skip to content

Commit 952be13

Browse files
committed
docs: remove stale compressor config stanza (#204)
The LLM-compressor subsystem was deleted with the historian v2 decay-render rewrite, and the JSON schema no longer accepts a `compressor` key — so IDEs validating against the schema flag the stanza CONFIGURATION.md still documented. Remove the compressor row and section, and reunite the commit_cluster_trigger field table (stranded below the compressor section) with its own section.
1 parent fdc0b23 commit 952be13

1 file changed

Lines changed: 7 additions & 58 deletions

File tree

CONFIGURATION.md

Lines changed: 7 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,6 @@ Higher-tier models with longer cache windows benefit from a longer TTL. Setting
110110
| `history_budget_percentage` | `number` (0.05–0.5) | `0.15` | Fraction of usable context (`context_limit × execute_threshold`) reserved for the history block. Triggers compression when exceeded. |
111111
| `commit_cluster_trigger` | `object` | See below | Controls the commit-cluster historian trigger. |
112112
| `sqlite` | `object` | See below | Per-connection SQLite tuning for Magic Context's own `context.db`. |
113-
| `compressor` | `object` | See below | Controls the background compressor that merges older compartments when the history block exceeds its budget. |
114113

115114
### `language`
116115

@@ -139,6 +138,13 @@ A **commit cluster** is a distinct work phase where the agent made one or more g
139138
}
140139
```
141140

141+
| Field | Type | Default | Description |
142+
|-------|------|---------|-------------|
143+
| `enabled` | `boolean` | `true` | Enable commit-cluster based historian triggering. |
144+
| `min_clusters` | `number` | `3` | Minimum number of commit clusters in the unsummarized tail before historian fires. The tail must also contain at least one `trigger_budget` worth of tokens, where `trigger_budget = main_context × execute_threshold × 5%` clamped to `[5K, 50K]`. |
145+
146+
Set `enabled: false` to disable this trigger entirely and rely only on pressure-based and tail-size triggers for historian.
147+
142148
### `sqlite`
143149

144150
Per-connection PRAGMAs applied to Magic Context's own `context.db` at open. These tune SQLite's runtime behaviour only — they do not change the schema or what is stored, and they do not touch OpenCode's or Pi's databases.
@@ -157,63 +163,6 @@ Per-connection PRAGMAs applied to Magic Context's own `context.db` at open. Thes
157163

158164
Separately, Magic Context runs `PRAGMA optimize` (bounded by `PRAGMA analysis_limit=400`) on its 15-minute maintenance tick. This is self-gating — it re-analyses a table only when its row count has drifted enough to matter — so the query planner keeps choosing good indexes as the database grows. There is no config knob for it.
159165

160-
### `compressor`
161-
162-
Compressor is a background pass that runs when the rendered `<session-history>` block exceeds its budget. It merges older compartments using progressively aggressive **caveman-style** compression at each depth level, enforcing style consistency via a deterministic post-process after the historian LLM call. Each compartment range can be compressed at most `max_merge_depth` times.
163-
164-
**Depth tiers** (applied progressively as compartments are re-compressed):
165-
166-
| Depth | Style | What happens |
167-
|---|---|---|
168-
| 1 | **Merge only** | Preserve narrative and all U: lines. Drop only duplicates spanning compartments. |
169-
| 2 | **Lite caveman** | Drop filler words (just, really, basically) and hedging. Keep grammar. |
170-
| 3 | **Full caveman** | Drop articles (the, a, an), weak auxiliaries. Fragments OK. Single paragraph per compartment. |
171-
| 4 | **Ultra caveman** | Telegraphic. Symbol connectives (``, `+`, `//`, `\|`). Pattern: `[thing] [action] [reason]`. |
172-
| 5 | **Title-only collapse** | Content cleared (no LLM call). Raw messages recoverable via `ctx_expand`. |
173-
174-
Inspired by the [caveman Claude Code skill](https://github.com/JuliusBrussee/caveman) which validated telegraph-style compression as LLM-friendly (and saves tokens without tokenizer fallback issues that character-dropping causes).
175-
176-
```jsonc
177-
{
178-
"compressor": {
179-
"enabled": true, // default: true
180-
"min_compartment_ratio": 1000, // default: 1000 (floor = ceil(total_raw_messages / ratio))
181-
"max_merge_depth": 5, // default: 5 (1-5, deeper = more aggressive)
182-
"cooldown_ms": 600000, // default: 600000 (10 min between background runs)
183-
"max_compartments_per_pass": 15, // default: 15 (LLM batch cap)
184-
"grace_compartments": 10 // default: 10 (newest N compartments never compressed)
185-
}
186-
}
187-
```
188-
189-
**Merge ratios per depth** (applied per LLM pass — small ratios preserve more narrative):
190-
191-
| Depth transition | Ratio | Shape |
192-
|---|---|---|
193-
| 0 → 1 | 1.33× (4:3) | Narrative merge; preserve all `U:` lines |
194-
| 1 → 2 | 1.5× (3:2) | Drop filler, keep grammar (caveman-lite) |
195-
| 2 → 3 | 2× (2:1) | Paragraph, fragments OK (caveman-full) |
196-
| 3 → 4 | 2× (2:1) | Telegraph + symbol connectives (caveman-ultra) |
197-
| 4 → 5 || Title-only collapse (no LLM, recoverable via `ctx_expand`) |
198-
199-
**Selection strategy:** The compressor picks the oldest contiguous run of compartments that share the SAME rounded compression depth (up to `max_compartments_per_pass`). This progresses naturally: depth-0 bands get compressed first → depth-1 bands compressed next → and so on. Each run goes through one LLM call.
200-
201-
**Floor protection:** The compressor never reduces your session's compartment count below `ceil(total_raw_messages / min_compartment_ratio)`. For a 20K-message session with the default ratio, that's a floor of 20 compartments.
202-
203-
**Grace period:** The newest `grace_compartments` compartments are always excluded from compression. This protects freshly-published historian output from being re-compressed before it has been used. Default is 10, which works well even for long autonomous runs that publish many compartments per hour.
204-
205-
**Ordinal snap:** When the LLM drifts by ±1-2 ordinals on merged boundaries (e.g. outputs `start=8161` when the actual input boundary is `8160`), the runtime snaps those values to the enclosing input compartment's canonical boundary rather than rejecting the whole pass. Snaps are logged for observability.
206-
207-
**Disable entirely:** Set `compressor.enabled: false` to skip all background compression. Older sessions will simply carry a larger history footprint.
208-
209-
210-
| Field | Type | Default | Description |
211-
|-------|------|---------|-------------|
212-
| `enabled` | `boolean` | `true` | Enable commit-cluster based historian triggering. |
213-
| `min_clusters` | `number` | `3` | Minimum number of commit clusters in the unsummarized tail before historian fires. The tail must also contain at least one `trigger_budget` worth of tokens, where `trigger_budget = main_context × execute_threshold × 5%` clamped to `[5K, 50K]`. |
214-
215-
Set `enabled: false` to disable this trigger entirely and rely only on pressure-based and tail-size triggers for historian.
216-
217166
### `execute_threshold_tokens`
218167

219168
An absolute-tokens alternative to `execute_threshold_percentage`. Useful when you want a hard cap expressed in tokens rather than a percentage — for example, when a provider limits effective prompt size below its advertised context window.

0 commit comments

Comments
 (0)