feat(config): execute_threshold_tokens for absolute-token execute thresholds

Adds execute_threshold_tokens as an alternative to execute_threshold_percentage.
Per-model map (with optional 'default' key) of absolute token counts; when a
model matches, tokens config overrides percentage and is clamped to 80% of the
resolved context limit. Requires a known context_limit to convert to percentage.

Core:
- resolveExecuteThresholdDetail() is now the single source of truth returning
  { percentage, mode, absoluteTokens?, matchedKey? }. resolveExecuteThreshold()
  is a thin back-compat wrapper.
- Clamp warnings dedupe by (session, model, tokenVal, cap) so over-cap configs
  don't spam the log on every transform pass.
- Runtime guards (isFinitePositive) block NaN/negative/zero contextLimit and
  token values from poisoning the resolver; all bad inputs fall through to
  percentage config safely.

Call-site integration:
- scheduler.ts/transform.ts/event-handler.ts/hook.ts/create-session-hooks.ts:
  thread tokensConfig + contextLimit through the execute path.
- execute-status.ts: /ctx-status now uses the detail resolver for mode display
  (fixes progressive base-model lookup drift where e.g. gpt-5.4-fast matching
  gpt-5.4 was mislabeled as percentage mode).
- rpc-handlers.ts + rpc-types.ts: StatusDetail exposes executeThresholdMode and
  executeThresholdTokens so TUI can verify token mode and show the absolute
  trigger.
- nudger.ts: intentionally stays percentage-only (advisory only; scheduler is
  authoritative). Documented inline.

Config + docs:
- schema: execute_threshold_tokens per-model map (min 5K, max 2M).
- assets/magic-context.schema.json regenerated with drop_tool_structure,
  compaction_markers, and experimental fields that were previously missing.
- CONFIGURATION.md: documents both percentage and token modes and their
  interaction.
- dashboard ConfigEditor: new per-model text field for execute_threshold_tokens
  (alwaysObject + numericText so saved shape stays schema-valid); fixed
  drop_tool_structure default display, stale execute_threshold_percentage
  slider min (35→20), and added missing memory.retrieval_count_promotion_threshold.
- PerModelField: adds alwaysObject and numericText props for fields whose
  schema forbids bare scalars or requires numeric values.

Tests: +10 targeted cases for token-mode detail resolution, progressive
base-model matching, clamp behavior, NaN/zero/negative guards, and dedupe
stability. Suite is 571 pass (was 561).

Author: ualtinok

CONFIGURATION.md

@@ -65,11 +65,13 @@ Higher-tier models with longer cache windows benefit from a longer TTL. Setting
 | `protected_tags` | `number` (1–100) | `20` | Last N active tags immune from immediate dropping. |
 | `nudge_interval_tokens` | `number` | `10000` | Minimum token growth between rolling nudges. |
 | `execute_threshold_percentage` | `number` (20–80) or `object` | `65` | Context usage that forces queued ops to execute. Capped at 80% max for cache safety. Supports per-model map. |
+| `execute_threshold_tokens` | `object` (per-model map) | — | **Optional absolute-tokens variant of `execute_threshold_percentage`.** Per-model map (e.g. `{ "default": 150000, "github-copilot/gpt-5.2-codex": 40000 }`). When set for a model, overrides the percentage-based threshold for that model. Clamped to `80% × context_limit` with a warn log. Requires a resolvable context limit — falls through to percentage if unavailable. See below. |
 | `auto_drop_tool_age` | `number` | `100` | Auto-drop tool outputs older than N tags during execution. |
+| `drop_tool_structure` | `boolean` | `true` | When `true`, dropped tool parts are fully removed from the transformed prompt. When `false`, tool call structure is preserved: tool name kept, tool inputs truncated to 5 chars + `...[truncated]`, tool output replaced with `[truncated]`. Preserving structure keeps the agent aware that prior tools ran (preventing hallucinated re-calls) at the cost of ~4K additional tokens per ~60 dropped tools. |
 | `clear_reasoning_age` | `number` | `50` | Clear thinking/reasoning blocks older than N tags. |
 | `iteration_nudge_threshold` | `number` | `15` | Consecutive assistant turns without user input before an iteration nudge. |
 | `historian_timeout_ms` | `number` | `300000` | Timeout per historian call (ms). |
-| `history_budget_percentage` | `number` (0–1) | `0.15` | Fraction of usable context reserved for the history block. Triggers compression when exceeded. |
+| `history_budget_percentage` | `number` (0.05–0.5) | `0.15` | Fraction of usable context (`context_limit × execute_threshold`) reserved for the history block. Triggers compression when exceeded. |
 | `compaction_markers` | `boolean` | `true` | Inject compaction boundaries into OpenCode's DB after historian publishes. Reduces transform input size for long sessions. |
 | `commit_cluster_trigger` | `object` | See below | Controls the commit-cluster historian trigger. |
 
@@ -93,6 +95,37 @@ A **commit cluster** is a distinct work phase where the agent made one or more g
 
 Set `enabled: false` to disable this trigger entirely and rely only on pressure-based and tail-size triggers for historian.
 
+### `execute_threshold_tokens`
+
+An absolute-tokens alternative to `execute_threshold_percentage`. Useful when you want a hard cap expressed in tokens rather than a percentage — for example, when a provider limits effective prompt size below its advertised context window.
+
+```jsonc
+{
+  "execute_threshold_tokens": {
+    "default": 150000,                          // fires at 150K for any model without an explicit entry
+    "github-copilot/gpt-5.2-codex": 40000       // fires at 40K specifically for gpt-5.2-codex
+  }
+}
+```
+
+**Behavior:**
+
+- Per-model map only — no bare-number form. All sessions are assumed to have different context limits, so the `default` key acts as a fallback inside the map.
+- **Tokens wins:** when a matching entry exists for the current model, it overrides the percentage-based threshold for that model. Other models continue to use `execute_threshold_percentage`.
+- **Progressive key lookup** just like percentage config — `openai/gpt-5.4-fast` matches `openai/gpt-5.4` if the derived key is absent.
+- **Clamped at 80% × context_limit** for the same cache-safety reason as percentage. If the clamp fires, a `log.warn` records the original and capped value.
+- Requires a **resolvable context limit** at runtime. On brand-new sessions before any response arrives, the context limit is unknown — in that case, resolution falls through to `execute_threshold_percentage`. Once the first response lands, the correct tokens-based threshold is applied on the following turn.
+
+**When to prefer tokens over percentage:**
+
+- You hit a provider-side prompt cap (like GitHub Copilot's `max_prompt_tokens` ignoring user config overrides — see the github-copilot interaction in the project KNOWN_ISSUES).
+- You want consistent compaction behavior across models with very different context window sizes.
+
+**When to prefer percentage:**
+
+- You want the threshold to scale proportionally with the model's window (bigger window → compacts later in absolute terms).
+- You're not targeting a specific provider cap.
+
 ---
 
 ## Model Resolution
@@ -109,6 +142,34 @@ Setting `model` in any agent config overrides the fallback chain entirely. Setti
 
 > **Tip — Dreamer with local models:** Since the dreamer runs during idle time (typically overnight), it works well with local models. Even slower ones like `ollama/mlx-qwen3.5-27b-claude-4.6-opus-reasoning-distilled` are fine — there's no user waiting.
 
+### Advanced agent fields
+
+All three agents (`historian`, `dreamer`, `sidekick`) accept these additional fields beyond the common `model`, `fallback_models`, `temperature`, `variant`, `prompt`. Most map directly to OpenCode's `AgentConfig` and pass through unchanged.
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `tools` | `{ [toolName: string]: boolean }` | Restrict which tools the agent can use. `{ "bash": false, "write": false }` disables those tools for this agent only. |
+| `permission` | `object` | Per-agent permission overrides. Sub-fields: `edit`, `bash`, `webfetch`, `doom_loop`, `external_directory`. Each accepts `"ask"`, `"allow"`, or `"deny"`. `bash` additionally accepts a record form for per-command rules. |
+| `disable` | `boolean` | Disable the agent without removing its config. Useful for toggling on/off during testing. |
+| `description` | `string` | Agent description shown in OpenCode UI. |
+| `mode` | `"subagent"` \| `"primary"` \| `"all"` | OpenCode agent mode. Magic Context internal agents run as `subagent`. |
+| `top_p` | `number` (0–1) | Nucleus sampling. |
+| `maxSteps` | `number` | Max reasoning steps per agent call. |
+| `maxTokens` | `number` | Max output tokens. ⚠️ OpenCode does not currently consume this field for plugin-registered agents — setting it has no effect. Tracked in the project as a known limitation. |
+| `color` | `string` (`#RRGGBB`) | Display color in OpenCode UI. |
+
+Example — restricting historian to read-only tools and denying bash:
+
+```jsonc
+{
+  "historian": {
+    "model": "github-copilot/gpt-5.4",
+    "tools": { "bash": false, "write": false, "edit": false },
+    "permission": { "bash": "deny", "webfetch": "deny" }
+  }
+}
+```
+
 ---
 
 ## `historian`
@@ -201,11 +262,18 @@ Controls semantic search for cross-session memories.
 
 | Field | Type | Default | Description |
 |-------|------|---------|-------------|
-| `provider` | `"local"` \| `"openai-compatible"` \| `"off"` | `"local"` | `"local"` runs `Xenova/all-MiniLM-L6-v2` in-process. |
+| `provider` | `"local"` \| `"openai-compatible"` \| `"off"` | `"local"` | `"local"` runs `Xenova/all-MiniLM-L6-v2` in-process. `"off"` disables semantic ranking entirely — see below. |
 | `model` | `string` | `"Xenova/all-MiniLM-L6-v2"` | Embedding model. |
 | `endpoint` | `string` | — | Required for `"openai-compatible"`. |
 | `api_key` | `string` | — | Optional API key for remote endpoints. |
 
+When `provider: "off"`:
+
+- No embeddings are generated. `ctx_memory(write)` skips embedding inline and the background embedding sweep becomes a no-op.
+- `ctx_search` and memory injection fall back to FTS5 (BM25) ranking only. Keyword matches still work; semantic similarity does not.
+- Session-start memory injection still happens when `memory.enabled` is `true` — memories are ordered by utility tier plus `seen_count` rather than semantic similarity to the current turn.
+- Memories written while `off` is active will have no embedding row; if you later re-enable `"local"` or `"openai-compatible"`, the background sweep embeds them on the next 15-minute tick.
+
 ```jsonc
 {
   "embedding": {
@@ -256,13 +324,17 @@ It is useful when starting a new session. It's better to choose a fast and cheap
 | `fallback_models` | `string` or `string[]` | Fallback models. |
 | `temperature` | `number` (0–2) | Sampling temperature. |
 | `variant` | `string` | Agent variant. |
-| `prompt` | `string` | Agent prompt override. |
+| `prompt` | `string` | Persistent agent-level system prompt override. Applies to every sidekick run. |
+
+### Operational fields
 
 | Field | Type | Default | Description |
 |-------|------|---------|-------------|
 | `enabled` | `boolean` | `false` | Enable sidekick. |
 | `timeout_ms` | `number` | `30000` | Timeout per run (ms). |
-| `system_prompt` | `string` | — | Per-run system prompt override for the sidekick child session. |
+| `system_prompt` | `string` | — | Per-invocation system prompt prepended to the sidekick child session for this `/ctx-aug` call only. Layered on top of `prompt` if both are set. |
+
+> **`prompt` vs `system_prompt`:** `prompt` is the persistent agent definition applied to every sidekick run. `system_prompt` is a per-call override injected into that specific child session — useful when a single `/ctx-aug` invocation needs different guidance than the default.
 
 ---
 
@@ -323,6 +395,7 @@ When enabled, dreamer analyzes which files each session's agent reads most frequ
   },
   "protected_tags": 10,
   "auto_drop_tool_age": 50,
+  "drop_tool_structure": true,
   "history_budget_percentage": 0.15,
   "compaction_markers": true,
 

assets/magic-context.schema.json

@@ -48,15 +48,15 @@
       "oneOf": [
         {
           "type": "number",
-          "minimum": 35,
+          "minimum": 20,
           "maximum": 80
         },
         {
           "type": "object",
           "properties": {
             "default": {
               "type": "number",
-              "minimum": 35,
+              "minimum": 20,
               "maximum": 80
             }
           },
@@ -65,7 +65,7 @@
           ],
           "additionalProperties": {
             "type": "number",
-            "minimum": 35,
+            "minimum": 20,
             "maximum": 80
           },
           "description": "Per-model threshold (e.g. { \"default\": 65, \"anthropic/claude-sonnet-4-6\": 40 })"
@@ -74,6 +74,22 @@
       "default": 65,
       "description": "Context usage percentage that forces queued operations to execute. Capped at 80% for cache safety. Number or per-model object."
     },
+    "execute_threshold_tokens": {
+      "type": "object",
+      "properties": {
+        "default": {
+          "type": "number",
+          "minimum": 5000,
+          "maximum": 2000000
+        }
+      },
+      "additionalProperties": {
+        "type": "number",
+        "minimum": 5000,
+        "maximum": 2000000
+      },
+      "description": "Absolute token threshold that forces queued operations to execute. Per-model map (e.g. { \"default\": 150000, \"github-copilot/gpt-5.2-codex\": 40000 }). When set for a model, overrides execute_threshold_percentage. Clamped to 80% × context_limit with a warn log. Requires a resolvable model context limit at runtime to convert to an effective percentage."
+    },
     "nudge_interval_tokens": {
       "type": "number",
       "minimum": 1000,
@@ -93,6 +109,11 @@
       "default": 100,
       "description": "Auto-drop tool outputs older than N tags during queue execution"
     },
+    "drop_tool_structure": {
+      "type": "boolean",
+      "default": true,
+      "description": "When true, dropped tool parts are fully removed. When false, tool call structure is preserved (name kept, inputs truncated to 5 chars + '...[truncated]', output replaced with '[truncated]') to prevent agents from hallucinating re-calls."
+    },
     "clear_reasoning_age": {
       "type": "number",
       "minimum": 10,
@@ -140,6 +161,83 @@
       },
       "description": "Fire historian when enough commit clusters accumulate in the unsummarized tail"
     },
+    "compaction_markers": {
+      "type": "boolean",
+      "default": true,
+      "description": "Inject compaction boundaries into OpenCode's DB after historian publishes. Reduces transform input size for long sessions by letting OpenCode's filterCompacted skip older messages."
+    },
+    "experimental": {
+      "type": "object",
+      "properties": {
+        "user_memories": {
+          "type": "object",
+          "properties": {
+            "enabled": {
+              "type": "boolean",
+              "default": false,
+              "description": "Extract user behavior observations from historian runs and promote recurring patterns to stable user memories injected into all sessions. Requires dreamer."
+            },
+            "promotion_threshold": {
+              "type": "number",
+              "minimum": 2,
+              "maximum": 20,
+              "default": 3,
+              "description": "Minimum candidate observations before dreamer considers promotion to stable user memory"
+            }
+          },
+          "additionalProperties": false,
+          "default": {
+            "enabled": false,
+            "promotion_threshold": 3
+          },
+          "description": "User memory extraction and promotion (experimental)."
+        },
+        "pin_key_files": {
+          "type": "object",
+          "properties": {
+            "enabled": {
+              "type": "boolean",
+              "default": false,
+              "description": "Pin frequently-read key files into the system prompt. Dreamer identifies key files per session based on read patterns. Requires dreamer."
+            },
+            "token_budget": {
+              "type": "number",
+              "minimum": 2000,
+              "maximum": 30000,
+              "default": 10000,
+              "description": "Total token budget for all pinned key files"
+            },
+            "min_reads": {
+              "type": "number",
+              "minimum": 2,
+              "maximum": 20,
+              "default": 4,
+              "description": "Minimum full-read count before a file is considered for pinning"
+            }
+          },
+          "additionalProperties": false,
+          "default": {
+            "enabled": false,
+            "token_budget": 10000,
+            "min_reads": 4
+          },
+          "description": "Pin frequently-read key files into system prompt (experimental)."
+        }
+      },
+      "additionalProperties": false,
+      "default": {
+        "user_memories": {
+          "enabled": false,
+          "promotion_threshold": 3
+        },
+        "pin_key_files": {
+          "enabled": false,
+          "token_budget": 10000,
+          "min_reads": 4
+        }
+      },
+      "description": "Experimental features — gated behind flags, may change between releases."
+    },
     "historian": {
       "type": "object",
       "properties": {

packages/dashboard/src/components/ConfigEditor/ConfigEditor.tsx

@@ -64,7 +64,7 @@ const FIELD_DEFS: FieldDef[] = [
   // General
   { key: "enabled", label: "Enabled", type: "boolean", description: "Enable the magic-context plugin", section: "General" },
   { key: "ctx_reduce_enabled", label: "Agent Controlled Reduction", type: "boolean", description: "Enable agent controlled reductions via ctx_reduce tool. When enabled, agent is prompted and nudged to choose what messages and tool calls to drop periodically. If disabled the system still works via auto drops based on message ages.", section: "General" },
-  { key: "drop_tool_structure", label: "Drop Tool Structure", type: "boolean", description: "When enabled, dropped tool calls are fully removed. When disabled, tool input/output is truncated in place so tool structure stays visible.", section: "General", defaultValue: false },
+  { key: "drop_tool_structure", label: "Drop Tool Structure", type: "boolean", description: "When enabled, dropped tool calls are fully removed. When disabled, tool input/output is truncated in place so tool structure stays visible.", section: "General", defaultValue: true },
   { key: "compaction_markers", label: "Compaction Markers", type: "boolean", description: "Inject boundary into OpenCode's DB so transform only processes the live tail after historian compaction. Significantly reduces transform input size for long sessions.", section: "General", defaultValue: true },
   // Thresholds
   // cache_ttl and execute_threshold_percentage are rendered as custom PerModelField components
@@ -81,6 +81,7 @@ const FIELD_DEFS: FieldDef[] = [
   { key: "memory.enabled", label: "Memory Enabled", type: "boolean", description: "Enable cross-session project memory.", section: "Memory" },
   { key: "memory.injection_budget_tokens", label: "Injection Budget (tokens)", type: "number", description: "Max tokens for memory injection into session history.", section: "Memory" },
   { key: "memory.auto_promote", label: "Auto Promote", type: "boolean", description: "Automatically promote session facts to project memory.", section: "Memory" },
+  { key: "memory.retrieval_count_promotion_threshold", label: "Retrieval Count Promotion Threshold", type: "number", description: "Minimum ctx_search retrieval count before a session fact is auto-promoted to project memory.", section: "Memory" },
 ];
 
 // ── Nested value access helpers ─────────────────────────────
@@ -205,7 +206,7 @@ function ConfigForm(props: {
   const getRangeConfig = (fieldKey: string): { min: number; max: number; step: number; suffix: string; defaultValue: number } => {
     switch (fieldKey) {
       case "execute_threshold_percentage":
-        return { min: 35, max: 80, step: 1, suffix: "%", defaultValue: 65 };
+        return { min: 20, max: 80, step: 1, suffix: "%", defaultValue: 65 };
       case "history_budget_percentage":
         return { min: 0.05, max: 0.5, step: 0.01, suffix: "", defaultValue: 0.15 };
       case "nudge_interval_tokens":
@@ -592,14 +593,26 @@ function ConfigForm(props: {
                       <PerModelField
                         label="Execute Threshold %"
                         configKey="execute_threshold_percentage"
-                        description="Context usage percentage (35–80) at which queued drops execute. Max 80."
+                        description="Context usage percentage (20–80) at which queued drops execute. Max 80."
                         value={getNestedValue(formData(), "execute_threshold_percentage") ?? getNestedValue(parsed(), "execute_threshold_percentage")}
                         onChange={(v) => handleFieldChange("execute_threshold_percentage", v)}
                         models={models() ?? []}
                         inputType="slider"
-                        sliderConfig={{ min: 35, max: 80, step: 1, suffix: "%", defaultValue: 65 }}
+                        sliderConfig={{ min: 20, max: 80, step: 1, suffix: "%", defaultValue: 65 }}
                         defaultPlaceholder="65"
                       />
+                      <PerModelField
+                        label="Execute Threshold (tokens)"
+                        configKey="execute_threshold_tokens"
+                        description="Optional absolute-tokens threshold. When set for a model, overrides the percentage above. Per-model map only (use 'default' key for a fallback across all unlisted models). Clamped to 80% × context_limit at runtime."
+                        value={getNestedValue(formData(), "execute_threshold_tokens") ?? getNestedValue(parsed(), "execute_threshold_tokens")}
+                        onChange={(v) => handleFieldChange("execute_threshold_tokens", v)}
+                        models={models() ?? []}
+                        inputType="text"
+                        alwaysObject
+                        numericText
+                        defaultPlaceholder="150000"
+                      />
                     </div>
                   </div>
                 )}