feat(bench): add preregistered v9 efficiency controller

Author: tbitcs

.github/workflows/bench.yml

@@ -28,6 +28,9 @@ on:
           - publication-open-admission
           - publication-open-screen
           - publication-open-release
+          - literature-v9-ablation
+          - literature-v9-screen
+          - literature-v9-release
         default: custom
       experiment:
         description: "Versioned controller protocol; use one admission per causal variant."
@@ -60,7 +63,20 @@ on:
           - scalar-milestone-packet-authority-v6
           - scalar-milestone-packet-authority-v7
           - scalar-milestone-packet-authority-v8
+          - scalar-milestone-packet-authority-v9
         default: control
+      controller_features:
+        description: "v9 feature set: all, retrieval, working-context, lanes, early-stop, or early-stop,critical-replay."
+        required: false
+        default: "all"
+      escalation_model:
+        description: "Optional stronger model used only after a v9 milestone handoff."
+        required: false
+        default: ""
+      escalation_provider:
+        description: "Provider for the optional escalation model."
+        required: false
+        default: "openai-responses"
       groups:
         description: "Model groups to run (comma-separated: open,open-qwen,open-mid-admission,open-mid-route-recovery,open-mid-followup,open-frontier,open-direct,openai,openai-native,anthropic,google). Default: open."
         required: false
@@ -415,15 +431,25 @@ jobs:
           BENCH_DRY_RUN:     ${{ github.event.inputs.dry_run == 'true' && '1' || '' }}
           BENCH_MAX_TURNS:   "12"
           BENCH_CONTROLLER_EXPERIMENT: ${{ github.event.inputs.experiment || 'control' }}
+          BENCH_CONTROLLER_FEATURES: ${{ github.event.inputs.controller_features || 'all' }}
+          ESCALATION_MODEL: ${{ github.event.inputs.escalation_model || '' }}
+          ESCALATION_PROVIDER: ${{ github.event.inputs.escalation_provider || 'openai-responses' }}
         run: |
           DRY_FLAG=""
           [ "${{ github.event.inputs.dry_run }}" = "true" ] && DRY_FLAG="--dry-run"
+          ESCALATION_ARGS=""
+          if [ -n "$ESCALATION_MODEL" ]; then
+            ESCALATION_ARGS="--escalation-model $ESCALATION_MODEL --escalation-provider $ESCALATION_PROVIDER"
+          fi
           python scripts/govern_bench/run_bench.py \
             ${{ steps.build-args.outputs.args }} \
             --provider ${{ matrix.provider }} \
             --model "${{ matrix.model }}" \
+            --controller-features "$BENCH_CONTROLLER_FEATURES" \
             --output   "bench-report-${{ matrix.label }}.md" \
             --json-output "bench-results-${{ matrix.label }}.json" \
+            --policy-output "bench-policy-${{ matrix.label }}.jsonl" \
+            $ESCALATION_ARGS \
             $DRY_FLAG
 
       - name: Upload per-model artifact
@@ -435,6 +461,7 @@ jobs:
             bench-report-${{ matrix.label }}.md
             bench-results-${{ matrix.label }}.json
             bench-results-${{ matrix.label }}.audit.json
+            bench-policy-${{ matrix.label }}.jsonl
           retention-days: 30
           if-no-files-found: ignore
 

CHANGELOG.md

@@ -11,6 +11,11 @@ consolidated into the next published release.
 ## [Unreleased]
 
 ### Added
+- Add the preregistered literature-driven v9 benchmark controller without
+  changing frozen v8: role-aware source-on-demand retrieval, lossless bounded
+  working context, deterministic controller lanes, execution-derived early
+  stop, focused critical replay, optional same-fixture model escalation,
+  trace-derived policy exports, and locked n=1/n=5/n=10 experiment profiles.
 - Freeze preregistration `GB-PREPRINT-2026-07-30-V1` before paid inference.
   It adds the locked T28/T29/T30 three-condition n=10 publication grid,
   sequential n=1/n=5/n=10 admissions for six exact 20B-35B open-model routes,

README.md

@@ -136,32 +136,18 @@ the [`examples/policies`](examples/policies) directory.
 
 ## Governance efficiency benchmark
 
-In the same-commit GPT-5.6 Sol n=10 replication, Specsmith FULL passed 80/80
-cells at 10.7k tokens per correct answer (TPCA); the versioned Cursor-style
-condition passed 70/80 at 24.8k. A separate preregistered mixed-suite run found
-Terra + FULL at 80/80 and 11.7k TPCA versus raw Sol at 65/80 and 28.0k; its
-coding-only correctness gate remained inconclusive.
-
-The fresh synthetic polyglot T29 package later passed an independent 10/10
-confirmation first-pass at 17.6k TPCA, five turns, and 1.62% CV. These are
-task-, route-, and benchmark-specific results. Published-size open models have
-not passed the repeated frontier-replacement gates, so Specsmith does not claim
-that small models generally replace frontier models.
-
-The publication-readiness round adds a commit-pinned independent upstream
-repository, matched raw/Cursor/Specsmith conditions, explicit failed-run
-expenditure, and sequential 20B–35B open-model admission. Its self-audit
-invalidated the original T30 evaluator before publication; corrected frozen
-protocol `GB-PREPRINT-2026-07-30-V2` is the only scored real-repository
-stratum. Full receipts and limitations stay in the benchmark report rather
-than this quick start.
-
-On corrected T30, Sol FULL passed 8/10 at 105.5k TPCA versus raw Sol at 5/10
-and 234.5k and the versioned Cursor-style condition at 1/10 and 1,027.1k.
-Terra FULL improved over Terra controls but passed only 2/10, so the
-lower-tier substitution claim does not transfer to this repository. The
-within-model point estimates favor FULL, but their joint uncertainty gates
-remain unconfirmed at n=10.
+Same-commit repeated runs show that deterministic requirements, bounded
+context, and executable verification can materially reduce tokens per correct
+answer on the evaluated task/route combinations. The result is conditional:
+it transfers strongly on some synthetic suites, but current 20B–35B routes and
+the independent upstream task do not substantiate a general claim that small
+models replace frontier models.
+
+The preregistered v9 experiment now isolates role-aware retrieval, lossless
+working-context bounds, minimal controller lanes, execution-derived early
+stopping, focused replay, and optional stronger-model handoff. These are
+experimental controls, not published gains, until they clear n=1, n=5, and
+n=10 correctness and efficiency gates.
 
 See the
 [full benchmark report](https://specsmith.readthedocs.io/stable/efficiency-benchmark/),

docs/ARCHITECTURE.md

@@ -41,6 +41,15 @@ The free default ESDB uses SQLite. The optional ChronoMemory backend adds its
 licensed ChronoStore implementation. Both expose the same Specsmith evidence
 boundary so integrations do not depend on a proprietary backend.
 
+`specsmith.efficiency_controller` is the reusable experimental boundary for
+long-horizon model efficiency. It selects the smallest deterministic lane that
+can satisfy a task, keeps only five recent tool exchanges in working context,
+and archives every evicted exchange losslessly under a content-addressed
+evidence reference. Role/symbol/dependency repository maps identify likely
+files without preloading raw source. Repeated execution signatures trigger one
+focused replay and then a machine-readable milestone handoff; an optional
+stronger route resumes the same fixture and diff rather than restarting.
+
 ### Interfaces
 
 - The focused CLI supports project adoption, requirements, tests, preflight,
@@ -82,6 +91,8 @@ flowchart LR
 7. Windows, Linux, and macOS use equivalent paths and command semantics.
 8. Specsmith publication occurs only through reviewed release branches and
    repository-local CI.
+9. Experimental benchmark controllers never mutate a frozen publication
+   controller, and hidden acceptance runs only after the final route stops.
 
 ## Release architecture
 

docs/LEDGER.md

@@ -561,3 +561,41 @@
 - **Status**: complete
 - **Epistemic status**: high
 - **Chain hash**: `e148e0000c209daf...`
+
+## 2026-07-31T07:44 — KILL SWITCH ACTIVATED: emergency stop
+- **Author**: specsmith-operator
+- **Type**: kill-switch
+- **REQs affected**: REG-005
+- **Status**: complete
+- **Epistemic status**: high
+- **Chain hash**: `d90f093acee88be2...`
+
+## 2026-07-31T09:09 — wi_close WI-9EEA67280570: REQ-484 v9 implementation and release-path verification complete
+- **Author**: specsmith
+- **Type**: wi_close
+- **Status**: complete
+- **Chain hash**: `23f0c39ccfa156a5...`
+
+## 2026-07-31T09:09 — wi_close WI-9D3192B60628: Accepted efficiency improvements implemented and verified
+- **Author**: specsmith
+- **Type**: wi_close
+- **Status**: complete
+- **Chain hash**: `d33ec2683ac0bf60...`
+
+## 2026-07-31T09:09 — wi_close WI-3E78F9B1FE6B: Canonical requirements, tests, docs, and preprint updated and verified
+- **Author**: specsmith
+- **Type**: wi_close
+- **Status**: complete
+- **Chain hash**: `27d2379ef500efe4...`
+
+## 2026-07-31T09:09 — wi_archive WI-96A3586F61CD: Read-only research assessment completed and subsumed by REQ-484 v9
+- **Author**: specsmith
+- **Type**: wi_archive
+- **Status**: complete
+- **Chain hash**: `c90aefea916d9e55...`
+
+## 2026-07-31T09:09 — wi_archive WI-EC9713B9BF5A: Read-only implementation planning completed and subsumed by REQ-484 v9
+- **Author**: specsmith
+- **Type**: wi_archive
+- **Status**: complete
+- **Chain hash**: `7859f9465d9eaacb...`

docs/requirements/govern_bench.yml

@@ -35,6 +35,18 @@
     including simulated dispatch, MUST be excluded from published comparisons.
     A deterministic single-file repair handoff SHOULD provide bounded current
     content and a write-only tool surface so compaction does not force a reread.
+    New controller changes MUST be isolated behind a versioned experiment and
+    MUST NOT mutate a frozen publication controller. The literature-driven v9
+    experiment MUST support role-aware source-on-demand retrieval, lossless
+    evidence references with a bounded recent working history, deterministic
+    selection of the smallest sufficient controller lane, execution-derived
+    stall replay and milestone escalation, and an optional same-fixture model
+    cascade that evaluates the hidden oracle exactly once. Its retrieval,
+    working-context, lane-selection, early-stop, and critical-replay controls
+    MUST be independently ablatable before n=1 admission, n=5 screening, and
+    n=10 confirmation. Raw rows MUST retain controller lane, context pruning,
+    evidence references, handoff state, failed-token mass, and trace-derived
+    policy examples so efficiency decisions remain reproducible.
   source: scripts/govern_bench benchmark trustworthiness fixes (plan aebed8bd)
   status: implemented
 - id: REQ-485

docs/site/benchmark-audit.md

@@ -1,5 +1,22 @@
 # Long-Horizon Benchmark and Weakness Audit
 
+## V9 efficiency audit contract
+
+The literature-driven v9 controller adds two directly actionable audit signals:
+`failed_token_share` identifies how much spend ended in incorrect rows, and
+`milestone_escalation_rate` identifies how often a route could not recover
+within one focused replay. It also records peak/pruned working-context size,
+evidence references, deterministic lane choice, and the exact handoff used by
+an optional stronger model.
+
+A repeated execution signature now restores only the active file boundary once.
+Another identical signature stops the current route and emits completed,
+pending, changed-file, validator, and evidence state. The next model continues
+that work; it does not receive a clean restart or an early hidden-oracle hint.
+The audit recommends a cascade comparison when escalation is recurring and
+blocks repetition when failed runs hold at least half the observed token mass
+in an n≥5 slice.
+
 GovernanceBench `T28` is a 20-turn product task spanning a Python/FastAPI API,
 Go worker, TypeScript/React UI, Playwright journey, JSON Schema, CSS, public
 tests, and architecture documentation. Its result is reported separately as

docs/site/efficiency-benchmark.md

@@ -1,5 +1,31 @@
 # Specsmith Governance Efficiency Benchmark
 
+## Preregistered literature-driven v9 experiment
+
+The next controller study is preregistered in
+`scripts/govern_bench/LITERATURE_V9_PROTOCOL.yml`; it does not alter the frozen
+v8 publication controller and has no reported gain yet. Six cells isolate
+role-aware retrieval, lossless five-pair working context, smallest-lane
+selection, execution-derived early stopping, focused replay, and their combined
+effect on T28, T29, and the pinned upstream T30 repository. Promotion remains
+n=1 admission → n=5 screen → n=10 confirmation.
+
+The design is motivated by primary studies reporting that role-aware code
+representations can improve localization with much smaller footprints
+([Caumartin et al.](https://arxiv.org/abs/2607.11046)); five-pair pruning can
+lower token use while improving completion in a long-horizon tool workflow
+([Lodha et al.](https://arxiv.org/abs/2606.10209)); indexed external evidence
+can bound working context without discarding the underlying observations
+([Memex](https://arxiv.org/abs/2603.04257)); and critical trajectory replay can
+reduce test-time scaling cost ([SWE-Replay](https://arxiv.org/abs/2601.22129)).
+Those results motivate hypotheses; they are not evidence that the mechanisms
+will transfer to GovernanceBench.
+
+Raw v9 rows add controller lane, peak and pruned working-context characters,
+evidence-reference count, failed-token share, milestone escalation, handoff
+state, and trace-derived policy examples. The audit fails closed on missing or
+errored cells and treats high failed-token mass as an efficiency blocker.
+
 ## Current promotion and release protocol
 
 Benchmark spend now follows locked, versioned profiles:

docs/site/preprint.md

@@ -18,6 +18,28 @@ does not imply a parameter count or architecture that the provider has not
 published. “Small model” is reserved for a route with a published size, such
 as Llama 3.1 8B.
 
+## Publication hold and next evidence
+
+The preprint remains unsubmitted while the preregistered v9 controller is
+evaluated. Its mechanisms are literature-backed hypotheses—role-aware compact
+retrieval, five-pair lossless context, minimal controller lanes, one critical
+replay, and bounded model escalation—not claimed improvements. Each is first
+ablated on T28/T29/T30 at n=1; only a fully correct cell advances to n=5, and
+only a passing screen advances to independent n=10 confirmation.
+
+This also tests whether Specsmith's narrow integration direction is justified.
+[SWE-Skills-Bench](https://arxiv.org/abs/2603.15401) reports that most tested
+generic skills did not improve pass rate and sometimes added substantial token
+overhead, supporting measurement of specialized governance controls rather
+than expansion of the default skill catalogue. Training or distillation is a
+later stage: trace-derived controller labels can support focused verifiers or
+compression policies, while executable environments and diverse generated
+tasks follow the evidence strategy of
+[SWE-Gym](https://arxiv.org/abs/2412.21139),
+[SWE-smith](https://arxiv.org/abs/2504.21798), and
+[SWE-Dev](https://arxiv.org/abs/2506.07636). No broader small-model claim is
+approved until repeated fresh-repository gates pass.
+
 ## Claim ladder
 
 Public wording follows the strongest completed evidence gate, not the most

docs/tests/govern_bench.yml

@@ -15,10 +15,17 @@
     each selected model for a live provider before a full run, fail partial real
     runs, reject incomplete or mismatched comparison artifacts, and hand a
     single-file deterministic repair its bounded current content without an
-    avoidable reread turn.
+    avoidable reread turn. Tests also verify that the versioned v9 experiment
+    leaves frozen v8 behavior unchanged, ranks compact role/symbol/dependency
+    repository entries, preserves pruned evidence by exact reference, retains
+    only five recent tool pairs, selects deterministic controller lanes,
+    restores only the focused file during critical replay, escalates repeated
+    execution failures with a machine-readable handoff, defers hidden-oracle
+    evaluation until a cascade finishes, exposes isolated controller features,
+    exports trace-derived policy examples, and defines n=1, n=5, and n=10 gates.
   requirement_id: REQ-484
   type: integration
-  verification_method: pytest tests/test_govern_bench_integrity.py tests/test_benchmark_harness.py
+  verification_method: pytest tests/test_govern_bench_integrity.py tests/test_benchmark_harness.py tests/test_efficiency_controller.py tests/test_benchmark_v9_controller.py
   input: {}
   expected_behavior: {}
   confidence: 1.0