Codex dispatch admission: complete missing-step diagnostics, seam tests, ADR

[Trecek]

Summary

open_kitchen(name="implementation") on the codex backend returns dispatch_infeasible (infeasible_steps=["gate_backend_write"], message "backend 'codex' lacks provider overrides for ['retry_worktree']…"). A deep-mode investigation (18 subagents, 2 exploration batches, adversarial challenge round, blast-radius round, 3 validators; full report appended below) concluded the refusal is working-as-designed admission control — the blocker is partial provider-override config (only implement: minimax of 9 git-write-guarded steps) — but the episode exposed a diagnostics defect, a critical test-seam gap, and process failures that let four consecutive PRs (#4094 → #4130 → #4162 → #4164 → #4166) swing the same contract back and forth with green tests every time. This issue is the remediation package for those durable defects.

Remediation Scope

A. Diagnostics completeness in _provider_aware_capability_overrides (blast radius: LOW)

Prerequisite: applies on top of #4166 (origin/develop) — CapabilityResolutionDetail does not exist at 0.10.833.

1. Change the guarded-step resolution loop (src/autoskillit/server/tools/_auto_overrides.py) from bail-on-first-miss to full enumeration so missing_provider_steps reports ALL uncovered steps. Today the envelope names only the first uncovered step, forcing config whack-a-mole: a user covering retry_worktree is next told about the next uncovered step, one open_kitchen round trip at a time (8 total for a partial config). Verified: the three envelope branches (tools_kitchen.py, tools_recipe.py, tools_fleet_dispatch.py) read only resolution_path/missing_provider_steps — zero production consumers break; at most 3 mechanical test updates (tests/server/test_capability_admission_e2e.py:611-612,755,815) if bail_step is removed/renamed per the no-dead-fields rule; docstrings in _auto_overrides.py:62 and core/types/_type_results.py:225 go stale and must be updated.
2. Fleet fallback sentinel: in tools_fleet_dispatch.py, _capability_overrides starts {} and is populated in a try/except; on exception it stays falsy and fleet/_api.py's provider_capability_overrides or _build_capability_overrides(...) silently falls back to the non-provider-aware computation — fleet dispatches would see backend_supports_git_write=false even with a correct user config. Pre-existing defect; add an explicit sentinel or structured log.
3. Correct the record on issue _provider_aware_capability_overrides all-must-pass logic blocks Codex pipeline when only implement has provider override #4165: the [REMEDIATION] Capability Admission Diagnostics Remediation #4166 commit message claims an all→any logic change that the merged code does not implement (correctly so — any-pass was adversarially rejected as unsound). Future investigations must not believe any-pass shipped.

B. Test-seam and contract hardening

The structural hole: no test wires real ProvidersConfig → real _provider_aware_capability_overrides → real load_and_validate → open_kitchen. Server tests hardcode load_and_validate returns; the only real-recipe server test sets recipes.find=None (bypassing the provider-aware branch); recipe tests inject the capability answer directly; unit tests patch _resolve_provider_profile and use synthetic inline provider: fields real recipes don't have (Tier 3 vs the real Tiers 0–2). This is why three PRs shipped green while user-facing behavior flipped each time.

1. Seam tests (top-level functions in tests/server/test_capability_admission_e2e.py, mark medium): partial and full/wildcard config shapes — real ProvidersConfig (assert via resolved_profiles[...].raw_env["ANTHROPIC_BASE_URL"]), real recipe steps via autoskillit.recipe.io.load_recipe(...), real load_and_validate; no mocking of _resolve_provider_profile or load_and_validate. Parametrize to include remediation (guarded step names differ: assess/merge_gate_assess vs fix/merge_gate_fix).
2. Path-filter manifest fix: .autoskillit/test-filter-manifest.yaml maps src/autoskillit/recipes/*.yaml → recipe/ + contracts/ only; add server/test_capability_admission_e2e.py so recipe YAML changes run the admission tests (single-file entries are valid; the existence guard test_manifest_entry_path_exists passes).
3. Ratchet: decision-record docstring on the existing behavioral invariant test test_codex_backend_reachable_gate_returns_infeasible (tests/server/test_capability_admission_e2e.py:34) documenting the canonical contract and the 4-PR flip history, so any future assertion flip requires editing an explicit decision record. (A new arch-layer behavioral test is NOT the right home: tests/arch/test_capability_admission_control.py is small-marked/AST-structural, and duplicating the existing assertion violates redundancy rules.)
4. Config contract test (tests/config/test_providers_config.py): ANTHROPIC_BASE_URL in a profile propagates into resolved_profiles[...].raw_env — the key the entire admission path gates on, currently untested.
5. Static↔runtime key-consistency test: AST-form arch test asserting _auto_overrides.py and tools_execution.py (~line 762) gate on the same "ANTHROPIC_BASE_URL" key. A behavioral round-trip test requires extracting the inline predicate at tools_execution.py:762-767 into a helper — a production refactor to decide during planning.

C. Decision record + process

1. ADR (docs/decisions/0005-…): the codex dispatchability contract — codex + no overrides → infeasible; codex + all 9 guarded steps covered → feasible; codex + partial → infeasible with complete enumeration. Must also record the schema question any future relaxation (e.g. reachability-weighted admission) needs answered first: is review_pr.on_result[changes_requested] a happy-path or failure-path edge for admission purposes? No ADR currently answers any of this; every prior investigation re-derived the contract from the latest symptom.
2. Claimed-scope-vs-diff validation in audit-impl or review-pr: PR Remediate Missing Provider-Aware Capability Override Tests #4164 stated "All changes are test-only. No production code modifications" while modifying 5 production files (+666 lines, introducing _provider_aware_capability_overrides); the mislabel originated in issue Revert #4162: _gate_reachable_post_prune blocks working Codex+MiniMax pipeline — add provider-aware capability gate exemption #4163's framing, propagated through make-plan, and passed audit-impl (which validates diff-vs-plan, and the plan repeated the claim). Add a check that scope claims ("test-only") match changed paths.

Explicitly out of scope (evaluated and killed)

• Any-pass capability logic — unsound: uncovered guarded steps would survive pruning and run as codex workers that cannot write .git (resolve_review fires on ~75% of historical runs → mid-pipeline waste, the original [FIX] Rectify: Capability Admission Control at Recipe Load — Refuse DOA Pipelines #4094 problem). Already rejected in _provider_aware_capability_overrides all-must-pass logic blocks Codex pipeline when only implement has provider override #4165's adversarial addendum.
• Reachability-weighted admission (coverage required only for happy-path guarded steps) — HIGH breakage: the happy/recovery classification is not mechanically derivable from the recipe schema (resolve_review, merge_gate_fix, rebase_conflict_fix enter via ordinary on_result conditions indistinguishable from success routes in _build_success_step_graph); the global backend_supports_git_write ingredient unfences all guarded steps on flip; ~25 skip_when_false occurrences + contract regeneration across 3+ recipes. Blocked on the ADR question above.
• Wildcard "*" config as recommended guidance — works, but empirically reroutes 19 non-guarded steps (plan, verify, audit_impl, review_pr, diagnostics) to the override profile's model; keep the error-message example but the enumeration fix (A.1) makes surgical per-step config practical.

Context: user-side fix already identified (not part of this issue's code scope)

Immediate unblock is config-side: explicit per-step minimax overrides for all 9 guarded steps under providers.recipe_overrides.implementation (empirically validated: dispatch_feasible=True, zero effect on non-guarded steps). Remediation recipe needs its own block with assess/merge_gate_assess in place of fix/merge_gate_fix. Merge into the existing providers: block (do not duplicate the key). Version note: Codex runs autoskillit from the uv-tools install (0.10.834); the repo checkout was at 0.10.833 — git pull needed before implementing Part A.

Related

• Prior fix chain: PR [FIX] Rectify: Capability Admission Control at Recipe Load — Refuse DOA Pipelines #4094 (cfb6ecc), [FIX] Dispatch Feasibility Immunity — Capability-Aware Admission Control #4130 (1112c0f), [FIX] Rectify: Vacuous Gate Reachability Blind Spot Bypasses Dispatch Admission Control #4162 (f5d7724), Remediate Missing Provider-Aware Capability Override Tests #4164 (8f8c2d7), [REMEDIATION] Capability Admission Diagnostics Remediation #4166 (44b2ac0)
• Open staged issues in this area: _is_vacuous_gate reachability blind spot bypasses dispatch_infeasible admission control for Codex backend #4161, Revert #4162: _gate_reachable_post_prune blocks working Codex+MiniMax pipeline — add provider-aware capability gate exemption #4163, _provider_aware_capability_overrides all-must-pass logic blocks Codex pipeline when only implement has provider override #4165, Codex backend dispatch_infeasible blocks entire implementation recipe — hybrid dispatch via _preflight.py extension #4109, Capability admission control at recipe load — refuse DOA pipelines pre-claim #4090
• Full investigation report with evidence citations appended below.

Investigation

Prior investigation completed interactively. See below for root cause analysis.

Investigation: Codex dispatch_infeasible — gate_backend_write / Missing Provider Overrides

Date: 2026-07-01
Scope: open_kitchen(name="implementation") on the codex backend immediately returns {"success": false, "kitchen": "dispatch_infeasible", "infeasible_steps": ["gate_backend_write"]} with message "Cannot dispatch recipe: backend 'codex' lacks provider overrides for ['retry_worktree']…". Root cause of the refusal, the history of fix attempts (#4094 → #4130 → #4162 → #4164 → #4166), why the test suite never catches this class of immediate-runtime failure, and how to prevent recurrence.
Mode: Deep Analysis (2 exploration batches, 3-agent challenge round, 3-agent blast-radius round, 3 validators; 18 subagents total by user instruction to exceed the standard cap)

---

Summary

The dispatch_infeasible refusal is working-as-designed admission control, not a code defect — but it blocks a configuration the user reasonably expected to work, its error reporting was (until #4166) unactionable and is still incomplete, and the path to this state involved a four-PR pendulum in which tests were flipped to mirror each new behavior instead of encoding an invariant. The immediate blocker is the user's partial provider-override config: ~/.autoskillit/config.yaml covers only implement: minimax out of 9 git-write-guarded run_skill steps in implementation.yaml. The all-must-pass check in _provider_aware_capability_overrides bails at the first uncovered step (retry_worktree), keeps backend_supports_git_write="false", pruning removes all guarded steps, route-repair rewires create_impl_worktree.on_success → gate_backend_write, and #4162's reachability check correctly flags the gate as a reachable terminal-failure step. The validated fix is a config change: add explicit minimax overrides for all 9 guarded steps (empirically confirmed: dispatch_feasible=True, zero side effects on non-guarded steps). Two deeper problems remain: (1) the diagnostics name only the first uncovered step, creating config whack-a-mole, and (2) no test wires the real seam (ProvidersConfig → _provider_aware_capability_overrides → load_and_validate → open_kitchen), which is why three consecutive PRs shipped green while user-facing behavior flipped each time.

A version-provenance note that shaped the confusion: the user's Codex session ran v0.10.834 from the uv-tools install (~/.local/share/uv/tools/autoskillit/, auto-upgraded at 16:17 today, one minute before the session), while the repo checkout is at v0.10.833. The exact error text the user saw exists only in #4166's code, which is on origin/develop but not in the local working tree.

Root Cause

Immediate mechanical cause (SUPPORTED, reproduced locally at both v0.10.833 and v0.10.834):

1. CodexBackend.capabilities.git_metadata_writable=False and anthropic_provider_capable=False (src/autoskillit/execution/backends/codex.py:582,594) → base capability override backend_supports_git_write="false".
2. _provider_aware_capability_overrides (src/autoskillit/server/tools/_auto_overrides.py:32-89) iterates the 9 guarded run_skill steps (skip_when_false: inputs.backend_supports_git_write) and requires every one to resolve to a provider profile whose extras contain ANTHROPIC_BASE_URL. The user's config covers only implement (Tier 0 recipe_overrides.implementation.implement: minimax); retry_worktree resolves to the default anthropic profile with empty extras → bail on first miss → capability stays "false".
3. _prune_skipped_steps (src/autoskillit/recipe/_recipe_composition.py:434-555) prunes all 9 guarded steps; route-repair redirects create_impl_worktree.on_success from the pruned implement to implement.on_success = gate_backend_write (implementation.yaml:325).
4. _compute_capability_feasibility (_recipe_composition.py:109-176) finds gate_backend_write surviving with a falsy capability ingredient; _is_vacuous_gate (:179-223) would have exempted it pre-[FIX] Rectify: Vacuous Gate Reachability Blind Spot Bypasses Dispatch Admission Control #4162, but the _gate_reachable_post_prune check (:226-246, added by [FIX] Rectify: Vacuous Gate Reachability Blind Spot Bypasses Dispatch Admission Control #4162/f5d7724f3) detects create_impl_worktree routes to the gate → not vacuous → dispatch_feasible=False, infeasible_steps=["gate_backend_write"].
5. open_kitchen → _dispatch_infeasible_response (src/autoskillit/server/tools/tools_kitchen.py:133-160 at HEAD; extended with missing_provider_steps/escape_hatch on origin/develop by [REMEDIATION] Capability Admission Diagnostics Remediation #4166 commit e2883cc) closes the gate and returns the refusal.

This is a true positive: without the fix, the pipeline dispatches, executes side-effecting steps (clone, claim, branch publish, plan), and then deterministically fails at the runtime gate — the original 21-minute-waste incident that motivated #4094.

Architectural root causes (why this keeps recurring):

• Two capability systems, never reconciled. Load-time admission derives capability from the orchestrator backend globally (_auto_overrides.py), while runtime dispatch resolves providers per step (tools_execution.py:762-770 sets backend_override=claude-code when a step's extras contain ANTHROPIC_BASE_URL). The split-plane model (Codex orchestrates; Claude-compatible workers do git writes outside the Codex sandbox) was bolted on without an interface contract between the two systems.
• No canonical decision record. docs/decisions/ (4 ADRs) contains nothing answering "should codex+implementation be dispatchable, and under what configuration?" Every investigation re-derived the answer from the most recent symptom.
• Tests as mirrors, not invariants. The same assertions flipped three times (see Historical Context); test_codex_implementation_dispatch_infeasible asserted dispatch_feasible is True for a full PR cycle.
• A mock-severed test seam. No test connects real ProvidersConfig → real _provider_aware_capability_overrides → real load_and_validate → open_kitchen envelope (see Test Gap Analysis).

Affected Components

• src/autoskillit/server/tools/_auto_overrides.py — _provider_aware_capability_overrides all-must-pass, bail-on-first-miss loop [SUPPORTED]
• src/autoskillit/server/_guards.py:341-436 — _resolve_provider_profile six-tier waterfall (Tier 0 recipe+step, Tier 0W recipe wildcard "*", Tier 1/2 step_overrides, Tier 3 SKILL.md provider:, Tier 4 default) [SUPPORTED]
• src/autoskillit/recipe/_recipe_composition.py — _prune_skipped_steps route-repair, _compute_capability_feasibility, _is_vacuous_gate, _gate_reachable_post_prune [SUPPORTED]
• src/autoskillit/recipe/_api.py:378-412,605 — feasibility wiring inside load_and_validate [SUPPORTED]
• src/autoskillit/server/tools/tools_kitchen.py — open_kitchen override injection (:670-680) and _dispatch_infeasible_response (:133-160) [SUPPORTED]
• src/autoskillit/recipes/implementation.yaml — 9 guarded steps at lines 328 (implement), 355 (retry_worktree), 688 (fix), 745 (merge_gate_fix), 798 (rebase_conflict_fix), 1095 (resolve_review), 1139 (resolve_pre_review_conflicts), 2092/2108 (resolve_ci), 2148/2157 (resolve_pre_resolve_conflicts); gate_backend_write step at :356-365 [SUPPORTED]
• ~/.autoskillit/config.yaml — user config: recipe_overrides.implementation: {implement: minimax}; # retry_worktree: minimax commented out in step_overrides; minimax profile carries ANTHROPIC_BASE_URL [SUPPORTED]
• ~/.local/share/uv/tools/autoskillit/ — the install Codex actually runs (~/.codex/config.toml → command = "autoskillit" → PATH → uv tools symlink); upgraded to 0.10.834 at 16:17 on 2026-07-01 [SUPPORTED]
• tests/server/test_capability_admission_e2e.py, tests/recipe/test_bundled_recipes_dispatch_ready.py, tests/server/test_backend_ingredient_injection.py, tests/recipe/test_recipe_backend_composition_matrix.py — the flipped/mock-severed test surfaces [SUPPORTED]

Data Flow

open_kitchen("implementation")                              [tools_kitchen.py:645]
  → recipes.find/load (raw steps)                           [tools_kitchen.py:659-676]
  → _provider_aware_capability_overrides(backend, name,
        config.providers, raw_steps)                        [_auto_overrides.py:32-89]
        → _resolve_provider_profile per guarded step        [_guards.py:341-436]
        → implement: minimax (has ANTHROPIC_BASE_URL) ✓
        → retry_worktree: anthropic ({}) ✗ → BAIL
        ⇒ {"backend_supports_git_write": "false"}
  → _promote_capability_keys → merged overrides             [_auto_overrides.py:92-102]
  → load_and_validate(..., ingredient_overrides)            [_api.py]
        → _prune_skipped_steps: 123 → 101 steps; all 9
          guarded steps pruned; route-repair:
          create_impl_worktree.on_success → gate_backend_write
        → _compute_capability_feasibility
            → _is_vacuous_gate → _gate_reachable_post_prune = True (not vacuous)
        ⇒ dispatch_feasible=False, infeasible_steps=["gate_backend_write"]
  → _dispatch_infeasible_response: gate.disable(),
    disable kitchen tags, refusal envelope                  [tools_kitchen.py:133-160, 862-865]

At runtime (the counterfactual the gate prevents): a step with ANTHROPIC_BASE_URL extras gets backend_override=claude-code (tools_execution.py:762-770) and runs as a claude worker outside the Codex sandbox (which mounts .git/ read-only); a step without extras would run as a codex worker and fail on git writes.

Test Gap Analysis

Why three consecutive PRs shipped green while open_kitchen behavior flipped each time — the admission chain spans two layers, and every test mocks exactly the segment where the bug lives:

1. Server-layer tests hardcode the recipe layer. The 7 tests added by Remediate Missing Provider-Aware Capability Override Tests #4164 (test_capability_admission_e2e.py) set tool_ctx.recipes.load_and_validate.return_value = _make_feasible_load_result() (hardcoded dispatch_feasible: True) and patch _resolve_provider_profile globally. They cannot detect a wrong override computation, a broken wiring in open_kitchen, or a config-merge bug — the mocked producer always reports success.
2. The only real-recipe server test bypasses the provider-aware branch. test_codex_open_kitchen_blocks_on_reachable_gate (test_backend_ingredient_injection.py:499) sets recipes.find.return_value = None (line 524), so _provider_aware_capability_overrides receives recipe_steps=None and takes the graceful-degradation early return — right output, wrong code path.
3. Recipe-layer tests inject the answer. test_bundled_recipes_dispatch_ready.py and the backend-composition matrix pass ingredient_overrides={"backend_supports_git_write": "false"|"true"} directly, never invoking the server function that computes the value.
4. Fixture/reality divergence. Unit tests use synthetic steps with inline provider="minimax" fields (Tier 3 resolution); the real bundled recipe has no provider: fields — real users rely on config-level recipe_overrides/step_overrides (Tiers 0-2), a different resolution branch.
5. No partial-config case. The user's actual shape (implement covered, everything else not) was untested until Remediate Missing Provider-Aware Capability Override Tests #4164 added test_provider_aware_capability_override_partial_overrides_stays_false — a unit test that pins the refusal, which is precisely why "the tests pass" while the user is blocked: the blockage is the pinned behavior.
6. Path-filtering hole. .autoskillit/test-filter-manifest.yaml maps src/autoskillit/recipes/*.yaml → recipe/ + contracts/ only; recipe YAML edits never trigger the server admission tests.

The one missing test class: real ProvidersConfig (empty/partial/full shapes) → real _provider_aware_capability_overrides → real load_and_validate on bundled YAML → assert dispatch_feasible + infeasible_steps + envelope actionability. It would have caught every swing of the pendulum.

Similar Patterns

• merge-prs.yaml does not exhibit this failure: its implement.on_success routes to test (not a gate callable), so pruning on codex produces no reachable capability gate — feasibility passes. remediation.yaml, implementation-groups.yaml, and implement-findings.yaml share the gate_backend_write pattern and fail identically on codex without full override coverage (pinned by test_compute_capability_feasibility_returns_infeasible_for_codex_recipes).
• The preflight layer (_preflight.py:_check_dispatch_feasibility) already does per-step provider-aware exclusion for hook enforcement — the admission capability gate is the only layer that evaluates capability globally. (Note: the hook preflight is currently inert — HOOK_REGISTRY has zero fix-required entries — so it blocks nothing today.)
• The repo already has ratchet-style test patterns (INVARIANT_REGISTRY + tests/arch/test_invariant_registry_coverage.py, allowlist ratchets in tests/infra/test_schema_version_convention.py, behavioral pinning docstrings in test_recipe_composition_vacuous_gate.py) — none were applied to the dispatch-feasibility contract.

Design Intent Findings

• gate_backend_write (runtime gate): introduced 207deda91 (2026-06-09, Implementation Plan: Backend-Conditional Routing — gate_backend_write Step #3960) to convert DOA codex endings from false-success (done_no_changes) into honest failure (release_issue_failure). Registered in CAPABILITY_GATE_CALLABLES (core/types/_type_constants.py:114-118), which is what couples it to static admission. [SUPPORTED]
• _check_dispatch_feasibility preflight: introduced a72cb01d8 (2026-06-12, [FIX] Dispatch-Feasibility Preflight — Claim-Before-Validate Immunity #4087) — hook-enforcement feasibility at open_kitchen/dispatch_food_truck before irreversible side effects. [SUPPORTED]
• _compute_capability_feasibility (static admission): introduced cfb6ecc51 (2026-06-12, [FIX] Rectify: Capability Admission Control at Recipe Load — Refuse DOA Pipelines #4094) — "refuse DOA pipelines at recipe load time"; commit message notes 84 prior [FIX] commits had patched links in the capability chain without closing the architectural gap. [SUPPORTED]
• _is_vacuous_gate exemption: introduced 1112c0f3f (2026-06-27, [FIX] Dispatch Feasibility Immunity — Capability-Aware Admission Control #4130) to un-block codex after [FIX] Rectify: Capability Admission Control at Recipe Load — Refuse DOA Pipelines #4094 was judged over-blocking; lacked reachability awareness. [SUPPORTED]
• _gate_reachable_post_prune: introduced f5d7724f3 (2026-06-30, [FIX] Rectify: Vacuous Gate Reachability Blind Spot Bypasses Dispatch Admission Control #4162) closing the route-repair reachability blind spot. [SUPPORTED]
• _provider_aware_capability_overrides (split-plane escape hatch): introduced by the Remediate Missing Provider-Aware Capability Override Tests #4164 merge 8f8c2d762 (2026-07-01; branch commit a4a773485), despite the PR/commit message claiming "All changes are test-only. No production code modifications" — it modified 5 production files (+666 lines incl. fleet/_api.py, tools_kitchen.py, tools_recipe.py, tools_fleet_dispatch.py). The mislabel originated in issue Revert #4162: _gate_reachable_post_prune blocks working Codex+MiniMax pipeline — add provider-aware capability gate exemption #4163's "missing tests" framing and propagated through make-plan → audit-impl unchecked (audit-impl validates diff-vs-plan; the plan's own summary repeated the false claim while its architecture section depicted the production components). [SUPPORTED]
• _auto_overrides.py registry pattern: SERVER_AUTHORITATIVE_INGREDIENTS introduced d70a4b38b (2026-05-27, Rectify: Defense-in-Depth — Server-Authoritative Ingredient Override Registry #3185) — capability keys are server-derived and must win the ingredient merge (_promote_capability_keys). [SUPPORTED]
• Layered defense as intended: L0 ingredient truth (_auto_overrides), L1 static DOA admission (_compute_capability_feasibility, 6 consuming surfaces incl. fleet), L2 hook preflight (_preflight.py), L3 runtime gate (gate_backend_write). [SUPPORTED]

Historical Context

This is a recurring pattern — the fourth+ intervention on the same contract. Prior-fix falsifiability analysis:

Date	PR / commit	Swing	What it missed
Jun 9	#3960 207deda91	Added runtime gate	By design, fires only mid-pipeline (waste already incurred)
Jun 12	#4094 cfb6ecc51	Block (static admission)	No exemption for provider-routed steps → judged over-blocking
Jun 27	#4130 1112c0f3f	Unblock (vacuous-gate exemption)	Route-repair reachability → DOA pipelines re-admitted; tests flipped; test_codex_capability_gate_recipes diluted to a disjunctive assertion
Jun 30	#4162 f5d7724f3	Block (reachability check)	Correct on its own terms; didn't account for split-plane users with per-step providers (the user's setup)
Jul 1	#4164 8f8c2d762	Escape hatch (all-must-pass)	Mislabeled test-only; all-must-pass blocks partial configs — the user's exact case
Jul 1	#4166 44b2ac0ae (origin/develop)	Diagnostics only	Commit message claims all→any logic change; the code does not implement it (correctly so — issue #4165's adversarial addendum concluded any-pass is unsound and the sanctioned fix is config). Reports only the first uncovered step

Open issues: #4163 ("Revert #4162… add provider-aware capability gate exemption") and #4165 (all-must-pass blocks partial config; final correction: "the code IS correct — fix is config") remain OPEN. The "two previous failed attempts" the user experienced are #4164 (escape hatch that their partial config can't satisfy) and #4166 (announced logic change that shipped diagnostics only).

Process gaps sustaining the pendulum: no ADR for the dispatchability contract; investigations without flip-history access; audit-impl validates diff-vs-plan (plans inherit wrong framing from issues); review-pr does not validate PR-body scope claims against the diff; tests flipped freely with no ratchet.

Recurrence check Part A: no prior /investigate reports found in Claude project logs for this root cause (log mining returned no matches). Part B: extensive prior-fix history found (table above).

This is a recurring pattern — consider running /rectify for architectural immunity after resolving the immediate issue.

External Research

Applied to the remediation design (Batch-2 web research):

• "Don't mock what you don't own" / mock at the seam you control, not the interior of the unit under test — mocking owned interior functions (_resolve_provider_profile) creates silent mock drift: https://hynek.me/articles/what-to-mock-in-5-mins/
• Prefer fakes/real value objects over mocks for owned collaborators (ProvidersConfig is a plain dataclass — use it real): https://pytest-with-eric.com/mocking/pytest-common-mocking-problems/
• Characterization tests as ratchets: record intended behavior so changing it requires a deliberate edit: https://cloudamite.com/characterization-testing/
• Contract testing between internal layers (consumer-driven contracts intra-repo): https://www.tweag.io/blog/2025-01-23-contract-testing/
• MCP-server test patterns (schema/envelope contract assertions): https://dev.to/klement_gunndu/your-mcp-server-has-no-tests-here-are-4-patterns-to-fix-that-2k59
• Codex sandbox .git protection (basis of git_metadata_writable=False): codex-rs PROTECTED_METADATA_PATH_NAMES, enforced via Seatbelt (macOS) / bwrap+landlock (Linux) — cited in codex.py:587-594 comments.

Scope Boundary

Investigated: full admission-control code path (both v0.10.833 HEAD and v0.10.834 origin/develop, empirically reproduced); the 6-commit fix chain incl. branch commits; user + project config merge ground truth (Dynaconf deep-merge verified); recipe step-graph and route-repair behavior; runtime split-plane semantics (run_skill backend_override); test coverage across tests/server/, tests/recipe/, tests/arch/; process pipeline (issues #4163/#4165, plan files, audit-impl/review-pr skill definitions); version provenance (uv-tools vs editable install); design alternatives with blast-radius analysis; external testing best practices.

Not yet explored: whether other backends (future non-codex, non-claude) hit analogous gaps; implement-findings.yaml/implementation-groups.yaml guarded-step line-level enumeration (spot-checked only; remediation.yaml's 9 were enumerated during validation — assess/merge_gate_assess differ from implementation's fix/merge_gate_fix); the exact end-to-end runtime behavior of a full codex+minimax pipeline post-config-fix (validated to the load_and_validate + preflight boundary, not a live pipeline run); whether audit-impl should gain a general claimed-scope-vs-diff validator (flagged, not designed); Dynaconf edge cases for "*" keys in YAML (quoting requirements).

Explicitly out of scope: today's failing sessions in sessions.jsonl are backend=claude-code / recipe=remediation — a different root cause than this investigation (on claude-code, git_metadata_writable=True, so this dispatch_infeasible path cannot fire). If those failures matter, they need their own investigation.

Recommendations

Single converged recommendation — a three-part remediation (immediate config unblock + low-risk diagnostics fix + seam-test/ADR hardening):

Part 1 — Unblock now (config only, no code)

Add explicit per-step overrides for all 9 guarded steps to ~/.autoskillit/config.yaml (empirically validated: dispatch_feasible=True, zero effect on non-guarded steps):

providers:
  recipe_overrides:
    implementation:
      implement: minimax          # already present
      retry_worktree: minimax
      fix: minimax
      merge_gate_fix: minimax
      rebase_conflict_fix: minimax
      resolve_review: minimax
      resolve_pre_review_conflicts: minimax
      resolve_ci: minimax
      resolve_pre_resolve_conflicts: minimax

Merge-safety warning: ~/.autoskillit/config.yaml already has a populated providers: block (minimax profile with live credentials, step_overrides, recipe_overrides.remediation). Add ONLY the missing sub-keys under the existing providers.recipe_overrides.implementation: — do not paste a second providers: key (YAML duplicate-key handling would drop the existing profile/credentials).

For remediation, two step names differ — its 9 guarded steps are implement, retry_worktree, assess, merge_gate_assess, rebase_conflict_fix, resolve_review, resolve_pre_review_conflicts, resolve_ci, resolve_pre_resolve_conflicts (assess/merge_gate_assess replace fix/merge_gate_fix). Copying the implementation block verbatim would leave remediation blocked. Same treatment applies to implementation-groups/implement-findings if dispatched on codex (enumerate their guarded steps first).

Avoid the "*" wildcard the error message suggests unless routing everything to minimax is intended — it was empirically shown to reroute 19 non-guarded steps (plan, verify, audit_impl, review_pr, all diagnostics) to the minimax profile's active model (MiniMax-M3; the M2.7-highspeed entries in the profile are commented out).

After Part 1, open_kitchen("implementation") on codex returns {"success": true, "kitchen": "open"} with the ingredients table; the 9 guarded steps survive pruning and execute as minimax-routed claude workers (outside the codex sandbox), and gate_backend_write passes.

Also run git pull in the repo: the working tree (0.10.833) is behind the deployed uv-tools install (0.10.834), a skew that will produce confusing test-vs-runtime mismatches. Part 1 itself needs no pull — it takes effect against the installed 0.10.834 binary immediately.

Part 2 — Diagnostics completeness (small code fix on top of #4166; blast radius LOW)

Hard prerequisite: git pull first — CapabilityResolutionDetail and the tuple return type exist only on origin/develop (#4166), not in the working tree at HEAD.

Change _provider_aware_capability_overrides's loop from bail-on-first-miss to full enumeration so missing_provider_steps reports all uncovered steps (today a user fixing retry_worktree would next be told about the next uncovered step, one per attempt — eight sequential round trips). Verified: the three envelope branches read only resolution_path/missing_provider_steps (not bail_step); zero production consumers break; at most 3 mechanical test updates if bail_step is removed/renamed per the no-dead-fields rule. Also correct the misleading #4166 commit narrative in the tracking issue (#4165) so future investigations don't believe any-pass logic shipped.

Residual risk to address while in this code: in tools_fleet_dispatch.py, _capability_overrides starts as {} and is populated in a try/except; on exception it stays falsy, and fleet/_api.py's provider_capability_overrides or _build_capability_overrides(...) silently falls back to the non-provider-aware computation — fleet dispatches would see backend_supports_git_write=false even with a correct user config. Pre-existing defect; worth an explicit sentinel or log.

Part 3 — Seal it off architecturally (feed to /rectify / make-plan)

1. ADR (docs/decisions/0005-…): the codex dispatchability contract — codex + no overrides → infeasible; codex + all 9 guarded steps covered → feasible; codex + partial → infeasible with complete enumeration; and the schema-level question a future relaxation must answer first: is review_pr.on_result[changes_requested] a happy-path or failure-path edge for admission purposes?
2. Seam tests (adjusted for repo conventions): top-level functions in tests/server/test_capability_admission_e2e.py for partial and full/wildcard config shapes — real ProvidersConfig (assert via resolved_profiles[...].raw_env["ANTHROPIC_BASE_URL"]), real recipe steps via autoskillit.recipe.io.load_recipe(...), real load_and_validate — no mocking of _resolve_provider_profile or load_and_validate. Mark medium. Parametrize to include remediation (its guarded step names differ: assess/merge_gate_assess).
3. Ratchet: decision-record docstring on the existing test_codex_backend_reachable_gate_returns_infeasible in tests/server/test_capability_admission_e2e.py:34 — that is where the behavioral invariant is asserted today. A new arch-layer behavioral test is NOT feasible: tests/arch/test_capability_admission_control.py is small-marked (no persistent I/O) and contains only structural/AST checks, and duplicating the existing behavioral assertion would violate repo redundancy rules.
4. Manifest fix: add server/test_capability_admission_e2e.py to the src/autoskillit/recipes/*.yaml entry in .autoskillit/test-filter-manifest.yaml (verified real file; single-file entries are valid; existence guard passes).
5. Config contract test: tests/config/test_providers_config.py — ANTHROPIC_BASE_URL propagates into raw_env (drives the admission path; currently untested for this key).
6. Static↔runtime consistency: AST-form arch test asserting _auto_overrides.py and tools_execution.py gate on the same "ANTHROPIC_BASE_URL" key (behavioral round-trip test deferred — requires extracting the inline predicate at tools_execution.py:762-767, a production refactor to decide in the plan).
7. Process: add a claimed-scope-vs-diff check ("test-only" claims validated against changed paths) to audit-impl or review-pr — the Remediate Missing Provider-Aware Capability Override Tests #4164 mislabel passed every gate.

Killed alternatives (with reasons):

• Any-pass capability logic — killed. Uncovered guarded steps would survive pruning and run as codex workers that cannot write .git; resolve_review fires on ~75% of historical runs, so most pipelines would fail mid-run (5–15 min waste each). Already adversarially rejected in issue _provider_aware_capability_overrides all-must-pass logic blocks Codex pipeline when only implement has provider override #4165; [REMEDIATION] Capability Admission Diagnostics Remediation #4166's message claimed it but correctly did not ship it.
• Wildcard "*" config as the recommended fix — demoted to opt-in. Works (validated) but silently reroutes 19 non-guarded steps' model/provider.
• Reachability-weighted admission (require coverage only for happy-path guarded steps) — killed for now, HIGH breakage: the happy/recovery classification is not mechanically derivable from the recipe schema (resolve_review, merge_gate_fix, rebase_conflict_fix enter via ordinary on_result conditions indistinguishable from success routes; _build_success_step_graph cannot separate them), the global ingredient unfences all guarded steps on flip, ~25 YAML occurrences + contract regeneration across 3+ recipes would churn, and the 4-PR pendulum shows this surface is high-risk without the ADR question resolved first.
• Message-only tweak (note "other steps may also be missing") — superseded by Part 2, which costs little more and eliminates the whack-a-mole entirely.

Breakage Analysis

• Recommendation Part 1 (config): additive user-config change.
  • Breakage surface: none in code. Behavior change: the 9 guarded steps survive pruning and execute via minimax-routed claude workers — this is the intended split-plane mode; non-guarded steps unaffected (verified by exhaustive per-step resolution scan).
  • Prior reverts: none (config-side).
  • Downstream contract violations: none — gate_backend_write receives "true" and passes; hook preflight inert (0 fix-required hooks).
  • Risk: LOW.
• Recommendation Part 2 (enumeration): replaces bail-on-first-miss loop exit with accumulation.
  • Breakage surface: CapabilityResolutionDetail consumers — envelope branches in tools_kitchen.py:164-166, tools_recipe.py:265-267, tools_fleet_dispatch.py:357-358 (read only resolution_path/missing_provider_steps; unaffected); formatter _fmt_dispatch.py:53-55 (reactive; unaffected); tests asserting bail_step at test_capability_admission_e2e.py:611-612,755,815 (0 breaks if bail_step kept as first-miss; 3 mechanical rewrites if removed); docstrings in _auto_overrides.py:62 and _type_results.py:225 go stale and must be updated.
  • Prior reverts: none on _auto_overrides.py (git log --grep=revert empty).
  • Downstream contract violations: none — capability outcome ("false" on partial) unchanged; only detail completeness changes.
  • Risk: LOW (LOW-MEDIUM if bail_step is removed).
• Recommendation Part 3 (tests/ADR/manifest/process): additive; no mechanism removed.
  • Breakage surface: new tests could expose latent bugs (that is their purpose); manifest addition widens filtered-test selection for recipe YAML changes (longer CI on those diffs); ADR constrains future PRs by policy, not code.
  • Prior reverts: n/a.
  • Risk: LOW.
• Killed candidate (reachability-weighted admission), documented for the record: breakage surface spans _auto_overrides.py, fleet/_api.py parity fn, _type_constants.py registries, 3+ recipe YAMLs (~25 skip_when_false occurrences), contract cards under recipes/contracts/ (freshness hook blocks stale hashes), semantic rules in recipe/rules/rules_bypass.py, and the conservative-behavior pin test_provider_aware_capability_override_partial_overrides_stays_false (origin/develop test_capability_admission_e2e.py:269). Pendulum history = de-facto revert pattern. Risk: HIGH — do not attempt before the ADR resolves the edge-classification question.

---

Empirical artifacts (temp scripts + verbatim outputs backing the SUPPORTED findings): .autoskillit/temp/investigate/repro_codex_feasibility.py, full_investigation.py, adversarial_wildcard_fix.py, surgical_config_fix_nine_steps.py. (Pyright warnings on these throwaway scripts are expected — they duck-type backend fakes against a structural Protocol.)