Skip to content

Capability admission control at recipe load — refuse DOA pipelines pre-claim #4090

Open
Open
Capability admission control at recipe load — refuse DOA pipelines pre-claim#4090
Labels
enhancementNew feature or requestrecipe:implementationRoute: proceed directly to implementationstagedImplementation staged and waiting for promotion to main

Description

@Trecek
Trecek
opened on Jun 12, 2026

Problem

Implementation-family pipelines on the Codex backend are admitted at open_kitchen despite being provably unable to succeed. CodexBackend correctly declares git_metadata_writable=False (src/autoskillit/execution/backends/codex.py:521 — Codex CLI's workspace-write sandbox makes .git/ read-only by design; headless sessions never receive the bypass flag). The derived hidden ingredient backend_supports_git_write=false prunes implement/retry_worktree via skip_when_false (recipe/_recipe_composition.py:294-415), rerouting into gate_backend_write (recipes/implementation.yaml:330,333,360,361-370), whose only outcome on the normal verdict=plan path is release_issue_failure. Nothing refuses to start the run: the orchestrator claims the issue, publishes a branch, runs make-plan and dry-walkthrough, creates a worktree, then dies ~21 minutes in.

Latest casualty: TalonT-Org/the-token-reserve#149 (2026-06-12) on autoskillit 0.10.772 == repo HEAD — all 84 recent [FIX] commits were active, including #4087's dispatch-feasibility preflight, which passes vacuously on Codex (zero fix-required hooks exist; it checks only the hook class and never inspects backend_supports_git_write). 48 abandoned impl-* clones accumulated in ~6 days. The Codex backend has never completed an end-to-end implementation pipeline.

Root Cause

Missing admission invariant: everything needed to conclude the run is dead-on-arrival is known at recipe load time (static backend capability, resolved ingredient, completed pruning), but load_and_validate blesses the gutted recipe and side effects (claim, branch publish, clone, worktree) execute before the in-pipeline gate fires. The capability flag is correct; it is consumed mid-pipeline instead of at the door.

Proximate cause: #4068 (8734c6355) removed the accidental admission blocker (pre-prune validation errors used to block Codex open_kitchen) without a precise replacement, and added tests pinning valid=True/success=True for Codex+implementation.

Evidence chain: codex.py:521 → server/tools/_auto_overrides.py:19-28 (mirrored at fleet/_api.py:112-120) → server/tools/tools_kitchen.py:574-581 → pruning → gate. The appended investigation report contains the complete analysis, adversarial challenge, and fix-history falsification (#3844 → #4087: every prior fix operated downstream of admission, or implemented admission for a different failure class).

Prescribed Mechanism

  1. Purpose-step annotation — mark purpose-critical steps (e.g. implement) in implementation-family recipes (implementation.yaml, remediation.yaml, implementation-groups.yaml). Schema plumbing is required in three coordinated places: RecipeStep (recipe/schema.py), _PARSE_STEP_HANDLED_FIELDS, and _parse_step (recipe/io.py) — the import-time assertion at io.py:391-398 raises RuntimeError on divergence, and an unplumbed YAML key is silently dropped by the parser.
  2. Infeasibility computation in load_and_validate — after _prune_skipped_steps (its resolutions dict already names every pruned step, threaded at recipe/_api.py:374; the triggering ingredient is derivable from _pre_prune_steps[name].skip_when_false): if any purpose-critical step was pruned, the recipe is infeasible on this backend. Produce a machine-readable refusal naming the backend, the missing capability, and the pruned purpose steps. Key the check on backend_name (already a load_and_validate parameter, passed even by fleet's inner _run_dispatch load at fleet/_api.py:354) — NOT on the presence of capability ingredient overrides, which that call site omits. Bridge constraint (adversarially validated): recipe/ (IL-2) is forbidden from importing execution/ (pyproject import-linter contract "IL-2 recipe imports only IL-0 and workspace", pyproject.toml:282-290), so the backend_name→capability bridge must be either an IL-0 capability mapping in core/types or an injected capability_overrides parameter. Fleet's _run_dispatch has _build_capability_overrides available (fleet/_api.py:112-120) but does not pass overrides to its load_and_validate call (:349-355) — with no overrides the recipe default 'true' applies, nothing is pruned, and a pruning-keyed check passes vacuously there (tolerable only because dispatch_food_truck's preflight runs upstream with correct overrides; the bridge must still be explicit).
  3. Fail-closed refusal at all four recipe-content-serving surfaces — both open_kitchen paths (normal at tools_kitchen.py:684-796 and deferred-recall at :595-683; the new refusal branches must follow the preflight-failure shape at :660-661/:765-766 — gate.disable() AND await ctx.disable_components(tags={"kitchen"}) — not the partial validation-failure blocks at :647/:751, and they need a NEW response builder because _recipe_validation_error_response (tools_kitchen.py:104-126) carries no ingredients_table), get_recipe (tools_kitchen.py:425 gates only on valid; infeasibility is a separate signal, so a dispatch_feasible gate must be added), and load_recipe (tools_recipe.py:237-238 currently returns full recipe content with only a validation_failed: True flag — needs an explicit refusal branch). The refusal fires before the kitchen gate enables side-effect tools — and therefore before any claim, branch publish, or clone.
  4. Discovery flows preserved — the refusal response must still carry ingredients_table so open_kitchen(ingredients_only=True) discovery (_prompts_kitchen.py:42) keeps working: callers get the schema plus dispatch_feasible=false, not a bare error. Signal design: structural valid semantics stay unchanged; infeasibility is carried in new dispatch_feasible: bool + infeasible_steps: list[str] fields. LoadRecipeResult is TypedDict(total=False) and already carries ingredients_table, so the extension is additive; mirror the new fields in OpenKitchenResult, and every serving surface gates on both valid AND dispatch_feasible.
  5. Defense-in-depth retained — gate_backend_write stays untouched; _check_dispatch_feasibility ([FIX] Dispatch-Feasibility Preflight — Claim-Before-Validate Immunity #4087, already wired into open_kitchen and dispatch_food_truck) consumes the same infeasibility computation so fleet dispatch refuses before claiming.
  6. Rot prevention — an arch test cross-references SKILL_CAPABILITY_REGISTRY / skill uses_capabilities so that every bundled-recipe step invoking a git_metadata_write-class skill carries an explicit classification — purpose-critical (refuse at admission) or degraded-ok (pruned silently by design) — with no unclassified git-write step allowed (same enforcement pattern as tests/arch/test_skill_backend_annotations.py). Adversarial validation found two recipes beyond the implementation family that DO prune git-write steps on Codex and need explicit classification decisions: merge-prs.yaml (implement, skill implement-worktree-no-merge declares git_metadata_write) and research-implement.yaml:230 (implement_phase, skill implement-experiment declares git_metadata_write). User-authored recipes without annotations are fail-open (admitted vacuously) — a deliberate default that must be documented. An integration test asserts every content-serving surface refuses an infeasible recipe — computing the signal in one place is necessary but not sufficient (load_recipe proves surfaces must also act on it).

Tests to Update (currently pin the wrong behavior)

  • tests/server/test_backend_ingredient_injection.py:494-495,559-565 — valid=True pin and test_codex_open_kitchen_returns_success
  • tests/recipe/test_bundled_recipes_dispatch_ready.py:125-148 — codex × dispatch-gate recipes asserted dispatch-ready
  • tests/recipe/test_recipe_backend_composition_matrix.py — (implementation, codex) cell

Killed Alternatives (do not re-litigate without new evidence)

  • backend_supports_git_write check inside _check_dispatch_feasibility — hand-keyed flag check is the same rot class as prior fixes; merge-prs/research-* recipes prune git steps by design and would need per-recipe exemptions.
  • --sandbox danger-full-access for headless Codex — inverts the deliberate security posture (Codex .git protection is intentional, Enforce workspace metadata protections in Linux sandbox openai/codex#19852) and trades a clean failure for known headless instability (--full-auto and --sandbox workspace-write flags cause indefinite hang with orphaned child processes openai/codex#7852 hangs, #14345 trust prompts).
  • Host-side commit-on-behalf (backend writes files, orchestrator commits) — the correct long-term feature to make Codex implementation-capable and consistent with the existing host-side-git architecture, but separate-ticket scope; does not remove the need for admission control for other capability gaps.
  • Naive "any success terminal reachable?" BFS — adversarially verified as defeated by the false_positive/already_done exception paths, which legitimately succeed on Codex because their git work is host-side. Purpose-step pruning is the semantics that survives that counterexample.

Split Assessment

Single ticket. The three stages (schema plumbing → infeasibility computation → serving-surface refusal branches) are sequentially coupled through shared files (recipe/_api.py, recipe/io.py, server/tools/tools_kitchen.py); a parallel split would guarantee merge conflicts, failing the parallel-implementation criterion.

Related Issues

Sibling of #4083 (hook-class dispatch-feasibility preflight → fixed by #4087) — same "claim-before-validate" anti-pattern, different capability class that #4087's implementation does not cover. Related: #4059 (pre-prune validity; fix #4068 removed the accidental blocker that previously masked this gap), #3971 (fleet backend_supports_git_write authority contract), #3976 (resolve_ci guard gap).

Breakage risk: MEDIUM — concentrated in the pinned-test updates, the discovery-flow response shape, and the four serving-surface refusal branches.

Adversarial Validation Addendum (2026-06-12)

Two independent adversarial agents (fact-checker + gap analyst) attacked this issue body; every finding was then re-verified by hand against the actual files before being applied. Confirmed corrections integrated above: _prune_skipped_steps spans :294-415; clone count 48; strict [FIX] count 84; recipe/_rule_helpers.py path; the IL-2 import constraint forcing an explicit backend_name→capability bridge; the valid-vs-dispatch_feasible signal split (get_recipe/load_recipe need new gates); the ingredients_table-capable refusal builder; the disable_components asymmetry; and the merge-prs.yaml/research-implement.yaml git-write steps requiring explicit purpose-critical vs degraded-ok classification. Adversary claims rejected after hand-verification: "94 FIX-grep unreproducible" (it reproduces exactly via case-sensitive --grep="FIX"), "skip_when_false count is 227 not 53" (227 counts all ingredients; the 53 figure is correctly scoped to backend_supports_git_write), and the scaffold-date discrepancy (2026-05-20 is the author date).

Investigation

Prior investigation completed interactively (deep mode: 2 exploration batches, adversarial challenge round, breakage analysis, 2-validator post-report validation). Full report follows.

Investigation: Recurring Codex-Backend Implementation Pipeline Failures (gate_backend_write Family)

Date: 2026-06-12
Scope: Why Codex-backend implementation pipelines keep dying at gate_backend_write (latest casualty: TalonT-Org/the-token-reserve#149, run impl-20260612-071816-761474), why ~3 weeks / 84 [FIX] commits of remediation have not stopped the family, the true root cause, and an approach immune to the failure mode that defeated prior fixes.
Mode: Deep Analysis (2 exploration batches + adversarial challenge + breakage analysis + post-report validation; 9-subagent budget)

Summary

The failed run behaved exactly as the current code is designed to behave. The Codex backend correctly and statically declares that it cannot write .git/ metadata (git commit, rebase, worktree add are blocked by Codex CLI's workspace-write sandbox by design). That capability flag becomes the hidden ingredient backend_supports_git_write=false, which silently prunes the implement/retry_worktree steps from the recipe at load time and redirects their routes into gate_backend_write — a step added (#3960) purely to make the inevitable failure honest instead of a false success. Nothing at any entry point refuses to start the doomed run: open_kitchen returns success: true, valid: true for Codex + implementation (a behavior deliberately pinned by tests since #4068), so the orchestrator claims the issue, publishes a branch, runs make-plan and dry-walkthrough, creates a worktree (~21 minutes, two headless sessions), and only then hits the gate, which routes to release_issue_failure and marks the issue fail. The run executed autoskillit 0.10.772 — identical to repo HEAD — so all 84 recent [FIX] commits, including the newest dispatch-feasibility preflight (#4087), were active and none of them addresses this class. The root cause is a missing admission-control invariant: no layer enforces "never begin side-effecting pipeline work that the active backend provably cannot finish." Every prior fix operated downstream of admission, each one relocating or beautifying the death rather than preventing it.

Root Cause

Primary (architectural): There is no admission control derived from the recipe artifact itself. At recipe load time the system already knows everything needed to conclude the run is dead on arrival — CodexBackend.capabilities.git_metadata_writable=False is static, the ingredient is resolved, pruning has already removed the implement path, and gate_backend_write's input is a load-time constant that deterministically routes to release_issue_failure on the normal (verdict=plan) path. Yet load_and_validate treats the pruned, dead-on-arrival recipe as valid, open_kitchen opens the kitchen, and side-effecting steps (issue claim, branch publish, clone, worktree) execute before the gate fires. [SUPPORTED]

Proximate (why this run, today): Commit 8734c6355 (#4068, 2026-06-11) fixed "Pre-Prune Validity Computation Breaks Codex open_kitchen." Before it, Codex + implementation was accidentally blocked at open_kitchen (13 pre-prune ERROR findings → valid=False). #4068 correctly removed that imprecise structural blocker — but installed no precise replacement, and added tests that pin valid=True / success=True for Codex + implementation (tests/server/test_backend_ingredient_injection.py:494-495, 559-565; tests/recipe/test_bundled_recipes_dispatch_ready.py:125-148). The accidental admission control was removed deliberately; deliberate admission control was never added. [SUPPORTED]

Why the backend flag is correct (not the bug): Codex CLI protects .git as read-only inside writable workspace roots at the OS-sandbox layer, by explicit design (official sandboxing docs; openai/codex PR #19852, merged 2026-04-29, hardened this across macOS/Linux/Windows adapters). Headless AutoSkillit Codex sessions always pass --sandbox workspace-write (implementation sessions, codex.py:583 hardcoded; codex.py:620 default sandbox_mode="workspace-write") or --sandbox read-only (orchestrator sessions, codex.py:818); the --dangerously-bypass-approvals-and-sandbox flag is used only by the interactive command builder (codex.py:860-867), never headless. The interactive Codex CLI's "YOLO mode" visible in the user's transcript does not propagate to pipeline sessions. [SUPPORTED]

Affected Components

  • src/autoskillit/execution/backends/codex.py:521 — CodexBackend.capabilities returns BackendCapabilities(git_metadata_writable=False, ...); static declaration, no runtime probe. Also lines 191–197: Codex backend injects AGENT_BACKEND_DYNACONF_ENV_VAR=codex so the MCP server resolves config.agent_backend.backend="codex" even when the consumer project config has no agent_backend entry. [SUPPORTED]
  • src/autoskillit/core/types/_type_backend.py:142,240 — git_metadata_writable: bool = True field; CLAUDE_CODE_CAPABILITIES sets True. [SUPPORTED]
  • src/autoskillit/server/tools/_auto_overrides.py:19-28 — maps capability → backend_supports_git_write ingredient override. Mirrored (IL-layering) by src/autoskillit/fleet/_api.py:112-120. [SUPPORTED]
  • src/autoskillit/server/tools/tools_kitchen.py:574-581 (also 410) — injects and promotes the override into the config layer during open_kitchen/get_recipe. [SUPPORTED]
  • src/autoskillit/recipe/_recipe_composition.py:294-415 — _prune_skipped_steps() removes skip_when_false steps and redirects incoming routes to the pruned step's on_success (so create_impl_worktree → implement → gate_backend_write collapses to create_impl_worktree → gate_backend_write). [SUPPORTED]
  • src/autoskillit/recipes/implementation.yaml:40-44 (hidden ingredient), :330 (implement.on_success: gate_backend_write), :333,360 (skip_when_false), :361-370 (gate routes backend_capable == false → release_issue_failure). Same pattern in remediation.yaml:396 and implementation-groups.yaml:314. [SUPPORTED]
  • src/autoskillit/smoke_utils/_git.py:215-218 — gate_backend_write() pure string compare. [SUPPORTED]
  • src/autoskillit/server/tools/_preflight.py — _check_dispatch_feasibility ([FIX] Dispatch-Feasibility Preflight — Claim-Before-Validate Immunity #4087) checks only fix-required HOOK_REGISTRY coverage; zero hooks currently carry codex_status="fix-required", so it passes vacuously on Codex and never inspects backend_supports_git_write. [SUPPORTED]
  • tests/recipe/test_backend_reachability.py — asserts pruned-graph satisfiability but treats failure terminals as acceptable; a DOA recipe passes. [SUPPORTED]

Data Flow

  1. Codex CLI (interactive orchestrator, token-reserve project) launches the AutoSkillit MCP server with AGENT_BACKEND=codex in the environment (codex.py:191-197) → get_backend("codex") → git_metadata_writable=False.
  2. open_kitchen(name="implementation") → _backend_capability_overrides() injects backend_supports_git_write="false" → _promote_capability_keys makes it config-authoritative → load_and_validate prunes implement/retry_worktree/fix-family steps and redirects routes → post-prune validation passes ([FIX] Pre-Prune Validity Computation Breaks Codex open_kitchen #4068) → valid=True, success=True. The ingredient is hidden: true, so the orchestrator's ingredients table omits it (recipe/_recipe_ingredients.py:59); no warning or suggestion is emitted.
  3. The orchestrator executes the served recipe: bootstrap_clone → claim_and_resolve_issue (issue Package and build infrastructure for public release #149 labeled in-progress) → create_and_publish_branch → make-plan (headless Codex session, ~13 min) → dry-walkthrough → create_impl_worktree (host-side run_cmd, succeeds — git ops at orchestrator level are unsandboxed by design, per the step's own note).
  4. Pruned routing sends it straight to gate_backend_write → {"backend_capable": "false"} → release_issue_failure (issue labeled fail) → register_clone_status(error) → failure terminal. Wall-clock ≈ 21 minutes; artifacts: dead clone, dead worktree, published branch, failed issue.

Test Gap Analysis

Tests do not merely miss the defect — they pin it as correct behavior:

  • tests/server/test_backend_ingredient_injection.py:494-495,559-565 assert Codex + implementation open_kitchen returns valid=True / success=True (added with [FIX] Pre-Prune Validity Computation Breaks Codex open_kitchen #4068). [SUPPORTED]
  • tests/recipe/test_bundled_recipes_dispatch_ready.py:125-148 asserts every dispatch-gate recipe is "dispatch ready" on Codex with backend_supports_git_write=false. [SUPPORTED]
  • tests/recipe/test_backend_reachability.py checks the pruned graph is satisfiable but counts failure stops as satisfying — no test asks "can a success terminal be reached on the normal path?" [SUPPORTED]
  • Unit tests for the gate (tests/test_smoke_utils.py:3243-3271) verify the gate's string parsing, i.e., that the pipeline dies correctly — not that it shouldn't have started. [SUPPORTED]

The pattern: each defensive layer has tests asserting its local contract, and the composition of locally-correct layers produces a globally absurd outcome (start work that provably cannot finish) that no test observes end-to-end.

Similar Patterns

  • The codebase already solved the analogous problem for a different failure class: _check_dispatch_feasibility ([FIX] Dispatch-Feasibility Preflight — Claim-Before-Validate Immunity #4087) refuses dispatch when fix-required hooks are unenforceable on the backend — admission control keyed on the hook registry. The git-write class has no equivalent.
  • The recipe-graph rule failure-verdict-bypass-reachable (Rectify: Failure-Verdict Bypass Immunity via Static Reachability Rule #3630) already performs terminal-classified reachability analysis using is_success_stop() / is_failure_stop() (recipe/_rule_helpers.py:115,132) and bfs_reachable() / _build_step_graph() (_analysis_bfs.py:17,58). The machinery for the missing invariant exists. [SUPPORTED]
  • Host-side git is the established in-repo workaround pattern: bootstrap_clone, create_and_publish_branch, create_impl_worktree (run_cmd), merge_worktree, push_to_remote all execute at the orchestrator/server level, unsandboxed. The exception success paths (false_positive, already_done) legitimately complete on Codex because all their git work is host-side. The irreducible gap is only the implement skill committing inside the sandboxed session. [SUPPORTED]

Design Intent Findings

Historical Context

This is an unambiguous recurring pattern.

External Research

  • Codex CLI workspace-write sandbox protects <writable_root>/.git as read-only by design, recursively, including pointer-file resolution; justification is hook-based privilege-escalation prevention. Official: developers.openai.com/codex/concepts/sandboxing, /codex/agent-approvals-security. Hardening PR: github.com/Enforce workspace metadata protections in Linux sandbox openai/codex#19852 (merged 2026-04-29). [SUPPORTED]
  • features.codex_git_commit only controls commit attribution trailers; it is not a sandbox carveout (developers.openai.com/codex/config-reference). No writable_roots workaround for .git exists (config-advanced docs). The only git-write path is --sandbox danger-full-access / --yolo — with known headless caveats: workspace-write codex exec hang bug (--full-auto and --sandbox workspace-write flags cause indefinite hang with orphaned child processes openai/codex#7852), app-server ignoring the bypass flag (#14068), automations silently falling back to workspace-write (#15310), trust-prompt regression in ephemeral dirs (#14345), and a v0.115.0 regression making .git read-only even where it previously worked (#15505). [SUPPORTED]
  • Framework survey: Aider performs all git operations host-side (the LLM never touches .git) — aider.chat/docs/git.html; OpenHands and SWE-agent assume container-local git with no capability declaration. AutoSkillit's static capability declaration is the most robust pattern of the three for multi-backend orchestration, but it is currently consumed only mid-pipeline, not at admission. [SUPPORTED]

Scope Boundary

Investigated: capability resolution chain (backend → ingredient → pruning → gate), all pipeline entry points and their preflights, the 14-commit Codex fix chain, recipe graph analysis machinery, session logs / clone-registry / abandoned-run forensics, installed-vs-HEAD version comparison, Codex CLI sandbox semantics (external), framework comparisons (external), adversarial challenge of the root-cause hypothesis, and breakage surface of the recommendation.
Not yet explored: the full content of the 897 prior /investigate session logs (sampled via git history instead); exact per-issue enumeration of fail-labeled GitHub issues; the L3 planner/campaign layers' assumptions about backend capability (only dispatch paths were traced); whether research.yaml-family recipes silently degrade on Codex in ways users haven't noticed yet (they prune git steps with no gate — flagged below as a latent sibling defect).

Recommendations

Single recommendation: capability admission control at the recipe-load choke point (fail-closed open_kitchen/dispatch for purpose-critical pruning).

Mechanism (design constraints, not an implementation):

  1. Mark the steps that constitute a recipe's purpose (e.g., implement in implementation-family recipes) — either a purpose: core step annotation or a recipe-level declaration. An arch test cross-references SKILL_CAPABILITY_REGISTRY / skill uses_capabilities to ensure every bundled-recipe step invoking git_metadata_write-class skills carries an explicit purpose-critical or degraded-ok classification, so the annotation cannot rot silently (the same enforcement pattern as tests/arch/test_skill_backend_annotations.py). Schema plumbing constraint (validated): a new step field must be added in three coordinated places — RecipeStep in recipe/schema.py, _PARSE_STEP_HANDLED_FIELDS, and _parse_step in recipe/io.py — because io.py:391-398 raises an import-time RuntimeError if the dataclass and the handled-fields list diverge, and an unplumbed YAML purpose: key is silently dropped by the parser.
  2. In load_and_validate, after _prune_skipped_steps (its resolutions dict already names every pruned step, threaded at _api.py:374; the triggering ingredient is derivable from _pre_prune_steps[name].skip_when_false): if any purpose-critical step was pruned, the recipe is infeasible on this backend. Return a machine-readable refusal naming the backend, the missing capability, and the pruned steps (e.g., "backend codex lacks git_metadata_write required by purpose step implement"). Key the check on backend_name (already a load_and_validate parameter, passed even by fleet's inner _run_dispatch load at fleet/_api.py:354) rather than on the presence of capability ingredient overrides — the _run_dispatch call site omits capability overrides, so an override-keyed check would silently skip there. Because recipe/ (IL-2) is forbidden from importing execution/ (pyproject import-linter contract "IL-2 recipe imports only IL-0 and workspace"), the backend_name→capability bridge must be an IL-0 capability mapping in core/types or an injected capability_overrides parameter.
  3. All recipe-content-serving surfaces fail closed on infeasibility before the kitchen gate enables side-effect tools — and therefore before any claim, branch publish, or clone. There are four such surfaces (validated): both open_kitchen paths (normal at tools_kitchen.py:684-796 and deferred-recall at :595-683 — new refusal branches must call both gate.disable() and await ctx.disable_components(tags={"kitchen"}) per the preflight-failure shape at :660-661/:765-766, and need a new response builder since _recipe_validation_error_response (:104-126) carries no ingredients_table), get_recipe (tools_kitchen.py:425 gates only on valid — add a dispatch_feasible gate), and the load_recipe tool — which today does NOT fail closed (tools_recipe.py:237-238 returns full recipe content with only a validation_failed: True flag) and needs an explicit refusal branch. The refusal response must still include the ingredients schema so ingredients_only=true discovery flows (_prompts_kitchen.py:42) keep working: discovery callers get the table plus dispatch_feasible=false instead of a bare error (LoadRecipeResult is TypedDict(total=False) and already carries ingredients_table — extension is additive; mirror the new fields in OpenKitchenResult).
  4. _check_dispatch_feasibility ([FIX] Dispatch-Feasibility Preflight — Claim-Before-Validate Immunity #4087's preflight, already wired into open_kitchen and dispatch_food_truck) consumes the same computation, so fleet dispatch refuses before claiming — extending [FIX] Dispatch-Feasibility Preflight — Claim-Before-Validate Immunity #4087's "claim-before-validate immunity" from the hook class to the capability class with one shared source of truth.
  5. gate_backend_write remains untouched as defense-in-depth.

Why this will not fail the way the previous ~100 fixes did — each property below negates a specific defeat mechanism observed in the fix history:

  • Prior fixes were per-entry-point; new entry points bypassed them. This check lives in load_and_validate — the function every surface that serves recipe content to a pipeline executor calls (verified: both open_kitchen paths, the get_recipe resource, the load_recipe tool, and the fleet dispatch_food_truck preflight all traverse it; the lifespan gate-boot paths serve no recipe content, and validate_recipe's parallel validate_from_path is not a pipeline-start surface). A future entry point cannot skip it without skipping the recipe itself. Precision matters here: computing the signal in one place is necessary but not sufficient — each serving surface must also act on it (load_recipe currently doesn't fail closed), which is why the implementation must include an arch/integration test asserting every content-serving tool refuses an infeasible recipe, not just that load_and_validate flags it.
  • Prior fixes were keyed on hand-enumerated classes (specific hooks, specific skills, specific flags); the next unlisted case recreated the gap. This check is derived from the recipe artifact plus the backend capability set — a new capability flag, recipe, or backend is covered automatically, and the arch test forces purpose annotations to track skill capability declarations.
  • Prior fixes fired after side effects (claim, branch, clone) had already happened. This check fires before the kitchen gate enables side-effecting tools; the claim is a recipe step that can no longer be reached. The 21-minute waste, the fail-labeled issue, the dead clone, and the published branch all become impossible, not merely better-reported.
  • Prior fixes were validated by layer-local tests that pinned globally wrong behavior ([FIX] Pre-Prune Validity Computation Breaks Codex open_kitchen #4068's valid=True pins). This change replaces those pins with end-to-end assertions of the actual invariant ("Codex + implementation is refused pre-claim with a machine-readable reason"), so future regressions fail loudly.
  • The naive version of this very idea would also have failed: a global "any success terminal reachable?" BFS is defeated by the false_positive/already_done exception paths (adversarially verified — those paths legitimately succeed on Codex because their git work is host-side). Keying on purpose-critical pruning rather than global reachability is what survives that counterexample.

Killed alternatives (deep-mode convergence):

  • Add a backend_supports_git_write check to _check_dispatch_feasibility — killed: a hand-keyed flag check is the same rot class as prior fixes; merge-prs/research-* recipes prune git steps by design and would need per-recipe exemptions, reintroducing hand-maintenance. (It also would not generalize to the next capability flag.)
  • Run Codex headless sessions with --sandbox danger-full-access to make the flag true — killed: inverts the deliberate security posture (Codex's .git protection is intentional, PR Enforce workspace metadata protections in Linux sandbox openai/codex#19852; AutoSkillit deliberately chose workspace-write), and trades a clean failure for known headless instability (#7852 hangs, #14345 trust prompts). Risk HIGH.
  • Host-side commit-on-behalf (Aider pattern): backend writes files, orchestrator commits — killed as the remediation: it is the correct long-term direction to make Codex implementation-capable (it matches the repo's existing host-side-git architecture), but it is a feature with large scope (implement-skill redesign, commit protocol, audit attribution) and does not remove the need for admission control for every other capability gap. Worth filing as a separate feature issue.
  • Unhide the ingredient / add a warning to the open_kitchen response — killed as primary: visibility that depends on an LLM orchestrator noticing and voluntarily stopping is not an invariant. Acceptable as a complement.
  • Constant-folding gate steps at load time (evaluate gate_backend_write statically and prune dead edges, then check reachability) — killed: more general but semantically entangled with the exception-verdict branches; the purpose-step declaration achieves the same refusal with far less machinery.

Breakage Analysis

  • Recommendation: capability admission control at recipe load (fail-closed open_kitchen/dispatch on purpose-critical pruning)
    • Breakage surface: tests pinning current Codex admission — tests/server/test_backend_ingredient_injection.py:494-495 (REQ-PRUNE-001 valid=True), :559-565 (test_codex_open_kitchen_returns_success), tests/recipe/test_bundled_recipes_dispatch_ready.py:125-148 (codex × dispatch-gate recipes), tests/recipe/test_recipe_backend_composition_matrix.py ((implementation, codex) cell). All must be inverted to assert the precise refusal. The ingredients_only=true discovery flow (_prompts_kitchen.py:42) must receive schema-plus-infeasibility rather than a bare failure, or fleet dispatch discovery breaks.
    • Prior reverts: none found on admission/validity blocking (git log --grep="revert" -i). [FIX] Pre-Prune Validity Computation Breaks Codex open_kitchen #4068 removed an accidental blocker; this recommendation is compatible with [FIX] Pre-Prune Validity Computation Breaks Codex open_kitchen #4068's stated intent because the refusal is precise (named capability, named steps) rather than a pre-prune structural false positive — but implementers must treat [FIX] Pre-Prune Validity Computation Breaks Codex open_kitchen #4068's tests as deliberate pins to be consciously replaced, not incidental breakage.
    • Downstream contract violations: none for gate_backend_write (retained) or skip_when_false pruning (unchanged for non-purpose steps). Correction from adversarial validation: merge-prs.yaml (implement/retry_worktree) and research-implement.yaml:230 (implement_phase) DO prune git-write steps on Codex — their skills declare git_metadata_write — so each such step needs an explicit purpose-critical vs degraded-ok classification decision; only purpose-critical classification triggers refusal. Fleet dispatch gains an earlier refusal; the L3 orchestrator prompt contract already instructs stopping on open_kitchen success:false.
    • Implementation sites easy to miss (validator-identified): the deferred-recall open_kitchen path (tools_kitchen.py:595-683) is a second, separate refusal site; load_recipe (tools_recipe.py:237-238) needs a new fail-closed branch; fleet's inner _run_dispatch load (fleet/_api.py:349-355) omits capability overrides, so the check must derive capabilities from backend_name; validate_from_path (_api_listing.py) is a parallel validation pipeline that won't carry the check — acceptable (not a pipeline-start surface) but should be documented. Gate timing note: _open_kitchen_handler enables the gate (tools_kitchen.py:237) before recipe load, but the orchestrator cannot invoke tools until open_kitchen returns, so the refusal still lands pre-side-effect in practice.
    • Risk level: MEDIUM — manageable, concentrated in test updates, the discovery-flow response shape, and the four serving-surface refusal branches.

Confidence Levels

  • Static capability flag, injection chain, pruning redirect, gate routing, claim-after-open ordering: SUPPORTED (file:line evidence throughout).
  • Codex sandbox .git protection is by-design and headless sessions never bypass it: SUPPORTED (official docs + code).
  • [FIX] Fix-Required Dispatch Gate Cross-Registry Immunity #4086/[FIX] Dispatch-Feasibility Preflight — Claim-Before-Validate Immunity #4087 do not cover this class; today's run executed HEAD-equivalent code: SUPPORTED (vacuous-pass verified against hook registry; installed 0.10.772 == HEAD).
  • "DOA on the normal path; success reachable only via exception verdicts whose git work is host-side": SUPPORTED (adversarial trace).
  • "No prior fix implemented admission control for this class; none reverted": SUPPORTED (14-commit classification).
  • Exact count of GitHub fail-labeled issues attributable to this family: NEEDS-EVIDENCE (48 abandoned clones and 4 gate-firing logs are direct; per-issue attribution was not enumerated).
  • Research-family recipes silently degrading on Codex (gate-less pruning) being a live user-facing problem: NEEDS-EVIDENCE (structural observation; no observed incident examined).

Requirements

REQ-SCHEMA-001: RecipeStep must support a purpose-critical annotation parsed from recipe YAML, plumbed through recipe/schema.py, _PARSE_STEP_HANDLED_FIELDS, and _parse_step in recipe/io.py so the import-time handled-fields assertion passes.

REQ-SCHEMA-002: All bundled implementation-family recipes (implementation.yaml, remediation.yaml, implementation-groups.yaml) must declare their implement-family steps as purpose-critical; every other bundled-recipe step invoking a git_metadata_write-class skill (merge-prs.yaml implement/retry_worktree, research-implement.yaml implement_phase) must carry an explicit purpose-critical or degraded-ok classification — no unclassified git-write step may remain. User-authored recipes without annotations are fail-open (admitted vacuously); this default must be documented.

REQ-SCHEMA-003: LoadRecipeResult must gain dispatch_feasible: bool and infeasible_steps: list[str] fields, mirrored in OpenKitchenResult; structural valid semantics must remain unchanged.

REQ-ADMIT-001: load_and_validate must classify a recipe as infeasible when any purpose-critical step is pruned by skip_when_false, keyed on backend_name rather than the presence of capability ingredient overrides.

REQ-ADMIT-002: The infeasibility result must be machine-readable, naming the backend, the missing capability, and the pruned purpose-critical steps.

REQ-ADMIT-003: The infeasibility result must include the ingredients_table so ingredients_only=True discovery flows continue to function (the existing _recipe_validation_error_response builder does not carry it — a new refusal response builder is required).

REQ-ADMIT-004: The refusal's user_visible_message must name the backend, the missing capability, and the pruned purpose-critical steps, and must include an actionable escape hatch (e.g., switch the backend to claude-code), following the _preflight.py:95-110 hook-class precedent.

REQ-SURF-001: Both open_kitchen paths (normal and deferred-recall) must fail closed on an infeasible recipe — calling both gate.disable() and disable_components(tags={"kitchen"}), returning a structured refusal envelope — before any side-effect tool is callable.

REQ-SURF-002: get_recipe must gate on dispatch_feasible in addition to its existing valid gate and refuse to serve an infeasible recipe's content.

REQ-SURF-003: load_recipe must fail closed on infeasibility instead of returning recipe content with only a validation_failed flag.

REQ-SURF-004: The dispatch-feasibility preflight consumed by dispatch_food_truck must refuse infeasible recipes before any issue claim or branch publication.

REQ-SURF-005: The gate_backend_write recipe step must remain in place as defense-in-depth.

REQ-TEST-001: An arch test must cross-reference SKILL_CAPABILITY_REGISTRY to require that every bundled-recipe step invoking a git_metadata_write-class skill carries an explicit purpose-critical or degraded-ok classification.

REQ-TEST-002: An integration test must assert that every recipe-content-serving surface refuses an infeasible recipe.

REQ-TEST-003: The tests currently pinning Codex+implementation admission (test_backend_ingredient_injection.py, test_bundled_recipes_dispatch_ready.py, test_recipe_backend_composition_matrix.py) must assert the precise refusal instead.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestrecipe:implementationRoute: proceed directly to implementationstagedImplementation staged and waiting for promotion to main

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions