Skip to content

Commit cfb6ecc

Browse files
Trecekclaude
andauthored
[FIX] Rectify: Capability Admission Control at Recipe Load — Refuse DOA Pipelines (#4094)
## Summary Codex-backend implementation pipelines repeatedly die at `gate_backend_write` (step 12) after executing 11 irreversible side-effect steps (clone, claim, branch publish, implement). 84 `[FIX]` commits over ~3 weeks have patched individual links in the capability injection chain without closing the architectural gap: **there is no admission control that refuses a dead-on-arrival (DOA) pipeline at recipe load time**. The `gate_backend_write` step is a mid-pipeline runtime firewall — by the time it fires, the damage (wasted time, orphaned branches, claimed issues) is done. The immunity approach: extend `load_and_validate` to compute a `dispatch_feasible` signal based on whether capability-gated steps that route to terminal failure are reachable in the post-prune step graph. All four content-serving surfaces (`open_kitchen` normal, `open_kitchen` deferred-recall, `get_recipe`, `load_recipe`) gate on this signal. The pipeline is refused before any tool is revealed. ## Requirements **REQ-SCHEMA-001:** `RecipeStep` must support a purpose-critical annotation parsed from recipe YAML, plumbed through `recipe/schema.py`, `_PARSE_STEP_HANDLED_FIELDS`, and `_parse_step` in `recipe/io.py` so the import-time handled-fields assertion passes. **REQ-SCHEMA-002:** All bundled implementation-family recipes (`implementation.yaml`, `remediation.yaml`, `implementation-groups.yaml`) must declare their implement-family steps as purpose-critical; every other bundled-recipe step invoking a `git_metadata_write`-class skill (`merge-prs.yaml` `implement`/`retry_worktree`, `research-implement.yaml` `implement_phase`) must carry an explicit purpose-critical or degraded-ok classification — no unclassified git-write step may remain. User-authored recipes without annotations are fail-open (admitted vacuously); this default must be documented. **REQ-SCHEMA-003:** `LoadRecipeResult` must gain `dispatch_feasible: bool` and `infeasible_steps: list[str]` fields, mirrored in `OpenKitchenResult`; structural `valid` semantics must remain unchanged. **REQ-ADMIT-001:** `load_and_validate` must classify a recipe as infeasible when any purpose-critical step is pruned by `skip_when_false`, keyed on `backend_name` rather than the presence of capability ingredient overrides. **REQ-ADMIT-002:** The infeasibility result must be machine-readable, naming the backend, the missing capability, and the pruned purpose-critical steps. **REQ-ADMIT-003:** The infeasibility result must include the `ingredients_table` so `ingredients_only=True` discovery flows continue to function (the existing `_recipe_validation_error_response` builder does not carry it — a new refusal response builder is required). **REQ-ADMIT-004:** The refusal's `user_visible_message` must name the backend, the missing capability, and the pruned purpose-critical steps, and must include an actionable escape hatch (e.g., switch the backend to claude-code), following the `_preflight.py:95-110` hook-class precedent. **REQ-SURF-001:** Both `open_kitchen` paths (normal and deferred-recall) must fail closed on an infeasible recipe — calling both `gate.disable()` and `disable_components(tags={"kitchen"})`, returning a structured refusal envelope — before any side-effect tool is callable. **REQ-SURF-002:** `get_recipe` must gate on `dispatch_feasible` in addition to its existing `valid` gate and refuse to serve an infeasible recipe's content. **REQ-SURF-003:** `load_recipe` must fail closed on infeasibility instead of returning recipe content with only a `validation_failed` flag. **REQ-SURF-004:** The dispatch-feasibility preflight consumed by `dispatch_food_truck` must refuse infeasible recipes before any issue claim or branch publication. **REQ-SURF-005:** The `gate_backend_write` recipe step must remain in place as defense-in-depth. **REQ-TEST-001:** An arch test must cross-reference `SKILL_CAPABILITY_REGISTRY` to require that every bundled-recipe step invoking a `git_metadata_write`-class skill carries an explicit purpose-critical or degraded-ok classification. **REQ-TEST-002:** An integration test must assert that every recipe-content-serving surface refuses an infeasible recipe. **REQ-TEST-003:** The tests currently pinning Codex+implementation admission (`test_backend_ingredient_injection.py`, `test_bundled_recipes_dispatch_ready.py`, `test_recipe_backend_composition_matrix.py`) must assert the precise refusal instead. Closes #4090 ## Implementation Plan Plan file: `/home/talon/projects/autoskillit-runs/remediation-20260612-092948-265956/.autoskillit/temp/rectify/rectify_capability_admission_control_2026-06-12_093500.md` 🤖 Generated with [Claude Code](https://claude.com/claude-code) via AutoSkillit <!-- autoskillit:pipeline-signature steps=prepare_pr,run_arch_lenses,compose_pr,annotate_pr_diff,review_pr --> ## Token Usage Summary | Step | Model | count | uncached | output | cache_read | peak_ctx | turns | cache_write | time | |------|-------|-------|----------|--------|------------|----------|-------|-------------|------| | rectify* | opus[1m] | 1 | 7.4k | 25.2k | 1.7M | 126.5k | 61 | 105.2k | 18m 38s | | review_approach* | opus[1m] | 1 | 24 | 6.2k | 264.7k | 70.9k | 14 | 28.4k | 9m 44s | | dry_walkthrough* | opus | 2 | 4.1k | 22.5k | 2.7M | 100.6k | 109 | 163.3k | 13m 37s | | implement* | MiniMax-M3 | 2 | 10.7M | 24.6k | 0 | 0 | 182 | 0 | 16m 4s | | assess* | opus[1m] | 1 | 46 | 9.7k | 1.1M | 81.0k | 53 | 59.4k | 9m 10s | | audit_impl* | opus[1m] | 2 | 71 | 35.7k | 788.0k | 76.8k | 43 | 109.0k | 22m 19s | | make_plan* | opus[1m] | 1 | 77 | 15.2k | 1.9M | 112.4k | 50 | 91.3k | 16m 50s | | retry_worktree* | opus[1m] | 1 | 28 | 2.4k | 385.2k | 44.2k | 19 | 22.7k | 2m 0s | | prepare_pr* | MiniMax-M3 | 1 | 349.4k | 5.0k | 0 | 0 | 19 | 0 | 1m 22s | | compose_pr* | MiniMax-M3 | 1 | 312.6k | 2.1k | 0 | 0 | 15 | 0 | 1m 1s | | **Total** | | | 11.4M | 148.5k | 8.9M | 126.5k | | 579.2k | 1h 50m | \* *Step used a non-Anthropic provider; caching behavior may differ.* ## Token Efficiency | Step | LoC Changed | cache_read/LoC | cache_write/LoC | output/LoC | |------|-------------|----------------|-----------------|------------| | rectify | 0 | — | — | — | | review_approach | 0 | — | — | — | | dry_walkthrough | 0 | — | — | — | | implement | 492 | 0.0 | 0.0 | 50.0 | | assess | 22 | 50917.0 | 2700.9 | 441.6 | | audit_impl | 0 | — | — | — | | make_plan | 0 | — | — | — | | retry_worktree | 0 | — | — | — | | prepare_pr | 0 | — | — | — | | compose_pr | 0 | — | — | — | | **Total** | **514** | 17378.8 | 1126.9 | 289.0 | ## Model Usage Breakdown | Model | steps | uncached | output | cache_read | cache_write | time | |-------|-------|----------|--------|------------|-------------|------| | opus[1m] | 6 | 7.7k | 94.4k | 6.2M | 416.0k | 1h 18m | | opus | 1 | 4.1k | 22.5k | 2.7M | 163.3k | 13m 37s | | MiniMax-M3 | 3 | 11.3M | 31.7k | 0 | 0 | 18m 29s | --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent cadfc1e commit cfb6ecc

22 files changed

Lines changed: 531 additions & 27 deletions

src/autoskillit/config/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,6 @@
99
_MAX_CONCURRENT_DISPATCHES as _MAX_CONCURRENT_DISPATCHES,
1010
)
1111
from autoskillit.config.ingredient_defaults import (
12-
BACKEND_CAPABILITY_INGREDIENTS,
1312
SERVER_AUTHORITATIVE_INGREDIENTS,
1413
apply_config_authoritative_overrides,
1514
build_config_authoritative_layer,
@@ -57,6 +56,7 @@
5756
validate_layer_keys,
5857
write_config_layer,
5958
)
59+
from autoskillit.core import BACKEND_CAPABILITY_INGREDIENTS
6060

6161
__all__ = [
6262
"AgentBackendConfig",

src/autoskillit/config/ingredient_defaults.py

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -125,12 +125,6 @@ def iter_display_categories(
125125
"backend_supports_git_write",
126126
}
127127

128-
BACKEND_CAPABILITY_INGREDIENTS: frozenset[str] = frozenset(
129-
{
130-
"backend_supports_git_write",
131-
}
132-
)
133-
134128

135129
def resolve_ingredient_defaults(project_dir: Path) -> dict[str, str]:
136130
"""Resolve auto-detect ingredient values from the project environment."""

src/autoskillit/core/__init__.pyi

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,9 @@ from .types import AUTOSKILLIT_INSTALLED_VERSION as AUTOSKILLIT_INSTALLED_VERSIO
131131
from .types import AUTOSKILLIT_PRIVATE_ENV_VARS as AUTOSKILLIT_PRIVATE_ENV_VARS
132132
from .types import AUTOSKILLIT_SKILL_PREFIX as AUTOSKILLIT_SKILL_PREFIX
133133
from .types import AUTOSKILLIT_WRITE_GUARD_TOOL_NAMES as AUTOSKILLIT_WRITE_GUARD_TOOL_NAMES
134+
from .types import BACKEND_CAPABILITY_INGREDIENTS as BACKEND_CAPABILITY_INGREDIENTS
134135
from .types import CAMPAIGN_ID_ENV_VAR as CAMPAIGN_ID_ENV_VAR
136+
from .types import CAPABILITY_GATE_CALLABLES as CAPABILITY_GATE_CALLABLES
135137
from .types import CAPTURE_VALID_VALUE_TYPES as CAPTURE_VALID_VALUE_TYPES
136138
from .types import CATEGORY_TAGS as CATEGORY_TAGS
137139
from .types import CLAUDE_CODE_CAPABILITIES as CLAUDE_CODE_CAPABILITIES

src/autoskillit/core/types/_type_constants.py

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,8 @@
3333
"RUN_PYTHON_SENTINEL_KEYS",
3434
"SCOPE_DIRECTION_SOURCE_TYPES",
3535
"WORKTREE_SKILLS",
36+
"BACKEND_CAPABILITY_INGREDIENTS",
37+
"CAPABILITY_GATE_CALLABLES",
3638
"SkillFamilyDef",
3739
"GITHUB_API_SKILL_FAMILIES",
3840
"CODEX_SESSIONS_SUBDIR",
@@ -83,6 +85,30 @@
8385
}
8486
)
8587

88+
# Ingredient keys that are derived from a live runtime backend's capabilities
89+
# (not from LLM-supplied values). Used to gate recipe admission control — a
90+
# recipe is refused at load time when an ingredient in this set resolves to a
91+
# falsy value while a surviving step depends on it. Moved to IL-0 so it is
92+
# importable by the recipe layer (IL-2) without violating the IL-006 contract
93+
# that forbids recipe → config imports.
94+
BACKEND_CAPABILITY_INGREDIENTS: frozenset[str] = frozenset(
95+
{
96+
"backend_supports_git_write",
97+
}
98+
)
99+
100+
# Bare (un-dotted) callable names whose run_python steps are capability gates.
101+
# Each entry corresponds to a callable in smoke_utils that reads a
102+
# BACKEND_CAPABILITY_INGREDIENTS key and returns a verdict dict. When a
103+
# surviving step's with_args["callable"] final component matches an entry here
104+
# and the corresponding capability ingredient resolves falsy, the recipe is
105+
# marked dispatch-infeasible at load time.
106+
CAPABILITY_GATE_CALLABLES: frozenset[str] = frozenset(
107+
{
108+
"gate_backend_write",
109+
}
110+
)
111+
86112

87113
class SkillFamilyDef(NamedTuple):
88114
"""Skill family definition — groups sibling skills that must share API patterns."""

src/autoskillit/hooks/formatters/_fmt_recipe.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,8 @@
4444
"stop_step_semantics", # delivered via open_kitchen response Channel B; not redisplayed
4545
"deferred_guards", # internal deferral metadata; not displayed to agent
4646
"post_prune_step_names", # internal preflight field; not displayed to agent
47+
"dispatch_feasible", # internal admission control signal; surfaced via refusal envelopes
48+
"infeasible_steps", # internal admission control detail; surfaced via refusal envelopes
4749
}
4850
)
4951

@@ -185,6 +187,8 @@ def _fmt_load_recipe(data: LoadRecipeResult, pipeline: bool) -> str:
185187
"hook_warning", # edge-case diagnostic; not rendered in standard path
186188
"deferred_guards", # internal deferral metadata; not displayed to agent
187189
"post_prune_step_names", # internal preflight field; not displayed to agent
190+
"dispatch_feasible", # internal admission control signal; surfaced via refusal envelopes
191+
"infeasible_steps", # internal admission control detail; surfaced via refusal envelopes
188192
}
189193
)
190194

src/autoskillit/recipe/_api.py

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,8 @@
1313
from typing import Any, cast
1414

1515
from autoskillit.core import (
16+
BACKEND_CAPABILITY_INGREDIENTS,
17+
CAPABILITY_GATE_CALLABLES,
1618
ProcessStaleError,
1719
RecipeNotFoundError,
1820
RecipeSource,
@@ -45,6 +47,7 @@
4547
from autoskillit.recipe._recipe_composition import (
4648
_assert_content_integrity,
4749
_build_active_recipe,
50+
_compute_capability_feasibility,
4851
_prune_skipped_steps,
4952
_resolve_hidden_inputs_in_content,
5053
_resolve_skip_guards_in_content,
@@ -318,6 +321,8 @@ def load_and_validate(
318321
active_recipe = None
319322
_skip_resolutions: dict[str, bool | None] = {}
320323
_pre_prune_steps: dict[str, Any] = {}
324+
_dispatch_feasible = True
325+
_infeasible_steps: list[str] = []
321326

322327
# Determine recipes_dir from source
323328
if match.source == RecipeSource.BUILTIN:
@@ -393,6 +398,18 @@ def load_and_validate(
393398
raw = ""
394399
t0 = _t("prune_skipped_steps", t0, name)
395400

401+
# Stage: capability feasibility — detect DOA pipelines where a
402+
# surviving capability-gated run_python step has no recovery route
403+
# for a falsy capability ingredient. This refuses the pipeline
404+
# before any side-effect step is executed.
405+
_dispatch_feasible, _infeasible_steps = _compute_capability_feasibility(
406+
active_recipe,
407+
ingredient_overrides or {},
408+
capability_ingredient_keys=BACKEND_CAPABILITY_INGREDIENTS,
409+
capability_gate_callables=CAPABILITY_GATE_CALLABLES,
410+
)
411+
t0 = _t("capability_feasibility", t0, name)
412+
396413
# Stage: semantic rules (builds ValidationContext from post-prune recipe)
397414
if lister is None:
398415
from autoskillit.workspace import DefaultSkillResolver # noqa: PLC0415
@@ -582,6 +599,9 @@ def load_and_validate(
582599
result["deferred_guards"] = _deferred_guard_list
583600
if active_recipe is not None:
584601
result["post_prune_step_names"] = list(active_recipe.steps.keys())
602+
result["dispatch_feasible"] = _dispatch_feasible
603+
if _infeasible_steps:
604+
result["infeasible_steps"] = _infeasible_steps
585605

586606
# Write to cache (only when recipe was found and fully processed)
587607
if match is not None:

src/autoskillit/recipe/_api_cache.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@ def copy_result(self, result: Any) -> dict[str, Any]:
6363
"requires_packs",
6464
"requires_features",
6565
"deferred_guards",
66+
"infeasible_steps",
6667
):
6768
if list_key in r:
6869
r[list_key] = list(r[list_key])

src/autoskillit/recipe/_recipe_composition.py

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,60 @@ def _is_ingredient_truthy(value: str) -> bool:
106106
return bool(value) and value.lower() not in FALSY_STRINGS
107107

108108

109+
def _compute_capability_feasibility(
110+
post_prune_recipe: Any,
111+
ingredient_overrides: dict[str, str],
112+
*,
113+
capability_ingredient_keys: frozenset[str],
114+
capability_gate_callables: frozenset[str],
115+
) -> tuple[bool, list[str]]:
116+
"""Detect dead-on-arrival pipelines caused by capability-gated run_python steps.
117+
118+
A pipeline is DOA when a surviving post-prune step:
119+
1. Has tool="run_python" with step.with_args["callable"] whose final dotted
120+
component matches an entry in capability_gate_callables.
121+
2. The step reads an ingredient (via with_args keys that overlap
122+
capability_ingredient_keys) that resolves to a falsy value under the
123+
current ingredient_overrides.
124+
3. The step's on_result / on_failure routes the falsy case to a terminal
125+
failure target ("escalate") rather than recovery.
126+
127+
Returns (is_feasible, infeasible_step_names).
128+
"""
129+
if not post_prune_recipe or not getattr(post_prune_recipe, "steps", None):
130+
return True, []
131+
132+
infeasible: list[str] = []
133+
steps = post_prune_recipe.steps
134+
for step_name, step in steps.items():
135+
if step is None:
136+
continue
137+
if getattr(step, "tool", None) != "run_python":
138+
continue
139+
with_args = getattr(step, "with_args", None) or {}
140+
callable_dotted = with_args.get("callable", "")
141+
if not callable_dotted or "." not in callable_dotted:
142+
continue
143+
bare_name = callable_dotted.rsplit(".", 1)[-1]
144+
if bare_name not in capability_gate_callables:
145+
continue
146+
gate_input_keys = {k for k in with_args.keys() if k in capability_ingredient_keys}
147+
if not gate_input_keys:
148+
continue
149+
all_falsy = all(
150+
not _is_ingredient_truthy(str(ingredient_overrides.get(k, "")))
151+
for k in gate_input_keys
152+
)
153+
if not all_falsy:
154+
continue
155+
on_failure = getattr(step, "on_failure", None) or "escalate"
156+
on_exhausted = getattr(step, "on_exhausted", None) or "escalate"
157+
if on_failure == "escalate" or on_exhausted == "escalate":
158+
infeasible.append(step_name)
159+
160+
return (not infeasible), infeasible
161+
162+
109163
def _drop_sub_recipe_step(recipe: Any, step_name: str) -> Any:
110164
"""Return a new Recipe with the named sub_recipe placeholder step removed."""
111165
new_steps = {k: v for k, v in recipe.steps.items() if k != step_name}

src/autoskillit/recipe/_recipe_ingredients.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -127,6 +127,8 @@ class LoadRecipeResult(TypedDict, total=False):
127127
recipe_version: str | None
128128
deferred_guards: list[DeferredGuard]
129129
post_prune_step_names: list[str]
130+
dispatch_feasible: bool
131+
infeasible_steps: list[str]
130132

131133

132134
class OpenKitchenResult(TypedDict, total=False):
@@ -135,7 +137,7 @@ class OpenKitchenResult(TypedDict, total=False):
135137
Extends LoadRecipeResult with three post-return keys injected by the handler.
136138
"""
137139

138-
# Inherited from LoadRecipeResult (17 keys)
140+
# Inherited from LoadRecipeResult (19 keys)
139141
content: str
140142
errors: list[str]
141143
diagram: str | None
@@ -153,6 +155,8 @@ class OpenKitchenResult(TypedDict, total=False):
153155
recipe_version: str | None
154156
deferred_guards: list[DeferredGuard]
155157
post_prune_step_names: list[str]
158+
dispatch_feasible: bool
159+
infeasible_steps: list[str]
156160
# Post-return keys injected by open_kitchen handler (4 keys)
157161
success: bool
158162
kitchen: str

src/autoskillit/server/tools/tools_kitchen.py

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,36 @@ def _recipe_validation_error_response(name: str, result: dict[str, Any]) -> str:
126126
)
127127

128128

129+
async def _dispatch_infeasible_response(
130+
result: dict[str, Any],
131+
backend: Any,
132+
gate: Any,
133+
ctx: Context,
134+
) -> str:
135+
"""Refuse a dead-on-arrival pipeline before the gate is enabled.
136+
137+
Used by the open_kitchen (both normal and deferred-recall) paths when
138+
load_and_validate reports dispatch_feasible=False.
139+
"""
140+
gate.disable()
141+
await ctx.disable_components(tags={"kitchen"})
142+
_infeasible = result.get("infeasible_steps", [])
143+
_backend_name = backend.name if backend is not None else "unknown"
144+
return json.dumps(
145+
{
146+
"success": False,
147+
"kitchen": "dispatch_infeasible",
148+
"infeasible_steps": _infeasible,
149+
"ingredients_table": result.get("ingredients_table"),
150+
"user_visible_message": (
151+
f"Cannot dispatch recipe: backend {_backend_name!r} "
152+
f"causes steps {_infeasible} to route to terminal failure. "
153+
f"Use a backend with git_metadata_writable=True (e.g. claude-code)."
154+
),
155+
}
156+
)
157+
158+
129159
class QuotaGuardHookPayload(TypedDict):
130160
cache_max_age: int
131161
cache_path: str
@@ -433,6 +463,14 @@ def get_recipe(name: str) -> str:
433463
"suggestions": result.get("suggestions", []),
434464
}
435465
)
466+
if not result.get("dispatch_feasible", True):
467+
return json.dumps(
468+
{
469+
"error": "Recipe is infeasible on current backend",
470+
"dispatch_feasible": False,
471+
"infeasible_steps": result.get("infeasible_steps", []),
472+
}
473+
)
436474
return result.get("content", json.dumps({"error": "Recipe composition failed."}))
437475

438476

@@ -673,6 +711,10 @@ async def open_kitchen(
673711
tool_ctx.gate.disable()
674712
tool_ctx.gate_infrastructure_ready = False
675713
return _recipe_validation_error_response(name, result)
714+
if not result.get("dispatch_feasible", True):
715+
return await _dispatch_infeasible_response(
716+
result, tool_ctx.backend, tool_ctx.gate, ctx
717+
)
676718
# Dispatch-feasibility preflight: verify the backend can enforce
677719
# all fix-required hooks for the recipe's run_skill steps.
678720
if tool_ctx.active_recipe_steps is not None:
@@ -782,6 +824,11 @@ async def open_kitchen(
782824
tool_ctx.gate_infrastructure_ready = False
783825
return _recipe_validation_error_response(name, result)
784826

827+
if not result.get("dispatch_feasible", True):
828+
return await _dispatch_infeasible_response(
829+
result, tool_ctx.backend, tool_ctx.gate, ctx
830+
)
831+
785832
# Dispatch-feasibility preflight: verify the backend can enforce
786833
# all fix-required hooks for the recipe's run_skill steps.
787834
if tool_ctx.active_recipe_steps is not None:

0 commit comments

Comments
 (0)