[FIX] Pre-Prune Validity Computation Breaks Codex open_kitchen#4068
Merged
Trecek merged 13 commits intoJun 11, 2026
Merged
Conversation
Trecek
force-pushed
the
pre-prune-validity-computation-breaks-codex-open-kitchen-and/4059
branch
from
61f65d0 to
b9324e7
Compare
Pre-prune semantic findings (notably backend-incompatible-skill for guarded steps) poisoned compute_recipe_validity, causing load_and_validate to return valid=False for codex-pruned recipes. open_kitchen then fell through to the "unknown structural error" envelope because _recipe_validation_error_response only reads result.errors, not error-severity suggestions. Move the prune block before the semantic rules block in load_and_validate so pruned steps are never seen by semantic rules — self-enforcing against future rules that produce step-specific findings. Also surface error-severity suggestions in the validation error response with .get() fallbacks so the actual rule and message are visible instead of the generic "unknown structural error" fallback. Add an AST structural test (test_semantic_rules_run_after_pruning) to prevent future regression of the ordering invariant. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
_prune_skipped_steps clears skip_when_false on truthy-resolved steps but left optional=True. When semantic rules run after pruning, this causes false optional-without-skip-when findings. Also fix: pipeline ordering test path (doubled autoskillit), ruff violations, schema version allowlist line shifts, parametrize ids lambda on empty dict, and open_kitchen test wrapper signature. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… test overrides Pruning changes the step graph reachability, causing dead-output findings for captures that were live pre-prune. Compare pre-prune vs post-prune dead-output sets and suppress findings introduced by pruning. Also add open_pr=false override to codex tests since merge-conflict steps are guarded by open_pr, not backend_supports_git_write. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace dead-output-only filter with general approach: run semantic rules on the pre-prune recipe as baseline, then intersect with post-prune findings. Only findings present in BOTH survive — pruning-induced false positives (dead-output, capture-inversion-detection) are suppressed. Also fix open_kitchen test: set project_dir, pass open_pr=false via resolve_ingredient_defaults. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The pre-prune baseline adds a first run_semantic_rules call before _prune_skipped_steps. The AST ordering test must find the last (post-prune) call. Also fix backend name: canonical is 'claude-code' (hyphen), not 'claude_code' (underscore). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ng support After Part A fixed load_and_validate(), validate_from_path() still ran semantic rules on the raw unpruned recipe, producing false-positive backend-incompatible errors for Codex sessions. This change threads ingredient_overrides through the full caller chain — Protocol, DefaultRecipeRepository, InMemoryRecipeRepository, and the validate_recipe MCP tool — and calls _prune_skipped_steps() before semantic rule execution when overrides are provided. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…date arch catalog validate_from_path now snapshots pre-prune semantic findings before pruning, then filters post-prune findings to exclude pruning-induced false positives (dead-output, capture-inversion) — mirroring the load_and_validate pipeline. Also adds test_pipeline_ordering.py to the resolve-review Architectural Constraint Catalog. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Trecek
force-pushed
the
pre-prune-validity-computation-breaks-codex-open-kitchen-and/4059
branch
from
b9324e7 to
f337bca
Compare
…yright allowlist _recipe_validation_error_response now prefers structural errors when present, falling back to semantic findings only when errors list is empty. Uses 'unknown-rule' as the .get() default for missing rule keys. Updates pyright suppression allowlist for recipe/_api.py line shift (275→276). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Trecek
deleted the
pre-prune-validity-computation-breaks-codex-open-kitchen-and/4059
branch
This was referenced Jun 12, 2026
This was referenced Jun 30, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The recipe loading pipeline in
load_and_validate()computes validity from semantic rule findings that were generated against the pre-prune recipe. When the Codex backend is used, thebackend-incompatible-skillrule fires ERROR-severity findings against steps that haveskip_when_falseguards — steps that would be pruned immediately after. The stale pre-prune findings poisoncompute_recipe_validity(), marking the recipe asvalid=False. The error response function then reads the emptyerrors[]list (structural errors only) and falls back to "unknown structural error", hiding the true cause entirely.Part A fixes the core pipeline ordering bug in
load_and_validate()and surfaces semantic findings in the validation error response. Part B will cover adding pruning support tovalidate_from_pathand its full caller chain — implement as a separate task.Requirements
open_kitchen(name="implementation")fails on the first orchestrator message of every Codex-backend session in v0.10.752 with:The fail-closed guard added by PR #3995 (
61802dce4) gates onresult["valid"], butload_and_validatecomputesvalidfrom semantic-rule findings evaluated on the unpruned recipe. With the Codex capability override (backend_supports_git_write="false"), thebackend-incompatible-skillrule emits 13 ERROR-severity findings — 9 on steps that are pruned immediately afterward (skip_when_false: inputs.backend_supports_git_write) and 4 oninputs.open_pr-route-guarded steps that the conformance test explicitly exempts via_ROUTE_GUARDED_COMPAT_EXCEPTIONSbutcompute_recipe_validitydoes not. Sovalid=Falseeven though the served (pruned) content is structurally sound.Root cause chain (all empirically confirmed on installed v0.10.752)
CodexBackenddeclaresgit_metadata_writable=False(execution/backends/codex.py:521);_backend_capability_overridesyields{"backend_supports_git_write": "false"}, merged last/unoverridable (tools_kitchen.py:560-561via_promote_capability_keys).recipe/_api.py(load_and_validate):run_semantic_rulesruns at line 397 on the unpruned recipe →_prune_skipped_stepsat line 408 →compute_recipe_validityat line 457 consumes the pre-prune findings.backend-incompatible-skill(rules/rules_backend_compat.py:27-68) has no awareness ofskip_when_falseguards or route-guard exemptions → 13 ERROR findings for codex.compute_recipe_validity(registry.py:245-254) folds any ERROR-severity semantic/contract finding intovalid→valid=False,contentnon-empty,errors=[].not result.get("valid", False); the envelope renders only the structuralerrorslist → misleading "unknown structural error".Scope of fix
_prune_skipped_steps. TheValidationContextmust be rebuilt with the pruned recipe — not reused from the pre-prune build._ROUTE_GUARDED_COMPAT_EXCEPTIONS(currently test-only) into a production constant consumed by both the validity computation path and the conformance test.fleet/_api.pydispatch validation must pass the backend capability overrides so fleet's pruning matches what would actually be served.load_and_validatewith real capability overrides.Related work
valid=Truefor codex × implementation/planner viaload_and_validatewith no xfail machineryxfail(strict=True)markers must be removed after this fix landsCloses #4059
Implementation Plan
Plan file:
/home/talon/projects/autoskillit-runs/remediation-20260611-063523-547628/.autoskillit/temp/rectify/rectify_pre_prune_validity_2026-06-11_063523_part_a.md🤖 Generated with Claude Code via AutoSkillit
Token Usage Summary
* Step used a non-Anthropic provider; caching behavior may differ.
Token Efficiency
Model Usage Breakdown