Fleet L2 run_skill calls omit order_id, triggering ingredient-lock rejection before investigate

[Trecek]

Problem

Fleet L2 dispatches can invoke run_skill without order_id, causing ingredient-lock enforcement to reject the first recipe run_skill step before investigation starts.

This blocks the pipeline-hardening queue: item #4207 could not reach its investigate step because run_skill rejected the call with an empty pipeline scope.

Reproduction

From /home/talon/projects/generic_automation_mcp, dispatch #4207 with the queue-mandated command:

env -u AUTOSKILLIT_SESSION_TYPE AUTOSKILLIT_CORE__DEFAULT_MODEL='opus[1m]' \
  AUTOSKILLIT_RUN_SKILL__STREAM_IDLE_TIMEOUT_MS=7200000 \
  AUTOSKILLIT_RUN_SKILL__MAX_SUPPRESSION_SECONDS=7200 \
  autoskillit fleet run remediation-fable \
  -t "recipe_read_guard false positives on git add/diff/wc referencing recipe files (fix GitHub issue #4207)" \
  -i issue_url=https://github.com/TalonT-Org/AutoSkillit/issues/4207 \
  -i source_dir=/home/talon/projects/generic_automation_mcp \
  -i post_run_diagnostics=true --timeout-sec 14400

Evidence

Dispatch envelope:

{"kind":"completed","success":false,"dispatch_status":"failure","dispatch_id":"613cb784-aaf0-4f53-858e-e4257b4a7f59","dispatched_session_id":"019f42e1-eef5-7e81-a708-ed084b4695ce","reason":"degraded_tool_response","l3_payload":{"success":false,"reason":"degraded_tool_response","summary":"run_skill investigate failed with an ingredient lock enforcement error and returned no content field."}}

Raw rollout: /home/talon/.codex/sessions/2026/07/08/rollout-2026-07-08T10-58-50-019f42e1-eef5-7e81-a708-ed084b4695ce.jsonl

The L2 invoked run_skill for the investigate step with no order_id:

{"cwd":"/home/talon/projects/autoskillit-runs/remediation-20260708-110009-819754","model":"fable","skill_command":"/autoskillit:investigate ...","step_name":"investigate"}

The MCP result rejected it:

INGREDIENT LOCK ENFORCED: Step 'investigate' is locked out by pipeline ''. Pass order_id to scope the check, or call lock_ingredients(unlock=[...]) to release.

Session artifacts:

• Dispatch log: /home/talon/projects/generic_automation_mcp/.autoskillit/temp/codex-loop/4207-dispatch-20260708-105848.log
• Session directory: /home/talon/.local/share/autoskillit/logs/sessions/019f42e1-eef5-7e81-a708-ed084b4695ce/
• Clone: /home/talon/projects/autoskillit-runs/remediation-20260708-110009-819754
• Created branch: recipe-read-guard-false-positives-on-git-add-diff-wc-referen/4207 with 0 commits ahead of origin/develop; no implementation worktree existed.

Expected behavior

Fleet-orchestrated recipe steps should pass a stable per-dispatch scoping key, likely order_id=<dispatch_id>, to every run_skill invocation so ingredient locks are scoped to the active dispatch instead of the empty pipeline id.

If order_id is required for lock enforcement, recipe execution should inject it automatically rather than requiring every recipe author or L2 model to remember it.

Prior-attempt history

This was discovered while routing #4207 in the pipeline-hardening queue. #4207 itself has not been investigated or implemented yet; the failure happened before the first engineering step. This bug blocks re-dispatching #4207 and likely any other remediation-fable dispatch in the current environment.

Unclogger Round 1 — Stale Overlay Cleanup

Timestamp: 2026-07-08 11:39 PDT.

Driver entered unclogger mode because #4207 default dispatch could not reach investigate and the forced Claude L2 workaround is externally blocked by Anthropic weekly quota until 2026-07-12 10:00 America/Los_Angeles.

Pre-edit checks: no fleet dispatch was running (pgrep -af autoskillit fleet run|fleet run remediation|remediation-fable returned no live process; a second ps scan matched only the scan command).

Backup: copied .autoskillit/temp/.hook_config_overlay.json to .autoskillit/temp/codex-loop/hook_config_overlay.20260708T113815-0700.bak.json before editing.

Evidence: the overlay contained an empty-string/global lock scope with investigate=false. src/autoskillit/server/tools/tools_execution.py checks order_id or AUTOSKILLIT_DISPATCH_ID; when both are empty it scans every locked_steps scope and denies the matching step. That behavior matches #4207 round-1: the L2 called run_skill without order_id, and the empty scope rejected investigate.

Change: removed only the empty-string entries from locked_ingredients and locked_steps, leaving named historical pipeline lock scopes untouched. Verification: jq reported has_empty_locked_ingredients=false and has_empty_locked_steps=false after the edit.

This is a state cleanup, not a structural code fix. #4212 remains open because fleet L2 calls should still pass a stable order_id; clearing the stale global scope only allows the queue to test whether default dispatch can proceed now. Next action: retry #4207 with the default remediation-fable command and route the outcome.

Unclogger Round 2 — Named Stale Investigate Locks

Timestamp: 2026-07-08 11:45 PDT.

After the empty-string/global lock cleanup, #4207 default retry still failed before investigate: dispatch 9760a52e-49a9-4f44-a0e4-9d8163fa0ced, session 019f4307-f496-7ca1-a3ff-52448b3acd60. Raw rollout: /home/talon/.codex/sessions/2026/07/08/rollout-2026-07-08T11-40-22-019f4307-f496-7ca1-a3ff-52448b3acd60.jsonl. The denial was now from named stale scope 4171: INGREDIENT LOCK ENFORCED: Step 'investigate' is locked out by pipeline '4171'.

Evidence from current code: when run_skill has neither order_id nor AUTOSKILLIT_DISPATCH_ID, _check_ingredient_locks() scans every locked_steps entry and denies the first matching False for that step. With the global scope gone, old named investigate=false locks were still enough to block unscoped L2 calls.

Pre-edit check: no fleet dispatch process was running. Backup: .autoskillit/temp/codex-loop/hook_config_overlay.20260708T114218-0700.pre-named-investigate-clear.bak.json.

Change: removed stale investigate lock entries from all named historical scopes in locked_steps and locked_ingredients, leaving non-denying entries intact. Verification after cleanup: no locked_steps entries contain false, and neither locked_ingredients nor locked_steps has an empty-string scope.

This is still only state cleanup. The structural fix remains: fleet L2 recipe execution should pass a stable order_id to every run_skill call so unrelated historical lock scopes cannot affect an active dispatch. Next action: retry #4207 default remediation-fable and route the outcome.

Unclogger Round 3 — Route Update After Lock Cleanup

Timestamp: 2026-07-08 12:05 PDT.

After the second overlay cleanup, #4207 default dispatch no longer failed on ingredient-lock enforcement. Dispatch ed699bb3-80ab-46e6-9fc0-9fb6193aeb39, session 019f430d-d168-7270-bd58-191009e4475a, reached clone/claim/create-branch and launched the investigate child. This confirms the stale-lock state was unclogged enough for the default Codex L2 to pass the prior lock checks.

New blocker: the child failed because the local remediation-fable recipe pins investigate to model=fable, and Codex cannot run that model for this account. Child session 019f430e-d4c8-7201-9105-6773fa46baa6 exited in 4.6s with root error: The 'fable' model is not supported when using Codex with a ChatGPT account. Child rollout: /home/talon/.local/share/autoskillit/logs/codex-sessions/2026/07/08/rollout-2026-07-08T11-47-53-019f430e-d4c8-7201-9105-6773fa46baa6.jsonl.

Operator correction captured in the handoff: Anthropic-backed L2/model routes are unavailable and should not be used as the active path. Because the Codex backend is the system being hardened, Codex/default fleet dispatch may itself be the broken component. When remediation-fable would require Codex to launch fable, or Claude L2 to spend Anthropic quota, do not repeat dispatch; use UNCLOGGER mode and run recipe steps manually through existing MCP tools/skills/CLI.

Structural #4212 root cause remains open: fleet food-truck prompts still do not require L2 sessions to pass order_id=<dispatch_id> to inner run_skill calls. A read-only explorer scoped a minimal future fix: add a prompt contract in src/autoskillit/fleet/_prompts.py requiring order_id on every run_skill call and add a contract test in tests/fleet/test_food_truck_prompt.py. Do not change unscoped lock semantics; existing server tests intentionally cover unscoped denial.

Route Update After #4207 Manual Landing

Timestamp: 2026-07-08 14:25 PDT.

#4207 was completed through the manual UNCLOGGER recipe tail because #4212/default dispatch and Anthropic-backed Claude L2 routes were both unavailable. The driver manually ran prepare-pr, compose-pr, review-pr, and resolve-review through run_skill with the MiniMax provider route, then queued PR #4213.

Outcome:

• recipe_read_guard false positives on git add/diff/wc referencing recipe files — burns context, contributed to assess exhaustion #4207 PR Recipe Read Guard Metadata Command Remediation #4213 merged into develop at 2026-07-08T21:20:00Z.
• Merge commit: 61d7df986e41d6349363b42361056b281d341d3c.
• Installed AutoSkillit advanced to 0.10.856 after autoskillit update.
• Temporary quota_guard.disabled=true overlay state was removed after backing up the overlay; this was needed because the session-level override made quota hook tests fail by design.

This does not close #4212. The stale-lock cleanup and manual #4207 landing only bypassed the symptom. The structural issue remains: default/Codex L2 run_skill calls can omit order_id, allowing stale named locks to affect unscoped calls before the recipe reaches engineering work.

Route Update After #4199 Manual Landing — 2026-07-08/09

• fleet run resume of timed-out dispatch crashes: FAILURE→RUNNING invalid transition — timeout should classify RESUMABLE, resume should reset or reject cleanly #4199 could not safely use normal remediation-fable fleet dispatch because Codex rejects the fable-pinned child model and Anthropic-backed routes remain unavailable.
• The driver completed fleet run resume of timed-out dispatch crashes: FAILURE→RUNNING invalid transition — timeout should classify RESUMABLE, resume should reset or reject cleanly #4199 by preserving recipe order manually through MCP run_skill steps with step_provider=minimax and order_id=manual-4199, then ran the normal PR tail (prepare-pr, compose-pr, review-pr, resolve-review, diagnose-ci, resolve-failures).
• PR [FIX] Rectify: Fleet Dispatch Resume — Centralized Precondition Chokepoint #4214 merged into develop at 2026-07-09T01:52:27Z; fleet run resume of timed-out dispatch crashes: FAILURE→RUNNING invalid transition — timeout should classify RESUMABLE, resume should reset or reject cleanly #4199 is staged. This does not close the Fleet L2 run_skill calls omit order_id, triggering ingredient-lock rejection before investigate #4212 structural problem: Fleet L2 prompts still need to pass scoped order_id/dispatch identity to child run_skill calls so stale/global ingredient locks cannot affect unrelated work.
• Current overlay has no quota_guard key and no false ingredient locks; the fleet run resume of timed-out dispatch crashes: FAILURE→RUNNING invalid transition — timeout should classify RESUMABLE, resume should reset or reject cleanly #4199 temporary quota-guard overlay was backed up to .autoskillit/temp/codex-loop/hook_config_overlay.20260708-182056.pre-quota-guard-removal.bak.json and removed before final tests.

Post-#4215 Overlay Verification (2026-07-09 UTC)

Before retrying the #4202 Codex self-test after PR #4216, the driver rechecked fleet liveness and the active overlay. No live fleet dispatch process was running. .autoskillit/temp/.hook_config_overlay.json had no quota_guard key, no empty locked-step scope, and no false locked steps.

No additional overlay edit was evidence-supported. #4212 remains open as a structural issue: fleet L2 recipe execution still needs to pass a stable scoped order_id/dispatch identity to child run_skill calls so old historical lock scopes cannot affect an unscoped active dispatch.

July 10 backend-selection recurrence and ownership split

While continuing #4150 manually, a Codex parent invoked run_skill for an investigate step with literal model=fable. The child remained on the Codex backend and failed immediately because run_skill exposes a provider selector but no direct backend selector, and the investigate capability set did not activate worker routing. A corrected attempt required launching a separate Claude Code autoskillit cook parent first.

That recurrence is operationally related to the older remediation-fable failures recorded here, but it is not the order_id/ingredient-lock defect owned by #4212. The backend-selection defect now has a focused source of truth in #4227. The transient Codex capacity classification discovered in the same investigation is tracked in #4226.

#4212 remains scoped to requiring a stable dispatch identity on every fleet-L2 child run_skill call. #4227 owns capability-driven worker backend selection. Both must be installed before the final unattended Codex self-test.