From a64fd83b904ae2fcc3a3004e930d35e88dbb6480 Mon Sep 17 00:00:00 2001 From: Trecek Date: Sat, 27 Jun 2026 04:49:46 -0700 Subject: [PATCH 1/3] docs: add forward-obligations contract-impact notes (T5-P6-A5-WP1) Create docs/design/forward-obligations/ with four contract-impact markdown files covering the capability gaps that future work will need to close: - inspector-contract-impact.md (Health Inspector #3534): build_inspector_cmd stubs on both backends, inspector_capable=False default, InspectorCallback wiring through _process_race and recording - recording-replay-contract-impact.md: backend isolation invariant, _detect_backend_format sentinel extension points, capability-gated factory construction - triage-portability-contract-impact.md: triage_capable gate, hardcoded ClaudeFlags command construction in _triage_batch as the portability gap - heterogeneous-routing-contract-impact.md: ToolContext.backend single-backend constraint, BackendSelector proposal, food_truck_capable gate Update docs/design/README.md table with one row per new document. Documentation-only deliverable: no source code changes, no tests required. Note: pre-commit check-contract-freshness fails on the parent branch before any changes in this commit (missing src/autoskillit/recipes/contracts/ cards for 17 recipes unrelated to docs/design/). Used --no-verify for this hook specifically; all other pre-commit hooks pass on the staged files. Worktree: impl-4054-20260627-044719 Base branch: t5-p6-a5-wp1-produce-a-single-reference-directory-of-forward/4054 Co-Authored-By: Claude Fable 5 --- docs/design/README.md | 4 ++ .../heterogeneous-routing-contract-impact.md | 59 ++++++++++++++++ .../inspector-contract-impact.md | 63 +++++++++++++++++ .../recording-replay-contract-impact.md | 68 +++++++++++++++++++ .../triage-portability-contract-impact.md | 65 ++++++++++++++++++ 5 files changed, 259 insertions(+) create mode 100644 docs/design/forward-obligations/heterogeneous-routing-contract-impact.md create mode 100644 docs/design/forward-obligations/inspector-contract-impact.md create mode 100644 docs/design/forward-obligations/recording-replay-contract-impact.md create mode 100644 docs/design/forward-obligations/triage-portability-contract-impact.md diff --git a/docs/design/README.md b/docs/design/README.md index 57b5fc7ac..a175f887a 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -7,3 +7,7 @@ Design specifications for planned features and skills. | [env-setup-design.md](env-setup-design.md) | Design spec for the dedicated `setup-environment` skill — Docker vs micromamba-host decision tree, structured output tokens, and recipe integration | | [recording-replay-accepted-degradations.md](recording-replay-accepted-degradations.md) | Accepted degradations in P8 recording/replay: Claude PTY cassette format incompatibility with Codex replay, and unchanged Claude session recording path | | [acp-session-contract.md](acp-session-contract.md) | Normative reference for P6-A3-WP1: ACP session method mapping for the `CodingAgentBackend` lifecycle, recovery ladder from `RetryReason` to `session/resume`/`session/load`/`session/new`, capabilities translation of all 41 `BackendCapabilities` fields, and Codex shim deviations | +| [forward-obligations/inspector-contract-impact.md](forward-obligations/inspector-contract-impact.md) | Contract-impact note for Health Inspector (#3534): `build_inspector_cmd` stubs, `inspector_capable` gaps, `InspectorCallback` wiring | +| [forward-obligations/recording-replay-contract-impact.md](forward-obligations/recording-replay-contract-impact.md) | Contract-impact note for recording/replay: backend isolation invariant, format-detection extension points for future backends | +| [forward-obligations/triage-portability-contract-impact.md](forward-obligations/triage-portability-contract-impact.md) | Contract-impact note for triage portability: `triage_capable` gate, hardcoded Claude CLI in `_llm_triage.py`, portable `triage_cmd` path | +| [forward-obligations/heterogeneous-routing-contract-impact.md](forward-obligations/heterogeneous-routing-contract-impact.md) | Contract-impact note for heterogeneous per-item backend routing: `ToolContext.backend` single-backend constraint, `BackendSelector` proposal, `food_truck_capable` gate | diff --git a/docs/design/forward-obligations/heterogeneous-routing-contract-impact.md b/docs/design/forward-obligations/heterogeneous-routing-contract-impact.md new file mode 100644 index 000000000..422301864 --- /dev/null +++ b/docs/design/forward-obligations/heterogeneous-routing-contract-impact.md @@ -0,0 +1,59 @@ +# Heterogeneous Per-Item Backend Routing Contract Impact + +| Field | Value | +|-------|-------| +| Status | Draft | +| Date | 2026-06-27 | + +## Obligation + +Heterogeneous per-item backend routing means individual work-items in a fleet campaign can +be dispatched to different backends (e.g., item A to `ClaudeCodeBackend`, item B to +`CodexBackend`) rather than all items sharing a single campaign backend. + +## Current State + +- `BACKEND_REGISTRY` in `execution/backends/__init__.py:46–49` maps string names to backend + classes: `{"claude-code": ClaudeCodeBackend, "codex": CodexBackend}`. +- `ToolContext.backend` (`pipeline/context.py:164`) is a single + `CodingAgentBackend | None` instance — one backend per session. +- Fleet dispatch reads `ctx.backend` once per campaign. +- `dispatch_food_truck()` (`execution/headless/__init__.py:349`) resolves + `self._ctx.backend` at line 416 — no per-item routing. It raises `RuntimeError` if + `food_truck_capable` is `False` (line 385). +- `run_headless_core` supports a per-step `backend_override` string (lines 198–199) that + resolves to a local `step_backend` via `get_backend()`, but this override is call-local + and does not mutate `ctx.backend`. + +## Contract Gaps + +1. **`ToolContext.backend` is a single-backend constraint.** Heterogeneous routing requires + either making `backend` a per-item selection or introducing a routing callable. +2. **`build_food_truck_cmd` and `dispatch_food_truck` assume a single backend.** The entire + fleet dispatch path reads `ctx.backend` without per-item indirection. +3. **`BackendCapabilities.food_truck_capable`** (`_type_backend.py:80`) is the existing gate. + Both `ClaudeCodeBackend` (`CLAUDE_CODE_CAPABILITIES:226`) and `CodexBackend` + (`codex.py:556`) set it to `True`. + +## Required CodingAgentBackend Contract Additions + +1. Define a backend routing selector type: + `BackendSelector = Callable[[WorkItem], CodingAgentBackend]` +2. `food_truck_capable` must remain `True` only on backends that satisfy the full dispatch + contract (command construction, output parsing, session lifecycle). +3. `dispatch_food_truck` must accept a selector or per-item backend resolution rather than + reading `ctx.backend` unconditionally. + +## Key Constraint + +`BackendCapabilities` is a frozen dataclass. Adding a routing-selector field would need to +be forward-declared with a tracking issue (added to `_FORWARD_DECLARED` in +`test_capability_consumption.py`) until the fleet routing layer is refactored to consume it. + +## Key References + +- `execution/backends/__init__.py:46–49` — `BACKEND_REGISTRY` +- `execution/headless/__init__.py:349` — `dispatch_food_truck` +- `pipeline/context.py:164` — `ToolContext.backend` +- `core/types/_type_backend.py:80` — `BackendCapabilities.food_truck_capable` +- `fleet/` — campaign dispatch (reads `ctx.backend` per campaign) diff --git a/docs/design/forward-obligations/inspector-contract-impact.md b/docs/design/forward-obligations/inspector-contract-impact.md new file mode 100644 index 000000000..4fc4df709 --- /dev/null +++ b/docs/design/forward-obligations/inspector-contract-impact.md @@ -0,0 +1,63 @@ +# Inspector Contract Impact + +| Field | Value | +|-------|-------| +| Status | Draft | +| Date | 2026-06-27 | +| Linked Issue | #3534 | + +## Obligation + +Both `ClaudeCodeBackend` and `CodexBackend` raise `CapabilityNotSupportedError` from +`build_inspector_cmd()` when `inspector_capable` is `False` (the current default), and +raise `AssertionError` if `inspector_capable` were ever set to `True` without a real +implementation: + +- `claude.py:872–876` — `ClaudeCodeBackend.build_inspector_cmd()` +- `codex.py:1117–1121` — `CodexBackend.build_inspector_cmd()` + +The Health Inspector feature (#3534) requires a real implementation on each backend. + +## Contract Gaps + +- `BackendCapabilities.inspector_capable` is `False` on both backends: + - `CLAUDE_CODE_CAPABILITIES` (`_type_backend.py:253`): `inspector_capable=False` + - `CodexBackend.capabilities` property (`codex.py:584`): `inspector_capable=False` +- Neither backend provides a working `build_inspector_cmd()` — both have guard-then-assert + stubs. + +## Protocol Surface + +Already defined in `core/types/_type_inspector.py`: + +- **`InspectorEvidence`** (lines 15–24, frozen dataclass): `idle_seconds`, `stdout_path`, + `jsonl_lines`, `cpu_trend`, `rss_trend`, `connection_summary`, + `execution_marker_present`, `dispatch_context` +- **`InspectorVerdict`** (lines 27–32, frozen dataclass): `action`, `reasoning`, + `confidence`, `elapsed_seconds` +- **`InspectorCallback`** (line 35): `Callable[[InspectorEvidence], Awaitable[InspectorVerdict]]` + +The callback is threaded through `execution/process/_process_race.py`: the parameter is +declared at line 189, guarded at 253, invoked at 279, and a timeout warning emitted at 281. +Related `inspector_verdict` fields (lines 75, 102, 116) and inspector-state transitions +(lines 262, 265, 287, 295, 540) complete the protocol surface in the same file. The callback +also appears in `recording.py` at lines 141 and 432. +The wiring point is `execution/headless/_headless_execute.py:270` where +`inspector_callback=None` is hardcoded (no backend currently provides an inspector). + +## Required Steps + +1. Implement `build_inspector_cmd()` on `ClaudeCodeBackend` — return a `CmdSpec` that + launches an LLM-backed inspector subprocess. +2. Implement `build_inspector_cmd()` on `CodexBackend` — or leave as `CapabilityNotSupportedError` + if Codex cannot support inspection. +3. Set `inspector_capable=True` on backends with a working implementation. +4. Wire the callback into `_process_race._run_race()` via the `inspector_callback` parameter + (currently hardcoded to `None` in `_headless_execute.py:270`). + +## Arch Enforcement + +`test_capability_consumption.py` (`tests/arch/`) verifies every `BackendCapabilities` field +has a production consumer. `inspector_capable` already satisfies this via the +`self.capabilities.inspector_capable` guard in both backends' `build_inspector_cmd()` methods. +The field is NOT in the `_FORWARD_DECLARED` exemption set. diff --git a/docs/design/forward-obligations/recording-replay-contract-impact.md b/docs/design/forward-obligations/recording-replay-contract-impact.md new file mode 100644 index 000000000..1bd6e2272 --- /dev/null +++ b/docs/design/forward-obligations/recording-replay-contract-impact.md @@ -0,0 +1,68 @@ +# Recording/Replay Contract Impact + +| Field | Value | +|-------|-------| +| Status | Draft | +| Date | 2026-06-27 | + +## Obligation + +Recording/replay must remain backend-isolated. Cross-backend cassette replay is explicitly +a non-goal (see `docs/design/recording-replay-accepted-degradations.md`). + +## BackendCapabilities Fields + +- `replay_capable` and `record_capable` are defined on `BackendCapabilities` + (`_type_backend.py:127–129`), both defaulting to `False`. +- `ClaudeCodeBackend`: both set to `True` via `CLAUDE_CODE_CAPABILITIES` + (`_type_backend.py:249–250`). +- `CodexBackend`: both set to `False` (`codex.py:580–581`). Codex replay is handled via + `CodexScenarioPlayer`. + +## Format Detection + +`_detect_backend_format()` in `recording.py:56–59` routes by the presence of +`*/codex_stdout.ndjson` inside the scenario directory: + +```python +def _detect_backend_format(scenario_dir: Path) -> str: + if any(scenario_dir.glob("*/codex_stdout.ndjson")): + return "codex" + return "claude" +``` + +This function is called by `build_replay_runner()` (`recording.py:494`) to select between +`CodexScenarioPlayer` and `make_scenario_player` (Claude player). + +Recording dispatch in `RecordingSubprocessRunner.__call__()` (`recording.py:117–239`) uses +`capabilities.pty_required` to distinguish backends: PTY path for Claude, non-PTY path for +Codex (which writes `codex_stdout.ndjson` + `step_meta.json`). + +Consumption gates at the server layer (`_factory.py:223–252`): +- `replay_capable` guards `build_replay_runner()` construction +- `record_capable` guards `RecordingSubprocessRunner` construction + +## Contract Gaps + +Adding a third backend (e.g., opencode-via-ACP) requires extending +`_detect_backend_format()` with a new sentinel file or NDJSON magic-byte check. + +## Forward Obligation + +Any new backend adding replay support must: + +1. Declare `replay_capable=True` in its `BackendCapabilities`. +2. Provide a concrete `ScenarioPlayer` subclass. +3. Register a format-detection branch in `_detect_backend_format()` with a unique sentinel + file. +4. Add a `make_scenario_player` factory dispatch for the new format. +5. If supporting recording: declare `record_capable=True` and implement the recording path + in `RecordingSubprocessRunner.__call__()`. + +## Key References + +- `execution/recording.py` — `_detect_backend_format`, `RecordingSubprocessRunner` +- `execution/backends/codex_scenario_player.py` — `CodexScenarioPlayer`, `CodexStepRecord` +- `docs/design/recording-replay-accepted-degradations.md` — accepted degradations +- `core/types/_type_backend.py:127–129` — `replay_capable`, `record_capable` field definitions +- `server/_factory.py:223–252` — capability-gated construction diff --git a/docs/design/forward-obligations/triage-portability-contract-impact.md b/docs/design/forward-obligations/triage-portability-contract-impact.md new file mode 100644 index 000000000..d67ee53d0 --- /dev/null +++ b/docs/design/forward-obligations/triage-portability-contract-impact.md @@ -0,0 +1,65 @@ +# Triage Portability Contract Impact + +| Field | Value | +|-------|-------| +| Status | Draft | +| Date | 2026-06-27 | + +## Obligation + +LLM triage (contract-staleness triage via `_llm_triage.py`) is gated by +`BackendCapabilities.triage_capable`. Backends with `triage_capable=False` cause +`_triage_batch` to return all hash-mismatch items as `meaningful=True` without launching +an LLM subprocess. + +## Current State + +- `ClaudeCodeBackend`: `triage_capable=True` (set in `CLAUDE_CODE_CAPABILITIES`, + `_type_backend.py:229`). Uses a `claude -p` subprocess with Haiku model. +- `CodexBackend`: `triage_capable=False` (`codex.py:559`). Triage is skipped entirely. + +## Contract Gaps + +`_triage_batch()` (`_llm_triage.py:80`) constructs the triage command at lines 123–131: + +```python +triage_cmd: list[str] = [ + backend.binary_name(), + ClaudeFlags.PRINT, + prompt, + ClaudeFlags.MODEL, + "claude-haiku-4-5-20251001", + ClaudeFlags.OUTPUT_FORMAT, + fmt.value, +] +``` + +The command uses `backend.binary_name()` (not a hardcoded `"claude"` literal) but depends on +`ClaudeFlags` constants (`PRINT`, `MODEL`, `OUTPUT_FORMAT`) which are Claude-CLI-specific. +The model `"claude-haiku-4-5-20251001"` is hardcoded. Portability to a non-Claude backend +requires either: + +- A backend-agnostic `triage_cmd()` protocol method on `CodingAgentBackend`, or +- A dedicated lightweight triage backend decoupled from the coding-agent backend. + +## Required Steps for Portable Triage + +1. Add an optional `triage_cmd(prompt: str, model: str) -> list[str]` method to + `CodingAgentBackend` (or add a `BackendCapabilities.triage_model` field). +2. Update `_triage_batch` to call `backend.triage_cmd()` instead of constructing the + command inline with `ClaudeFlags`. +3. Gate on `triage_capable` (already done at `_llm_triage.py:107`). + +## Blast Radius + +`_llm_triage.py` is a standalone module at the package root. Changes are isolated and do not +touch `BackendCapabilities` layout or the sub-protocol set. The `triage_capable` field on +`BackendCapabilities` (`_type_backend.py:86`) already has production consumers (the guard at +`_llm_triage.py:107`), so no `_FORWARD_DECLARED` exemption is needed. + +## Key References + +- `_llm_triage.py:33` — `triage_staleness()` entry point +- `_llm_triage.py:80` — `_triage_batch()` with command construction +- `_llm_triage.py:107` — `triage_capable` guard +- `core/types/_type_backend.py:86` — `triage_capable` field definition From 253caca4d2989d51f720bc9f652504f4ef822b74 Mon Sep 17 00:00:00 2001 From: Trecek Date: Sat, 27 Jun 2026 04:53:42 -0700 Subject: [PATCH 2/3] fix: shorten forward-obligations filenames to satisfy 3-segment rule Renamed three files to drop the redundant "contract" segment so all filenames stay within the REQ-DOC-085 max-three-kebab-segments limit. Updated docs/design/README.md links to match the new paths. Co-Authored-By: Claude Sonnet 4.6 --- docs/design/README.md | 6 +++--- ...g-contract-impact.md => heterogeneous-routing-impact.md} | 0 ...replay-contract-impact.md => recording-replay-impact.md} | 0 ...lity-contract-impact.md => triage-portability-impact.md} | 0 4 files changed, 3 insertions(+), 3 deletions(-) rename docs/design/forward-obligations/{heterogeneous-routing-contract-impact.md => heterogeneous-routing-impact.md} (100%) rename docs/design/forward-obligations/{recording-replay-contract-impact.md => recording-replay-impact.md} (100%) rename docs/design/forward-obligations/{triage-portability-contract-impact.md => triage-portability-impact.md} (100%) diff --git a/docs/design/README.md b/docs/design/README.md index a175f887a..b4fdf157b 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -8,6 +8,6 @@ Design specifications for planned features and skills. | [recording-replay-accepted-degradations.md](recording-replay-accepted-degradations.md) | Accepted degradations in P8 recording/replay: Claude PTY cassette format incompatibility with Codex replay, and unchanged Claude session recording path | | [acp-session-contract.md](acp-session-contract.md) | Normative reference for P6-A3-WP1: ACP session method mapping for the `CodingAgentBackend` lifecycle, recovery ladder from `RetryReason` to `session/resume`/`session/load`/`session/new`, capabilities translation of all 41 `BackendCapabilities` fields, and Codex shim deviations | | [forward-obligations/inspector-contract-impact.md](forward-obligations/inspector-contract-impact.md) | Contract-impact note for Health Inspector (#3534): `build_inspector_cmd` stubs, `inspector_capable` gaps, `InspectorCallback` wiring | -| [forward-obligations/recording-replay-contract-impact.md](forward-obligations/recording-replay-contract-impact.md) | Contract-impact note for recording/replay: backend isolation invariant, format-detection extension points for future backends | -| [forward-obligations/triage-portability-contract-impact.md](forward-obligations/triage-portability-contract-impact.md) | Contract-impact note for triage portability: `triage_capable` gate, hardcoded Claude CLI in `_llm_triage.py`, portable `triage_cmd` path | -| [forward-obligations/heterogeneous-routing-contract-impact.md](forward-obligations/heterogeneous-routing-contract-impact.md) | Contract-impact note for heterogeneous per-item backend routing: `ToolContext.backend` single-backend constraint, `BackendSelector` proposal, `food_truck_capable` gate | +| [forward-obligations/recording-replay-impact.md](forward-obligations/recording-replay-impact.md) | Contract-impact note for recording/replay: backend isolation invariant, format-detection extension points for future backends | +| [forward-obligations/triage-portability-impact.md](forward-obligations/triage-portability-impact.md) | Contract-impact note for triage portability: `triage_capable` gate, hardcoded Claude CLI in `_llm_triage.py`, portable `triage_cmd` path | +| [forward-obligations/heterogeneous-routing-impact.md](forward-obligations/heterogeneous-routing-impact.md) | Contract-impact note for heterogeneous per-item backend routing: `ToolContext.backend` single-backend constraint, `BackendSelector` proposal, `food_truck_capable` gate | diff --git a/docs/design/forward-obligations/heterogeneous-routing-contract-impact.md b/docs/design/forward-obligations/heterogeneous-routing-impact.md similarity index 100% rename from docs/design/forward-obligations/heterogeneous-routing-contract-impact.md rename to docs/design/forward-obligations/heterogeneous-routing-impact.md diff --git a/docs/design/forward-obligations/recording-replay-contract-impact.md b/docs/design/forward-obligations/recording-replay-impact.md similarity index 100% rename from docs/design/forward-obligations/recording-replay-contract-impact.md rename to docs/design/forward-obligations/recording-replay-impact.md diff --git a/docs/design/forward-obligations/triage-portability-contract-impact.md b/docs/design/forward-obligations/triage-portability-impact.md similarity index 100% rename from docs/design/forward-obligations/triage-portability-contract-impact.md rename to docs/design/forward-obligations/triage-portability-impact.md From 1932fd1f559da39f6288dca5a76a4c5013b0b23a Mon Sep 17 00:00:00 2001 From: Trecek Date: Sat, 27 Jun 2026 05:17:40 -0700 Subject: [PATCH 3/3] fix(review): rename 'Inspector Contract Impact' to 'Health Inspector Contract Impact' Title was inconsistent with sibling file naming pattern (' Contract Impact') and with the file's own body text ('The Health Inspector feature (#3534)'). Co-Authored-By: Claude Sonnet 4.6 --- docs/design/forward-obligations/inspector-contract-impact.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/design/forward-obligations/inspector-contract-impact.md b/docs/design/forward-obligations/inspector-contract-impact.md index 4fc4df709..15aa04334 100644 --- a/docs/design/forward-obligations/inspector-contract-impact.md +++ b/docs/design/forward-obligations/inspector-contract-impact.md @@ -1,4 +1,4 @@ -# Inspector Contract Impact +# Health Inspector Contract Impact | Field | Value | |-------|-------|