diff --git a/docs/design/README.md b/docs/design/README.md index 57b5fc7ac..b4fdf157b 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -7,3 +7,7 @@ Design specifications for planned features and skills. | [env-setup-design.md](env-setup-design.md) | Design spec for the dedicated `setup-environment` skill — Docker vs micromamba-host decision tree, structured output tokens, and recipe integration | | [recording-replay-accepted-degradations.md](recording-replay-accepted-degradations.md) | Accepted degradations in P8 recording/replay: Claude PTY cassette format incompatibility with Codex replay, and unchanged Claude session recording path | | [acp-session-contract.md](acp-session-contract.md) | Normative reference for P6-A3-WP1: ACP session method mapping for the `CodingAgentBackend` lifecycle, recovery ladder from `RetryReason` to `session/resume`/`session/load`/`session/new`, capabilities translation of all 41 `BackendCapabilities` fields, and Codex shim deviations | +| [forward-obligations/inspector-contract-impact.md](forward-obligations/inspector-contract-impact.md) | Contract-impact note for Health Inspector (#3534): `build_inspector_cmd` stubs, `inspector_capable` gaps, `InspectorCallback` wiring | +| [forward-obligations/recording-replay-impact.md](forward-obligations/recording-replay-impact.md) | Contract-impact note for recording/replay: backend isolation invariant, format-detection extension points for future backends | +| [forward-obligations/triage-portability-impact.md](forward-obligations/triage-portability-impact.md) | Contract-impact note for triage portability: `triage_capable` gate, hardcoded Claude CLI in `_llm_triage.py`, portable `triage_cmd` path | +| [forward-obligations/heterogeneous-routing-impact.md](forward-obligations/heterogeneous-routing-impact.md) | Contract-impact note for heterogeneous per-item backend routing: `ToolContext.backend` single-backend constraint, `BackendSelector` proposal, `food_truck_capable` gate | diff --git a/docs/design/forward-obligations/heterogeneous-routing-impact.md b/docs/design/forward-obligations/heterogeneous-routing-impact.md new file mode 100644 index 000000000..422301864 --- /dev/null +++ b/docs/design/forward-obligations/heterogeneous-routing-impact.md @@ -0,0 +1,59 @@ +# Heterogeneous Per-Item Backend Routing Contract Impact + +| Field | Value | +|-------|-------| +| Status | Draft | +| Date | 2026-06-27 | + +## Obligation + +Heterogeneous per-item backend routing means individual work-items in a fleet campaign can +be dispatched to different backends (e.g., item A to `ClaudeCodeBackend`, item B to +`CodexBackend`) rather than all items sharing a single campaign backend. + +## Current State + +- `BACKEND_REGISTRY` in `execution/backends/__init__.py:46–49` maps string names to backend + classes: `{"claude-code": ClaudeCodeBackend, "codex": CodexBackend}`. +- `ToolContext.backend` (`pipeline/context.py:164`) is a single + `CodingAgentBackend | None` instance — one backend per session. +- Fleet dispatch reads `ctx.backend` once per campaign. +- `dispatch_food_truck()` (`execution/headless/__init__.py:349`) resolves + `self._ctx.backend` at line 416 — no per-item routing. It raises `RuntimeError` if + `food_truck_capable` is `False` (line 385). +- `run_headless_core` supports a per-step `backend_override` string (lines 198–199) that + resolves to a local `step_backend` via `get_backend()`, but this override is call-local + and does not mutate `ctx.backend`. + +## Contract Gaps + +1. **`ToolContext.backend` is a single-backend constraint.** Heterogeneous routing requires + either making `backend` a per-item selection or introducing a routing callable. +2. **`build_food_truck_cmd` and `dispatch_food_truck` assume a single backend.** The entire + fleet dispatch path reads `ctx.backend` without per-item indirection. +3. **`BackendCapabilities.food_truck_capable`** (`_type_backend.py:80`) is the existing gate. + Both `ClaudeCodeBackend` (`CLAUDE_CODE_CAPABILITIES:226`) and `CodexBackend` + (`codex.py:556`) set it to `True`. + +## Required CodingAgentBackend Contract Additions + +1. Define a backend routing selector type: + `BackendSelector = Callable[[WorkItem], CodingAgentBackend]` +2. `food_truck_capable` must remain `True` only on backends that satisfy the full dispatch + contract (command construction, output parsing, session lifecycle). +3. `dispatch_food_truck` must accept a selector or per-item backend resolution rather than + reading `ctx.backend` unconditionally. + +## Key Constraint + +`BackendCapabilities` is a frozen dataclass. Adding a routing-selector field would need to +be forward-declared with a tracking issue (added to `_FORWARD_DECLARED` in +`test_capability_consumption.py`) until the fleet routing layer is refactored to consume it. + +## Key References + +- `execution/backends/__init__.py:46–49` — `BACKEND_REGISTRY` +- `execution/headless/__init__.py:349` — `dispatch_food_truck` +- `pipeline/context.py:164` — `ToolContext.backend` +- `core/types/_type_backend.py:80` — `BackendCapabilities.food_truck_capable` +- `fleet/` — campaign dispatch (reads `ctx.backend` per campaign) diff --git a/docs/design/forward-obligations/inspector-contract-impact.md b/docs/design/forward-obligations/inspector-contract-impact.md new file mode 100644 index 000000000..15aa04334 --- /dev/null +++ b/docs/design/forward-obligations/inspector-contract-impact.md @@ -0,0 +1,63 @@ +# Health Inspector Contract Impact + +| Field | Value | +|-------|-------| +| Status | Draft | +| Date | 2026-06-27 | +| Linked Issue | #3534 | + +## Obligation + +Both `ClaudeCodeBackend` and `CodexBackend` raise `CapabilityNotSupportedError` from +`build_inspector_cmd()` when `inspector_capable` is `False` (the current default), and +raise `AssertionError` if `inspector_capable` were ever set to `True` without a real +implementation: + +- `claude.py:872–876` — `ClaudeCodeBackend.build_inspector_cmd()` +- `codex.py:1117–1121` — `CodexBackend.build_inspector_cmd()` + +The Health Inspector feature (#3534) requires a real implementation on each backend. + +## Contract Gaps + +- `BackendCapabilities.inspector_capable` is `False` on both backends: + - `CLAUDE_CODE_CAPABILITIES` (`_type_backend.py:253`): `inspector_capable=False` + - `CodexBackend.capabilities` property (`codex.py:584`): `inspector_capable=False` +- Neither backend provides a working `build_inspector_cmd()` — both have guard-then-assert + stubs. + +## Protocol Surface + +Already defined in `core/types/_type_inspector.py`: + +- **`InspectorEvidence`** (lines 15–24, frozen dataclass): `idle_seconds`, `stdout_path`, + `jsonl_lines`, `cpu_trend`, `rss_trend`, `connection_summary`, + `execution_marker_present`, `dispatch_context` +- **`InspectorVerdict`** (lines 27–32, frozen dataclass): `action`, `reasoning`, + `confidence`, `elapsed_seconds` +- **`InspectorCallback`** (line 35): `Callable[[InspectorEvidence], Awaitable[InspectorVerdict]]` + +The callback is threaded through `execution/process/_process_race.py`: the parameter is +declared at line 189, guarded at 253, invoked at 279, and a timeout warning emitted at 281. +Related `inspector_verdict` fields (lines 75, 102, 116) and inspector-state transitions +(lines 262, 265, 287, 295, 540) complete the protocol surface in the same file. The callback +also appears in `recording.py` at lines 141 and 432. +The wiring point is `execution/headless/_headless_execute.py:270` where +`inspector_callback=None` is hardcoded (no backend currently provides an inspector). + +## Required Steps + +1. Implement `build_inspector_cmd()` on `ClaudeCodeBackend` — return a `CmdSpec` that + launches an LLM-backed inspector subprocess. +2. Implement `build_inspector_cmd()` on `CodexBackend` — or leave as `CapabilityNotSupportedError` + if Codex cannot support inspection. +3. Set `inspector_capable=True` on backends with a working implementation. +4. Wire the callback into `_process_race._run_race()` via the `inspector_callback` parameter + (currently hardcoded to `None` in `_headless_execute.py:270`). + +## Arch Enforcement + +`test_capability_consumption.py` (`tests/arch/`) verifies every `BackendCapabilities` field +has a production consumer. `inspector_capable` already satisfies this via the +`self.capabilities.inspector_capable` guard in both backends' `build_inspector_cmd()` methods. +The field is NOT in the `_FORWARD_DECLARED` exemption set. diff --git a/docs/design/forward-obligations/recording-replay-impact.md b/docs/design/forward-obligations/recording-replay-impact.md new file mode 100644 index 000000000..1bd6e2272 --- /dev/null +++ b/docs/design/forward-obligations/recording-replay-impact.md @@ -0,0 +1,68 @@ +# Recording/Replay Contract Impact + +| Field | Value | +|-------|-------| +| Status | Draft | +| Date | 2026-06-27 | + +## Obligation + +Recording/replay must remain backend-isolated. Cross-backend cassette replay is explicitly +a non-goal (see `docs/design/recording-replay-accepted-degradations.md`). + +## BackendCapabilities Fields + +- `replay_capable` and `record_capable` are defined on `BackendCapabilities` + (`_type_backend.py:127–129`), both defaulting to `False`. +- `ClaudeCodeBackend`: both set to `True` via `CLAUDE_CODE_CAPABILITIES` + (`_type_backend.py:249–250`). +- `CodexBackend`: both set to `False` (`codex.py:580–581`). Codex replay is handled via + `CodexScenarioPlayer`. + +## Format Detection + +`_detect_backend_format()` in `recording.py:56–59` routes by the presence of +`*/codex_stdout.ndjson` inside the scenario directory: + +```python +def _detect_backend_format(scenario_dir: Path) -> str: + if any(scenario_dir.glob("*/codex_stdout.ndjson")): + return "codex" + return "claude" +``` + +This function is called by `build_replay_runner()` (`recording.py:494`) to select between +`CodexScenarioPlayer` and `make_scenario_player` (Claude player). + +Recording dispatch in `RecordingSubprocessRunner.__call__()` (`recording.py:117–239`) uses +`capabilities.pty_required` to distinguish backends: PTY path for Claude, non-PTY path for +Codex (which writes `codex_stdout.ndjson` + `step_meta.json`). + +Consumption gates at the server layer (`_factory.py:223–252`): +- `replay_capable` guards `build_replay_runner()` construction +- `record_capable` guards `RecordingSubprocessRunner` construction + +## Contract Gaps + +Adding a third backend (e.g., opencode-via-ACP) requires extending +`_detect_backend_format()` with a new sentinel file or NDJSON magic-byte check. + +## Forward Obligation + +Any new backend adding replay support must: + +1. Declare `replay_capable=True` in its `BackendCapabilities`. +2. Provide a concrete `ScenarioPlayer` subclass. +3. Register a format-detection branch in `_detect_backend_format()` with a unique sentinel + file. +4. Add a `make_scenario_player` factory dispatch for the new format. +5. If supporting recording: declare `record_capable=True` and implement the recording path + in `RecordingSubprocessRunner.__call__()`. + +## Key References + +- `execution/recording.py` — `_detect_backend_format`, `RecordingSubprocessRunner` +- `execution/backends/codex_scenario_player.py` — `CodexScenarioPlayer`, `CodexStepRecord` +- `docs/design/recording-replay-accepted-degradations.md` — accepted degradations +- `core/types/_type_backend.py:127–129` — `replay_capable`, `record_capable` field definitions +- `server/_factory.py:223–252` — capability-gated construction diff --git a/docs/design/forward-obligations/triage-portability-impact.md b/docs/design/forward-obligations/triage-portability-impact.md new file mode 100644 index 000000000..d67ee53d0 --- /dev/null +++ b/docs/design/forward-obligations/triage-portability-impact.md @@ -0,0 +1,65 @@ +# Triage Portability Contract Impact + +| Field | Value | +|-------|-------| +| Status | Draft | +| Date | 2026-06-27 | + +## Obligation + +LLM triage (contract-staleness triage via `_llm_triage.py`) is gated by +`BackendCapabilities.triage_capable`. Backends with `triage_capable=False` cause +`_triage_batch` to return all hash-mismatch items as `meaningful=True` without launching +an LLM subprocess. + +## Current State + +- `ClaudeCodeBackend`: `triage_capable=True` (set in `CLAUDE_CODE_CAPABILITIES`, + `_type_backend.py:229`). Uses a `claude -p` subprocess with Haiku model. +- `CodexBackend`: `triage_capable=False` (`codex.py:559`). Triage is skipped entirely. + +## Contract Gaps + +`_triage_batch()` (`_llm_triage.py:80`) constructs the triage command at lines 123–131: + +```python +triage_cmd: list[str] = [ + backend.binary_name(), + ClaudeFlags.PRINT, + prompt, + ClaudeFlags.MODEL, + "claude-haiku-4-5-20251001", + ClaudeFlags.OUTPUT_FORMAT, + fmt.value, +] +``` + +The command uses `backend.binary_name()` (not a hardcoded `"claude"` literal) but depends on +`ClaudeFlags` constants (`PRINT`, `MODEL`, `OUTPUT_FORMAT`) which are Claude-CLI-specific. +The model `"claude-haiku-4-5-20251001"` is hardcoded. Portability to a non-Claude backend +requires either: + +- A backend-agnostic `triage_cmd()` protocol method on `CodingAgentBackend`, or +- A dedicated lightweight triage backend decoupled from the coding-agent backend. + +## Required Steps for Portable Triage + +1. Add an optional `triage_cmd(prompt: str, model: str) -> list[str]` method to + `CodingAgentBackend` (or add a `BackendCapabilities.triage_model` field). +2. Update `_triage_batch` to call `backend.triage_cmd()` instead of constructing the + command inline with `ClaudeFlags`. +3. Gate on `triage_capable` (already done at `_llm_triage.py:107`). + +## Blast Radius + +`_llm_triage.py` is a standalone module at the package root. Changes are isolated and do not +touch `BackendCapabilities` layout or the sub-protocol set. The `triage_capable` field on +`BackendCapabilities` (`_type_backend.py:86`) already has production consumers (the guard at +`_llm_triage.py:107`), so no `_FORWARD_DECLARED` exemption is needed. + +## Key References + +- `_llm_triage.py:33` — `triage_staleness()` entry point +- `_llm_triage.py:80` — `_triage_batch()` with command construction +- `_llm_triage.py:107` — `triage_capable` guard +- `core/types/_type_backend.py:86` — `triage_capable` field definition