Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/design/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,7 @@ Design specifications for planned features and skills.
| [env-setup-design.md](env-setup-design.md) | Design spec for the dedicated `setup-environment` skill β€” Docker vs micromamba-host decision tree, structured output tokens, and recipe integration |
| [recording-replay-accepted-degradations.md](recording-replay-accepted-degradations.md) | Accepted degradations in P8 recording/replay: Claude PTY cassette format incompatibility with Codex replay, and unchanged Claude session recording path |
| [acp-session-contract.md](acp-session-contract.md) | Normative reference for P6-A3-WP1: ACP session method mapping for the `CodingAgentBackend` lifecycle, recovery ladder from `RetryReason` to `session/resume`/`session/load`/`session/new`, capabilities translation of all 41 `BackendCapabilities` fields, and Codex shim deviations |
| [forward-obligations/inspector-contract-impact.md](forward-obligations/inspector-contract-impact.md) | Contract-impact note for Health Inspector (#3534): `build_inspector_cmd` stubs, `inspector_capable` gaps, `InspectorCallback` wiring |
| [forward-obligations/recording-replay-impact.md](forward-obligations/recording-replay-impact.md) | Contract-impact note for recording/replay: backend isolation invariant, format-detection extension points for future backends |
| [forward-obligations/triage-portability-impact.md](forward-obligations/triage-portability-impact.md) | Contract-impact note for triage portability: `triage_capable` gate, hardcoded Claude CLI in `_llm_triage.py`, portable `triage_cmd` path |
| [forward-obligations/heterogeneous-routing-impact.md](forward-obligations/heterogeneous-routing-impact.md) | Contract-impact note for heterogeneous per-item backend routing: `ToolContext.backend` single-backend constraint, `BackendSelector` proposal, `food_truck_capable` gate |
59 changes: 59 additions & 0 deletions docs/design/forward-obligations/heterogeneous-routing-impact.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Heterogeneous Per-Item Backend Routing Contract Impact

| Field | Value |
|-------|-------|
| Status | Draft |
| Date | 2026-06-27 |

## Obligation

Heterogeneous per-item backend routing means individual work-items in a fleet campaign can
be dispatched to different backends (e.g., item A to `ClaudeCodeBackend`, item B to
`CodexBackend`) rather than all items sharing a single campaign backend.

## Current State

- `BACKEND_REGISTRY` in `execution/backends/__init__.py:46–49` maps string names to backend
classes: `{"claude-code": ClaudeCodeBackend, "codex": CodexBackend}`.
- `ToolContext.backend` (`pipeline/context.py:164`) is a single
`CodingAgentBackend | None` instance β€” one backend per session.
- Fleet dispatch reads `ctx.backend` once per campaign.
- `dispatch_food_truck()` (`execution/headless/__init__.py:349`) resolves
`self._ctx.backend` at line 416 β€” no per-item routing. It raises `RuntimeError` if
`food_truck_capable` is `False` (line 385).
- `run_headless_core` supports a per-step `backend_override` string (lines 198–199) that
resolves to a local `step_backend` via `get_backend()`, but this override is call-local
and does not mutate `ctx.backend`.

## Contract Gaps

1. **`ToolContext.backend` is a single-backend constraint.** Heterogeneous routing requires
either making `backend` a per-item selection or introducing a routing callable.
2. **`build_food_truck_cmd` and `dispatch_food_truck` assume a single backend.** The entire
fleet dispatch path reads `ctx.backend` without per-item indirection.
3. **`BackendCapabilities.food_truck_capable`** (`_type_backend.py:80`) is the existing gate.
Both `ClaudeCodeBackend` (`CLAUDE_CODE_CAPABILITIES:226`) and `CodexBackend`
(`codex.py:556`) set it to `True`.

## Required CodingAgentBackend Contract Additions

1. Define a backend routing selector type:
`BackendSelector = Callable[[WorkItem], CodingAgentBackend]`
2. `food_truck_capable` must remain `True` only on backends that satisfy the full dispatch
contract (command construction, output parsing, session lifecycle).
3. `dispatch_food_truck` must accept a selector or per-item backend resolution rather than
reading `ctx.backend` unconditionally.

## Key Constraint

`BackendCapabilities` is a frozen dataclass. Adding a routing-selector field would need to
be forward-declared with a tracking issue (added to `_FORWARD_DECLARED` in
`test_capability_consumption.py`) until the fleet routing layer is refactored to consume it.

## Key References

- `execution/backends/__init__.py:46–49` β€” `BACKEND_REGISTRY`
- `execution/headless/__init__.py:349` β€” `dispatch_food_truck`
- `pipeline/context.py:164` β€” `ToolContext.backend`
- `core/types/_type_backend.py:80` β€” `BackendCapabilities.food_truck_capable`
- `fleet/` β€” campaign dispatch (reads `ctx.backend` per campaign)
63 changes: 63 additions & 0 deletions docs/design/forward-obligations/inspector-contract-impact.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Health Inspector Contract Impact

| Field | Value |
|-------|-------|
| Status | Draft |
| Date | 2026-06-27 |
| Linked Issue | #3534 |

## Obligation

Both `ClaudeCodeBackend` and `CodexBackend` raise `CapabilityNotSupportedError` from
`build_inspector_cmd()` when `inspector_capable` is `False` (the current default), and
raise `AssertionError` if `inspector_capable` were ever set to `True` without a real
implementation:

- `claude.py:872–876` β€” `ClaudeCodeBackend.build_inspector_cmd()`
- `codex.py:1117–1121` β€” `CodexBackend.build_inspector_cmd()`

The Health Inspector feature (#3534) requires a real implementation on each backend.

## Contract Gaps

- `BackendCapabilities.inspector_capable` is `False` on both backends:
- `CLAUDE_CODE_CAPABILITIES` (`_type_backend.py:253`): `inspector_capable=False`
- `CodexBackend.capabilities` property (`codex.py:584`): `inspector_capable=False`
- Neither backend provides a working `build_inspector_cmd()` β€” both have guard-then-assert
stubs.

## Protocol Surface

Already defined in `core/types/_type_inspector.py`:

- **`InspectorEvidence`** (lines 15–24, frozen dataclass): `idle_seconds`, `stdout_path`,
`jsonl_lines`, `cpu_trend`, `rss_trend`, `connection_summary`,
`execution_marker_present`, `dispatch_context`
- **`InspectorVerdict`** (lines 27–32, frozen dataclass): `action`, `reasoning`,
`confidence`, `elapsed_seconds`
- **`InspectorCallback`** (line 35): `Callable[[InspectorEvidence], Awaitable[InspectorVerdict]]`

The callback is threaded through `execution/process/_process_race.py`: the parameter is
declared at line 189, guarded at 253, invoked at 279, and a timeout warning emitted at 281.
Related `inspector_verdict` fields (lines 75, 102, 116) and inspector-state transitions
(lines 262, 265, 287, 295, 540) complete the protocol surface in the same file. The callback
also appears in `recording.py` at lines 141 and 432.
The wiring point is `execution/headless/_headless_execute.py:270` where
`inspector_callback=None` is hardcoded (no backend currently provides an inspector).

## Required Steps

1. Implement `build_inspector_cmd()` on `ClaudeCodeBackend` β€” return a `CmdSpec` that
launches an LLM-backed inspector subprocess.
2. Implement `build_inspector_cmd()` on `CodexBackend` β€” or leave as `CapabilityNotSupportedError`
if Codex cannot support inspection.
3. Set `inspector_capable=True` on backends with a working implementation.
4. Wire the callback into `_process_race._run_race()` via the `inspector_callback` parameter
(currently hardcoded to `None` in `_headless_execute.py:270`).

## Arch Enforcement

`test_capability_consumption.py` (`tests/arch/`) verifies every `BackendCapabilities` field
has a production consumer. `inspector_capable` already satisfies this via the
`self.capabilities.inspector_capable` guard in both backends' `build_inspector_cmd()` methods.
The field is NOT in the `_FORWARD_DECLARED` exemption set.
68 changes: 68 additions & 0 deletions docs/design/forward-obligations/recording-replay-impact.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Recording/Replay Contract Impact

| Field | Value |
|-------|-------|
| Status | Draft |
| Date | 2026-06-27 |

## Obligation

Recording/replay must remain backend-isolated. Cross-backend cassette replay is explicitly
a non-goal (see `docs/design/recording-replay-accepted-degradations.md`).

## BackendCapabilities Fields

- `replay_capable` and `record_capable` are defined on `BackendCapabilities`
(`_type_backend.py:127–129`), both defaulting to `False`.
- `ClaudeCodeBackend`: both set to `True` via `CLAUDE_CODE_CAPABILITIES`
(`_type_backend.py:249–250`).
- `CodexBackend`: both set to `False` (`codex.py:580–581`). Codex replay is handled via
`CodexScenarioPlayer`.

## Format Detection

`_detect_backend_format()` in `recording.py:56–59` routes by the presence of
`*/codex_stdout.ndjson` inside the scenario directory:

```python
def _detect_backend_format(scenario_dir: Path) -> str:
if any(scenario_dir.glob("*/codex_stdout.ndjson")):
return "codex"
return "claude"
```

This function is called by `build_replay_runner()` (`recording.py:494`) to select between
`CodexScenarioPlayer` and `make_scenario_player` (Claude player).

Recording dispatch in `RecordingSubprocessRunner.__call__()` (`recording.py:117–239`) uses
`capabilities.pty_required` to distinguish backends: PTY path for Claude, non-PTY path for
Codex (which writes `codex_stdout.ndjson` + `step_meta.json`).

Consumption gates at the server layer (`_factory.py:223–252`):
- `replay_capable` guards `build_replay_runner()` construction
- `record_capable` guards `RecordingSubprocessRunner` construction

## Contract Gaps

Adding a third backend (e.g., opencode-via-ACP) requires extending
`_detect_backend_format()` with a new sentinel file or NDJSON magic-byte check.

## Forward Obligation

Any new backend adding replay support must:

1. Declare `replay_capable=True` in its `BackendCapabilities`.
2. Provide a concrete `ScenarioPlayer` subclass.
3. Register a format-detection branch in `_detect_backend_format()` with a unique sentinel
file.
4. Add a `make_scenario_player` factory dispatch for the new format.
5. If supporting recording: declare `record_capable=True` and implement the recording path
in `RecordingSubprocessRunner.__call__()`.

## Key References

- `execution/recording.py` β€” `_detect_backend_format`, `RecordingSubprocessRunner`
- `execution/backends/codex_scenario_player.py` β€” `CodexScenarioPlayer`, `CodexStepRecord`
- `docs/design/recording-replay-accepted-degradations.md` β€” accepted degradations
- `core/types/_type_backend.py:127–129` β€” `replay_capable`, `record_capable` field definitions
- `server/_factory.py:223–252` β€” capability-gated construction
65 changes: 65 additions & 0 deletions docs/design/forward-obligations/triage-portability-impact.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Triage Portability Contract Impact

| Field | Value |
|-------|-------|
| Status | Draft |
| Date | 2026-06-27 |

## Obligation

LLM triage (contract-staleness triage via `_llm_triage.py`) is gated by
`BackendCapabilities.triage_capable`. Backends with `triage_capable=False` cause
`_triage_batch` to return all hash-mismatch items as `meaningful=True` without launching
an LLM subprocess.

## Current State

- `ClaudeCodeBackend`: `triage_capable=True` (set in `CLAUDE_CODE_CAPABILITIES`,
`_type_backend.py:229`). Uses a `claude -p` subprocess with Haiku model.
- `CodexBackend`: `triage_capable=False` (`codex.py:559`). Triage is skipped entirely.

## Contract Gaps

`_triage_batch()` (`_llm_triage.py:80`) constructs the triage command at lines 123–131:

```python
triage_cmd: list[str] = [
backend.binary_name(),
ClaudeFlags.PRINT,
prompt,
ClaudeFlags.MODEL,
"claude-haiku-4-5-20251001",
ClaudeFlags.OUTPUT_FORMAT,
fmt.value,
]
```

The command uses `backend.binary_name()` (not a hardcoded `"claude"` literal) but depends on
`ClaudeFlags` constants (`PRINT`, `MODEL`, `OUTPUT_FORMAT`) which are Claude-CLI-specific.
The model `"claude-haiku-4-5-20251001"` is hardcoded. Portability to a non-Claude backend
requires either:

- A backend-agnostic `triage_cmd()` protocol method on `CodingAgentBackend`, or
- A dedicated lightweight triage backend decoupled from the coding-agent backend.

## Required Steps for Portable Triage

1. Add an optional `triage_cmd(prompt: str, model: str) -> list[str]` method to
`CodingAgentBackend` (or add a `BackendCapabilities.triage_model` field).
2. Update `_triage_batch` to call `backend.triage_cmd()` instead of constructing the
command inline with `ClaudeFlags`.
3. Gate on `triage_capable` (already done at `_llm_triage.py:107`).

## Blast Radius

`_llm_triage.py` is a standalone module at the package root. Changes are isolated and do not
touch `BackendCapabilities` layout or the sub-protocol set. The `triage_capable` field on
`BackendCapabilities` (`_type_backend.py:86`) already has production consumers (the guard at
`_llm_triage.py:107`), so no `_FORWARD_DECLARED` exemption is needed.

## Key References

- `_llm_triage.py:33` β€” `triage_staleness()` entry point
- `_llm_triage.py:80` β€” `_triage_batch()` with command construction
- `_llm_triage.py:107` β€” `triage_capable` guard
- `core/types/_type_backend.py:86` β€” `triage_capable` field definition
Loading