Skip to content

Commit 455601d

Browse files
Trecekclaude
andauthored
Implementation Plan: T5-P6-A5-WP1 Forward-Obligation Readiness Notes (#4125)
## Summary Create the `docs/design/forward-obligations/` directory containing four contract-impact markdown files (inspector, recording/replay, triage portability, heterogeneous routing) and update `docs/design/README.md` with table rows for each new document. This is a documentation-only deliverable — no source code files are created or modified. Closes #4054 ## Implementation Plan Plan file: `/home/talon/projects/autoskillit-runs/impl-20260627-043714-780997/.autoskillit/temp/make-plan/T5-P6-A5-WP1-forward-obligation-readiness-notes_plan_2026-06-27_044600.md` 🤖 Generated with [Claude Code](https://claude.com/claude-code) via AutoSkillit <!-- autoskillit:pipeline-signature steps=prepare_pr,run_arch_lenses,compose_pr,annotate_pr_diff,review_pr --> ## Token Usage Summary | Step | Model | count | uncached | output | cache_read | peak_ctx | turns | cache_write | time | |------|-------|-------|----------|--------|------------|----------|-------|-------------|------| | plan* | opus[1m] | 1 | 4.3k | 9.6k | 411.2k | 55.1k | 20 | 62.3k | 4m 35s | | verify* | sonnet | 1 | 1.8k | 9.5k | 545.8k | 55.5k | 27 | 36.9k | 5m 17s | | implement* | MiniMax-M3 | 1 | 53.0k | 9.3k | 1.4M | 0 | 47 | 0 | 2m 40s | | fix* | sonnet | 1 | 150 | 6.2k | 869.5k | 59.0k | 48 | 40.5k | 3m 30s | | audit_impl* | sonnet | 1 | 44 | 11.5k | 164.3k | 43.0k | 14 | 37.2k | 5m 34s | | prepare_pr* | MiniMax-M3 | 1 | 44.1k | 2.1k | 195.0k | 45.5k | 13 | 0 | 44s | | compose_pr* | MiniMax-M3 | 1 | 36.2k | 1.2k | 283.4k | 0 | 13 | 0 | 40s | | **Total** | | | 139.6k | 49.3k | 3.9M | 59.0k | | 176.9k | 23m 2s | \* *Step used a non-Anthropic provider; caching behavior may differ.* ## Token Efficiency | Step | LoC Changed | cache_read/LoC | cache_write/LoC | output/LoC | |------|-------------|----------------|-----------------|------------| | plan | 0 | — | — | — | | verify | 0 | — | — | — | | implement | 259 | 5521.8 | 0.0 | 35.7 | | fix | 6 | 144912.8 | 6747.2 | 1026.3 | | audit_impl | 0 | — | — | — | | prepare_pr | 0 | — | — | — | | compose_pr | 0 | — | — | — | | **Total** | **265** | 14714.5 | 667.7 | 186.0 | ## Model Usage Breakdown | Model | steps | uncached | output | cache_read | cache_write | time | |-------|-------|----------|--------|------------|-------------|------| | opus[1m] | 1 | 4.3k | 9.6k | 411.2k | 62.3k | 4m 35s | | sonnet | 3 | 2.0k | 27.1k | 1.6M | 114.7k | 14m 23s | | MiniMax-M3 | 3 | 133.3k | 12.6k | 1.9M | 0 | 4m 3s | --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent ce1a067 commit 455601d

5 files changed

Lines changed: 259 additions & 0 deletions

File tree

docs/design/README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,3 +7,7 @@ Design specifications for planned features and skills.
77
| [env-setup-design.md](env-setup-design.md) | Design spec for the dedicated `setup-environment` skill — Docker vs micromamba-host decision tree, structured output tokens, and recipe integration |
88
| [recording-replay-accepted-degradations.md](recording-replay-accepted-degradations.md) | Accepted degradations in P8 recording/replay: Claude PTY cassette format incompatibility with Codex replay, and unchanged Claude session recording path |
99
| [acp-session-contract.md](acp-session-contract.md) | Normative reference for P6-A3-WP1: ACP session method mapping for the `CodingAgentBackend` lifecycle, recovery ladder from `RetryReason` to `session/resume`/`session/load`/`session/new`, capabilities translation of all 41 `BackendCapabilities` fields, and Codex shim deviations |
10+
| [forward-obligations/inspector-contract-impact.md](forward-obligations/inspector-contract-impact.md) | Contract-impact note for Health Inspector (#3534): `build_inspector_cmd` stubs, `inspector_capable` gaps, `InspectorCallback` wiring |
11+
| [forward-obligations/recording-replay-impact.md](forward-obligations/recording-replay-impact.md) | Contract-impact note for recording/replay: backend isolation invariant, format-detection extension points for future backends |
12+
| [forward-obligations/triage-portability-impact.md](forward-obligations/triage-portability-impact.md) | Contract-impact note for triage portability: `triage_capable` gate, hardcoded Claude CLI in `_llm_triage.py`, portable `triage_cmd` path |
13+
| [forward-obligations/heterogeneous-routing-impact.md](forward-obligations/heterogeneous-routing-impact.md) | Contract-impact note for heterogeneous per-item backend routing: `ToolContext.backend` single-backend constraint, `BackendSelector` proposal, `food_truck_capable` gate |
Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# Heterogeneous Per-Item Backend Routing Contract Impact
2+
3+
| Field | Value |
4+
|-------|-------|
5+
| Status | Draft |
6+
| Date | 2026-06-27 |
7+
8+
## Obligation
9+
10+
Heterogeneous per-item backend routing means individual work-items in a fleet campaign can
11+
be dispatched to different backends (e.g., item A to `ClaudeCodeBackend`, item B to
12+
`CodexBackend`) rather than all items sharing a single campaign backend.
13+
14+
## Current State
15+
16+
- `BACKEND_REGISTRY` in `execution/backends/__init__.py:46–49` maps string names to backend
17+
classes: `{"claude-code": ClaudeCodeBackend, "codex": CodexBackend}`.
18+
- `ToolContext.backend` (`pipeline/context.py:164`) is a single
19+
`CodingAgentBackend | None` instance — one backend per session.
20+
- Fleet dispatch reads `ctx.backend` once per campaign.
21+
- `dispatch_food_truck()` (`execution/headless/__init__.py:349`) resolves
22+
`self._ctx.backend` at line 416 — no per-item routing. It raises `RuntimeError` if
23+
`food_truck_capable` is `False` (line 385).
24+
- `run_headless_core` supports a per-step `backend_override` string (lines 198–199) that
25+
resolves to a local `step_backend` via `get_backend()`, but this override is call-local
26+
and does not mutate `ctx.backend`.
27+
28+
## Contract Gaps
29+
30+
1. **`ToolContext.backend` is a single-backend constraint.** Heterogeneous routing requires
31+
either making `backend` a per-item selection or introducing a routing callable.
32+
2. **`build_food_truck_cmd` and `dispatch_food_truck` assume a single backend.** The entire
33+
fleet dispatch path reads `ctx.backend` without per-item indirection.
34+
3. **`BackendCapabilities.food_truck_capable`** (`_type_backend.py:80`) is the existing gate.
35+
Both `ClaudeCodeBackend` (`CLAUDE_CODE_CAPABILITIES:226`) and `CodexBackend`
36+
(`codex.py:556`) set it to `True`.
37+
38+
## Required CodingAgentBackend Contract Additions
39+
40+
1. Define a backend routing selector type:
41+
`BackendSelector = Callable[[WorkItem], CodingAgentBackend]`
42+
2. `food_truck_capable` must remain `True` only on backends that satisfy the full dispatch
43+
contract (command construction, output parsing, session lifecycle).
44+
3. `dispatch_food_truck` must accept a selector or per-item backend resolution rather than
45+
reading `ctx.backend` unconditionally.
46+
47+
## Key Constraint
48+
49+
`BackendCapabilities` is a frozen dataclass. Adding a routing-selector field would need to
50+
be forward-declared with a tracking issue (added to `_FORWARD_DECLARED` in
51+
`test_capability_consumption.py`) until the fleet routing layer is refactored to consume it.
52+
53+
## Key References
54+
55+
- `execution/backends/__init__.py:46–49``BACKEND_REGISTRY`
56+
- `execution/headless/__init__.py:349``dispatch_food_truck`
57+
- `pipeline/context.py:164``ToolContext.backend`
58+
- `core/types/_type_backend.py:80``BackendCapabilities.food_truck_capable`
59+
- `fleet/` — campaign dispatch (reads `ctx.backend` per campaign)
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Health Inspector Contract Impact
2+
3+
| Field | Value |
4+
|-------|-------|
5+
| Status | Draft |
6+
| Date | 2026-06-27 |
7+
| Linked Issue | #3534 |
8+
9+
## Obligation
10+
11+
Both `ClaudeCodeBackend` and `CodexBackend` raise `CapabilityNotSupportedError` from
12+
`build_inspector_cmd()` when `inspector_capable` is `False` (the current default), and
13+
raise `AssertionError` if `inspector_capable` were ever set to `True` without a real
14+
implementation:
15+
16+
- `claude.py:872–876``ClaudeCodeBackend.build_inspector_cmd()`
17+
- `codex.py:1117–1121``CodexBackend.build_inspector_cmd()`
18+
19+
The Health Inspector feature (#3534) requires a real implementation on each backend.
20+
21+
## Contract Gaps
22+
23+
- `BackendCapabilities.inspector_capable` is `False` on both backends:
24+
- `CLAUDE_CODE_CAPABILITIES` (`_type_backend.py:253`): `inspector_capable=False`
25+
- `CodexBackend.capabilities` property (`codex.py:584`): `inspector_capable=False`
26+
- Neither backend provides a working `build_inspector_cmd()` — both have guard-then-assert
27+
stubs.
28+
29+
## Protocol Surface
30+
31+
Already defined in `core/types/_type_inspector.py`:
32+
33+
- **`InspectorEvidence`** (lines 15–24, frozen dataclass): `idle_seconds`, `stdout_path`,
34+
`jsonl_lines`, `cpu_trend`, `rss_trend`, `connection_summary`,
35+
`execution_marker_present`, `dispatch_context`
36+
- **`InspectorVerdict`** (lines 27–32, frozen dataclass): `action`, `reasoning`,
37+
`confidence`, `elapsed_seconds`
38+
- **`InspectorCallback`** (line 35): `Callable[[InspectorEvidence], Awaitable[InspectorVerdict]]`
39+
40+
The callback is threaded through `execution/process/_process_race.py`: the parameter is
41+
declared at line 189, guarded at 253, invoked at 279, and a timeout warning emitted at 281.
42+
Related `inspector_verdict` fields (lines 75, 102, 116) and inspector-state transitions
43+
(lines 262, 265, 287, 295, 540) complete the protocol surface in the same file. The callback
44+
also appears in `recording.py` at lines 141 and 432.
45+
The wiring point is `execution/headless/_headless_execute.py:270` where
46+
`inspector_callback=None` is hardcoded (no backend currently provides an inspector).
47+
48+
## Required Steps
49+
50+
1. Implement `build_inspector_cmd()` on `ClaudeCodeBackend` — return a `CmdSpec` that
51+
launches an LLM-backed inspector subprocess.
52+
2. Implement `build_inspector_cmd()` on `CodexBackend` — or leave as `CapabilityNotSupportedError`
53+
if Codex cannot support inspection.
54+
3. Set `inspector_capable=True` on backends with a working implementation.
55+
4. Wire the callback into `_process_race._run_race()` via the `inspector_callback` parameter
56+
(currently hardcoded to `None` in `_headless_execute.py:270`).
57+
58+
## Arch Enforcement
59+
60+
`test_capability_consumption.py` (`tests/arch/`) verifies every `BackendCapabilities` field
61+
has a production consumer. `inspector_capable` already satisfies this via the
62+
`self.capabilities.inspector_capable` guard in both backends' `build_inspector_cmd()` methods.
63+
The field is NOT in the `_FORWARD_DECLARED` exemption set.
Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Recording/Replay Contract Impact
2+
3+
| Field | Value |
4+
|-------|-------|
5+
| Status | Draft |
6+
| Date | 2026-06-27 |
7+
8+
## Obligation
9+
10+
Recording/replay must remain backend-isolated. Cross-backend cassette replay is explicitly
11+
a non-goal (see `docs/design/recording-replay-accepted-degradations.md`).
12+
13+
## BackendCapabilities Fields
14+
15+
- `replay_capable` and `record_capable` are defined on `BackendCapabilities`
16+
(`_type_backend.py:127–129`), both defaulting to `False`.
17+
- `ClaudeCodeBackend`: both set to `True` via `CLAUDE_CODE_CAPABILITIES`
18+
(`_type_backend.py:249–250`).
19+
- `CodexBackend`: both set to `False` (`codex.py:580–581`). Codex replay is handled via
20+
`CodexScenarioPlayer`.
21+
22+
## Format Detection
23+
24+
`_detect_backend_format()` in `recording.py:56–59` routes by the presence of
25+
`*/codex_stdout.ndjson` inside the scenario directory:
26+
27+
```python
28+
def _detect_backend_format(scenario_dir: Path) -> str:
29+
if any(scenario_dir.glob("*/codex_stdout.ndjson")):
30+
return "codex"
31+
return "claude"
32+
```
33+
34+
This function is called by `build_replay_runner()` (`recording.py:494`) to select between
35+
`CodexScenarioPlayer` and `make_scenario_player` (Claude player).
36+
37+
Recording dispatch in `RecordingSubprocessRunner.__call__()` (`recording.py:117–239`) uses
38+
`capabilities.pty_required` to distinguish backends: PTY path for Claude, non-PTY path for
39+
Codex (which writes `codex_stdout.ndjson` + `step_meta.json`).
40+
41+
Consumption gates at the server layer (`_factory.py:223–252`):
42+
- `replay_capable` guards `build_replay_runner()` construction
43+
- `record_capable` guards `RecordingSubprocessRunner` construction
44+
45+
## Contract Gaps
46+
47+
Adding a third backend (e.g., opencode-via-ACP) requires extending
48+
`_detect_backend_format()` with a new sentinel file or NDJSON magic-byte check.
49+
50+
## Forward Obligation
51+
52+
Any new backend adding replay support must:
53+
54+
1. Declare `replay_capable=True` in its `BackendCapabilities`.
55+
2. Provide a concrete `ScenarioPlayer` subclass.
56+
3. Register a format-detection branch in `_detect_backend_format()` with a unique sentinel
57+
file.
58+
4. Add a `make_scenario_player` factory dispatch for the new format.
59+
5. If supporting recording: declare `record_capable=True` and implement the recording path
60+
in `RecordingSubprocessRunner.__call__()`.
61+
62+
## Key References
63+
64+
- `execution/recording.py``_detect_backend_format`, `RecordingSubprocessRunner`
65+
- `execution/backends/codex_scenario_player.py``CodexScenarioPlayer`, `CodexStepRecord`
66+
- `docs/design/recording-replay-accepted-degradations.md` — accepted degradations
67+
- `core/types/_type_backend.py:127–129``replay_capable`, `record_capable` field definitions
68+
- `server/_factory.py:223–252` — capability-gated construction
Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
# Triage Portability Contract Impact
2+
3+
| Field | Value |
4+
|-------|-------|
5+
| Status | Draft |
6+
| Date | 2026-06-27 |
7+
8+
## Obligation
9+
10+
LLM triage (contract-staleness triage via `_llm_triage.py`) is gated by
11+
`BackendCapabilities.triage_capable`. Backends with `triage_capable=False` cause
12+
`_triage_batch` to return all hash-mismatch items as `meaningful=True` without launching
13+
an LLM subprocess.
14+
15+
## Current State
16+
17+
- `ClaudeCodeBackend`: `triage_capable=True` (set in `CLAUDE_CODE_CAPABILITIES`,
18+
`_type_backend.py:229`). Uses a `claude -p` subprocess with Haiku model.
19+
- `CodexBackend`: `triage_capable=False` (`codex.py:559`). Triage is skipped entirely.
20+
21+
## Contract Gaps
22+
23+
`_triage_batch()` (`_llm_triage.py:80`) constructs the triage command at lines 123–131:
24+
25+
```python
26+
triage_cmd: list[str] = [
27+
backend.binary_name(),
28+
ClaudeFlags.PRINT,
29+
prompt,
30+
ClaudeFlags.MODEL,
31+
"claude-haiku-4-5-20251001",
32+
ClaudeFlags.OUTPUT_FORMAT,
33+
fmt.value,
34+
]
35+
```
36+
37+
The command uses `backend.binary_name()` (not a hardcoded `"claude"` literal) but depends on
38+
`ClaudeFlags` constants (`PRINT`, `MODEL`, `OUTPUT_FORMAT`) which are Claude-CLI-specific.
39+
The model `"claude-haiku-4-5-20251001"` is hardcoded. Portability to a non-Claude backend
40+
requires either:
41+
42+
- A backend-agnostic `triage_cmd()` protocol method on `CodingAgentBackend`, or
43+
- A dedicated lightweight triage backend decoupled from the coding-agent backend.
44+
45+
## Required Steps for Portable Triage
46+
47+
1. Add an optional `triage_cmd(prompt: str, model: str) -> list[str]` method to
48+
`CodingAgentBackend` (or add a `BackendCapabilities.triage_model` field).
49+
2. Update `_triage_batch` to call `backend.triage_cmd()` instead of constructing the
50+
command inline with `ClaudeFlags`.
51+
3. Gate on `triage_capable` (already done at `_llm_triage.py:107`).
52+
53+
## Blast Radius
54+
55+
`_llm_triage.py` is a standalone module at the package root. Changes are isolated and do not
56+
touch `BackendCapabilities` layout or the sub-protocol set. The `triage_capable` field on
57+
`BackendCapabilities` (`_type_backend.py:86`) already has production consumers (the guard at
58+
`_llm_triage.py:107`), so no `_FORWARD_DECLARED` exemption is needed.
59+
60+
## Key References
61+
62+
- `_llm_triage.py:33``triage_staleness()` entry point
63+
- `_llm_triage.py:80``_triage_batch()` with command construction
64+
- `_llm_triage.py:107``triage_capable` guard
65+
- `core/types/_type_backend.py:86``triage_capable` field definition

0 commit comments

Comments
 (0)