Skip to content

Commit f7e5980

Browse files
TrecekTrecek
authored
Implementation Plan: T5-P3-A3-WP1 Establish Committed Fixture Foundation for B3a Deterministic Conformance (#4137)
## Summary Create 21 committed JSON fixture files and 2 Python package init files under `tests/execution/backends/fixtures/codex_ndjson/` to seal the contract boundary between Codex CLI NDJSON output and the production parser. This includes 9 event schemas (one per non-UNKNOWN `CodexEventType`), 10 item schemas (one per non-UNKNOWN `CodexItemType`), a hook event format snapshot bound to `HOOK_REGISTRY_HASH`, and a config.toml schema template. Also updates `.autoskillit/test-filter-manifest.yaml` with a glob pattern for the new JSON fixture files. No source code or test files are modified — this WP produces only fixture data consumed by P3-A3-WP3's conformance tests. Closes #4010 ## Implementation Plan Plan file: `/home/talon/projects/autoskillit-runs/impl-20260628-005721-828122/.autoskillit/temp/make-plan/t5-p3-a3-wp1-establish-fixture-foundation_plan_2026-06-28_010600.md` 🤖 Generated with [Claude Code](https://claude.com/claude-code) via AutoSkillit <!-- autoskillit:pipeline-signature steps=prepare_pr,run_arch_lenses,compose_pr,annotate_pr_diff,review_pr --> ## Token Usage Summary | Step | Model | count | uncached | output | cache_read | peak_ctx | turns | cache_write | time | |------|-------|-------|----------|--------|------------|----------|-------|-------------|------| | plan* | opus[1m] | 1 | 1.3k | 22.1k | 1.4M | 105.1k | 40 | 86.5k | 14m 4s | | verify* | sonnet | 1 | 68 | 7.3k | 356.6k | 58.3k | 23 | 39.8k | 4m 6s | | implement* | MiniMax-M3 | 1 | 68.3k | 12.4k | 1.6M | 0 | 108 | 0 | 5m 29s | | audit_impl* | sonnet | 1 | 62 | 14.3k | 250.9k | 51.9k | 19 | 47.6k | 7m 47s | | prepare_pr* | MiniMax-M3 | 1 | 47.4k | 2.5k | 161.7k | 49.1k | 14 | 0 | 1m 32s | | compose_pr* | MiniMax-M3 | 1 | 36.7k | 1.3k | 177.2k | 0 | 14 | 0 | 31s | | **Total** | | | 153.9k | 59.9k | 4.0M | 105.1k | | 173.9k | 33m 32s | \* *Step used a non-Anthropic provider; caching behavior may differ.* ## Token Efficiency | Step | LoC Changed | cache_read/LoC | cache_write/LoC | output/LoC | |------|-------------|----------------|-----------------|------------| | plan | 0 | — | — | — | | verify | 0 | — | — | — | | implement | 719 | 2245.0 | 0.0 | 17.2 | | audit_impl | 0 | — | — | — | | prepare_pr | 0 | — | — | — | | compose_pr | 0 | — | — | — | | **Total** | **719** | 5520.8 | 241.9 | 83.3 | ## Model Usage Breakdown | Model | steps | uncached | output | cache_read | cache_write | time | |-------|-------|----------|--------|------------|-------------|------| | opus[1m] | 1 | 1.3k | 22.1k | 1.4M | 86.5k | 14m 4s | | sonnet | 2 | 130 | 21.6k | 607.6k | 87.4k | 11m 54s | | MiniMax-M3 | 3 | 152.4k | 16.2k | 2.0M | 0 | 7m 33s |
1 parent b68db97 commit f7e5980

24 files changed

Lines changed: 719 additions & 0 deletions

‎.autoskillit/test-filter-manifest.yaml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,9 @@ tests/recipe/fixtures/*:
153153
tests/fixtures/codex/*.ndjson:
154154
- execution/
155155

156+
tests/execution/backends/fixtures/codex_ndjson/*.json:
157+
- execution/
158+
156159
# Local .autoskillit/ config and state files
157160
.autoskillit/test-source-map.json:
158161
- infra/

‎tests/execution/backends/fixtures/__init__.py‎

Whitespace-only changes.

‎tests/execution/backends/fixtures/codex_ndjson/__init__.py‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
"""Codex NDJSON deterministic conformance fixture schemas.
2+
3+
Versions the sealed JSON Schema fixtures for B3a conformance testing.
4+
FIXTURE_SCHEMA_VERSION tracks the JSON Schema fixture format and is distinct
5+
from CODEX_SCHEMA_VERSION (in tests/fixtures/codex/ and core/types/) which
6+
versions the NDJSON session replay fixture format.
7+
"""
8+
9+
from __future__ import annotations
10+
11+
from pathlib import Path
12+
13+
FIXTURE_SCHEMA_VERSION: int = 1
14+
15+
16+
def fixture_schema_path(name: str) -> Path:
17+
"""Return the absolute path to a fixture file in this directory."""
18+
return Path(__file__).parent / name
19+
20+
21+
__all__ = [
22+
"FIXTURE_SCHEMA_VERSION",
23+
"fixture_schema_path",
24+
]

‎tests/execution/backends/fixtures/codex_ndjson/config_toml_schema_template.json‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
{
2+
"_codex_mcp_required_keys": [
3+
"command",
4+
"env_vars",
5+
"startup_timeout_sec",
6+
"tool_timeout_sec"
7+
],
8+
"mcp_server_entry_required_keys": {
9+
"command": {
10+
"expected_type": "str"
11+
},
12+
"env_vars": {
13+
"expected_type": "list"
14+
},
15+
"startup_timeout_sec": {
16+
"expected_type": "float"
17+
},
18+
"tool_timeout_sec": {
19+
"expected_type": "float"
20+
}
21+
},
22+
"top_level_keys": {
23+
"model_auto_compact_token_limit": {
24+
"constraint": "minimum",
25+
"expected_type": "int",
26+
"floor_value": 999999999
27+
},
28+
"tool_output_token_limit": {
29+
"constraint": "minimum",
30+
"expected_type": "int",
31+
"floor_value": 50000
32+
}
33+
}
34+
}

‎tests/execution/backends/fixtures/codex_ndjson/event_error.json‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
"additionalProperties": false,
4+
"properties": {
5+
"type": {
6+
"const": "error",
7+
"type": "string"
8+
}
9+
},
10+
"required": [
11+
"type"
12+
],
13+
"type": "object"
14+
}

‎tests/execution/backends/fixtures/codex_ndjson/event_item_completed.json‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
"additionalProperties": false,
4+
"properties": {
5+
"item": {
6+
"properties": {
7+
"type": {
8+
"type": "string"
9+
}
10+
},
11+
"required": [
12+
"type"
13+
],
14+
"type": "object"
15+
},
16+
"type": {
17+
"const": "item.completed",
18+
"type": "string"
19+
}
20+
},
21+
"required": [
22+
"type",
23+
"item"
24+
],
25+
"type": "object"
26+
}

‎tests/execution/backends/fixtures/codex_ndjson/event_item_started.json‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
"additionalProperties": false,
4+
"properties": {
5+
"type": {
6+
"const": "item.started",
7+
"type": "string"
8+
}
9+
},
10+
"required": [
11+
"type"
12+
],
13+
"type": "object"
14+
}

‎tests/execution/backends/fixtures/codex_ndjson/event_item_updated.json‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
"additionalProperties": false,
4+
"properties": {
5+
"type": {
6+
"const": "item.updated",
7+
"type": "string"
8+
}
9+
},
10+
"required": [
11+
"type"
12+
],
13+
"type": "object"
14+
}

‎tests/execution/backends/fixtures/codex_ndjson/event_session_meta.json‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
"additionalProperties": false,
4+
"properties": {
5+
"payload": {
6+
"additionalProperties": false,
7+
"properties": {
8+
"id": {
9+
"type": "string"
10+
}
11+
},
12+
"required": [
13+
"id"
14+
],
15+
"type": "object"
16+
},
17+
"type": {
18+
"const": "session_meta",
19+
"type": "string"
20+
}
21+
},
22+
"required": [
23+
"type",
24+
"payload"
25+
],
26+
"type": "object"
27+
}

‎tests/execution/backends/fixtures/codex_ndjson/event_thread_started.json‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
"additionalProperties": false,
4+
"properties": {
5+
"thread_id": {
6+
"type": "string"
7+
},
8+
"type": {
9+
"const": "thread.started",
10+
"type": "string"
11+
}
12+
},
13+
"required": [
14+
"type",
15+
"thread_id"
16+
],
17+
"type": "object"
18+
}

0 commit comments

Comments
 (0)