Skip to content

Document stderr regex branch unreachability and add turn.failed fixture-driven tests#4122

Merged
Trecek merged 2 commits into
developfrom
t5-p6-a14-wp1-document-that-the-stderr-regex-branch-is-unrea/4050
Jun 27, 2026
Merged

Document stderr regex branch unreachability and add turn.failed fixture-driven tests#4122
Trecek merged 2 commits into
developfrom
t5-p6-a14-wp1-document-that-the-stderr-regex-branch-is-unrea/4050

Conversation

@Trecek

@Trecek Trecek commented Jun 27, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add a clarifying comment in _exit_classification.py documenting that the stderr regex branch (_CODEX_CONTEXT_EXHAUSTION_PATTERN.search(result.stderr)) is unreachable for Codex because turn.failed events arrive via stdout NDJSON, not stderr. Extend the _turn_failed_ndjson test helper with an optional error_code kwarg and add five new test methods exercising all error.code / error.message variants of turn.failed classification plus a stderr boundary test.

Implementation Plan

Plan file: /home/talon/projects/autoskillit-runs/impl-20260627-014523-646887/.autoskillit/temp/make-plan/t5_p6_a14_wp1_stderr_regex_unreachable_plan_2026-06-27_015000.md

Closes #4050

🤖 Generated with Claude Code via AutoSkillit

Token Usage Summary

Step Model count uncached output cache_read peak_ctx turns cache_write time
plan* opus[1m] 1 3.5k 7.6k 672.2k 68.1k 26 54.1k 4m 17s
verify* sonnet 1 70 7.1k 390.3k 70.7k 23 52.2k 4m 9s
implement* MiniMax-M3 1 62.7k 6.2k 1.3M 0 68 0 2m 13s
audit_impl* sonnet 1 1.6k 6.3k 159.7k 38.4k 14 27.7k 2m 56s
prepare_pr* MiniMax-M3 1 40.1k 1.7k 110.1k 0 11 0 38s
compose_pr* MiniMax-M3 1 36.5k 1.5k 249.1k 0 14 0 55s
review_pr* sonnet 1 124 16.4k 731.1k 62.2k 37 43.9k 4m 20s
resolve_review* sonnet 1 239 19.0k 2.0M 90.1k 72 72.2k 7m 34s
Total 144.9k 65.7k 5.7M 90.1k 250.2k 27m 5s

* Step used a non-Anthropic provider; caching behavior may differ.

Token Efficiency

Step LoC Changed cache_read/LoC cache_write/LoC output/LoC
plan 0
verify 0
implement 49 27125.6 0.0 126.1
audit_impl 0
prepare_pr 0
compose_pr 0
review_pr 0
resolve_review 8 255755.6 9028.8 2375.4
Total 57 99783.1 4388.6 1152.3

Model Usage Breakdown

Model steps uncached output cache_read cache_write time
opus[1m] 1 3.5k 7.6k 672.2k 54.1k 4m 17s
sonnet 4 2.1k 48.8k 3.3M 196.1k 19m 1s
MiniMax-M3 3 139.4k 9.3k 1.7M 0 3m 46s

Trecek and others added 2 commits June 27, 2026 01:56
…ssification

The stderr regex branch (_CODEX_CONTEXT_EXHAUSTION_PATTERN.search) is
unreachable for Codex because turn.failed events arrive via stdout NDJSON
and are handled by _adapt_agent_result. This branch is a non-Codex
fallback path only.
- Replace empty error_message with realistic non-matching message in
  test_turn_failed_error_code_only_classify_returns_context_exhausted
- Change `!= CONTEXT_EXHAUSTED` to `== API_ERROR` in
  test_turn_failed_non_exhaustion_error_code_not_context_exhausted
- Change `!= CONTEXT_EXHAUSTED` to `== COMPLETED` in
  test_stderr_non_exhaustion_not_context_exhausted

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@Trecek Trecek added this pull request to the merge queue Jun 27, 2026
Merged via the queue into develop with commit 82ac931 Jun 27, 2026
3 checks passed
@Trecek Trecek deleted the t5-p6-a14-wp1-document-that-the-stderr-regex-branch-is-unrea/4050 branch June 27, 2026 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant