Skip to content

Implementation Plan: Deferred-Recall Fail-Closed Validity Guard Tests#4074

Merged
Trecek merged 2 commits into
developfrom
t5-p1-a5-wp1-provide-explicit-regression-coverage-for-the-fa/4002
Jun 11, 2026
Merged

Implementation Plan: Deferred-Recall Fail-Closed Validity Guard Tests#4074
Trecek merged 2 commits into
developfrom
t5-p1-a5-wp1-provide-explicit-regression-coverage-for-the-fa/4002

Conversation

@Trecek

@Trecek Trecek commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add 3 regression test functions to tests/server/test_open_kitchen_deferred_recall.py covering the fail-closed validity guard on the _is_deferred_recall=True branch of open_kitchen. The guard (at tools_kitchen.py:631) rejects recipes when valid is False or content is empty, returning a structured error envelope via _recipe_validation_error_response. No production code changes.

Closes #4002

Implementation Plan

Plan file: /home/talon/projects/autoskillit-runs/impl-20260611-133942-160970/.autoskillit/temp/make-plan/t5_p1_a5_wp1_deferred_recall_fail_closed_tests_plan_2026-06-11_134500.md

🤖 Generated with Claude Code via AutoSkillit

Token Usage Summary

Step Model count uncached output cache_read peak_ctx turns cache_write time
plan* opus[1m] 1 2.0k 6.4k 594.9k 68.7k 24 50.2k 6m 29s
verify* sonnet 1 92 9.3k 469.4k 53.4k 24 32.5k 4m 50s
implement* MiniMax-M3 1 725.2k 5.9k 0 0 34 0 3m 16s
audit_impl* sonnet 1 44 7.0k 164.9k 40.3k 15 26.5k 2m 48s
prepare_pr* MiniMax-M3 1 310.6k 2.5k 0 0 18 0 1m 15s
compose_pr* MiniMax-M3 1 175.5k 1.1k 0 0 12 0 40s
review_pr* sonnet 1 164 14.0k 940.7k 60.0k 43 39.5k 4m 17s
resolve_review* sonnet 1 157 5.5k 909.2k 61.9k 44 87.0k 2m 56s
Total 1.2M 51.6k 3.1M 68.7k 235.8k 26m 35s

* Step used a non-Anthropic provider; caching behavior may differ.

Token Efficiency

Step LoC Changed cache_read/LoC cache_write/LoC output/LoC
plan 0
verify 0
implement 89 0.0 0.0 65.8
audit_impl 0
prepare_pr 0
compose_pr 0
review_pr 0
resolve_review 1 909176.0 87046.0 5510.0
Total 90 34211.8 2619.8 573.1

Model Usage Breakdown

Model steps uncached output cache_read cache_write time
opus[1m] 1 2.0k 6.4k 594.9k 50.2k 6m 29s
sonnet 4 457 35.8k 2.5M 185.5k 14m 53s
MiniMax-M3 3 1.2M 9.4k 0 0 5m 12s

Trecek added 2 commits June 11, 2026 13:54
…ue branch

Cover the not-valid-or-not-content guard at tools_kitchen.py:631 that returns
_recipe_validation_error_response when load_and_validate produces a
recipe whose 'valid' key is False (or absent) or whose 'content' is empty.
Three regression tests:

- valid=False with empty content
- valid=False with non-empty content (guard still fires)
- valid key absent (treated as False via result.get('valid', False))

Also updates the test file module docstring and tests/server/AGENTS.md
table entry to reflect the expanded scope.
…guard test

Adds the missing assertion on parsed["errors"] in
test_deferred_recall_fails_closed_when_valid_missing, making it
consistent with the sibling tests that both assert their specific
error values.
@Trecek Trecek added this pull request to the merge queue Jun 11, 2026
Merged via the queue into develop with commit 05dff6a Jun 11, 2026
3 checks passed
@Trecek Trecek deleted the t5-p1-a5-wp1-provide-explicit-regression-coverage-for-the-fa/4002 branch June 11, 2026 21:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant