fix: stale RUNNING dispatch recovery with session-type-agnostic liveness#4136
Merged
Trecek merged 4 commits intoJun 28, 2026
Merged
Conversation
Extract resolve_stale_running helper to centralize the RUNNING→check liveness pattern. reset_dispatch now verifies process liveness before blocking RUNNING dispatches, open_kitchen runs reap_stale_dispatches_async for interactive sessions, and structural guards prevent future regressions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…mentation - Extract find_locked_dispatch helper to fleet/_reset.py domain layer to satisfy REQ-CNST-008 (no comprehensions in tool handlers) - Hoist CampaignStateMutator import to module level to satisfy IL-3 cross-layer import hoisting and cross-package submodule rules - Fix test_reset_dispatch_running_dead_process_succeeds to mock resolve_stale_running returning False (dead) and assert success - Fix test_reset_dispatch_running_alive_process_still_blocked to mock resolve_stale_running returning True (alive) and assert block - Update _LEGACY_JSON_WRITES line numbers for tools_kitchen.py shifts - Bump tools_kitchen.py line limit exemption to 1290 for reaper call Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ess fields in reset_dispatch tests Exercise the full liveness detection path (resolve_stale_running → is_dispatch_session_alive → is_session_alive) in test_reset_dispatch_running_dead_process_succeeds and test_reset_dispatch_running_alive_process_still_blocked by writing real PID/boot-id/starttime fields onto the dispatch record via CampaignStateMutator instead of mocking resolve_stale_running. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
resolve_stale_running only calls mark_dirty() on the dead-dispatch path. The reaper unconditionally mutates dispatch fields afterward (ended_at, reaper_reason, reaper_dispatch_id, tombstone, reaper event), so mark_dirty() must be called unconditionally to ensure persistence. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Trecek
force-pushed
the
interactive-sessions-cannot-recover-stale-running-dispatch-s/4133
branch
from
096c939 to
ea37283
Compare
Trecek
deleted the
interactive-sessions-cannot-recover-stale-running-dispatch-s/4133
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Interactive sessions have zero path to recover stale RUNNING dispatch state. The bug is a convergence of two gaps: (1)
reset_dispatchblocks on RUNNING status without checking process liveness, and (2) neitheropen_kitchennor any interactive boot path runs the dispatch reaper. The architectural weakness is that liveness verification before status-based blocking is enforced by convention, not by structure — three callers do it correctly (resume_campaign_from_state,_try_claim_with_liveness,reap_stale_dispatches) whilereset_dispatchdoes not.The immunity plan extracts a shared
resolve_stale_runninghelper that centralizes the "RUNNING → check liveness → classify if dead" pattern, adds the reaper toopen_kitchen, extends the existing AST boot-step symmetry guard to coveropen_kitchen, and adds integration tests that would have caught this bug immediately.The remediation plan replaces mocked
resolve_stale_runningin two tests (test_reset_dispatch_running_dead_process_succeedsandtest_reset_dispatch_running_alive_process_still_blocked) with real PID/boot-id fields on the dispatch record, exercising the full liveness detection path:reset_dispatch→find_locked_dispatch→resolve_stale_running→is_dispatch_session_alive→is_session_alive.Closes #4133
Implementation Plan
Plan files:
/home/talon/projects/autoskillit-runs/remediation-20260627-224639-242652/.autoskillit/temp/rectify/rectify_stale_running_dispatch_recovery_2026-06-27_224639.md/home/talon/projects/autoskillit-runs/remediation-20260627-224639-242652/.autoskillit/temp/make-plan/remediation_stale_running_dispatch_recovery_plan_2026-06-28_001600.md🤖 Generated with Claude Code via AutoSkillit
Token Usage Summary
* Step used a non-Anthropic provider; caching behavior may differ.
Token Efficiency
Model Usage Breakdown