fix(instance): release resources at worktree boundaries#1709
Merged
Conversation
wqymi
added a commit
that referenced
this pull request
Jul 14, 2026
- Add RECLAIM_WORKTREE_TIMEOUT_MS (10s) to worktree.remove during reclaim - Add RECLAIM_ACTOR_TIMEOUT_MS (5s) to actor.cancel during reclaim - Add FIBER_INTERRUPT_TIMEOUT_MS (5s) to Fiber.interrupt in cancelEntry These timeouts prevent cancel from hanging indefinitely when: 1. git worktree remove --force blocks on processes using the worktree 2. A child actor is stuck in an uninterruptible operation 3. Fiber.interrupt stalls on a hung LLM fetch The specific fix addresses the 30s timeout in: 'WorkflowRuntime worktree isolation > cancel removes worktrees of in-flight isolated agents' Root cause: PR #1709 added Instance.disposeDirectory calls in worktree.remove which increased the time for cancel cleanup. Combined with git worktree remove --force potentially blocking, the cancel operation could exceed the test timeout.
wqymi
added a commit
that referenced
this pull request
Jul 14, 2026
- Add RECLAIM_WORKTREE_TIMEOUT_MS (10s) to worktree.remove during reclaim - Add RECLAIM_ACTOR_TIMEOUT_MS (5s) to actor.cancel during reclaim - Add FIBER_INTERRUPT_TIMEOUT_MS (5s) to Fiber.interrupt in cancelEntry These timeouts prevent cancel from hanging indefinitely when: 1. git worktree remove --force blocks on processes using the worktree 2. A child actor is stuck in an uninterruptible operation 3. Fiber.interrupt stalls on a hung LLM fetch The specific fix addresses the 30s timeout in: 'WorkflowRuntime worktree isolation > cancel removes worktrees of in-flight isolated agents' Root cause: PR #1709 added Instance.disposeDirectory calls in worktree.remove which increased the time for cancel cleanup. Combined with git worktree remove --force potentially blocking, the cancel operation could exceed the test timeout.
4d8c592 to
5c1e1f8
Compare
wqymi
added a commit
that referenced
this pull request
Jul 14, 2026
- Add RECLAIM_WORKTREE_TIMEOUT_MS (10s) to worktree.remove during reclaim - Add RECLAIM_ACTOR_TIMEOUT_MS (5s) to actor.cancel during reclaim - Add FIBER_INTERRUPT_TIMEOUT_MS (5s) to Fiber.interrupt in cancelEntry These timeouts prevent cancel from hanging indefinitely when: 1. git worktree remove --force blocks on processes using the worktree 2. A child actor is stuck in an uninterruptible operation 3. Fiber.interrupt stalls on a hung LLM fetch The specific fix addresses the 30s timeout in: 'WorkflowRuntime worktree isolation > cancel removes worktrees of in-flight isolated agents' Root cause: PR #1709 added Instance.disposeDirectory calls in worktree.remove which increased the time for cancel cleanup. Combined with git worktree remove --force potentially blocking, the cancel operation could exceed the test timeout.
- Add RECLAIM_WORKTREE_TIMEOUT_MS (10s) to worktree.remove during reclaim - Add RECLAIM_ACTOR_TIMEOUT_MS (5s) to actor.cancel during reclaim - Add FIBER_INTERRUPT_TIMEOUT_MS (5s) to Fiber.interrupt in cancelEntry These timeouts prevent cancel from hanging indefinitely when: 1. git worktree remove --force blocks on processes using the worktree 2. A child actor is stuck in an uninterruptible operation 3. Fiber.interrupt stalls on a hung LLM fetch The specific fix addresses the 30s timeout in: 'WorkflowRuntime worktree isolation > cancel removes worktrees of in-flight isolated agents' Root cause: PR #1709 added Instance.disposeDirectory calls in worktree.remove which increased the time for cancel cleanup. Combined with git worktree remove --force potentially blocking, the cancel operation could exceed the test timeout.
…ngUntil Two 30s-timeout tests used llm.hang (Stream.never + real TCP) and relied on Fiber.interrupt to unwind through the HTTP client stack within 30s. Under CI load the TCP cleanup alone can exceed 10s, pushing the total past deadline. Design-level fix: - llm-server: add hangUntil(Deferred) that blocks on Effect-level Deferred.await instead of Stream.never. Fiber.interrupt unwinds the Deferred at the Effect layer without touching TCP — fast and deterministic. - runtime-worktree cancel test: replace llm.hang with hangUntil; after cancel the Deferred is resolved so the agent unwinds cleanly. - spawn-notification woken-peer test: increase polling budget (600×50ms = 30s) to accommodate unbounded woken-turn LLM latency under CI load.
5c1e1f8 to
1e10597
Compare
… deadlock In cancelEntry, Fiber.interrupt was called AFTER reclaim (which does worktree.remove → disposeDirectory → ScopedCache.invalidate). The scope-close finalizer waits for the agent fiber to finish, but the fiber was still running in Stream.never → 30s deadlock. Reorder so Fiber.interrupt runs first, letting the scope-close finalize without waiting on a dead fiber. PR #1709 — fixes the 30s timeout in 'cancel removes worktrees' test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Isolated worktrees can finish or be removed while their directory-scoped Instance remains cached. The stale Instance keeps subprocesses and other scoped resources alive, including after the directory itself has disappeared.
Fix
Add bounded, directory-addressable Instance teardown and invoke it when an isolated agent settles and before worktree removal. The captured cache entry is removed before cleanup, replacement instances are protected from late teardown, missing paths are still reclaimed, and session CWD overrides are cleared only for the owning directory.
Verification
Post-Deploy Monitoring & Validation
instance dispose did not completeand unexpected increases in long-lived child processes grouped by removed worktree directories.🤖 Generated with GPT-5 (context window not disclosed, extended reasoning) via Codex