Skip to content

fix(session): add idle timeout to LLM streaming response#29348

Draft
WonderJL wants to merge 1 commit into
anomalyco:devfrom
WonderJL:feat/llm-stream-idle-timeout
Draft

fix(session): add idle timeout to LLM streaming response#29348
WonderJL wants to merge 1 commit into
anomalyco:devfrom
WonderJL:feat/llm-stream-idle-timeout

Conversation

@WonderJL
Copy link
Copy Markdown

@WonderJL WonderJL commented May 26, 2026

Summary

  • Adds an idle-timeout guard on the LLM event stream so a silently-stalled provider/transport no longer hangs the session loop indefinitely.
  • Applies to both the native runtime and ai-sdk runtime paths.
  • Configurable via experimental.llm_stream_idle_timeout (ms); defaults to 120000 (2 min). Set to 0 to disable.

Why

The session loop consumes the LLM stream via Stream.fromAsyncIterable(result.fullStream, …) with no read/idle deadline on either runtime path. If the upstream stops sending bytes without delivering [DONE], an error frame, or a TCP close — e.g. an intermittent provider hiccup or a half-open SSE connection — the loop blocks on the next chunk forever. Symptom in practice: the session never advances step, no finishReason is emitted, and the UI shows it as "not processing and not stopping." Because there is no idle deadline anywhere in the stream pipeline, any provider-side stall is unrecoverable without killing the process.

What changes

  • packages/opencode/src/session/llm.ts
    • New exported LLMStreamIdleTimeout error (_tag: "LLMStreamIdleTimeout") and withIdleTimeout(stream, idleMs) helper built on Stream.timeoutOrElse.
    • Wraps the unified LLMEvent stream (both native + ai-sdk paths) after the runtime selection so the failure surface is symmetric.
    • Reads the configured idle window from cfg.experimental?.llm_stream_idle_timeout with the default constant DEFAULT_LLM_STREAM_IDLE_TIMEOUT_MS = 120_000. 0 disables.
  • packages/opencode/src/config/config.ts
    • Adds experimental.llm_stream_idle_timeout: PositiveInt (optional), documented inline.
  • packages/opencode/test/session/llm-idle-timeout.test.ts (new)
    • Stalled stream → fails with LLMStreamIdleTimeout.
    • Partial stream (emits some events, then stalls) → emits the early events, then fails.
    • Healthy stream that completes before the window → passes through unchanged.

Test plan

  • bun run typecheck (packages/opencode) — clean.
  • bun test test/session/llm-idle-timeout.test.ts — 3/3 pass.
  • bun test test/session/{llm,llm-native,retry,session}.test.ts — 78/78 pass; no regressions in the existing LLM/session suites.
  • Manual smoke against openai/gpt-5.4 with a non-trivial prompt; confirm normal streaming + finish path is unaffected (default 120s window is well above expected gaps).

Notes

  • Stream.timeoutOrElse from Effect 4 is per-element/idle, not total — matches the symptom we want to catch.
  • The wrap happens outside the AbortController lifetime, so on timeout the controller's acquireRelease finalizer still aborts the underlying request/transport.
  • No behavior change when the default is in effect for healthy sessions; the only observable difference is that previously-hung sessions now fail fast with a tagged error the loop/retry layer can react to.

If the model provider's SSE stream stalls without sending [DONE] or
closing the socket (proxy drop, transport half-open, upstream hang),
the session loop blocks on the next chunk forever and the session
appears stuck with no error.

Wrap the LLM event stream in Stream.timeoutOrElse so that an idle
window with no events fails the stream with a tagged
LLMStreamIdleTimeout error, allowing the session loop to surface the
failure to the user. Applies to both the native runtime and ai-sdk
runtime paths.

Configurable via experimental.llm_stream_idle_timeout (ms); defaults
to 120000ms. Set to 0 to disable.
@github-actions
Copy link
Copy Markdown
Contributor

ghost commented May 26, 2026

Thanks for your contribution!

This PR doesn't have a linked issue. All PRs must reference an existing issue.

Please:

  1. Open an issue describing the bug/feature (if one doesn't exist)
  2. Add Fixes #<number> or Closes #<number> to this PR description

See CONTRIBUTING.md for details.

@github-actions
Copy link
Copy Markdown
Contributor

ghost commented May 26, 2026

This PR doesn't fully meet our contributing guidelines and PR template.

What needs to be fixed:

  • PR description is missing required template sections. Please use the PR template.

Please edit this PR description to address the above within 2 hours, or it will be automatically closed.

If you believe this was flagged incorrectly, please let a maintainer know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs:compliance This means the issue will auto-close after 2 hours. needs:issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant