Skip to content

fix(llmcore): suppress intermediate error yield during retry and handle ChunkedEncodingError#648

Open
Kailigithub wants to merge 1 commit into
lsdefine:mainfrom
Kailigithub:fix/issue-571-llm-retry-ssl-proxy
Open

fix(llmcore): suppress intermediate error yield during retry and handle ChunkedEncodingError#648
Kailigithub wants to merge 1 commit into
lsdefine:mainfrom
Kailigithub:fix/issue-571-llm-retry-ssl-proxy

Conversation

@Kailigithub

Copy link
Copy Markdown
Contributor

What problem this solves

When the LLM stream hits a transient network error (e.g. wifi blip walking to a meeting room, VPN reconnect, or proxy hiccup), the retry path yielded the raw !!!Error: ... string back to the UI on every attempt. The user then saw a wall of stacked error messages like:

!!!Error: ProxyError!!!Error: ConnectionError!!!Error: SSLError!!!Error: SSLError
!!!Error: SSLError!!!Error: SSLError!!!Error: SSLError!!!Error: SSLError
LLM Running (Turn 19) ...
!!!Error: SSLError!!!Error: SSLError!!!Error: SSLError!!!Error: SSLError

even when the eventual retry succeeded and the assistant was still producing a response. The user reported this as issue #571 with the expected behavior: exponential-backoff retry rather than a visible fail.

There was also a secondary gap: requests.exceptions.ChunkedEncodingError was not in the retryable-exception tuple, so a truncated chunked response would fall through to the bare except Exception branch and abort the stream with no retry, even though it is a transient transport-level error.

What changed

llmcore.py:_stream_with_retry:

  • Added requests.exceptions.ChunkedEncodingError to the connection-error exception tuple. SSLError and ProxyError already inherit from requests.ConnectionError and were already caught - only ChunkedEncodingError was actually missing.
  • Removed the yield err from the intermediate-retry branch, matching the existing HTTP-retry path (which time.sleeps without yielding). The retry is still visible via the existing [LLM Retry] ... stdout log; the user-facing stream now stays silent during retries and only surfaces a final error if all attempts are exhausted.

Evidence

Behavior before vs after, mocked retry sequence (max_retries=3):

  • Before: 2 ConnectionErrors -> 200 with content -> yielded 2 intermediate !!!Error: ... chunks + content.
  • After: 2 ConnectionErrors -> 200 with content -> yielded 0 intermediate error chunks + content.
  • Before: ChunkedEncodingError -> no retry, immediate abort.
  • After: ChunkedEncodingError -> caught, retried with backoff, only final error yielded if exhausted.

Local validation:

$ python3 -m py_compile llmcore.py
[no output]   # exit 0

$ ruff check llmcore.py --select F821,E9
All checks passed!

$ python3 -c "import ast; ast.parse(open('llmcore.py').read())"
[no output]   # exit 0

Behavioral tests (unittest.mock) verified that on a 2-error-then-success sequence the user-visible stream contains zero intermediate error chunks, while on exhaustion exactly one final error chunk is produced.

Risk

Minimal: 2 lines changed in one function. The removed yield err only affected the connection-error retry path; the HTTP-retry path was already yield-free, so this is just bringing the two paths into consistency. All three call sites of _stream_with_retry (_openai_stream, _claude_stream, BaseSession.ask) consume the generator without depending on intermediate yield count.

Closes #571

…le ChunkedEncodingError

When the LLM stream encounters a transient network error (ConnectionError,
Timeout, SSLError, ProxyError), the retry path yielded the error string
back to the caller on every attempt. This caused the UI to display a wall
of '!!!Error: ProxyError!!!Error: ConnectionError!!!Error: SSLError'
strings during a network blip, even when the retry ultimately succeeded.

Match the existing HTTP retry path (which sleeps without yielding) by
removing the intermediate yield so the caller only sees the final error
if retries are exhausted. Retry attempts remain visible via the existing
'[LLM Retry] ...' stdout log.

Also extend the exception tuple to include ChunkedEncodingError. Without
it, a truncated chunked response falls through to the bare 'except
Exception' branch and aborts the stream with no retry, even though the
transport-level error is transient and recoverable.

Closes lsdefine#571
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

网络波动的“用户友好提示”

1 participant