fix: convert requestOptions.timeout to milliseconds for OpenAI SDK#12451
Open
ScrewTSW wants to merge 1 commit into
Open
fix: convert requestOptions.timeout to milliseconds for OpenAI SDK#12451ScrewTSW wants to merge 1 commit into
ScrewTSW wants to merge 1 commit into
Conversation
…OpenAI SDK The OpenAI JS SDK expects timeout in milliseconds, but users configure requestOptions.timeout in seconds. Passing the value directly results in sub-second timeouts (e.g. timeout: 300 becomes 300ms instead of 5 minutes), causing "Connection error" for any model that takes more than a fraction of a second to respond. Fixes continuedev#12450
dosubot
Bot
added
the
size:XS
Contributor
|
All contributors have signed the CLA ✍️ ✅ |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Author
|
I have read the CLA Document and I hereby sign the CLA |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
requestOptions.timeoutis configured in seconds by users, but the OpenAI JS SDK constructor expects milliseconds. The value was passed through without conversion, sotimeout: 300(5 minutes) became a 300ms timeout.One-line fix: multiply by 1000 before passing to the SDK.
Fixes #12450
Likely root cause for many reports in #11818
Checklist
Tests
Tested with local llama.cpp models behind an orchestrator (
timeout: 300in config). Before: immediate "Connection error". After: requests complete normally with model load + prompt eval taking 30-60s.Summary by cubic
Convert
requestOptions.timeout(seconds) to milliseconds before passing to theopenaiclient. Fixes unintended sub-second timeouts and restores expected behavior for long-running requests.requestOptions.timeoutby 1000 in theOpenAIApiconstructor sotimeout: 300becomes 300000 ms, not 300 ms.Written for commit 4a456a4. Summary will update on new commits. Review in cubic