Feat/retry mechanism#1504
Open
xiaoyouqiang wants to merge 6 commits into
Open
Conversation
Add EnableThinking and ThinkingBudget fields to ChatRequest for Qwen deep thinking mode via OpenAI-compatible API. This allows toggling Qwen's reasoning mode and limiting thinking token budget.
Add opt-in automatic retry for transient HTTP failures across all LLM
providers. The mechanism handles:
- HTTP 429 (rate limit), 500, 502, 503, 504 (server errors)
- Network errors (connection refused, DNS failure, TLS timeout)
- Provider body-level errors (e.g., ERNIE HTTP 200 + error_code:18)
- Retry-After header: waits server-specified duration instead of backoff
Two internal retry strategies, unified under the same RetryConfig API:
- RetryOnError (application-layer): OpenAI, Anthropic, ERNIE — checks
network errors, HTTP status codes, AND body-level errors via MapError
- RetryTransport (transport-layer): Ollama, Cohere, Cloudflare,
HuggingFace, Llamafile, Maritaca — checks HTTP status codes and
network errors at the RoundTripper level
Usage:
llm, _ := openai.New(
openai.WithToken("sk-xxx"),
openai.WithRetryConfig(httputil.DefaultRetryConfig()),
)
Also includes:
- ERNIE error_code:17 fix (daily quota was wrongly classified as rate limit)
- ERNIE error_code:19 coverage (total request limit)
- 46 unit/integration tests
- Documentation in doc.go and httputil/doc.go
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- errcheck: discard WithRetryAfter return value with _ = - gosec G404: add nolint comments for jitter rand.Float64 - gofmt: format retry.go, retry_test.go, anthropicllm_option.go Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Checklist
memory: add interfaces for X, Yorutil: add whizzbang helpers).Fixes #123).golangci-lintchecks.