Skip to content

Commit b747d56

Browse files
daewoongohOh Daewoongnavedmerchant
authored
fix: apply apiRequestTimeout consistently across providers (#567)
* fix(api): apply apiRequestTimeout consistently across OpenAI/Anthropic SDK providers Wire getApiRequestTimeout() into every provider that instantiates an OpenAI or Anthropic SDK client so the user-configured apiRequestTimeout setting applies uniformly. Previously only openai/lm-studio and providers extending BaseOpenAiCompatibleProvider honored it. OpenAI SDK: openai-native, openai-codex, openrouter, router-provider (lite-llm, zoo-gateway, opencode-go, vercel-ai-gateway), requesty, unbound, xai, qwen-code. Anthropic SDK: anthropic, minimax, anthropic-vertex. * docs(i18n): list providers where apiRequestTimeout has no effect Append the list of unsupported providers (Amazon Bedrock, Google Gemini, GCP Vertex AI, Mistral, Ollama, VS Code LM API, Poe) to the apiRequestTimeout setting description across all package.nls locales so users can see at a glance which providers ignore the value. * test(api): align provider tests with apiRequestTimeout wiring Assert SDK clients receive a timeout option in providers updated by 4c82de4 (anthropic-vertex, openrouter, requesty, vercel-ai-gateway, zoo-gateway), and stub vscode.workspace.getConfiguration in tests that mock vscode as an empty object so getApiRequestTimeout() can run during provider construction. * docs(i18n): drop provider examples from apiRequestTimeout recommendation The previous description recommended raising the timeout for "local providers like LM Studio and Ollama" while also listing Ollama among the providers where the setting has no effect. Drop the provider examples from the recommendation so the two parts no longer contradict; the unsupported list still spells out where the setting has no effect. * chore: fix typo in Turkish apiRequestTimeout description * fix(api): round timeout ms and restrict apiRequestTimeout to 1-3600s - Math.round() prevents float ms from Anthropic SDK validatePositiveInteger throw - minimum changed from 0 to 1; removed unreachable <= 0 branch in getApiRequestTimeout - return type narrowed from number | undefined to number * docs(i18n): remove '0 = no timeout', scope GCP Vertex AI, add Moonshot to unsupported list - removed '0 = no timeout' from all locales; minimum is now 1 - scoped GCP Vertex AI to '(Gemini models)' since Claude models on Vertex are supported - added Moonshot to unsupported providers (uses Vercel AI SDK, no client-level timeout) * docs(i18n): tighten apiRequestTimeout description across all locales Restructured into 3 concise sentences: timeout info, local provider recommendation, and alphabetized unsupported providers list. Fixed Korean phrasing and unified Japanese fullwidth punctuation. * refactor(api): extract timeout constants and validate range bounds Pull the timeout bounds (1-3600s) and default (600s) into named constants and a small isValidTimeout type-guard, so out-of-range values fall back to the default just like NaN/non-number inputs. Tests updated to cover the new boundary and out-of-range fallback cases. * test(api): cover timeout passthrough for Vertex auth variants Add cases verifying timeout is forwarded to AnthropicVertex when constructing with vertexJsonCredentials or vertexKeyFile, and mock GoogleAuth so the auth-bound paths are exercised. * test(api): make GoogleAuth mock newable in Vertex tests Arrow functions cannot be invoked with `new`, so the mock threw TypeError when the handler did `new GoogleAuth(...)`. Switch to a regular function expression to match the AnthropicVertex mock. * Add changeset for API request timeout * refactor: standardize API request timeout handling across providers - Refactor timeout handling logic in multiple API providers - Improve consistency in timeout configuration and error handling - Reduce code duplication across provider implementations * refactor: use class field initialization for timeoutMs in BaseProvider - Replace constructor-based initialization with class field initialization - Removes unnecessary constructor when only initializing properties - Follows modern TypeScript/JavaScript conventions for cleaner code * fix(tests): add getConfiguration mock to vscode provider tests Add workspace.getConfiguration mock to vertex.spec.ts and vscode-lm.spec.ts to fix test failures * chore(i18n): update apiRequestTimeout description for Google Gemini clarification - Update translation files to clarify that Google Gemini is not supported either directly or through the Vertex AI platform * test: add timeout config mock to provider tests - Mock timeout-config module in anthropic-vertex, openai, openrouter, requesty, and zoo-gateway tests - Add MOCK_TIMEOUT_MS constant for consistent timeout assertions - Replace expect.any(Number) with explicit MOCK_TIMEOUT_MS value in timeout assertions --------- Co-authored-by: Oh Daewoong <dw.oh@samsung.com> Co-authored-by: Naved Merchant <naved.merchant@gmail.com>
1 parent 0306f2b commit b747d56

48 files changed

Lines changed: 252 additions & 66 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.changeset/api-request-timeout.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"zoo-code": patch
3+
---
4+
5+
Updated `apiRequestTimeout` validation. Values must be integers between 1 and 3600 seconds; invalid or out-of-range values, including `0`, now fall back to 600 seconds. This aligns with the SDK's default timeout value.

src/api/providers/__tests__/anthropic-vertex.spec.ts

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,26 @@
22

33
import { Anthropic } from "@anthropic-ai/sdk"
44
import { AnthropicVertex } from "@anthropic-ai/vertex-sdk"
5+
import { GoogleAuth } from "google-auth-library"
56

67
import { VERTEX_1M_CONTEXT_MODEL_IDS } from "@roo-code/types"
78

89
import { ApiStreamChunk } from "../../transform/stream"
910

1011
import { AnthropicVertexHandler } from "../anthropic-vertex"
1112

13+
vitest.mock("../utils/timeout-config", () => ({
14+
getApiRequestTimeout: vitest.fn().mockReturnValue(300_000),
15+
}))
16+
17+
const MOCK_TIMEOUT_MS = 300_000
18+
19+
vitest.mock("google-auth-library", () => ({
20+
GoogleAuth: vitest.fn().mockImplementation(function (opts) {
21+
return { __googleAuthOptions: opts }
22+
}),
23+
}))
24+
1225
vitest.mock("@anthropic-ai/vertex-sdk", () => ({
1326
AnthropicVertex: vitest.fn().mockImplementation(function () {
1427
return {
@@ -56,6 +69,11 @@ describe("VertexHandler", () => {
5669
let handler: AnthropicVertexHandler
5770

5871
describe("constructor", () => {
72+
beforeEach(() => {
73+
;(AnthropicVertex as any).mockClear()
74+
;(GoogleAuth as any).mockClear()
75+
})
76+
5977
it("should initialize with provided config for Claude", () => {
6078
handler = new AnthropicVertexHandler({
6179
apiModelId: "claude-3-5-sonnet-v2@20241022",
@@ -66,7 +84,58 @@ describe("VertexHandler", () => {
6684
expect(AnthropicVertex).toHaveBeenCalledWith({
6785
projectId: "test-project",
6886
region: "us-central1",
87+
timeout: MOCK_TIMEOUT_MS,
88+
})
89+
})
90+
91+
it("should pass timeout when initializing with vertexJsonCredentials", () => {
92+
const credentials = {
93+
type: "service_account",
94+
client_email: "test@test-project.iam.gserviceaccount.com",
95+
private_key: "-----BEGIN PRIVATE KEY-----\nfake\n-----END PRIVATE KEY-----\n",
96+
}
97+
98+
handler = new AnthropicVertexHandler({
99+
apiModelId: "claude-3-5-sonnet-v2@20241022",
100+
vertexProjectId: "test-project",
101+
vertexRegion: "us-central1",
102+
vertexJsonCredentials: JSON.stringify(credentials),
69103
})
104+
105+
expect(GoogleAuth).toHaveBeenCalledWith({
106+
scopes: ["https://www.googleapis.com/auth/cloud-platform"],
107+
credentials,
108+
})
109+
expect(AnthropicVertex).toHaveBeenCalledWith(
110+
expect.objectContaining({
111+
projectId: "test-project",
112+
region: "us-central1",
113+
googleAuth: expect.any(Object),
114+
timeout: MOCK_TIMEOUT_MS,
115+
}),
116+
)
117+
})
118+
119+
it("should pass timeout when initializing with vertexKeyFile", () => {
120+
handler = new AnthropicVertexHandler({
121+
apiModelId: "claude-3-5-sonnet-v2@20241022",
122+
vertexProjectId: "test-project",
123+
vertexRegion: "us-central1",
124+
vertexKeyFile: "/tmp/sa-key.json",
125+
})
126+
127+
expect(GoogleAuth).toHaveBeenCalledWith({
128+
scopes: ["https://www.googleapis.com/auth/cloud-platform"],
129+
keyFile: "/tmp/sa-key.json",
130+
})
131+
expect(AnthropicVertex).toHaveBeenCalledWith(
132+
expect.objectContaining({
133+
projectId: "test-project",
134+
region: "us-central1",
135+
googleAuth: expect.any(Object),
136+
timeout: MOCK_TIMEOUT_MS,
137+
}),
138+
)
70139
})
71140
})
72141

src/api/providers/__tests__/lite-llm.spec.ts

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,13 @@ import { ApiHandlerOptions } from "../../../shared/api"
66
import { litellmDefaultModelId, litellmDefaultModelInfo } from "@roo-code/types"
77

88
// Mock vscode first to avoid import errors
9-
vi.mock("vscode", () => ({}))
9+
vi.mock("vscode", () => ({
10+
workspace: {
11+
getConfiguration: () => ({
12+
get: (_key: string, defaultValue?: unknown) => defaultValue,
13+
}),
14+
},
15+
}))
1016

1117
// Mock OpenAI
1218
const mockCreate = vi.fn()

src/api/providers/__tests__/openai.spec.ts

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,12 @@ import { openAiModelInfoSaneDefaults, DEEP_SEEK_DEFAULT_TEMPERATURE } from "@roo
88
import { Package } from "../../../shared/package"
99
import axios from "axios"
1010

11+
vitest.mock("../utils/timeout-config", () => ({
12+
getApiRequestTimeout: vitest.fn().mockReturnValue(300_000),
13+
}))
14+
15+
const MOCK_TIMEOUT_MS = 300_000
16+
1117
const mockCreate = vitest.fn()
1218

1319
vitest.mock("openai", () => {
@@ -117,7 +123,7 @@ describe("OpenAiHandler", () => {
117123
"X-Title": "Zoo Code",
118124
"User-Agent": `ZooCode/${Package.version}`,
119125
},
120-
timeout: expect.any(Number),
126+
timeout: MOCK_TIMEOUT_MS,
121127
})
122128
})
123129
})

src/api/providers/__tests__/opencode-go.spec.ts

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,13 @@
11
// npx vitest run src/api/providers/__tests__/opencode-go.spec.ts
22

33
// Mock vscode first to avoid import errors
4-
vitest.mock("vscode", () => ({}))
4+
vitest.mock("vscode", () => ({
5+
workspace: {
6+
getConfiguration: () => ({
7+
get: (_key: string, defaultValue?: unknown) => defaultValue,
8+
}),
9+
},
10+
}))
511

612
import { Anthropic } from "@anthropic-ai/sdk"
713
import OpenAI from "openai"

src/api/providers/__tests__/openrouter.spec.ts

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,18 @@
11
// pnpm --filter roo-cline test api/providers/__tests__/openrouter.spec.ts
22

3-
vitest.mock("vscode", () => ({}))
3+
vitest.mock("vscode", () => ({
4+
workspace: {
5+
getConfiguration: () => ({
6+
get: (_key: string, defaultValue?: unknown) => defaultValue,
7+
}),
8+
},
9+
}))
10+
11+
vitest.mock("../utils/timeout-config", () => ({
12+
getApiRequestTimeout: vitest.fn().mockReturnValue(300_000),
13+
}))
14+
15+
const MOCK_TIMEOUT_MS = 300_000
416

517
import { Anthropic } from "@anthropic-ai/sdk"
618
import OpenAI from "openai"
@@ -108,6 +120,7 @@ describe("OpenRouterHandler", () => {
108120
"X-Title": "Zoo Code",
109121
"User-Agent": `ZooCode/${Package.version}`,
110122
},
123+
timeout: MOCK_TIMEOUT_MS,
111124
})
112125
})
113126

src/api/providers/__tests__/requesty.spec.ts

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
// npx vitest run api/providers/__tests__/requesty.spec.ts
22

3+
vitest.mock("../utils/timeout-config", () => ({
4+
getApiRequestTimeout: vitest.fn().mockReturnValue(300_000),
5+
}))
6+
7+
const MOCK_TIMEOUT_MS = 300_000
8+
39
import { Anthropic } from "@anthropic-ai/sdk"
410
import OpenAI from "openai"
511

@@ -82,6 +88,7 @@ describe("RequestyHandler", () => {
8288
"X-Title": "Zoo Code",
8389
"User-Agent": `ZooCode/${Package.version}`,
8490
},
91+
timeout: MOCK_TIMEOUT_MS,
8592
})
8693
})
8794

@@ -97,6 +104,7 @@ describe("RequestyHandler", () => {
97104
"X-Title": "Zoo Code",
98105
"User-Agent": `ZooCode/${Package.version}`,
99106
},
107+
timeout: MOCK_TIMEOUT_MS,
100108
})
101109
})
102110

src/api/providers/__tests__/vercel-ai-gateway.spec.ts

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,13 @@
11
// npx vitest run src/api/providers/__tests__/vercel-ai-gateway.spec.ts
22

33
// Mock vscode first to avoid import errors
4-
vitest.mock("vscode", () => ({}))
4+
vitest.mock("vscode", () => ({
5+
workspace: {
6+
getConfiguration: () => ({
7+
get: (_key: string, defaultValue?: unknown) => defaultValue,
8+
}),
9+
},
10+
}))
511

612
import { Anthropic } from "@anthropic-ai/sdk"
713
import OpenAI from "openai"
@@ -118,6 +124,7 @@ describe("VercelAiGatewayHandler", () => {
118124
"X-Title": "Zoo Code",
119125
"User-Agent": expect.stringContaining("ZooCode/"),
120126
}),
127+
timeout: expect.any(Number),
121128
})
122129
})
123130

src/api/providers/__tests__/vertex-credentials.spec.ts

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,13 @@
22

33
// Mock vscode first to avoid import errors when the provider stack pulls
44
// transitive vscode-dependent modules during construction.
5-
vitest.mock("vscode", () => ({}))
5+
vitest.mock("vscode", () => ({
6+
workspace: {
7+
getConfiguration: () => ({
8+
get: (_key: string, defaultValue?: unknown) => defaultValue,
9+
}),
10+
},
11+
}))
612

713
vitest.mock("@roo-code/telemetry", () => ({
814
TelemetryService: {

src/api/providers/__tests__/vertex.spec.ts

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,13 @@
11
// npx vitest run src/api/providers/__tests__/vertex.spec.ts
22

33
// Mock vscode first to avoid import errors
4-
vitest.mock("vscode", () => ({}))
4+
vitest.mock("vscode", () => ({
5+
workspace: {
6+
getConfiguration: vitest.fn(() => ({
7+
get: vitest.fn((key: string, defaultValue: any) => defaultValue),
8+
})),
9+
},
10+
}))
511

612
const mockCaptureException = vitest.fn()
713

0 commit comments

Comments
 (0)