fix(anthropic): clamp non-streaming structured-output max_tokens default (#849)

The #849 default of the model's full output ceiling broke the non-streaming
`structuredOutput()` path: the Anthropic SDK refuses a non-streaming request
whose `max_tokens` could exceed its 10-minute timeout (~21,333 tokens), so
`chat({ outputSchema })` on any fallback-path model threw "Streaming is
required for operations that may take longer than 10 minutes".

`getAnthropicDefaultMaxTokens(model, { stream })` now clamps the default to
`ANTHROPIC_MAX_NONSTREAMING_TOKENS` when `stream: false`; the streaming chat
path keeps the model's full ceiling. An explicit oversized `max_tokens` still
surfaces the SDK's "use streaming" error.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: tombeckenham

.changeset/anthropic-max-tokens-default.md

@@ -13,3 +13,11 @@ The adapter also now logs a warning when a response is truncated while using the
 defaulted (caller-unspecified) cap, so the truncation isn't silently attributed
 to the model "doing nothing". Callers that set `modelOptions.max_tokens`
 explicitly are unaffected.
+
+The non-streaming structured-output path (`structuredOutput()`) clamps this
+default to the Anthropic SDK's non-streaming-safe limit (~21K tokens). The SDK
+refuses a non-streaming request whose `max_tokens` could exceed its 10-minute
+timeout, so without the clamp the full-ceiling default would make every
+`chat({ outputSchema })` call on a fallback-path model throw "Streaming is
+required for operations that may take longer than 10 minutes". The streaming
+chat path keeps the model's full ceiling.

docs/adapters/anthropic.md

@@ -140,6 +140,8 @@ const stream = chat({
 
 Anthropic's Messages API _requires_ `max_tokens` on every request, so the adapter always sends a value. When you don't set `modelOptions.max_tokens`, it defaults to the selected model's full output ceiling (`max_output_tokens` from the model metadata — e.g. 64K for Sonnet, 128K for Opus), falling back to a safe constant for unrecognized models. `max_tokens` is a ceiling, not a reservation — billing is on tokens actually generated — so this default costs nothing extra and avoids the silent mid-response truncation (`stop_reason: "max_tokens"`) that a low default would cause. Set `max_tokens` explicitly only when you want to _cap_ output below the model ceiling. If a response is truncated while using the default cap, the adapter logs a warning (visible with [debug logging](../advanced/debug-logging) enabled).
 
+One exception: structured output (`chat({ outputSchema })`) on models that use the non-streaming finalization path clamps this default to ~21K tokens. The Anthropic SDK rejects a non-streaming request whose `max_tokens` could exceed its 10-minute timeout, so the full ceiling can't be used there. Streaming chat is unaffected. To raise the structured-output ceiling toward a model's true max, stream the response.
+
 ### Thinking (Extended Thinking)
 
 Enable extended thinking with a token budget. This allows Claude to show its reasoning process, which is streamed as `thinking` chunks:

packages/ai-anthropic/src/adapters/text.ts

@@ -266,7 +266,12 @@ export class AnthropicTextAdapter<
     const { chatOptions, outputSchema } = options
     const { logger } = chatOptions
 
-    const requestParams = this.mapCommonOptionsToAnthropic(chatOptions)
+    // `structuredOutput()` issues a non-streaming `messages.create({ stream:
+    // false })` below, so the defaulted `max_tokens` must stay under the SDK's
+    // non-streaming 10-minute guard (issue #849) — pass `stream: false`.
+    const requestParams = this.mapCommonOptionsToAnthropic(chatOptions, {
+      stream: false,
+    })
 
     // Create a tool that will capture the structured output
     // Anthropic's SDK requires input_schema with type: 'object' literal
@@ -355,6 +360,7 @@ export class AnthropicTextAdapter<
 
   private mapCommonOptionsToAnthropic(
     options: TextOptions<AnthropicTextProviderOptions>,
+    { stream = true }: { stream?: boolean } = {},
   ) {
     const modelOptions = options.modelOptions
 
@@ -429,8 +435,12 @@ export class AnthropicTextAdapter<
     // that silently truncates long responses with `stop_reason: "max_tokens"`
     // (issue #849). `max_tokens` is a ceiling, not a reservation — billing is on
     // tokens actually generated, so a higher default costs nothing extra.
+    // For non-streaming requests (the `structuredOutput()` path) the default is
+    // clamped to the SDK's non-streaming-safe limit so it doesn't trip the
+    // "streaming required" 10-minute guard — see getAnthropicDefaultMaxTokens.
     const defaultMaxTokens =
-      modelOptions?.max_tokens ?? getAnthropicDefaultMaxTokens(this.model)
+      modelOptions?.max_tokens ??
+      getAnthropicDefaultMaxTokens(this.model, { stream })
     const maxTokens =
       thinkingBudget && thinkingBudget >= defaultMaxTokens
         ? thinkingBudget + 1

packages/ai-anthropic/src/model-meta.ts

@@ -776,16 +776,42 @@ const ANTHROPIC_MODEL_MAX_OUTPUT_TOKENS: Record<string, number> = {
   [CLAUDE_OPUS_4_8_FAST.id]: CLAUDE_OPUS_4_8_FAST.max_output_tokens,
 }
 
+/**
+ * Largest `max_tokens` the Anthropic SDK permits on a **non-streaming**
+ * request. The SDK refuses to make a non-streaming call it estimates could
+ * exceed its 10-minute timeout, computed as
+ * `(60min * max_tokens) / 128_000 > 10min` — i.e. it throws
+ * `"Streaming is required for operations that may take longer than 10 minutes"`
+ * once `max_tokens > 128_000 * 10 / 60 ≈ 21_333`
+ * (`@anthropic-ai/sdk`'s `calculateNonstreamingTimeout`). The text adapter's
+ * only non-streaming call is the forced-tool `structuredOutput()` request, so
+ * its defaulted ceiling must stay at or below this; the streaming chat path
+ * keeps the model's full {@link getAnthropicDefaultMaxTokens} ceiling. We sit
+ * just under the boundary (`21_333` would round-trip to exactly 10min). This
+ * caps only the *default* — an explicit oversized `max_tokens` from the caller
+ * still surfaces the SDK's "use streaming" error, which is the correct signal.
+ */
+export const ANTHROPIC_MAX_NONSTREAMING_TOKENS = 21_000
+
 /**
  * Resolve the default `max_tokens` for a model: its known `max_output_tokens`
  * ceiling, or {@link ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS} for unknown models.
  * Callers that pass an explicit `max_tokens` bypass this entirely.
+ *
+ * Pass `stream: false` for non-streaming requests (the `structuredOutput()`
+ * path): the result is then clamped to {@link ANTHROPIC_MAX_NONSTREAMING_TOKENS}
+ * so the defaulted ceiling doesn't trip the SDK's non-streaming 10-minute guard
+ * (issue #849). Streaming requests (the default) are unaffected and get the
+ * model's full ceiling.
  */
-export function getAnthropicDefaultMaxTokens(model: string): number {
-  return (
+export function getAnthropicDefaultMaxTokens(
+  model: string,
+  { stream = true }: { stream?: boolean } = {},
+): number {
+  const ceiling =
     ANTHROPIC_MODEL_MAX_OUTPUT_TOKENS[model] ??
     ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS
-  )
+  return stream ? ceiling : Math.min(ceiling, ANTHROPIC_MAX_NONSTREAMING_TOKENS)
 }
 
 /**

packages/ai-anthropic/tests/anthropic-adapter.test.ts

@@ -2,6 +2,7 @@ import { describe, it, expect, beforeEach, vi } from 'vitest'
 import { chat, type Tool, type StreamChunk } from '@tanstack/ai'
 import { AnthropicTextAdapter } from '../src/adapters/text'
 import type { AnthropicTextProviderOptions } from '../src/adapters/text'
+import { ANTHROPIC_MAX_NONSTREAMING_TOKENS } from '../src/model-meta'
 import { z } from 'zod'
 
 const mocks = vi.hoisted(() => {
@@ -543,6 +544,51 @@ describe('Anthropic adapter option mapping', () => {
     expect(truncationWarning).toBeUndefined()
   })
 
+  it('clamps the default max_tokens on the non-streaming structured-output path so it never trips the SDK 10-minute guard (#849)', async () => {
+    // The structured-output fallback issues a NON-streaming
+    // `messages.create({ stream: false })`. The Anthropic SDK throws
+    // "Streaming is required for operations that may take longer than 10
+    // minutes" once max_tokens exceeds ~21_333, so the defaulted ceiling must
+    // be clamped here even though the streaming chat path keeps the full 64K.
+    mocks.betaMessagesCreate.mockResolvedValueOnce({
+      id: 'msg_structured',
+      type: 'message',
+      role: 'assistant',
+      model: 'claude-3-7-sonnet',
+      content: [
+        {
+          type: 'tool_use',
+          id: 'toolu_structured_output',
+          name: 'structured_output',
+          input: { recommendation: 'Strat', price: 1299 },
+        },
+      ],
+      stop_reason: 'tool_use',
+      usage: { input_tokens: 10, output_tokens: 20 },
+    })
+
+    const adapter = createAdapter('claude-3-7-sonnet')
+
+    for await (const _ of chat({
+      adapter,
+      messages: [{ role: 'user', content: 'recommend a guitar as json' }],
+      outputSchema: z.object({
+        recommendation: z.string(),
+        price: z.number(),
+      }),
+      stream: true,
+    })) {
+      // consume stream
+    }
+
+    const [payload] = mocks.betaMessagesCreate.mock.calls[0]!
+    expect(payload.stream).toBe(false)
+    // Clamped to the non-streaming limit — NOT claude-3-7-sonnet's full 64K
+    // streaming ceiling, which would make the SDK throw before the request.
+    expect(payload.max_tokens).toBe(ANTHROPIC_MAX_NONSTREAMING_TOKENS)
+    expect(payload.max_tokens).toBeLessThanOrEqual(21_333)
+  })
+
   it('native combined mode (#605): wires outputSchema into output_format alongside tools on Claude 4.5+', async () => {
     // Final-turn JSON the model emits when output_format is in play.
     const finalJson = JSON.stringify({ city: 'Berlin', temp: 18 })

packages/ai-anthropic/tests/model-meta.test.ts

@@ -1,6 +1,7 @@
 import { describe, expect, expectTypeOf, it } from 'vitest'
 import {
   ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS,
+  ANTHROPIC_MAX_NONSTREAMING_TOKENS,
   getAnthropicDefaultMaxTokens,
 } from '../src/model-meta'
 import type {
@@ -806,4 +807,41 @@ describe('getAnthropicDefaultMaxTokens (#849)', () => {
       1024,
     )
   })
+
+  it('clamps the default to the non-streaming limit for non-streaming requests (#849)', () => {
+    // The Anthropic SDK refuses non-streaming requests whose `max_tokens`
+    // could exceed its 10-minute timeout (~21_333). The streaming path keeps
+    // the full ceiling; the non-streaming (`structuredOutput`) path must clamp.
+    expect(ANTHROPIC_MAX_NONSTREAMING_TOKENS).toBeLessThanOrEqual(21_333)
+
+    // Opus 128K and Sonnet 64K both exceed the non-streaming limit → clamped.
+    expect(
+      getAnthropicDefaultMaxTokens('claude-opus-4.8', { stream: false }),
+    ).toBe(ANTHROPIC_MAX_NONSTREAMING_TOKENS)
+    expect(
+      getAnthropicDefaultMaxTokens('claude-sonnet-4-6', { stream: false }),
+    ).toBe(ANTHROPIC_MAX_NONSTREAMING_TOKENS)
+    // Unknown model fallback (64K) is also above the limit → clamped.
+    expect(
+      getAnthropicDefaultMaxTokens('some-future-claude-model', {
+        stream: false,
+      }),
+    ).toBe(ANTHROPIC_MAX_NONSTREAMING_TOKENS)
+  })
+
+  it('does not clamp a model whose ceiling is already below the non-streaming limit (#849)', () => {
+    // claude-3-haiku's 4K ceiling is under the non-streaming limit, so the
+    // non-streaming path returns the real ceiling, not the (larger) cap.
+    expect(
+      getAnthropicDefaultMaxTokens('claude-3-haiku', { stream: false }),
+    ).toBe(4_000)
+  })
+
+  it('keeps the full ceiling for streaming requests (default) (#849)', () => {
+    expect(
+      getAnthropicDefaultMaxTokens('claude-opus-4.8', { stream: true }),
+    ).toBe(128_000)
+    // Omitting the option defaults to streaming.
+    expect(getAnthropicDefaultMaxTokens('claude-opus-4.8')).toBe(128_000)
+  })
 })