@effect/ai-amazon-bedrock: withConfigOverride thinking config leaks into tool call handlers, breaking generateObject

[Chaoran-Huang]

What version of Effect is running?

• @effect/ai: ^0.33.0
• @effect/ai-amazon-bedrock: ^0.13.0

Description

When using AmazonBedrockLanguageModel.withConfigOverride to enable extended thinking on a generateText call that includes a toolkit, the Config service propagates into tool call handlers resolved by resolveToolCalls in LanguageModel.ts. If any tool handler internally calls generateObject (which sets toolChoice: { tool: { name: ... } } in the Bedrock provider), the Bedrock API rejects the request with:

The model returned the following errors: Thinking may not be enabled when tool_choice forces tool use.

This is because Anthropic's API does not allow extended thinking when toolChoice forces a specific tool. The @effect/ai-amazon-bedrock provider correctly maps generateObject to a forced tool call (lines 214-229 of AmazonBedrockLanguageModel.ts), but doesn't account for the fact that withConfigOverride may have injected thinking configuration from an outer scope.

Reproduction

import { Chat, Toolkit } from "@effect/ai"
import { LanguageModel } from "@effect/ai/LanguageModel"
import { AmazonBedrockLanguageModel } from "@effect/ai-amazon-bedrock"
import { Effect, Schema } from "effect"

const ResultSchema = Schema.Struct({
  outcome: Schema.Literal("Pass", "Fail"),
  reasoning: Schema.String,
})

// Define a tool whose handler uses generateObject
const CheckTool = /* ... tool definition ... */

const CheckToolkit = Toolkit.make(CheckTool)

const program = Effect.gen(function* () {
  const chat = yield* Chat.fromPrompt([{ role: "system", content: "You are a reviewer." }])

  // This call uses generateText with toolChoice: "auto" (compatible with thinking).
  // But the tool handler internally calls generateObject (forced toolChoice).
  // The thinking config leaks from the outer scope into the tool handler.
  const response = yield* chat.generateText({
    prompt: "Review this document",
    toolkit: /* resolved toolkit with handlers that call generateObject */,
  }).pipe(
    AmazonBedrockLanguageModel.withConfigOverride({
      additionalModelRequestFields: {
        thinking: { type: "enabled", budget_tokens: 4096 },
      },
    })
  )
})

Error: 400 Bad Request from Bedrock Converse API:

"Thinking may not be enabled when tool_choice forces tool use."

Root Cause

withConfigOverride provides the Config service via Effect.provideService:

// AmazonBedrockLanguageModel.ts
(self, overrides) =>
  Effect.flatMap(
    Config.getOrUndefined,
    (config) => Effect.provideService(self, Config, { ...config, ...overrides })
  )

This makes the config (including thinking) available to the entire effect tree of self. When resolveToolCalls (in LanguageModel.ts) invokes tool handlers, those handlers run within this scope and inherit the thinking config. If a handler calls generateObject, the Bedrock provider's makeRequest reads the Config:

const config = { modelId: options.model, ...options.config, ...context.unsafeMap.get(Config.key) }

It picks up additionalModelRequestFields.thinking from the outer scope, builds a request with both thinking enabled and forced toolChoice, and the Bedrock API rejects it.

Current Workaround

Explicitly clear the thinking config on each tool handler:

AmazonBedrockLanguageModel.withConfigOverride(handlerEffect, {
  additionalModelRequestFields: {},
})

Expected Behavior

Enabling thinking on a generateText call should not affect nested generateObject calls made by tool handlers. The provider could:

1. Automatically strip additionalModelRequestFields.thinking when building requests where responseFormat.type === "json" (i.e. generateObject), or when toolChoice forces a specific tool — since these are known to be incompatible with thinking per Anthropic's API.
2. Alternatively, scope the Config service more narrowly so it only applies to the provider's own API call, not to tool handler resolution.