fix(ai-gemini): read/write thoughtSignature at Part level for Gemini 3.x#459
fix(ai-gemini): read/write thoughtSignature at Part level for Gemini 3.x#459pemontto wants to merge 5 commits intoTanStack:mainfrom
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughReads Gemini Changes
Sequence Diagram(s)sequenceDiagram
participant Client as Client (StreamProcessor / UI)
participant Server as Server (chat() + Adapter)
participant Gemini as Gemini API
Client->>Server: Send conversation (ModelMessages with toolCalls and `metadata`)
Server->>Server: Adapter.formatMessages()\n(read part.thoughtSignature || functionCall.thoughtSignature)\n(attach `part.thoughtSignature` and `metadata`)
Server->>Gemini: POST chat request (parts[] with Part-level `thoughtSignature`)
Gemini-->>Server: Stream response (parts[] with part.thoughtSignature or functionCall.thoughtSignature)
Server->>Server: processStreamChunks()\n(store InternalToolCallState.metadata and thoughtSignature)\n(build assistant UI messages including `metadata`)
Server->>Client: Stream events / UIMessages (TOOL_CALL_START / updates) with `metadata`
Client->>Server: Send next request (history converted back to ModelMessage.toolCalls includes `metadata`)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Review rate limit: 7/8 reviews remaining, refill in 7 minutes and 30 seconds.Comment |
|
Re: the pre-merge check about #403 scope. This PR fixes the server-side adapter (the The client-side |
|
Can confirm this also fixes the issue we were experiencing missing thought signature |
|
View your CI Pipeline Execution ↗ for commit 48cf4a2
☁️ Nx Cloud last updated this comment at |
|
View your CI Pipeline Execution ↗ for commit 48cf4a2
☁️ Nx Cloud last updated this comment at |
|
Good call, addressed in ab05e4c. The
Only remaining All 66 tests still pass, typecheck clean. |
|
@pemontto After some further testing, it seems there are a few more places in the tool call life cycle where the provider meta data should be added to avoid more errors. See the following file locations: |
|
Good catch @mattsoltani, those locations are exactly what #404 (by @houmark, opened 2026-03-28) addresses. It threads That PR has been open for nearly a month without a maintainer review. Could we get it reviewed alongside this one? Without both, the adapter write-side fix here never actually has a signature to write because it's dropped upstream in the pipeline. If preferred, I'm happy to pull those changes into this PR (with attribution to @houmark) so it's a single comprehensive fix. Just let me know which you'd like. |
|
I see @pemontto and agree with your points. Don't know about the preference here, but would be good to get both fixes in so we can remove the local patches. @AlemTuzlak what do you think is the best path forward? |
|
Would you mind pulling it in? Also adding e2e tests if possible? |
|
also #404 should use modelOptions instead of providerOptions |
|
@AlemTuzlak before pulling #404 in: it threads |
|
@pemontto I think that's outdated, we moved away from providerMetadata and providerOptions to modelOptions and modelMetadata |
Gemini 3.x models emit thoughtSignature as a Part-level sibling of functionCall (per @google/genai Part type), not nested inside functionCall. The adapter was reading from functionCall.thoughtSignature (which does not exist in the SDK types) and writing it back nested, causing the API to reject subsequent tool-call turns with 400 INVALID_ARGUMENT: "Function call is missing a thought_signature". Read side: check part.thoughtSignature first, fall back to functionCall.thoughtSignature for Gemini 2.x compatibility. Write side: emit thoughtSignature as a Part-level sibling of functionCall instead of nesting it inside. Closes TanStack#403 Related: TanStack#218, TanStack#401, TanStack#404
Per review feedback: use the @google/genai typed `Part` interface directly instead of `as any` casts. - Read side: `part.thoughtSignature` is properly typed on `Part`, so the cast is removed entirely. The Gemini 2.x fallback to `functionCall.thoughtSignature` is also removed since the SDK has never typed it there and Gemini has always emitted it at Part level. - Write side: construct a typed `Part` and conditionally assign `thoughtSignature`, avoiding the `as Part` cast on a spread literal. The only remaining `as any` in this area is the pre-existing functionResponse cast, which is unrelated to this fix.
|
@AlemTuzlak I might be being obtuse, I can see |
|
Yeah then it should be metadata, i hallucinated providerOptions 💀 |
Renames `providerMetadata` to `metadata` and types it per-adapter via a new `TToolCallMetadata` generic on `BaseTextAdapter`, mirroring the existing typed-metadata pattern on content parts (`ImagePart<TMetadata>`, etc.). Also threads metadata through the client-side UIMessage pipeline so it round-trips with each tool call, fixing the silent drop surfaced in TanStack#403/TanStack#404 (incorporates that PR's plumbing under the new name). Gemini adapter now declares `TToolCallMetadata = GeminiToolCallMetadata`, giving consumers typed `toolCall.metadata?.thoughtSignature` end-to-end. Per maintainer feedback on TanStack#459 (AlemTuzlak): use `metadata` (typed per-adapter generic) rather than the previous `providerMetadata` bag. The `ToolCallStartEvent` event remains non-generic with `metadata?: Record<string, unknown>` because making it generic breaks the AGUIEvent discriminated-union narrowing. Breaking: consumers reading `toolCall.providerMetadata` or `toolCallStartEvent.providerMetadata` should rename to `metadata`. Co-authored-by: houmark <noreply@github.com>
pemontto
force-pushed
the
fix/gemini-thought-signature-part-level
branch
from
ab05e4c to
1ce6e29
Compare
Renames `providerMetadata` to `metadata` and types it per-adapter via a new `TToolCallMetadata` generic on `BaseTextAdapter`, mirroring the existing typed-metadata pattern on content parts (`ImagePart<TMetadata>`, etc.). Also threads metadata through the client-side UIMessage pipeline so it round-trips with each tool call, fixing the silent drop surfaced in TanStack#403/TanStack#404 (incorporates that PR's plumbing under the new name). Gemini adapter now declares `TToolCallMetadata = GeminiToolCallMetadata`, giving consumers typed `toolCall.metadata?.thoughtSignature` end-to-end. Per maintainer feedback on TanStack#459 (AlemTuzlak): use `metadata` (typed per-adapter generic) rather than the previous `providerMetadata` bag. The `ToolCallStartEvent` event remains non-generic with `metadata?: Record<string, unknown>` because making it generic breaks the AGUIEvent discriminated-union narrowing. Breaking: consumers reading `toolCall.providerMetadata` or `toolCallStartEvent.providerMetadata` should rename to `metadata`. Co-authored-by: houmark <noreply@github.com>
Added the new TToolCallMetadata generic to BaseTextAdapter requires mock adapters in tests/test-utils.ts and tests/type-check.test.ts to include toolCallMetadata in their `~types` block.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
packages/typescript/ai-event-client/src/index.ts (1)
89-100:ToolCallis now generic, butToolCallPartstill can’t carry typed metadata.Line 89 correctly introduces
ToolCall<TMetadata>, butToolCallPart(Line 52-64) still lacksmetadataand a matching generic, which leaves part-level metadata untyped and out of sync with the mirrored core types (packages/typescript/ai/src/types.ts:313-330).Proposed alignment patch
-export interface ToolCallPart { +export interface ToolCallPart<TMetadata = unknown> { type: 'tool-call' id: string name: string arguments: string state: ToolCallState @@ } output?: any + metadata?: TMetadata } @@ -export type MessagePart = +export type MessagePart<TMetadata = unknown> = | TextPart | ImagePart | AudioPart | VideoPart | DocumentPart - | ToolCallPart + | ToolCallPart<TMetadata> | ToolResultPart | ThinkingPart @@ -export interface TextMessageCreatedEvent extends BaseEventContext { +export interface TextMessageCreatedEvent<TMetadata = unknown> + extends BaseEventContext { @@ - parts?: Array<MessagePart> - toolCalls?: Array<ToolCall> + parts?: Array<MessagePart<TMetadata>> + toolCalls?: Array<ToolCall<TMetadata>> messageIndex?: number }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/typescript/ai-event-client/src/index.ts` around lines 89 - 100, ToolCall is generic (ToolCall<TMetadata>) but ToolCallPart is missing a matching generic and metadata field; update the ToolCallPart interface to accept the same generic (e.g. ToolCallPart<TMetadata = unknown>) and add metadata?: TMetadata to its shape so part-level metadata is typed and mirrors ToolCall; also update any related type references/usages in this file (and exports) to propagate the generic default where needed so signatures like ToolCallPart<T> and ToolCall<T> remain consistent.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@packages/typescript/ai-event-client/src/index.ts`:
- Around line 89-100: ToolCall is generic (ToolCall<TMetadata>) but ToolCallPart
is missing a matching generic and metadata field; update the ToolCallPart
interface to accept the same generic (e.g. ToolCallPart<TMetadata = unknown>)
and add metadata?: TMetadata to its shape so part-level metadata is typed and
mirrors ToolCall; also update any related type references/usages in this file
(and exports) to propagate the generic default where needed so signatures like
ToolCallPart<T> and ToolCall<T> remain consistent.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: cff574eb-ec93-48a4-a410-c1226235317c
📒 Files selected for processing (2)
.changeset/fix-gemini-thought-signature-part-level.mdpackages/typescript/ai-event-client/src/index.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- .changeset/fix-gemini-thought-signature-part-level.md
pemontto
force-pushed
the
fix/gemini-thought-signature-part-level
branch
from
6c68bd1 to
5de2d4c
Compare
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
packages/typescript/ai/src/activities/chat/stream/processor.ts (1)
1391-1397:⚠️ Potential issue | 🟠 Major | ⚡ Quick winPreserve metadata in
ProcessorResult.toolCallstoo.This keeps
metadataon the UI path, butgetCompletedToolCalls()still rebuilds eachToolCallwith only{ id, type, function }. Any caller usingprocess()/getResult().toolCallswill still lose Gemini'sthoughtSignaturebefore the next round-trip.Patch
result.push({ id: tc.id, type: 'function' as const, function: { name: tc.name, arguments: tc.arguments, }, + ...(tc.metadata !== undefined && { metadata: tc.metadata }), })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/typescript/ai/src/activities/chat/stream/processor.ts` around lines 1391 - 1397, getCompletedToolCalls() rebuilds ToolCall objects without carrying over metadata, so ProcessorResult.toolCalls loses Gemini's thoughtSignature; update getCompletedToolCalls() to include the metadata when reconstructing each ToolCall (preserve any existing toolCall.metadata) so process()/getResult().toolCalls retains metadata across round-trips—ensure the rebuilt objects include the metadata field (same shape used by updateToolCallPart call that sets state 'input-complete') and update any tests or callers that depend on ProcessorResult.toolCalls accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In `@packages/typescript/ai/src/activities/chat/stream/processor.ts`:
- Around line 1391-1397: getCompletedToolCalls() rebuilds ToolCall objects
without carrying over metadata, so ProcessorResult.toolCalls loses Gemini's
thoughtSignature; update getCompletedToolCalls() to include the metadata when
reconstructing each ToolCall (preserve any existing toolCall.metadata) so
process()/getResult().toolCalls retains metadata across round-trips—ensure the
rebuilt objects include the metadata field (same shape used by
updateToolCallPart call that sets state 'input-complete') and update any tests
or callers that depend on ProcessorResult.toolCalls accordingly.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 2c65d95f-6325-440f-93f8-033fffe71104
📒 Files selected for processing (15)
.changeset/fix-gemini-thought-signature-part-level.mdpackages/typescript/ai-event-client/src/index.tspackages/typescript/ai-gemini/src/adapters/text.tspackages/typescript/ai-gemini/src/message-types.tspackages/typescript/ai-gemini/tests/gemini-adapter.test.tspackages/typescript/ai/src/activities/chat/adapter.tspackages/typescript/ai/src/activities/chat/messages.tspackages/typescript/ai/src/activities/chat/stream/message-updaters.tspackages/typescript/ai/src/activities/chat/stream/processor.tspackages/typescript/ai/src/activities/chat/stream/types.tspackages/typescript/ai/src/activities/chat/tools/tool-calls.tspackages/typescript/ai/src/types.tspackages/typescript/ai/tests/strip-to-spec-middleware.test.tspackages/typescript/ai/tests/test-utils.tspackages/typescript/ai/tests/type-check.test.ts
✅ Files skipped from review due to trivial changes (5)
- packages/typescript/ai/tests/type-check.test.ts
- packages/typescript/ai/tests/test-utils.ts
- packages/typescript/ai/src/activities/chat/stream/types.ts
- packages/typescript/ai/tests/strip-to-spec-middleware.test.ts
- packages/typescript/ai-gemini/src/message-types.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- .changeset/fix-gemini-thought-signature-part-level.md
3 participants
Summary
Gemini 3.x models emit
thoughtSignatureas a Part-level sibling offunctionCall(per the@google/genaiParttype definition), not nested insidefunctionCall. TheFunctionCallinterface has nothoughtSignatureproperty at all.The adapter was:
functionCall.thoughtSignature(wrong location, doesn't exist in SDK types)functionCall(wrong location, API ignores it there)This causes Gemini 3.x to reject subsequent tool-call turns with:
The
@google/genaiPart type (for reference)Changes
processStreamChunks): readspart.thoughtSignaturefirst, falls back tofunctionCall.thoughtSignaturefor Gemini 2.x compatibilityformatMessages): emitsthoughtSignatureas a Part-level sibling offunctionCallinstead of nesting it insideTest plan
thoughtSignaturefrom Gemini 3.x streaming response and round-trips it at the Part levelfunctionCall.thoughtSignaturefor Gemini 2.x wire formatgemini-3.1-pro-previewandgemini-3.1-flash-lite-previewsessions (multi-turn tool calling with thinking enabled)Closes #403
Related: #218, #401, #404
Summary by CodeRabbit
Bug Fixes
Tests
Chores / Breaking Changes