Skip to content

Commit 5b46861

Browse files
authored
feat: Gemini Interactions API record/replay (issue #136) (#139)
## Summary Adds full record/replay support for Google's Gemini Interactions API (`POST /v1beta/interactions`), the stateful conversation endpoint used by TanStack AI's `geminiTextInteractions()` adapter. 12th LLM provider. - **New handler** (`src/gemini-interactions.ts`): request conversion, response builders, SSE stream writer with data-only format, multi-turn sequencing via `previous_interaction_id` - **SDK wire-format conformance**: prefers `content` over `parts` (Turn field) and `result` over `output` (function_result field), matching the official `@google/genai` SDK types with backwards-compat fallback - **Drift detection**: canonical drift tests with SDK shape triangulation - **Recorder integration**: `buildFixtureResponse` Interactions detection + stream collapse - **63 tests** (2682 total) covering input conversion, response builders, SSE streaming, stream collapse, fixture matching, wire-format conformance, and edge cases Closes #136 ## Test plan - [x] `pnpm test` — 2682 passed, 36 skipped, 0 failed - [x] `pnpm run format:check` — clean - [x] `pnpm run lint` — clean - [x] `npx tsc --noEmit` — clean - [x] CR converged (R1: 2 findings fixed, R2: 0 findings)
2 parents d6392c8 + be60203 commit 5b46861

31 files changed

Lines changed: 3206 additions & 66 deletions

File tree

DRIFT.md

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@ When a `critical` drift is detected:
7777
- OpenAI Responses API → `src/responses.ts` (`buildTextResponse`, `buildToolCallResponse`, `buildTextStreamEvents`, `buildToolCallStreamEvents`)
7878
- Anthropic Claude → `src/messages.ts` (`buildClaudeTextResponse`, `buildClaudeToolCallResponse`, `buildClaudeTextStreamEvents`, `buildClaudeToolCallStreamEvents`)
7979
- Google Gemini → `src/gemini.ts` (`buildGeminiTextResponse`, `buildGeminiToolCallResponse`, `buildGeminiTextStreamChunks`, `buildGeminiToolCallStreamChunks`)
80+
- Gemini Interactions → `src/gemini-interactions.ts` (`buildInteractionsTextResponse`, `buildInteractionsToolCallResponse`, `buildInteractionsTextSSEEvents`, `buildInteractionsToolCallSSEEvents`)
8081

8182
2. **Update the builder** — add or modify the field to match the real API shape.
8283

@@ -106,7 +107,18 @@ When a model is deprecated:
106107

107108
## WebSocket Drift Coverage
108109

109-
In addition to the 19 existing drift tests (16 HTTP response-shape + 3 model deprecation), WebSocket drift tests cover aimock's WS protocols (4 verified + 2 canary = 6 WS tests):
110+
In addition to the 23 existing drift tests (20 HTTP response-shape + 3 model deprecation), WebSocket drift tests cover aimock's WS protocols (4 verified + 2 canary = 6 WS tests):
111+
112+
### Gemini Interactions API (Beta)
113+
114+
The Gemini Interactions API (`/v1beta/interactions`) is covered by 4 drift tests in `gemini-interactions.drift.ts`:
115+
116+
- Non-streaming text shape
117+
- Streaming text event sequence
118+
- Non-streaming tool call shape
119+
- Streaming tool call event sequence
120+
121+
Uses `describe.skipIf(!GOOGLE_API_KEY)` like other Gemini tests. The Interactions API is in Beta — shapes may shift as Google iterates on the endpoint.
110122

111123
| Protocol | Text | Tool Call | Real Endpoint | Status |
112124
| ------------------- | ---- | --------- | ------------------------------------------------------------------- | ---------- |
@@ -163,4 +175,4 @@ The fix workflow also supports `workflow_dispatch` for manual runs.
163175

164176
## Cost
165177

166-
~25 API calls per run (16 HTTP response-shape + 3 model listing + 6 WS including canaries) using the cheapest available models (`gpt-4o-mini`, `gpt-4o-mini-realtime-preview`, `claude-haiku-4-5-20251001`, `gemini-2.5-flash`) with 10-100 max tokens each. Under $0.15/week at daily cadence. When Gemini Live text-capable models become available, the 2 canary tests will become full drift tests, increasing real WS connections from 4 to 6.
178+
~29 API calls per run (20 HTTP response-shape + 3 model listing + 6 WS including canaries) using the cheapest available models (`gpt-4o-mini`, `gpt-4o-mini-realtime-preview`, `claude-haiku-4-5-20251001`, `gemini-2.5-flash`) with 10-100 max tokens each. Under $0.20/week at daily cadence. When Gemini Live text-capable models become available, the 2 canary tests will become full drift tests, increasing real WS connections from 4 to 6.

README.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -35,22 +35,22 @@ await mock.stop();
3535

3636
aimock mocks everything your AI app talks to:
3737

38-
| Tool | What it mocks | Docs |
39-
| -------------- | ------------------------------------------------------------------------------------------------------- | --------------------------------------------------- |
40-
| **LLMock** | OpenAI (Chat/Responses/Realtime), Claude, Gemini (REST/Live), Bedrock, Azure, Vertex AI, Ollama, Cohere | [Providers](https://aimock.copilotkit.dev/docs) |
41-
| **MCPMock** | MCP tools, resources, prompts with session management | [MCP](https://aimock.copilotkit.dev/mcp-mock) |
42-
| **A2AMock** | Agent-to-agent protocol with SSE streaming | [A2A](https://aimock.copilotkit.dev/a2a-mock) |
43-
| **AGUIMock** | AG-UI agent-to-UI event streams for frontend testing | [AG-UI](https://aimock.copilotkit.dev/agui-mock) |
44-
| **VectorMock** | Pinecone, Qdrant, ChromaDB compatible endpoints | [Vector](https://aimock.copilotkit.dev/vector-mock) |
45-
| **Services** | Tavily search, Cohere rerank, OpenAI moderation | [Services](https://aimock.copilotkit.dev/services) |
38+
| Tool | What it mocks | Docs |
39+
| -------------- | -------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------- |
40+
| **LLMock** | OpenAI (Chat/Responses/Realtime), Claude, Gemini (REST/Live/Interactions), Bedrock, Azure, Vertex AI, Ollama, Cohere | [Providers](https://aimock.copilotkit.dev/docs) |
41+
| **MCPMock** | MCP tools, resources, prompts with session management | [MCP](https://aimock.copilotkit.dev/mcp-mock) |
42+
| **A2AMock** | Agent-to-agent protocol with SSE streaming | [A2A](https://aimock.copilotkit.dev/a2a-mock) |
43+
| **AGUIMock** | AG-UI agent-to-UI event streams for frontend testing | [AG-UI](https://aimock.copilotkit.dev/agui-mock) |
44+
| **VectorMock** | Pinecone, Qdrant, ChromaDB compatible endpoints | [Vector](https://aimock.copilotkit.dev/vector-mock) |
45+
| **Services** | Tavily search, Cohere rerank, OpenAI moderation | [Services](https://aimock.copilotkit.dev/services) |
4646

4747
Run them all on one port with `npx @copilotkit/aimock --config aimock.json`, or use the programmatic API to compose exactly what you need.
4848

4949
## Features
5050

5151
- **[Record & Replay](https://aimock.copilotkit.dev/record-replay)** — Proxy real APIs, save as fixtures, replay deterministically forever
5252
- **[Multi-turn Conversations](https://aimock.copilotkit.dev/multi-turn)** — Record and replay multi-turn traces with tool rounds; match distinct turns via `turnIndex`, `hasToolResult`, `toolCallId`, `sequenceIndex`, or custom predicates
53-
- **[11 LLM Providers](https://aimock.copilotkit.dev/docs)** — OpenAI Chat, OpenAI Responses, OpenAI Realtime, Claude, Gemini, Gemini Live, Azure, Bedrock, Vertex AI, Ollama, Cohere — full streaming support
53+
- **[12 LLM Providers](https://aimock.copilotkit.dev/docs)** — OpenAI Chat, OpenAI Responses, OpenAI Realtime, Claude, Gemini, Gemini Live, Gemini Interactions, Azure, Bedrock, Vertex AI, Ollama, Cohere — full streaming support
5454
- **Multimedia APIs**[image generation](https://aimock.copilotkit.dev/images) (DALL-E, Imagen), [text-to-speech](https://aimock.copilotkit.dev/speech), [audio transcription](https://aimock.copilotkit.dev/transcription), [video generation](https://aimock.copilotkit.dev/video)
5555
- **[MCP](https://aimock.copilotkit.dev/mcp-mock) / [A2A](https://aimock.copilotkit.dev/a2a-mock) / [AG-UI](https://aimock.copilotkit.dev/agui-mock) / [Vector](https://aimock.copilotkit.dev/vector-mock)** — Mock every protocol your AI agents use
5656
- **[Chaos Testing](https://aimock.copilotkit.dev/chaos-testing)** — 500 errors, malformed JSON, mid-stream disconnects at any probability

docs/docs/index.html

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -306,7 +306,10 @@ <h2>The Suite</h2>
306306
<tbody>
307307
<tr>
308308
<td>LLM Providers</td>
309-
<td>OpenAI, Claude, Gemini, Bedrock, Azure, Vertex AI, Ollama, Cohere</td>
309+
<td>
310+
OpenAI, Claude, Gemini, Gemini Interactions, Bedrock, Azure, Vertex AI, Ollama,
311+
Cohere
312+
</td>
310313
<td><a href="/chat-completions">Docs &rarr;</a></td>
311314
</tr>
312315
<tr>

docs/fixtures/index.html

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -547,6 +547,7 @@ <h2>Provider Support Matrix</h2>
547547
<th>OpenAI Responses</th>
548548
<th>Claude</th>
549549
<th>Gemini</th>
550+
<th>Gemini Int.</th>
550551
<th>Vertex AI</th>
551552
<th>Bedrock</th>
552553
<th>Azure</th>
@@ -566,6 +567,7 @@ <h2>Provider Support Matrix</h2>
566567
<td>Yes</td>
567568
<td>Yes</td>
568569
<td>Yes</td>
570+
<td>Yes</td>
569571
</tr>
570572
<tr>
571573
<td>Tool Calls</td>
@@ -578,6 +580,7 @@ <h2>Provider Support Matrix</h2>
578580
<td>Yes</td>
579581
<td>Yes</td>
580582
<td>Yes</td>
583+
<td>Yes</td>
581584
</tr>
582585
<tr>
583586
<td>Content + Tool Calls</td>
@@ -590,6 +593,7 @@ <h2>Provider Support Matrix</h2>
590593
<td>Yes</td>
591594
<td>Yes</td>
592595
<td>Yes</td>
596+
<td>Yes</td>
593597
</tr>
594598
<tr>
595599
<td>Streaming</td>
@@ -598,6 +602,7 @@ <h2>Provider Support Matrix</h2>
598602
<td>SSE</td>
599603
<td>SSE</td>
600604
<td>SSE</td>
605+
<td>SSE</td>
601606
<td>Binary EventStream</td>
602607
<td>SSE</td>
603608
<td>NDJSON</td>
@@ -609,6 +614,7 @@ <h2>Provider Support Matrix</h2>
609614
<td>Yes</td>
610615
<td>Yes</td>
611616
<td>Yes</td>
617+
<td>&mdash;</td>
612618
<td>Yes</td>
613619
<td>Yes</td>
614620
<td>Yes</td>
@@ -626,6 +632,7 @@ <h2>Provider Support Matrix</h2>
626632
<td>&mdash;</td>
627633
<td>&mdash;</td>
628634
<td>&mdash;</td>
635+
<td>&mdash;</td>
629636
</tr>
630637
<tr>
631638
<td>Response Overrides</td>
@@ -634,6 +641,7 @@ <h2>Provider Support Matrix</h2>
634641
<td>Yes</td>
635642
<td>Yes</td>
636643
<td>Yes</td>
644+
<td>Yes</td>
637645
<td>&mdash;</td>
638646
<td>Yes<sup>*</sup></td>
639647
<td>&mdash;</td>

0 commit comments

Comments
 (0)