Skip to content

Commit fdf36b3

Browse files
authored
feat: response merging, JSON auto-stringify, and openai-responses migration (#111)
## Summary - **Response template merging** — override `id`, `created`, `model`, `usage`, `finishReason`, `role`, `systemFingerprint` on fixture responses across all 4 provider formats (OpenAI, Claude, Gemini, Responses API), streaming and non-streaming - **JSON auto-stringify** — fixture `arguments` and `content` accept objects (auto-stringified by loader AND programmatic API), eliminating escaped JSON pain - **Repo-wide conversion** — all fixture files and docs examples converted to object syntax - **Migration guide** — `docs/migrate-from-openai-responses/` targeting ~8.5K weekly PyPI users with accurate before/after examples - **Docs accuracy fixes** — fixtures page field names corrected (latency, ttft, chaos), missing endpoint/ContentWithToolCalls documented, onTranscription signature fixed - **Type cleanup** — `role`/`finishReason` consolidated into `ResponseOverrides`, `FixtureFile*` types extend `ResponseOverrides`, `validateFixtures` recognizes all response types - **New exports** — `ResponseOverrides`, `ContentWithToolCallsResponse`, `FixtureFileResponse`, `FixtureFileToolCall` - **Updated guidance** — `skills/write-fixtures/SKILL.md` and docs pages updated for new features - **Version bump** — 1.14.0 ## Test plan - [x] 2377 tests pass (40 new: 29 response overrides + 8 auto-stringify + 3 programmatic) - [x] Build clean (1.8MB, 285 files) - [x] Exports clean (all 6 paths verified) - [x] All commit headers under 100 chars (commitlint) - [x] No escaped JSON strings remain in fixtures or docs - [x] Full MSAL CR loop converged (5 module reviews + fix round + 7-agent final gate)
2 parents 687a493 + 320d466 commit fdf36b3

File tree

44 files changed

+10838
-289
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

44 files changed

+10838
-289
lines changed

CHANGELOG.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,20 @@
11
# @copilotkit/aimock
22

3+
## 1.14.0
4+
5+
### Minor Changes
6+
7+
- Response template merging — override `id`, `created`, `model`, `usage`, `finishReason`, `role`, `systemFingerprint` on fixture responses across all 4 provider formats (OpenAI, Claude, Gemini, Responses API) (#111)
8+
- JSON auto-stringify — fixture `arguments` and `content` fields accept objects that are auto-stringified by the loader, eliminating escaped JSON pain (#111)
9+
- Migration guide from openai-responses-python (#111)
10+
- All fixture examples and docs converted to object syntax (#111)
11+
12+
### Patch Changes
13+
14+
- Fix `onTranscription` docs to show correct 1-argument signature
15+
- Fix `validateFixtures` to recognize ContentWithToolCalls and multimedia response types
16+
- Add `ResponseOverrides` field validation in `validateFixtures` — catches invalid types for `id`, `created`, `model`, `usage`, `finishReason`, `role`, `systemFingerprint`
17+
318
## 1.13.0
419

520
### Minor Changes

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ Run them all on one port with `npx aimock --config aimock.json`, or use the prog
5252
- **[Prometheus Metrics](https://aimock.copilotkit.dev/metrics)** — Request counts, latencies, fixture match rates
5353
- **[Docker + Helm](https://aimock.copilotkit.dev/docker)** — Container image and Helm chart for CI/CD
5454
- **[Vitest & Jest Plugins](https://aimock.copilotkit.dev/test-plugins)** — Zero-config `useAimock()` with auto lifecycle and env patching
55+
- **[Response Overrides](https://aimock.copilotkit.dev/fixtures)** — Control `id`, `model`, `usage`, `finishReason` in fixture responses
5556
- **Zero dependencies** — Everything from Node.js builtins
5657

5758
## GitHub Action
@@ -94,7 +95,7 @@ Test your AI agents with aimock — no API keys, no network calls: [LangChain](h
9495

9596
## Switching from other tools?
9697

97-
Step-by-step migration guides: [MSW](https://aimock.copilotkit.dev/migrate-from-msw) · [VidaiMock](https://aimock.copilotkit.dev/migrate-from-vidaimock) · [mock-llm](https://aimock.copilotkit.dev/migrate-from-mock-llm) · [piyook/llm-mock](https://aimock.copilotkit.dev/migrate-from-piyook) · [Python mocks](https://aimock.copilotkit.dev/migrate-from-python-mocks) · [Mokksy](https://aimock.copilotkit.dev/migrate-from-mokksy)
98+
Step-by-step migration guides: [MSW](https://aimock.copilotkit.dev/migrate-from-msw) · [VidaiMock](https://aimock.copilotkit.dev/migrate-from-vidaimock) · [mock-llm](https://aimock.copilotkit.dev/migrate-from-mock-llm) · [piyook/llm-mock](https://aimock.copilotkit.dev/migrate-from-piyook) · [Python mocks](https://aimock.copilotkit.dev/migrate-from-python-mocks) · [openai-responses](https://aimock.copilotkit.dev/migrate-from-openai-responses) · [Mokksy](https://aimock.copilotkit.dev/migrate-from-mokksy)
9899

99100
## Documentation
100101

docs/aws-bedrock/index.html

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -214,7 +214,7 @@ <h2>Fixture Examples</h2>
214214
<span class="prop">"response"</span>: {
215215
<span class="prop">"toolCalls"</span>: [{
216216
<span class="prop">"name"</span>: <span class="str">"get_weather"</span>,
217-
<span class="prop">"arguments"</span>: <span class="str">"{\"city\":\"SF\"}"</span>
217+
<span class="prop">"arguments"</span>: { <span class="prop">"city"</span>: <span class="str">"SF"</span> }
218218
}]
219219
}
220220
}

docs/chat-completions/index.html

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -120,9 +120,10 @@ <h2>Unit Test: Tool Calls</h2>
120120
<div class="code-block">
121121
<div class="code-block-header">tool-calls.test.ts <span class="lang-tag">ts</span></div>
122122
<pre><code><span class="fn">it</span>(<span class="str">"returns tool call in streaming mode"</span>, <span class="kw">async</span> () <span class="kw">=&gt;</span> {
123+
<span class="cm">// arguments accepts objects (auto-stringified) or JSON strings</span>
123124
<span class="op">mock</span>.<span class="fn">on</span>(
124125
{ <span class="prop">userMessage</span>: <span class="str">"weather"</span> },
125-
{ <span class="prop">toolCalls</span>: [{ <span class="prop">name</span>: <span class="str">"get_weather"</span>, <span class="prop">arguments</span>: <span class="str">'{"city":"SF"}'</span> }] }
126+
{ <span class="prop">toolCalls</span>: [{ <span class="prop">name</span>: <span class="str">"get_weather"</span>, <span class="prop">arguments</span>: { <span class="prop">city</span>: <span class="str">"SF"</span> } }] }
126127
);
127128

128129
<span class="kw">const</span> <span class="op">res</span> = <span class="kw">await</span> <span class="fn">fetch</span>(<span class="str">`${mock.url}/v1/chat/completions`</span>, {
@@ -147,7 +148,7 @@ <h2>Integration Test: Streaming SSE</h2>
147148
<div class="code-block-header">
148149
streaming-integration.test.ts <span class="lang-tag">ts</span>
149150
</div>
150-
<pre><code><span class="kw">import</span> { <span class="fn">createServer</span>, <span class="kw">type</span> <span class="type">ServerInstance</span> } <span class="kw">from</span> <span class="str">"@copilotkit/aimock/server"</span>;
151+
<pre><code><span class="kw">import</span> { <span class="fn">createServer</span>, <span class="kw">type</span> <span class="type">ServerInstance</span> } <span class="kw">from</span> <span class="str">"@copilotkit/aimock"</span>;
151152

152153
<span class="kw">const</span> <span class="op">instance</span> = <span class="kw">await</span> <span class="fn">createServer</span>(
153154
[{ <span class="prop">match</span>: { <span class="prop">userMessage</span>: <span class="str">"hello"</span> }, <span class="prop">response</span>: { <span class="prop">content</span>: <span class="str">"Hello! How can I help?"</span> } }],
@@ -192,7 +193,7 @@ <h2>JSON Fixture</h2>
192193
<span class="key">"response"</span>: {
193194
<span class="key">"toolCalls"</span>: [{
194195
<span class="key">"name"</span>: <span class="str">"get_weather"</span>,
195-
<span class="key">"arguments"</span>: <span class="str">"{\"city\":\"SF\"}"</span>
196+
<span class="key">"arguments"</span>: { <span class="key">"city"</span>: <span class="str">"SF"</span> }
196197
}]
197198
}
198199
}

docs/claude-messages/index.html

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -106,7 +106,7 @@ <h2>Unit Test: Tool Use</h2>
106106
<pre><code><span class="kw">const</span> <span class="op">toolFixture</span> = {
107107
<span class="prop">match</span>: { <span class="prop">userMessage</span>: <span class="str">"weather"</span> },
108108
<span class="prop">response</span>: {
109-
<span class="prop">toolCalls</span>: [{ <span class="prop">name</span>: <span class="str">"get_weather"</span>, <span class="prop">arguments</span>: <span class="str">'{"city":"NYC"}'</span> }]
109+
<span class="prop">toolCalls</span>: [{ <span class="prop">name</span>: <span class="str">"get_weather"</span>, <span class="prop">arguments</span>: { <span class="prop">city</span>: <span class="str">"NYC"</span> } }]
110110
},
111111
};
112112

docs/cohere/index.html

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -188,7 +188,7 @@ <h2>Fixture Examples</h2>
188188
"toolCalls": [
189189
{
190190
"name": "web_search",
191-
"arguments": "{\"query\":\"latest news\"}"
191+
"arguments": { "query": "latest news" }
192192
}
193193
]
194194
}

docs/docs/index.html

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -362,6 +362,8 @@ <h2>Switching from other tools?</h2>
362362
<a href="/migrate-from-mock-llm">mock-llm</a> <span class="sep">&middot;</span>
363363
<a href="/migrate-from-piyook">piyook/llm-mock</a> <span class="sep">&middot;</span>
364364
<a href="/migrate-from-python-mocks">Python mocks</a> <span class="sep">&middot;</span>
365+
<a href="/migrate-from-openai-responses">openai-responses</a>
366+
<span class="sep">&middot;</span>
365367
<a href="/migrate-from-mokksy">Mokksy</a>
366368
</div>
367369
</main>

docs/fixtures/index.html

Lines changed: 202 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -123,6 +123,13 @@ <h2>Match Fields</h2>
123123
<td>number</td>
124124
<td>Match on the Nth occurrence of this pattern</td>
125125
</tr>
126+
<tr>
127+
<td>endpoint</td>
128+
<td>string</td>
129+
<td>
130+
Restrict to endpoint type: chat, image, speech, transcription, video, embedding
131+
</td>
132+
</tr>
126133
<tr>
127134
<td>predicate</td>
128135
<td>function</td>
@@ -152,6 +159,11 @@ <h2>Response Types</h2>
152159
<td>toolCalls[], finishReason?</td>
153160
<td>Function call(s) with name + arguments</td>
154161
</tr>
162+
<tr>
163+
<td>Content + Tool Calls</td>
164+
<td>content, toolCalls[], reasoning?, finishReason?</td>
165+
<td>Text and tool calls in a single response</td>
166+
</tr>
155167
<tr>
156168
<td>Error</td>
157169
<td>error.message, error.type?, status?</td>
@@ -179,12 +191,99 @@ <h2>Response Types</h2>
179191
</tr>
180192
<tr>
181193
<td>Video</td>
182-
<td>video.url, video.duration?</td>
194+
<td>video.id, video.status, video.url?</td>
183195
<td>Generated video URL with async polling</td>
184196
</tr>
185197
</tbody>
186198
</table>
187199

200+
<div class="info-box">
201+
<p>
202+
<strong>Override fields:</strong> Text, Tool Call, and Content + Tool Calls responses
203+
also accept the override fields listed below (<code>id</code>, <code>model</code>,
204+
<code>usage</code>, <code>finishReason</code>, <code>role</code>,
205+
<code>systemFingerprint</code>, <code>created</code>).
206+
</p>
207+
</div>
208+
209+
<div class="info-box">
210+
<p>
211+
<strong>JSON auto-stringify:</strong> In fixture files and programmatic API,
212+
<code>arguments</code> and <code>content</code> fields accept both objects and strings.
213+
Objects are automatically stringified via <code>JSON.stringify()</code>. Use the object
214+
form for readability &mdash; no more escaped JSON strings.
215+
</p>
216+
</div>
217+
218+
<h2>Response Override Fields</h2>
219+
<p>
220+
Fixture responses can include optional fields to override auto-generated envelope values.
221+
These map correctly across all provider formats (OpenAI, Claude, Gemini, Responses API).
222+
</p>
223+
<table class="endpoint-table">
224+
<thead>
225+
<tr>
226+
<th>Field</th>
227+
<th>Type</th>
228+
<th>Description</th>
229+
</tr>
230+
</thead>
231+
<tbody>
232+
<tr>
233+
<td>id</td>
234+
<td>string</td>
235+
<td>Override auto-generated response ID</td>
236+
</tr>
237+
<tr>
238+
<td>created</td>
239+
<td>number</td>
240+
<td>Override Unix timestamp</td>
241+
</tr>
242+
<tr>
243+
<td>model</td>
244+
<td>string</td>
245+
<td>Override model name in response</td>
246+
</tr>
247+
<tr>
248+
<td>usage</td>
249+
<td>object</td>
250+
<td>
251+
Override token counts:
252+
<code>{ prompt_tokens, completion_tokens, total_tokens }</code>. Also accepts
253+
Anthropic field names (<code>input_tokens</code>, <code>output_tokens</code>) and
254+
Gemini field names (<code>promptTokenCount</code>,
255+
<code>candidatesTokenCount</code>, <code>totalTokenCount</code>). OpenAI Chat
256+
Completions includes usage in the response body; the Responses API uses a separate
257+
<code>response.usage</code> object. When omitted, token counts are auto-computed
258+
from content length
259+
</td>
260+
</tr>
261+
<tr>
262+
<td>finishReason</td>
263+
<td>string</td>
264+
<td>
265+
Override finish reason (default: "stop" or "tool_calls"). Provider mappings:
266+
<code>stop</code> &rarr; <code>end_turn</code> (Claude), <code>STOP</code> (Gemini);
267+
<code>tool_calls</code> &rarr; <code>tool_use</code> (Claude),
268+
<code>FUNCTION_CALL</code> (Gemini); <code>length</code> &rarr;
269+
<code>max_tokens</code> (Claude), <code>MAX_TOKENS</code> (Gemini);
270+
<code>content_filter</code> &rarr; <code>SAFETY</code> (Gemini),
271+
<code>failed</code> (Responses API)
272+
</td>
273+
</tr>
274+
<tr>
275+
<td>role</td>
276+
<td>string</td>
277+
<td>Override message role (default: "assistant")</td>
278+
</tr>
279+
<tr>
280+
<td>systemFingerprint</td>
281+
<td>string</td>
282+
<td>Add system_fingerprint to response</td>
283+
</tr>
284+
</tbody>
285+
</table>
286+
188287
<h2>Fixture Options</h2>
189288

190289
<table class="endpoint-table">
@@ -199,7 +298,7 @@ <h2>Fixture Options</h2>
199298
<tr>
200299
<td>latency</td>
201300
<td>number</td>
202-
<td>Milliseconds delay before first chunk</td>
301+
<td>Milliseconds delay between SSE chunks (streaming)</td>
203302
</tr>
204303
<tr>
205304
<td>chunkSize</td>
@@ -220,15 +319,16 @@ <h2>Fixture Options</h2>
220319
<td>streamingProfile</td>
221320
<td>object</td>
222321
<td>
223-
Streaming physics profile: <code>{ ttftMs, tps, jitter }</code>. See
322+
Streaming physics profile: <code>{ ttft, tps, jitter }</code>. See
224323
<a href="/streaming-physics">Streaming Physics</a>
225324
</td>
226325
</tr>
227326
<tr>
228327
<td>chaos</td>
229328
<td>object</td>
230329
<td>
231-
Per-fixture chaos config: <code>{ errorRate, latencyMs, ... }</code>. See
330+
Per-fixture chaos config: <code>{ dropRate, malformedRate, disconnectRate }</code>.
331+
See
232332
<a href="/chaos-testing">Chaos Testing</a>
233333
</td>
234334
</tr>
@@ -261,8 +361,8 @@ <h3>Programmatically</h3>
261361
<span class="op">mock</span>.<span class="fn">onEmbedding</span>(<span class="str">"my text"</span>, { <span class="prop">embedding</span>: [<span class="num">0.1</span>, <span class="num">0.2</span>] });
262362
<span class="op">mock</span>.<span class="fn">onImage</span>(<span class="str">"sunset"</span>, { <span class="prop">image</span>: { <span class="prop">url</span>: <span class="str">"https://example.com/sunset.png"</span> } });
263363
<span class="op">mock</span>.<span class="fn">onSpeech</span>(<span class="str">"hello"</span>, { <span class="prop">audio</span>: <span class="str">"SGVsbG8="</span> });
264-
<span class="op">mock</span>.<span class="fn">onTranscription</span>(<span class="str">"audio.mp3"</span>, { <span class="prop">transcription</span>: { <span class="prop">text</span>: <span class="str">"Hello"</span> } });
265-
<span class="op">mock</span>.<span class="fn">onVideo</span>(<span class="str">"cats"</span>, { <span class="prop">video</span>: { <span class="prop">url</span>: <span class="str">"https://example.com/cats.mp4"</span> } });
364+
<span class="op">mock</span>.<span class="fn">onTranscription</span>({ <span class="prop">transcription</span>: { <span class="prop">text</span>: <span class="str">"Hello"</span> } });
365+
<span class="op">mock</span>.<span class="fn">onVideo</span>(<span class="str">"cats"</span>, { <span class="prop">video</span>: { <span class="prop">id</span>: <span class="str">"vid-1"</span>, <span class="prop">status</span>: <span class="str">"completed"</span>, <span class="prop">url</span>: <span class="str">"https://example.com/cats.mp4"</span> } });
266366
<span class="op">mock</span>.<span class="fn">onJsonOutput</span>(<span class="str">"data"</span>, { <span class="prop">key</span>: <span class="str">"value"</span> });
267367
<span class="op">mock</span>.<span class="fn">onToolResult</span>(<span class="str">"call_123"</span>, { <span class="prop">content</span>: <span class="str">"Done"</span> });
268368

@@ -305,6 +405,102 @@ <h2>Routing Rules</h2>
305405
programmatic registration for predicate-based routing.
306406
</p>
307407
</div>
408+
409+
<h2>Provider Support Matrix</h2>
410+
<table class="endpoint-table">
411+
<thead>
412+
<tr>
413+
<th>Feature</th>
414+
<th>OpenAI Chat</th>
415+
<th>OpenAI Responses</th>
416+
<th>Claude</th>
417+
<th>Gemini</th>
418+
<th>Bedrock</th>
419+
<th>Azure</th>
420+
<th>Ollama</th>
421+
<th>Cohere</th>
422+
</tr>
423+
</thead>
424+
<tbody>
425+
<tr>
426+
<td>Text</td>
427+
<td>Yes</td>
428+
<td>Yes</td>
429+
<td>Yes</td>
430+
<td>Yes</td>
431+
<td>Yes</td>
432+
<td>Yes</td>
433+
<td>Yes</td>
434+
<td>Yes</td>
435+
</tr>
436+
<tr>
437+
<td>Tool Calls</td>
438+
<td>Yes</td>
439+
<td>Yes</td>
440+
<td>Yes</td>
441+
<td>Yes</td>
442+
<td>Yes</td>
443+
<td>Yes</td>
444+
<td>Yes</td>
445+
<td>Yes</td>
446+
</tr>
447+
<tr>
448+
<td>Content + Tool Calls</td>
449+
<td>Yes</td>
450+
<td>Yes</td>
451+
<td>Yes</td>
452+
<td>Yes</td>
453+
<td>Yes</td>
454+
<td>Yes</td>
455+
<td>Yes</td>
456+
<td>Yes</td>
457+
</tr>
458+
<tr>
459+
<td>Streaming</td>
460+
<td>SSE</td>
461+
<td>SSE</td>
462+
<td>SSE</td>
463+
<td>SSE</td>
464+
<td>Binary EventStream</td>
465+
<td>SSE</td>
466+
<td>NDJSON</td>
467+
<td>SSE</td>
468+
</tr>
469+
<tr>
470+
<td>Reasoning</td>
471+
<td>Yes</td>
472+
<td>Yes</td>
473+
<td>Yes</td>
474+
<td>Yes</td>
475+
<td>Yes</td>
476+
<td>Yes</td>
477+
<td>&mdash;</td>
478+
<td>&mdash;</td>
479+
</tr>
480+
<tr>
481+
<td>Web Searches</td>
482+
<td>&mdash;</td>
483+
<td>Yes</td>
484+
<td>&mdash;</td>
485+
<td>&mdash;</td>
486+
<td>&mdash;</td>
487+
<td>&mdash;</td>
488+
<td>&mdash;</td>
489+
<td>&mdash;</td>
490+
</tr>
491+
<tr>
492+
<td>Response Overrides</td>
493+
<td>Yes</td>
494+
<td>Yes</td>
495+
<td>Yes</td>
496+
<td>Yes</td>
497+
<td>&mdash;</td>
498+
<td>Yes</td>
499+
<td>&mdash;</td>
500+
<td>&mdash;</td>
501+
</tr>
502+
</tbody>
503+
</table>
308504
</main>
309505
<aside class="page-toc" id="page-toc"></aside>
310506
</div>

docs/gemini/index.html

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,7 @@ <h2>Unit Test: Tool Call</h2>
122122
<pre><code><span class="kw">const</span> <span class="op">toolFixture</span> = {
123123
<span class="prop">match</span>: { <span class="prop">userMessage</span>: <span class="str">"weather"</span> },
124124
<span class="prop">response</span>: {
125-
<span class="prop">toolCalls</span>: [{ <span class="prop">name</span>: <span class="str">"get_weather"</span>, <span class="prop">arguments</span>: <span class="str">'{"city":"NYC"}'</span> }]
125+
<span class="prop">toolCalls</span>: [{ <span class="prop">name</span>: <span class="str">"get_weather"</span>, <span class="prop">arguments</span>: { <span class="prop">city</span>: <span class="str">"NYC"</span> } }]
126126
},
127127
};
128128

docs/integrate-adk/index.html

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -138,7 +138,7 @@ <h2>Function Calling</h2>
138138
<span class="prop">"toolCalls"</span>: [
139139
{
140140
<span class="prop">"name"</span>: <span class="str">"get_weather"</span>,
141-
<span class="prop">"arguments"</span>: <span class="str">"{\"city\":\"San Francisco\",\"unit\":\"fahrenheit\"}"</span>
141+
<span class="prop">"arguments"</span>: { <span class="prop">"city"</span>: <span class="str">"San Francisco"</span>, <span class="prop">"unit"</span>: <span class="str">"fahrenheit"</span> }
142142
}
143143
]
144144
}

0 commit comments

Comments
 (0)