Skip to content

Commit 7c8eb4d

Browse files
authored
feat: context-based fixture routing via X-AIMock-Context header (#226)
## Summary - Adds `X-AIMock-Context` header support for scoping fixtures per integration — fixtures with `match.context` only match requests carrying that exact context value; fixtures without `context` remain shared - Injects context extraction in all 21 handler files (28 call sites) — HTTP handlers via `getContext(req)`, WebSocket handlers via `upgradeHeaders` extraction - Enriches recorder to auto-capture context in `buildFixtureMatch()` and route recorded fixtures into context subdirectories in `persistFixture()` - Adds context validation in fixture-loader (type check, dedup key, discriminator) - 15 new tests across 5 test files covering matching, helper extraction, fixture loading, recording, and strict-mode interaction - Bumps version to 1.26.0 (includes timing-replay from #223) ## Test plan - [x] 3151 tests pass (90 files) - [x] TypeScript clean (`tsc --noEmit` — 0 errors) - [x] Prettier clean - [x] ESLint clean - [x] Pre-commit hooks pass on all commits - [x] CR loop: 7-agent Round 1 + confirmation round — 0 bucket (a) findings on context routing code
2 parents 75dbb72 + 754c48b commit 7c8eb4d

37 files changed

Lines changed: 390 additions & 21 deletions

CHANGELOG.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
## [Unreleased]
44

5+
## [1.26.0] - 2026-05-18
6+
57
### Added
68

79
- **Timing-aware recording and replay** — proxy recording captures per-frame
@@ -13,6 +15,12 @@
1315
sources (recorded timings, streaming profiles, global latency). Per-fixture
1416
`replaySpeed` override. Covers SSE, NDJSON, Bedrock EventStream, and
1517
WebSocket protocols.
18+
- **Context-based fixture routing**`X-AIMock-Context` header scopes fixtures per integration. Fixtures with `match.context` only match requests carrying that context; fixtures without `context` remain shared. Recorder auto-captures context and routes recorded fixtures into context subdirectories.
19+
20+
## [1.25.0] - 2026-05-18
21+
22+
### Added
23+
1624
- **Gemini `embedContent` endpoint**`POST /v1beta/models/{model}:embedContent`
1725
with deterministic fallback embeddings and fixture matching
1826
- **`/v1/images/edit` and `/v1/images/variations` endpoints** — multipart

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ Run them all on one port with `npx @copilotkit/aimock --config aimock.json`, or
5656
- **[MCP](https://aimock.copilotkit.dev/mcp-mock) / [A2A](https://aimock.copilotkit.dev/a2a-mock) / [AG-UI](https://aimock.copilotkit.dev/agui-mock) / [Vector](https://aimock.copilotkit.dev/vector-mock)** — Mock every protocol your AI agents use
5757
- **[Chaos Testing](https://aimock.copilotkit.dev/chaos-testing)** — 500 errors, malformed JSON, mid-stream disconnects at any probability
5858
- **Per-Request Strict Mode**`X-AIMock-Strict` header overrides the server-level `--strict` flag per request (`true`/`1` = strict, `false`/`0` = lenient)
59+
- **Context-Based Fixture Routing**`X-AIMock-Context` header scopes fixtures per integration; fixtures with `match.context` only match requests carrying that context, fixtures without it remain shared
5960
- **[Drift Detection](https://aimock.copilotkit.dev/drift-detection)** — Daily CI validation against real APIs
6061
- **[Streaming Physics](https://aimock.copilotkit.dev/streaming-physics)** — Configurable `ttft`, `tps`, and `jitter`
6162
- **[WebSocket APIs](https://aimock.copilotkit.dev/websocket)** — OpenAI Realtime (GA protocol with models: gpt-realtime, gpt-realtime-2, gpt-realtime-1.5, gpt-realtime-mini; transcription/translation via gpt-4o-transcribe, gpt-4o-mini-transcribe, whisper-1; image input; commentary phase), Responses WS, Gemini Live

docs/docs/index.html

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -290,6 +290,14 @@ <h2>What's New</h2>
290290
<p class="highlight-card-desc">One-line CI setup for mock-backed test suites</p>
291291
<a href="/github-action" class="highlight-card-cta">Setup <span>&rarr;</span></a>
292292
</div>
293+
294+
<div class="highlight-card">
295+
<span class="highlight-card-title">Context Routing</span>
296+
<p class="highlight-card-desc">
297+
Scope fixtures per integration with <code>X-AIMock-Context</code> header
298+
</p>
299+
<a href="/fixtures#context" class="highlight-card-cta">Docs <span>&rarr;</span></a>
300+
</div>
293301
</div>
294302

295303
<!-- ─── The Suite (compact table) ───────────────────────────── -->

docs/fixtures/index.html

Lines changed: 42 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -160,6 +160,16 @@ <h2>Match Fields</h2>
160160
their own fixture APIs rather than via this field
161161
</td>
162162
</tr>
163+
<tr>
164+
<td>context</td>
165+
<td>string</td>
166+
<td>
167+
Restrict to a named context via <code>X-AIMock-Context</code> header. Fixtures with
168+
<code>context</code> only match requests carrying that exact value; fixtures without
169+
<code>context</code> match any request. Same opt-in semantics as
170+
<code>endpoint</code>
171+
</td>
172+
</tr>
163173
<tr>
164174
<td>predicate</td>
165175
<td>function</td>
@@ -242,14 +252,15 @@ <h3>5. Validation warnings surface shadowing at load time</h3>
242252
<code>duplicate userMessage 'hello' — shadows fixture 0</code>, where
243253
<code>'hello'</code> is the duplicated message and <code>0</code> is the zero-based
244254
index of the earlier fixture being shadowed. This is advisory, not a hard error: the
245-
check now factors in <code>turnIndex</code>, <code>hasToolResult</code>, and
246-
<code>sequenceIndex</code> when deciding whether two fixtures truly collide, but it does
247-
<em>not</em> consider <code>toolCallId</code>, <code>model</code>, or
248-
<code>predicate</code>, so the warning may still fire when those discriminators are
249-
present. Treat it as advisory: if a runtime differentiator is in place, the fixtures
250-
won't actually shadow each other at match time. Only fixtures with no differentiator at
251-
all will truly shadow on match &mdash; that's the case where the second is never reached
252-
because the first wins. Safe to ignore in the former case; investigate in the latter.
255+
check now factors in <code>turnIndex</code>, <code>hasToolResult</code>,
256+
<code>context</code>, and <code>sequenceIndex</code> when deciding whether two fixtures
257+
truly collide, but it does <em>not</em> consider <code>toolCallId</code>,
258+
<code>model</code>, or <code>predicate</code>, so the warning may still fire when those
259+
discriminators are present. Treat it as advisory: if a runtime differentiator is in
260+
place, the fixtures won't actually shadow each other at match time. Only fixtures with
261+
no differentiator at all will truly shadow on match &mdash; that's the case where the
262+
second is never reached because the first wins. Safe to ignore in the former case;
263+
investigate in the latter.
253264
</li>
254265
<li>
255266
<strong>Catch-all not last</strong> &mdash; a fixture with an empty <code>match</code>
@@ -511,6 +522,29 @@ <h3>From a directory</h3>
511522
</p>
512523
</div>
513524

525+
<h3 id="context">Context-scoped fixtures</h3>
526+
<div class="code-block">
527+
<div class="code-block-header">
528+
fixtures/context-example.json <span class="lang-tag">json</span>
529+
</div>
530+
<pre><code>{
531+
<span class="key">"fixtures"</span>: [
532+
{
533+
<span class="key">"match"</span>: { <span class="key">"userMessage"</span>: <span class="str">"hello"</span>, <span class="key">"context"</span>: <span class="str">"langgraph-python"</span> },
534+
<span class="key">"response"</span>: { <span class="key">"content"</span>: <span class="str">"Hi from LangGraph!"</span> }
535+
},
536+
{
537+
<span class="key">"match"</span>: { <span class="key">"userMessage"</span>: <span class="str">"hello"</span> },
538+
<span class="key">"response"</span>: { <span class="key">"content"</span>: <span class="str">"Hi from the shared fallback!"</span> }
539+
}
540+
]
541+
}</code></pre>
542+
</div>
543+
<p>
544+
Requests with <code>X-AIMock-Context: langgraph-python</code> match the first fixture; all
545+
other requests fall through to the shared fixture.
546+
</p>
547+
514548
<h3>Programmatically</h3>
515549
<div class="code-block">
516550
<div class="code-block-header">programmatic.ts <span class="lang-tag">ts</span></div>

docs/multi-turn/index.html

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -291,7 +291,10 @@ <h3>Turn 2 &mdash; client runs the tool, sends the result</h3>
291291
</p>
292292
</div>
293293

294-
<h2>Choosing between sequenceIndex, toolCallId, turnIndex, hasToolResult, and predicate</h2>
294+
<h2>
295+
Choosing between sequenceIndex, toolCallId, turnIndex, hasToolResult, context, and
296+
predicate
297+
</h2>
295298
<p>Five mechanisms handle different shapes of &ldquo;the same prompt twice&rdquo;:</p>
296299

297300
<table class="endpoint-table">
@@ -341,6 +344,15 @@ <h2>Choosing between sequenceIndex, toolCallId, turnIndex, hasToolResult, and pr
341344
pinning a specific <code>tool_call_id</code>.
342345
</td>
343346
</tr>
347+
<tr>
348+
<td>Same user prompt, different response per integration or caller identity</td>
349+
<td><code>context</code></td>
350+
<td>
351+
Exact match on the <code>X-AIMock-Context</code> header. Fixtures with
352+
<code>context</code> only match requests carrying that value; fixtures without it
353+
remain shared. Stateless.
354+
</td>
355+
</tr>
344356
<tr>
345357
<td>
346358
Arbitrary inspection &mdash; message count, specific content at any position, custom
@@ -442,6 +454,15 @@ <h2 id="gotchas">Gotchas</h2>
442454
instance. See <a href="/sequential-responses">Sequential Responses</a> for when
443455
<code>sequenceIndex</code> is the right tool.
444456
</li>
457+
<li>
458+
<strong><code>context</code> is an additional discriminator, not a replacement.</strong>
459+
<code>context</code> scopes fixtures by integration identity (<code
460+
>X-AIMock-Context</code
461+
>
462+
header). It combines with all other match fields via AND. Two fixtures with the same
463+
<code>userMessage</code> but different <code>context</code>
464+
values are not duplicates &mdash; the validator accounts for this.
465+
</li>
445466
</ul>
446467
</main>
447468
<aside class="page-toc" id="page-toc"></aside>

docs/record-replay/index.html

Lines changed: 47 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -556,6 +556,49 @@ <h2 id="model-aware-recording">Model-Aware Recording</h2>
556556
});</code></pre>
557557
</div>
558558

559+
<h2 id="context-aware-recording">Context-Aware Recording</h2>
560+
<p>
561+
When a request carries an <code>X-AIMock-Context</code> header, the recorder automatically
562+
captures the context value in <code>match.context</code>. On replay, fixtures with
563+
<code>context</code> only match requests carrying that exact header value &mdash; fixtures
564+
without <code>context</code> remain shared across all callers.
565+
</p>
566+
567+
<h3>Directory routing</h3>
568+
<p>
569+
Without snapshot-style recording (<code>X-Test-Id</code>), recorded fixtures for a given
570+
context are written to a <code>&lt;fixturePath&gt;/&lt;context&gt;/</code> subdirectory:
571+
</p>
572+
573+
<div class="code-block">
574+
<div class="code-block-header">
575+
Context directory layout <span class="lang-tag">text</span>
576+
</div>
577+
<pre><code>fixtures/recorded/
578+
openai-2026-05-18T10-30-00-000Z-a1b2c3d4.json # no context (shared)
579+
langgraph-python/
580+
openai-2026-05-18T10-30-01-000Z-e5f6a7b8.json # context = langgraph-python
581+
crewai/
582+
openai-2026-05-18T10-30-02-000Z-c9d0e1f2.json # context = crewai</code></pre>
583+
</div>
584+
585+
<p>
586+
When <code>X-Test-Id</code> is also present, snapshot-style paths take precedence and the
587+
context is captured only in <code>match.context</code> within the fixture file, not in the
588+
directory structure.
589+
</p>
590+
591+
<h3>Sending <code>X-AIMock-Context</code></h3>
592+
<div class="code-block">
593+
<div class="code-block-header">Header example <span class="lang-tag">ts</span></div>
594+
<pre><code><span class="cm">// Set as a default header on your LLM client</span>
595+
<span class="kw">const</span> <span class="op">client</span> = <span class="kw">new</span> <span class="fn">OpenAI</span>({
596+
<span class="prop">baseURL</span>: <span class="str">"http://localhost:4010/v1"</span>,
597+
<span class="prop">apiKey</span>: <span class="str">"mock"</span>,
598+
<span class="prop">defaultHeaders</span>: { <span class="str">"X-AIMock-Context"</span>: <span class="str">"langgraph-python"</span> },
599+
});</code></pre>
600+
</div>
601+
559602
<h2 id="upstream-timeouts">Upstream Timeouts</h2>
560603

561604
<p>
@@ -858,7 +901,10 @@ <h2 id="recording-multi-turn-conversations">Recording Multi-Turn Conversations</
858901
}
859902
<span class="cm">// Chat/multimedia — key on the LAST user message only</span>
860903
<span class="kw">const</span> lastUser = <span class="fn">getLastMessageByRole</span>(request.messages, <span class="str">"user"</span>);
861-
<span class="kw">return</span> { <span class="prop">userMessage</span>: <span class="fn">getTextContent</span>(lastUser.content) };
904+
<span class="kw">const</span> match = { <span class="prop">userMessage</span>: <span class="fn">getTextContent</span>(lastUser.content) };
905+
<span class="cm">// Capture context from X-AIMock-Context header if present</span>
906+
<span class="kw">if</span> (request._context) match.<span class="prop">context</span> = request._context;
907+
<span class="kw">return</span> match;
862908
}</code></pre>
863909
</div>
864910

src/__tests__/fixture-loader.test.ts

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1006,6 +1006,30 @@ describe("validateFixtures", () => {
10061006
).toBe(true);
10071007
});
10081008

1009+
// --- match.context type checks ---
1010+
1011+
it("validateFixtures reports error for non-string context", () => {
1012+
const fixtures = [
1013+
makeFixture({ match: { userMessage: "test", context: 42 as unknown as string } }),
1014+
];
1015+
const results = validateFixtures(fixtures);
1016+
expect(
1017+
results.some((r) => r.severity === "error" && r.message.includes("must be a string")),
1018+
).toBe(true);
1019+
});
1020+
1021+
it("context is a valid discriminator", () => {
1022+
const fixtures = [
1023+
makeFixture({ match: { context: "x" } }),
1024+
makeFixture({ match: { userMessage: "hello" } }),
1025+
];
1026+
const results = validateFixtures(fixtures);
1027+
const catchAllWarnings = results.filter(
1028+
(r) => r.severity === "warning" && r.message.includes("catch-all"),
1029+
);
1030+
expect(catchAllWarnings).toHaveLength(0);
1031+
});
1032+
10091033
// --- Warning checks ---
10101034

10111035
it("warning: duplicate userMessage", () => {
@@ -1545,6 +1569,15 @@ describe("auto-stringify JSON objects in fixture entries", () => {
15451569
expect((fixture.response as TextResponse).content).toBe("Hello, world!");
15461570
});
15471571

1572+
it("entryToFixture preserves context field", () => {
1573+
const entry: FixtureFileEntry = {
1574+
match: { context: "my-ctx", userMessage: "hi" },
1575+
response: { content: "ok" },
1576+
};
1577+
const fixture = entryToFixture(entry);
1578+
expect(fixture.match.context).toBe("my-ctx");
1579+
});
1580+
15481581
it("passes systemMessage through entryToFixture", () => {
15491582
const entry: FixtureFileEntry = {
15501583
match: { userMessage: "test", systemMessage: "name=Atai" },

src/__tests__/helpers.test.ts

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
import http from "node:http";
12
import { describe, it, expect } from "vitest";
23
import {
34
generateId,
@@ -11,6 +12,7 @@ import {
1112
buildToolCallChunks,
1213
buildTextCompletion,
1314
buildToolCallCompletion,
15+
getContext,
1416
} from "../helpers.js";
1517

1618
describe("generateId", () => {
@@ -292,6 +294,32 @@ describe("buildToolCallChunks", () => {
292294
});
293295
});
294296

297+
describe("getContext", () => {
298+
it("returns header value", () => {
299+
const req = {
300+
headers: { "x-aimock-context": "langgraph-python" },
301+
} as unknown as http.IncomingMessage;
302+
expect(getContext(req)).toBe("langgraph-python");
303+
});
304+
305+
it("returns undefined when header absent", () => {
306+
const req = { headers: {} } as unknown as http.IncomingMessage;
307+
expect(getContext(req)).toBeUndefined();
308+
});
309+
310+
it("returns first value from array header", () => {
311+
const req = {
312+
headers: { "x-aimock-context": ["first", "second"] },
313+
} as unknown as http.IncomingMessage;
314+
expect(getContext(req)).toBe("first");
315+
});
316+
317+
it("returns undefined for empty string", () => {
318+
const req = { headers: { "x-aimock-context": "" } } as unknown as http.IncomingMessage;
319+
expect(getContext(req)).toBeUndefined();
320+
});
321+
});
322+
295323
describe("buildTextCompletion", () => {
296324
it("returns a valid chat.completion object", () => {
297325
const result = buildTextCompletion("Hello!", "gpt-4");

src/__tests__/recorder.test.ts

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ import * as os from "node:os";
55
import * as path from "node:path";
66
import type { Fixture, FixtureFile } from "../types.js";
77
import { createServer, type ServerInstance } from "../server.js";
8-
import { proxyAndRecord, type ProxyCapturedResponse } from "../recorder.js";
8+
import { proxyAndRecord, buildFixtureMatch, type ProxyCapturedResponse } from "../recorder.js";
99
import type { RecordConfig } from "../types.js";
1010
import { Logger } from "../logger.js";
1111
import { LLMock } from "../llmock.js";
@@ -4105,6 +4105,29 @@ describe("buildFixtureMatch model recording", () => {
41054105
});
41064106
});
41074107

4108+
// ---------------------------------------------------------------------------
4109+
// buildFixtureMatch context
4110+
// ---------------------------------------------------------------------------
4111+
4112+
describe("buildFixtureMatch context", () => {
4113+
it("captures _context in match criteria", () => {
4114+
const match = buildFixtureMatch({
4115+
model: "gpt-4o",
4116+
messages: [{ role: "user", content: "hello" }],
4117+
_context: "langgraph-python",
4118+
});
4119+
expect(match.context).toBe("langgraph-python");
4120+
});
4121+
4122+
it("omits context when _context is absent", () => {
4123+
const match = buildFixtureMatch({
4124+
model: "gpt-4o",
4125+
messages: [{ role: "user", content: "hello" }],
4126+
});
4127+
expect(match.context).toBeUndefined();
4128+
});
4129+
});
4130+
41084131
async function setupUpstreamAndRecorder(
41094132
upstreamFixtures: Fixture[],
41104133
providerKey: string = "openai",

src/__tests__/router.test.ts

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1107,6 +1107,43 @@ describe("matchFixture — hasToolResult", () => {
11071107
});
11081108
});
11091109

1110+
// ---------------------------------------------------------------------------
1111+
// matchFixture — context matching
1112+
// ---------------------------------------------------------------------------
1113+
1114+
describe("matchFixture — context matching", () => {
1115+
it("matches fixture with matching context", () => {
1116+
const fixture = makeFixture({ context: "foo" });
1117+
const req = makeReq({ _context: "foo" });
1118+
expect(matchFixture([fixture], req)).toBe(fixture);
1119+
});
1120+
1121+
it("skips fixture with non-matching context", () => {
1122+
const fixture = makeFixture({ context: "foo" });
1123+
const req = makeReq({ _context: "bar" });
1124+
expect(matchFixture([fixture], req)).toBeNull();
1125+
});
1126+
1127+
it("matches fixture without context regardless of request context", () => {
1128+
const fixture = makeFixture({});
1129+
const req = makeReq({ _context: "bar" });
1130+
expect(matchFixture([fixture], req)).toBe(fixture);
1131+
});
1132+
1133+
it("skips context fixture when request has no context", () => {
1134+
const fixture = makeFixture({ context: "foo" });
1135+
const req = makeReq();
1136+
expect(matchFixture([fixture], req)).toBeNull();
1137+
});
1138+
1139+
it("context fixture wins over shared when listed first", () => {
1140+
const contextual = makeFixture({ context: "foo" }, { content: "contextual" });
1141+
const shared = makeFixture({}, { content: "shared" });
1142+
const req = makeReq({ _context: "foo" });
1143+
expect(matchFixture([contextual, shared], req)).toBe(contextual);
1144+
});
1145+
});
1146+
11101147
// ---------------------------------------------------------------------------
11111148
// matchFixture — first-match-wins
11121149
// ---------------------------------------------------------------------------

0 commit comments

Comments
 (0)