Skip to content

Commit 8f720d3

Browse files
jpr5jpr5
committed
docs: update docs, DRIFT.md, and competitive matrix for launch gaps
1 parent 0598e81 commit 8f720d3

4 files changed

Lines changed: 109 additions & 17 deletions

File tree

‎DRIFT.md‎

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,13 @@ When a `critical` drift is detected:
7777
- OpenAI Responses API → `src/responses.ts` (`buildTextResponse`, `buildToolCallResponse`, `buildTextStreamEvents`, `buildToolCallStreamEvents`)
7878
- Anthropic Claude → `src/messages.ts` (`buildClaudeTextResponse`, `buildClaudeToolCallResponse`, `buildClaudeTextStreamEvents`, `buildClaudeToolCallStreamEvents`)
7979
- Google Gemini → `src/gemini.ts` (`buildGeminiTextResponse`, `buildGeminiToolCallResponse`, `buildGeminiTextStreamChunks`, `buildGeminiToolCallStreamChunks`)
80+
- Gemini embedContent → `src/gemini.ts` (embedContent response builder)
8081
- Gemini Interactions → `src/gemini-interactions.ts` (`buildInteractionsTextResponse`, `buildInteractionsToolCallResponse`, `buildInteractionsTextSSEEvents`, `buildInteractionsToolCallSSEEvents`)
82+
- OpenAI Image Edit → `src/images.ts` (multipart `/v1/images/edit` handler)
83+
- OpenAI Audio Translation → `src/transcription.ts` (multipart `/v1/audio/translations` handler)
84+
- Ollama Embeddings → `src/ollama.ts` (`/api/embeddings` response builder)
85+
- Cohere Embed → `src/cohere.ts` (`/v2/embed` response builder)
86+
- ElevenLabs TTS → `src/elevenlabs-audio.ts` (`/v1/text-to-speech/{voice_id}` response builder)
8187

8288
2. **Update the builder** — add or modify the field to match the real API shape.
8389

@@ -107,7 +113,22 @@ When a model is deprecated:
107113

108114
## WebSocket Drift Coverage
109115

110-
In addition to the 23 existing drift tests (20 HTTP response-shape + 3 model deprecation), WebSocket drift tests cover aimock's WS protocols (6 verified + 2 canary = 8 WS tests):
116+
In addition to the 23 existing drift tests (20 HTTP response-shape + 3 model deprecation), the following new endpoint coverage has been added:
117+
118+
### New Endpoint Drift Coverage
119+
120+
| Endpoint | Provider | Type | Status |
121+
| ---------------------------------------- | ------------- | ----------------- | ------- |
122+
| POST /v1beta/models/{model}:embedContent | Gemini | HTTP | Covered |
123+
| POST /v1/images/edit | OpenAI | HTTP (multipart) | Covered |
124+
| POST /v1/audio/translations | OpenAI | HTTP (multipart) | Covered |
125+
| POST /api/embeddings | Ollama | HTTP | Covered |
126+
| POST /v2/embed | Cohere | HTTP | Covered |
127+
| POST /v1/text-to-speech/{voice_id} | ElevenLabs | HTTP | Covered |
128+
| stream_options.include_usage | OpenAI | Streaming feature | Covered |
129+
| x-ratelimit-\* / Retry-After 429 | All providers | Response headers | Covered |
130+
131+
WebSocket drift tests cover aimock's WS protocols (6 verified + 2 canary = 8 WS tests):
111132

112133
### Gemini Interactions API (Beta)
113134

‎README.md‎

Lines changed: 13 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
https://github.com/user-attachments/assets/76815122-574a-48e1-b275-edae0a014667
44

5-
Mock infrastructure for AI application testing — LLM APIs, image generation, text-to-speech, transcription, audio generation, video generation, MCP tools, A2A agents, AG-UI event streams, vector databases, search, rerank, and moderation. One package, one port, zero dependencies.
5+
Mock infrastructure for AI application testing — LLM APIs, image generation, image editing, text-to-speech, transcription, audio translation, audio generation, video generation, embeddings, MCP tools, A2A agents, AG-UI event streams, vector databases, search, rerank, and moderation. One package, one port, zero dependencies.
66

77
## Quick Start
88

@@ -35,23 +35,23 @@ await mock.stop();
3535

3636
aimock mocks everything your AI app talks to:
3737

38-
| Tool | What it mocks | Docs |
39-
| -------------- | ---------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------- |
40-
| **LLMock** | OpenAI (Chat/Responses/Realtime GA+Beta), Claude, Gemini (REST/Live/Interactions), Bedrock, Azure, Vertex AI, Ollama, Cohere | [Providers](https://aimock.copilotkit.dev/docs) |
41-
| **MCPMock** | MCP tools, resources, prompts with session management | [MCP](https://aimock.copilotkit.dev/mcp-mock) |
42-
| **A2AMock** | Agent-to-agent protocol with SSE streaming | [A2A](https://aimock.copilotkit.dev/a2a-mock) |
43-
| **AGUIMock** | AG-UI agent-to-UI event streams for frontend testing | [AG-UI](https://aimock.copilotkit.dev/agui-mock) |
44-
| **VectorMock** | Pinecone, Qdrant, ChromaDB compatible endpoints | [Vector](https://aimock.copilotkit.dev/vector-mock) |
45-
| **Services** | Tavily search, Cohere rerank, OpenAI moderation | [Services](https://aimock.copilotkit.dev/services) |
38+
| Tool | What it mocks | Docs |
39+
| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------- |
40+
| **LLMock** | OpenAI (Chat/Responses/Realtime GA+Beta), Claude, Gemini (REST/Live/Interactions/Embeddings), Bedrock, Azure, Vertex AI, Ollama (chat/embeddings), Cohere (chat/embed), ElevenLabs TTS | [Providers](https://aimock.copilotkit.dev/docs) |
41+
| **MCPMock** | MCP tools, resources, prompts with session management | [MCP](https://aimock.copilotkit.dev/mcp-mock) |
42+
| **A2AMock** | Agent-to-agent protocol with SSE streaming | [A2A](https://aimock.copilotkit.dev/a2a-mock) |
43+
| **AGUIMock** | AG-UI agent-to-UI event streams for frontend testing | [AG-UI](https://aimock.copilotkit.dev/agui-mock) |
44+
| **VectorMock** | Pinecone, Qdrant, ChromaDB compatible endpoints | [Vector](https://aimock.copilotkit.dev/vector-mock) |
45+
| **Services** | Tavily search, Cohere rerank, OpenAI moderation, ElevenLabs TTS | [Services](https://aimock.copilotkit.dev/services) |
4646

4747
Run them all on one port with `npx @copilotkit/aimock --config aimock.json`, or use the programmatic API to compose exactly what you need.
4848

4949
## Features
5050

5151
- **[Record & Replay](https://aimock.copilotkit.dev/record-replay)** — Proxy real APIs, save as fixtures, replay deterministically forever
5252
- **[Multi-turn Conversations](https://aimock.copilotkit.dev/multi-turn)** — Record and replay multi-turn traces with tool rounds; match distinct turns via `turnIndex`, `hasToolResult`, `toolCallId`, `sequenceIndex`, `systemMessage` (gate on host-supplied agent context), or custom predicates
53-
- **[12 LLM Providers](https://aimock.copilotkit.dev/docs)** — OpenAI Chat, OpenAI Responses, OpenAI Realtime (GA + Beta shim), Claude, Gemini, Gemini Live, Gemini Interactions, Azure, Bedrock, Vertex AI, Ollama, Cohere — full streaming support
54-
- **Multimedia APIs** — [image generation](https://aimock.copilotkit.dev/images) (DALL-E, Imagen), [text-to-speech](https://aimock.copilotkit.dev/speech), [audio transcription](https://aimock.copilotkit.dev/transcription), [video generation](https://aimock.copilotkit.dev/video), [fal.ai](https://aimock.copilotkit.dev/fal-ai) (image / video / audio with queue lifecycle)
53+
- **[14 LLM Providers](https://aimock.copilotkit.dev/docs)** — OpenAI Chat, OpenAI Responses, OpenAI Realtime (GA + Beta shim), Claude, Gemini (REST + embedContent), Gemini Live, Gemini Interactions, Azure, Bedrock, Vertex AI, Ollama (chat + embeddings), Cohere (chat + embed), ElevenLabs TTS — full streaming support
54+
- **Multimedia APIs** — [image generation](https://aimock.copilotkit.dev/images) (DALL-E, Imagen), [image editing](https://aimock.copilotkit.dev/images) (/v1/images/edit), [text-to-speech](https://aimock.copilotkit.dev/speech) (OpenAI + ElevenLabs), [audio transcription](https://aimock.copilotkit.dev/transcription), [audio translation](https://aimock.copilotkit.dev/transcription) (/v1/audio/translations), [video generation](https://aimock.copilotkit.dev/video), [fal.ai](https://aimock.copilotkit.dev/fal-ai) (image / video / audio with queue lifecycle)
5555
- **[MCP](https://aimock.copilotkit.dev/mcp-mock) / [A2A](https://aimock.copilotkit.dev/a2a-mock) / [AG-UI](https://aimock.copilotkit.dev/agui-mock) / [Vector](https://aimock.copilotkit.dev/vector-mock)** — Mock every protocol your AI agents use
5656
- **[Chaos Testing](https://aimock.copilotkit.dev/chaos-testing)** — 500 errors, malformed JSON, mid-stream disconnects at any probability
5757
- **Per-Request Strict Mode** — `X-AIMock-Strict` header overrides the server-level `--strict` flag per request (`true`/`1` = strict, `false`/`0` = lenient)
@@ -62,6 +62,8 @@ Run them all on one port with `npx @copilotkit/aimock --config aimock.json`, or
6262
- **[Docker + Helm](https://aimock.copilotkit.dev/docker)** — Container image and Helm chart for CI/CD
6363
- **[Vitest & Jest Plugins](https://aimock.copilotkit.dev/test-plugins)** — Zero-config `useAimock()` with auto lifecycle and env patching
6464
- **[Response Overrides](https://aimock.copilotkit.dev/fixtures)** — Control `id`, `model`, `usage`, `finishReason` in fixture responses
65+
- **[Streaming Usage Chunks](https://aimock.copilotkit.dev/streaming-physics)** — `stream_options.include_usage` support emits a final chunk with token counts, matching OpenAI's streaming usage protocol
66+
- **[Rate Limiting Headers](https://aimock.copilotkit.dev/chaos-testing)** — `x-ratelimit-*` headers on every response and `Retry-After` on 429 errors for testing retry/backoff logic
6567
- **Zero dependencies** — Everything from Node.js builtins
6668

6769
## GitHub Action

‎docs/index.html‎

Lines changed: 42 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1496,8 +1496,8 @@ <h2 class="fade-in">Everything you need</h2>
14961496
<div class="feature-icon">&#128225;</div>
14971497
<h3>Every Major LLM Provider</h3>
14981498
<p>
1499-
OpenAI, Claude, Gemini, Gemini Interactions, Bedrock, Azure, Vertex AI, Ollama, Cohere
1500-
&mdash; full streaming and embeddings support for every provider.
1499+
OpenAI, Claude, Gemini, Gemini Interactions, Bedrock, Azure, Vertex AI, Ollama,
1500+
Cohere, ElevenLabs &mdash; full streaming and embeddings support for every provider.
15011501
</p>
15021502
</div>
15031503

@@ -1547,8 +1547,9 @@ <h3>Chaos Testing</h3>
15471547
<div class="feature-icon">&#127912;</div>
15481548
<h3>Multimedia APIs</h3>
15491549
<p>
1550-
Image generation, text-to-speech, audio transcription, non-speech audio generation,
1551-
and video generation &mdash; mock every multimedia endpoint with fixtures.
1550+
Image generation and editing, text-to-speech (OpenAI + ElevenLabs), audio
1551+
transcription and translation, non-speech audio generation, and video generation
1552+
&mdash; mock every multimedia endpoint with fixtures.
15521553
</p>
15531554
</div>
15541555

@@ -1680,7 +1681,7 @@ <h2 class="fade-in">How aimock compares</h2>
16801681
</tr>
16811682
<tr>
16821683
<td>Multi-provider support</td>
1683-
<td class="col-aimock"><span class="yes">13 providers &#10003;</span></td>
1684+
<td class="col-aimock"><span class="yes">14 providers &#10003;</span></td>
16841685
<td><span class="manual">manual</span></td>
16851686
<td>12 providers</td>
16861687
<td>OpenAI only</td>
@@ -1714,6 +1715,15 @@ <h2 class="fade-in">How aimock compares</h2>
17141715
<td><span class="no">&#10007;</span></td>
17151716
<td><span class="no">&#10007;</span></td>
17161717
</tr>
1718+
<tr>
1719+
<td>Image editing</td>
1720+
<td class="col-aimock"><span class="yes">Built-in &#10003;</span></td>
1721+
<td><span class="no">&#10007;</span></td>
1722+
<td><span class="no">&#10007;</span></td>
1723+
<td><span class="no">&#10007;</span></td>
1724+
<td><span class="no">&#10007;</span></td>
1725+
<td><span class="no">&#10007;</span></td>
1726+
</tr>
17171727
<tr>
17181728
<td>Text-to-Speech</td>
17191729
<td class="col-aimock"><span class="yes">Built-in &#10003;</span></td>
@@ -1732,6 +1742,15 @@ <h2 class="fade-in">How aimock compares</h2>
17321742
<td><span class="no">&#10007;</span></td>
17331743
<td><span class="no">&#10007;</span></td>
17341744
</tr>
1745+
<tr>
1746+
<td>Audio translation</td>
1747+
<td class="col-aimock"><span class="yes">Built-in &#10003;</span></td>
1748+
<td><span class="no">&#10007;</span></td>
1749+
<td><span class="no">&#10007;</span></td>
1750+
<td><span class="no">&#10007;</span></td>
1751+
<td><span class="no">&#10007;</span></td>
1752+
<td><span class="no">&#10007;</span></td>
1753+
</tr>
17351754
<tr>
17361755
<td>Non-speech audio</td>
17371756
<td class="col-aimock"><span class="yes">Built-in &#10003;</span></td>
@@ -1921,6 +1940,24 @@ <h2 class="fade-in">How aimock compares</h2>
19211940
<td><span class="no">&#10007;</span></td>
19221941
<td><span class="no">&#10007;</span></td>
19231942
</tr>
1943+
<tr>
1944+
<td>Streaming usage chunks</td>
1945+
<td class="col-aimock"><span class="yes">Built-in &#10003;</span></td>
1946+
<td><span class="no">&#10007;</span></td>
1947+
<td><span class="no">&#10007;</span></td>
1948+
<td><span class="no">&#10007;</span></td>
1949+
<td><span class="no">&#10007;</span></td>
1950+
<td><span class="no">&#10007;</span></td>
1951+
</tr>
1952+
<tr>
1953+
<td>Rate limiting headers</td>
1954+
<td class="col-aimock"><span class="yes">Built-in &#10003;</span></td>
1955+
<td><span class="no">&#10007;</span></td>
1956+
<td><span class="no">&#10007;</span></td>
1957+
<td><span class="no">&#10007;</span></td>
1958+
<td><span class="no">&#10007;</span></td>
1959+
<td><span class="no">&#10007;</span></td>
1960+
</tr>
19241961
<tr>
19251962
<td>Dependencies</td>
19261963
<td class="col-aimock"><span class="yes">Zero &#10003;</span></td>

‎scripts/update-competitive-matrix.ts‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,10 @@ const FEATURE_RULES: FeatureRule[] = [
115115
rowLabel: "Image generation",
116116
keywords: ["dall-e", "dalle", "/v1/images", "image generation", "imagen", "generate.*image"],
117117
},
118+
{
119+
rowLabel: "Image editing",
120+
keywords: ["/v1/images/edit", "image edit", "image editing", "inpainting", "edit.*image"],
121+
},
118122
{
119123
rowLabel: "Text-to-Speech",
120124
keywords: ["text-to-speech", "/v1/audio/speech", "audio generation", "tts endpoint", "tts api"],
@@ -129,6 +133,15 @@ const FEATURE_RULES: FeatureRule[] = [
129133
"transcription api",
130134
],
131135
},
136+
{
137+
rowLabel: "Audio translation",
138+
keywords: [
139+
"/v1/audio/translations",
140+
"audio translation",
141+
"translate.*audio",
142+
"audio.*translate",
143+
],
144+
},
132145
{
133146
rowLabel: "Non-speech audio",
134147
keywords: [
@@ -217,6 +230,20 @@ const FEATURE_RULES: FeatureRule[] = [
217230
"test.*framework.*integrat",
218231
],
219232
},
233+
{
234+
rowLabel: "Streaming usage chunks",
235+
keywords: [
236+
"stream_options",
237+
"include_usage",
238+
"streaming.*usage",
239+
"usage.*chunk",
240+
"usage.*stream",
241+
],
242+
},
243+
{
244+
rowLabel: "Rate limiting headers",
245+
keywords: ["x-ratelimit", "rate.limit.*header", "retry-after", "429.*retry", "rate.limiting"],
246+
},
220247
];
221248

222249
/** Maps competitor display names to their migration page paths (relative to docs/) */
@@ -302,6 +329,7 @@ function countProviders(text: string): number {
302329
["groq"],
303330
["together"],
304331
["llama"],
332+
["elevenlabs"],
305333
];
306334

307335
let count = 0;
@@ -399,6 +427,10 @@ function buildMigrationRowPatterns(rowLabel: string): string[] {
399427
"Realtime translate/whisper": ["Realtime translate/whisper", "Translate/Whisper"],
400428
"Realtime image input": ["Realtime image input"],
401429
"Realtime commentary phase": ["Realtime commentary phase", "Commentary phase"],
430+
"Image editing": ["Image editing", "Image edit"],
431+
"Audio translation": ["Audio translation", "Audio translations"],
432+
"Streaming usage chunks": ["Streaming usage chunks", "Streaming usage"],
433+
"Rate limiting headers": ["Rate limiting headers", "Rate limiting"],
402434
};
403435

404436
if (variants[rowLabel]) {

0 commit comments

Comments
 (0)