Skip to content

Commit 2d5bd72

Browse files
tombeckenhamtombeckenham
committed
feat: support image + video responses and queue polling realism in fal handler
Closes #170. The fal handler previously rejected non-audio fixtures with HTTP 500 ("Fixture response is not an audio type") and always returned COMPLETED on submit. This broke any test that exercised fal image (flux, stable-diffusion) or video (kling, wan) endpoints, or that polled queue status realistically. Changes: - Rename src/fal-audio.ts → src/fal.ts (and matching test file). The handler now dispatches over AudioResponse | ImageResponse | VideoResponse and wraps each in the matching fal envelope. - Add onFalImage(prompt, ImageResponse, model?) and onFalVideo(prompt, VideoResponse, model?) helpers parallel to onFalAudio. - New endpoint discriminators "fal-image", "fal-video" with router support so generic fixtures pick a compatible response type. - Queue progression state machine: configure via MockServerOptions.falQueue to advance IN_QUEUE → IN_PROGRESS → COMPLETED over N status polls. Status responses include logs[] (one entry per transition) and metrics.inference_time once completed. Defaults preserve back-compat behavior (instant complete). - Cancel before completion returns 200 { status: "CANCELLED" }; after still returns 400 { status: "ALREADY_COMPLETED" }. - Result fetch before completion returns 202 with current status (instead of the legacy success path that exposed unfinished data). - Model-id-based endpoint inference picks fal-audio / fal-image / fal-video for fixture matching when callers don't tag fixtures. Tests: 10 new cases covering image + video lifecycle, sync run for both, polling progression with logs/metrics, cancel-before-complete, and audio/image endpoint isolation. All 2749 tests pass. Docs: new docs/fal/index.html, sidebar entry, README "Multimedia APIs" line. Deferred (separate follow-ups): - rest.alpha.fal.ai/storage/upload/initiate flow. - Hostname-aware proxy routing (fal.run / queue.fal.run / rest.fal.ai).
1 parent 78a629d commit 2d5bd72

12 files changed

Lines changed: 1399 additions & 632 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ Run them all on one port with `npx @copilotkit/aimock --config aimock.json`, or
5151
- **[Record & Replay](https://aimock.copilotkit.dev/record-replay)** — Proxy real APIs, save as fixtures, replay deterministically forever
5252
- **[Multi-turn Conversations](https://aimock.copilotkit.dev/multi-turn)** — Record and replay multi-turn traces with tool rounds; match distinct turns via `turnIndex`, `hasToolResult`, `toolCallId`, `sequenceIndex`, or custom predicates
5353
- **[12 LLM Providers](https://aimock.copilotkit.dev/docs)** — OpenAI Chat, OpenAI Responses, OpenAI Realtime, Claude, Gemini, Gemini Live, Gemini Interactions, Azure, Bedrock, Vertex AI, Ollama, Cohere — full streaming support
54-
- **Multimedia APIs** — [image generation](https://aimock.copilotkit.dev/images) (DALL-E, Imagen), [text-to-speech](https://aimock.copilotkit.dev/speech), [audio transcription](https://aimock.copilotkit.dev/transcription), [video generation](https://aimock.copilotkit.dev/video)
54+
- **Multimedia APIs** — [image generation](https://aimock.copilotkit.dev/images) (DALL-E, Imagen), [text-to-speech](https://aimock.copilotkit.dev/speech), [audio transcription](https://aimock.copilotkit.dev/transcription), [video generation](https://aimock.copilotkit.dev/video), [fal.ai](https://aimock.copilotkit.dev/fal) (image/video/audio with queue lifecycle)
5555
- **[MCP](https://aimock.copilotkit.dev/mcp-mock) / [A2A](https://aimock.copilotkit.dev/a2a-mock) / [AG-UI](https://aimock.copilotkit.dev/agui-mock) / [Vector](https://aimock.copilotkit.dev/vector-mock)** — Mock every protocol your AI agents use
5656
- **[Chaos Testing](https://aimock.copilotkit.dev/chaos-testing)** — 500 errors, malformed JSON, mid-stream disconnects at any probability
5757
- **[Drift Detection](https://aimock.copilotkit.dev/drift-detection)** — Daily CI validation against real APIs

‎docs/fal/index.html‎

Lines changed: 289 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,289 @@
1+
<!doctype html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="UTF-8" />
5+
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
6+
<title>fal.ai — aimock</title>
7+
<link rel="icon" type="image/svg+xml" href="../favicon.svg" />
8+
<link rel="preconnect" href="https://fonts.googleapis.com" />
9+
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
10+
<link
11+
href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:ital,wght@0,300;0,400;0,500;0,600;0,700;1,400&family=Instrument+Sans:wght@400;500;600;700&display=swap"
12+
rel="stylesheet"
13+
/>
14+
<link rel="stylesheet" href="../style.css" />
15+
<script src="/pixels.js" defer></script>
16+
</head>
17+
<body>
18+
<nav class="top-nav">
19+
<div class="nav-inner">
20+
<div style="display: flex; align-items: center; gap: 1rem">
21+
<button
22+
class="sidebar-toggle"
23+
onclick="document.querySelector('.sidebar').classList.toggle('open')"
24+
aria-label="Toggle sidebar"
25+
>
26+
&#9776;
27+
</button>
28+
<a href="/" class="nav-brand"> <span class="prompt">$</span> aimock </a>
29+
</div>
30+
<ul class="nav-links">
31+
<li><a href="/">Home</a></li>
32+
<li><a href="/docs" style="color: var(--accent)">Docs</a></li>
33+
<li>
34+
<a href="https://github.com/CopilotKit/aimock" class="gh-link" target="_blank"
35+
><svg width="16" height="16" viewBox="0 0 16 16" fill="currentColor">
36+
<path
37+
d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"
38+
/>
39+
</svg>
40+
GitHub</a
41+
>
42+
</li>
43+
</ul>
44+
</div>
45+
</nav>
46+
47+
<div class="docs-layout">
48+
<aside class="sidebar" id="sidebar"></aside>
49+
50+
<main class="docs-content">
51+
<h1>fal.ai</h1>
52+
<p class="lead">
53+
Mock fal.ai for image, video, and audio generation. aimock speaks fal's queue
54+
(<code>queue.fal.run</code>) and synchronous (<code>fal.run</code>) APIs, so callers using
55+
<code>@fal-ai/client</code> or <code>@tanstack/ai-fal</code> can be pointed at aimock
56+
without changing application code.
57+
</p>
58+
59+
<h2>Endpoints</h2>
60+
<table class="endpoint-table">
61+
<thead>
62+
<tr>
63+
<th>Method</th>
64+
<th>Path</th>
65+
<th>Purpose</th>
66+
</tr>
67+
</thead>
68+
<tbody>
69+
<tr>
70+
<td>POST</td>
71+
<td>/fal/queue/submit/{owner}/{model}</td>
72+
<td>
73+
Queue submit &mdash; returns <code>request_id</code> + status/response/cancel URLs
74+
</td>
75+
</tr>
76+
<tr>
77+
<td>GET</td>
78+
<td>/fal/queue/requests/{id}/status</td>
79+
<td>Poll job status (IN_QUEUE / IN_PROGRESS / COMPLETED / CANCELLED)</td>
80+
</tr>
81+
<tr>
82+
<td>GET</td>
83+
<td>/fal/queue/requests/{id}</td>
84+
<td>Fetch result once COMPLETED (else 202 with current status)</td>
85+
</tr>
86+
<tr>
87+
<td>PUT / DELETE</td>
88+
<td>/fal/queue/requests/{id}/cancel</td>
89+
<td>
90+
Cancel pending job &mdash; CANCELLED before completion, ALREADY_COMPLETED after
91+
</td>
92+
</tr>
93+
<tr>
94+
<td>POST</td>
95+
<td>/fal/run/{owner}/{model}</td>
96+
<td>Synchronous run &mdash; returns the result envelope directly, no queue</td>
97+
</tr>
98+
</tbody>
99+
</table>
100+
101+
<h2>Fixture Helpers</h2>
102+
<p>
103+
Three response shapes are supported. The handler dispatches on the fixture response type
104+
and wraps it in the matching fal envelope.
105+
</p>
106+
107+
<div class="code-block">
108+
<div class="code-block-header">fal.test.ts <span class="lang-tag">ts</span></div>
109+
<pre><code><span class="kw">import</span> { <span class="type">LLMock</span> } <span class="kw">from</span> <span class="str">"@copilotkit/aimock"</span>;
110+
111+
<span class="kw">const</span> <span class="op">mock</span> = <span class="kw">new</span> <span class="type">LLMock</span>({ <span class="prop">port</span>: <span class="num">0</span> });
112+
113+
<span class="cm">// Audio (e.g. fal-ai/stable-audio, fal-ai/elevenlabs/music)</span>
114+
<span class="op">mock</span>.<span class="fn">onFalAudio</span>(<span class="str">"drum loop"</span>, { <span class="prop">audio</span>: <span class="str">"SGVsbG8="</span>, <span class="prop">format</span>: <span class="str">"mp3"</span> });
115+
116+
<span class="cm">// Image (e.g. fal-ai/flux/dev, fal-ai/stable-diffusion)</span>
117+
<span class="op">mock</span>.<span class="fn">onFalImage</span>(<span class="str">"flux test"</span>, {
118+
<span class="prop">images</span>: [{ <span class="prop">url</span>: <span class="str">"https://mock.fal.media/files/x.png"</span> }],
119+
});
120+
121+
<span class="cm">// Video (e.g. fal-ai/kling-video, fal-ai/wan-2.5)</span>
122+
<span class="op">mock</span>.<span class="fn">onFalVideo</span>(<span class="str">"kling test"</span>, {
123+
<span class="prop">video</span>: { <span class="prop">id</span>: <span class="str">"v1"</span>, <span class="prop">status</span>: <span class="str">"completed"</span>, <span class="prop">url</span>: <span class="str">"https://mock.fal.media/files/clip.mp4"</span> },
124+
});
125+
126+
<span class="kw">await</span> <span class="op">mock</span>.<span class="fn">start</span>();</code></pre>
127+
</div>
128+
129+
<h2>Response Envelopes</h2>
130+
<p>
131+
Each fixture response is translated into fal's wire shape on the way out. The translations
132+
mirror what the real fal API returns for each model family.
133+
</p>
134+
135+
<div class="code-block">
136+
<div class="code-block-header">audio response <span class="lang-tag">json</span></div>
137+
<pre><code>{
138+
<span class="key">"audio"</span>: {
139+
<span class="key">"url"</span>: <span class="str">"https://mock.fal.media/files/generated_audio.mp3"</span>,
140+
<span class="key">"content_type"</span>: <span class="str">"audio/mpeg"</span>,
141+
<span class="key">"file_name"</span>: <span class="str">"generated_audio.mp3"</span>,
142+
<span class="key">"file_size"</span>: <span class="num">5</span>
143+
}
144+
}</code></pre>
145+
</div>
146+
147+
<div class="code-block">
148+
<div class="code-block-header">image response <span class="lang-tag">json</span></div>
149+
<pre><code>{
150+
<span class="key">"images"</span>: [
151+
{ <span class="key">"url"</span>: <span class="str">"..."</span>, <span class="key">"width"</span>: <span class="num">1024</span>, <span class="key">"height"</span>: <span class="num">1024</span>, <span class="key">"content_type"</span>: <span class="str">"image/png"</span> }
152+
],
153+
<span class="key">"timings"</span>: { <span class="key">"inference"</span>: <span class="num">0</span> },
154+
<span class="key">"seed"</span>: <span class="num">0</span>,
155+
<span class="key">"has_nsfw_concepts"</span>: [<span class="kw">false</span>],
156+
<span class="key">"prompt"</span>: <span class="str">"flux test"</span>
157+
}</code></pre>
158+
</div>
159+
160+
<div class="code-block">
161+
<div class="code-block-header">video response <span class="lang-tag">json</span></div>
162+
<pre><code>{
163+
<span class="key">"video"</span>: {
164+
<span class="key">"url"</span>: <span class="str">"https://mock.fal.media/files/clip.mp4"</span>,
165+
<span class="key">"content_type"</span>: <span class="str">"video/mp4"</span>,
166+
<span class="key">"file_name"</span>: <span class="str">"clip.mp4"</span>,
167+
<span class="key">"file_size"</span>: <span class="num">0</span>
168+
},
169+
<span class="key">"seed"</span>: <span class="num">0</span>
170+
}</code></pre>
171+
</div>
172+
173+
<h2>Queue Lifecycle</h2>
174+
<p>
175+
By default each submitted job is created in <code>COMPLETED</code> state &mdash; status
176+
polls and result fetches return immediately. This keeps tests fast and matches the legacy
177+
fal-handler shape.
178+
</p>
179+
<p>
180+
To exercise code that polls and reacts to intermediate states, configure
181+
<code>falQueue</code> with positive poll thresholds. The job advances
182+
<code>IN_QUEUE &rarr; IN_PROGRESS &rarr; COMPLETED</code> once each
183+
<code>/status</code> (or <code>/{id}</code>) call has been made the configured number of
184+
times.
185+
</p>
186+
187+
<div class="code-block">
188+
<div class="code-block-header">
189+
queue-progression.test.ts <span class="lang-tag">ts</span>
190+
</div>
191+
<pre><code><span class="kw">const</span> <span class="op">mock</span> = <span class="kw">new</span> <span class="type">LLMock</span>({
192+
<span class="prop">port</span>: <span class="num">0</span>,
193+
<span class="prop">falQueue</span>: { <span class="prop">pollsBeforeInProgress</span>: <span class="num">1</span>, <span class="prop">pollsBeforeCompleted</span>: <span class="num">2</span> },
194+
});
195+
<span class="op">mock</span>.<span class="fn">onFalImage</span>(<span class="str">"flux"</span>, { <span class="prop">images</span>: [{ <span class="prop">url</span>: <span class="str">"..."</span> }] });
196+
197+
<span class="cm">// Submit → IN_QUEUE with queue_position: 1</span>
198+
<span class="cm">// 1st status poll → IN_PROGRESS</span>
199+
<span class="cm">// 2nd status poll → COMPLETED with metrics.inference_time</span></code></pre>
200+
</div>
201+
202+
<p>
203+
Status responses always include a <code>logs</code> array (one entry per state transition)
204+
and, once <code>COMPLETED</code>, a <code>metrics.inference_time</code>
205+
field measuring wall-clock elapsed since submit.
206+
</p>
207+
208+
<h2>Cancellation</h2>
209+
<ul>
210+
<li>
211+
Cancel before completion &rarr; <code>200 { status: "CANCELLED" }</code>. Subsequent
212+
status polls keep reporting <code>CANCELLED</code>; result fetch returns
213+
<code>202</code> with status.
214+
</li>
215+
<li>Cancel after completion &rarr; <code>400 { status: "ALREADY_COMPLETED" }</code>.</li>
216+
<li>Unknown <code>request_id</code> &rarr; <code>404 { status: "NOT_FOUND" }</code>.</li>
217+
</ul>
218+
219+
<h2>Model Inference</h2>
220+
<p>
221+
When a fixture doesn't specify <code>endpoint</code>, aimock guesses the kind from the
222+
model path so it can pick a compatible fixture:
223+
</p>
224+
<ul>
225+
<li>
226+
Audio &mdash; matches <code>audio</code>, <code>music</code>, <code>tts</code>,
227+
<code>speech</code>, <code>voice</code>, <code>sound</code>, <code>lyria</code>.
228+
</li>
229+
<li>
230+
Video &mdash; matches <code>video</code>, <code>kling</code>, <code>wan</code>,
231+
<code>veo</code>, <code>sora</code>, <code>hunyuan</code>, <code>motion</code>,
232+
<code>ffmpeg</code>.
233+
</li>
234+
<li>
235+
Image &mdash; default for anything else (e.g. <code>flux</code>,
236+
<code>nano-banana</code>).
237+
</li>
238+
</ul>
239+
240+
<h2>Test Isolation</h2>
241+
<p>
242+
Queue state is keyed by <code>X-Test-Id</code> header (falls back to a shared bucket).
243+
Concurrent tests submitting against the same model never collide on
244+
<code>request_id</code>.
245+
</p>
246+
247+
<h2>Record &amp; Replay</h2>
248+
<p>
249+
Unmatched fal requests can proxy to the real fal upstream and record the response. Add
250+
<code>fal</code> to the providers map in <code>RecordConfig</code>:
251+
</p>
252+
253+
<div class="code-block">
254+
<div class="code-block-header">record-fal.ts <span class="lang-tag">ts</span></div>
255+
<pre><code><span class="kw">const</span> <span class="op">mock</span> = <span class="kw">new</span> <span class="type">LLMock</span>({
256+
<span class="prop">record</span>: {
257+
<span class="prop">providers</span>: { <span class="prop">fal</span>: <span class="str">"https://queue.fal.run"</span> },
258+
<span class="prop">fixturePath</span>: <span class="str">"./fixtures/recorded"</span>,
259+
},
260+
});</code></pre>
261+
</div>
262+
263+
<h2>Out of Scope</h2>
264+
<p>The following fal surfaces are not currently mocked (track in GitHub issues):</p>
265+
<ul>
266+
<li>File-blob uploads via <code>rest.alpha.fal.ai/storage/upload/initiate</code>.</li>
267+
<li>
268+
Hostname-aware proxy routing for callers that hit
269+
<code>fal.run</code> / <code>queue.fal.run</code> directly without rewriting URLs.
270+
</li>
271+
</ul>
272+
</main>
273+
<aside class="page-toc" id="page-toc"></aside>
274+
</div>
275+
<footer class="docs-footer">
276+
<div class="footer-inner">
277+
<div class="footer-left"><span>$</span> aimock &middot; MIT License</div>
278+
<ul class="footer-links">
279+
<li><a href="https://github.com/CopilotKit/aimock" target="_blank">GitHub</a></li>
280+
<li>
281+
<a href="https://www.npmjs.com/package/@copilotkit/aimock" target="_blank">npm</a>
282+
</li>
283+
</ul>
284+
</div>
285+
</footer>
286+
<script src="../sidebar.js"></script>
287+
<script src="../cli-tabs.js"></script>
288+
</body>
289+
</html>

‎docs/sidebar.js‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@
3636
{ label: "Text-to-Speech", href: "/speech" },
3737
{ label: "Audio Transcription", href: "/transcription" },
3838
{ label: "Video Generation", href: "/video" },
39+
{ label: "fal.ai (Image/Video/Audio)", href: "/fal" },
3940
],
4041
},
4142
{

0 commit comments

Comments
 (0)