Skip to content

Commit a707ee4

Browse files
committed
feat(sdk/js): add responses FFI fallback
Use the existing chat_completions native command for Responses create and streaming calls when a CoreInterop transport is available, with HTTP fallback for server-backed operations. Keep FFI-created stored responses available through the same client instance and cover the behavior with unit tests.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 88f0a0d commit a707ee4

7 files changed

Lines changed: 885 additions & 29 deletions

File tree

sdk/js/README.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -252,11 +252,12 @@ for await (const chunk of audioClient.transcribeStreaming('/path/to/audio.wav'))
252252

253253
### Responses API
254254

255-
Use the Responses API client for OpenAI-compatible text, tool, streaming, stored-response, and vision workflows over the embedded web service:
255+
Use the Responses API client for OpenAI-compatible text, tool, streaming, stored-response, and vision workflows. Clients created from `FoundryLocalManager` or a model use native FFI for `create()` and `createStreaming()` when possible, with HTTP fallback when a web service URL is available.
256256

257257
```typescript
258258
import { createImageContentFromFile, getOutputText } from 'foundry-local-sdk';
259259

260+
// Optional: start the web service for HTTP fallback and server-backed list/get/delete/cancel operations.
260261
manager.startWebService();
261262

262263
const client = manager.createResponsesClient(model.id);
@@ -285,7 +286,7 @@ const visionResponse = await client.create([
285286
console.log(getOutputText(visionResponse));
286287
```
287288

288-
Vision helpers support `.png`, `.jpg`, `.jpeg`, `.gif`, `.webp`, and `.bmp` files. `createImageContentFromFile()` sends Foundry Local's server contract (`image_data` plus `media_type`); `createImageContentFromUrl()` sends `image_url` and lets the server infer the media type.
289+
Vision helpers support `.png`, `.jpg`, `.jpeg`, `.gif`, `.webp`, and `.bmp` files. `createImageContentFromFile()` sends Foundry Local's `image_data` plus `media_type` contract; the FFI path converts that to the chat-completions data URL shape internally. `createImageContentFromUrl()` sends `image_url` and lets the runtime infer the media type.
289290

290291
### Embedded Web Service
291292

sdk/js/src/detail/model.ts

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -196,10 +196,11 @@ export class Model implements IModel {
196196

197197
/**
198198
* Creates a ResponsesClient for interacting with the model via the Responses API.
199-
* @param baseUrl - The base URL of the Foundry Local web service.
199+
* Uses native FFI for create/createStreaming and falls back to HTTP when baseUrl is provided.
200+
* @param baseUrl - Optional base URL of the Foundry Local web service for HTTP fallback and server-backed operations.
200201
* @returns A ResponsesClient instance.
201202
*/
202-
public createResponsesClient(baseUrl: string): ResponsesClient {
203+
public createResponsesClient(baseUrl?: string): ResponsesClient {
203204
return this.selectedVariant.createResponsesClient(baseUrl);
204205
}
205-
}
206+
}

sdk/js/src/detail/modelVariant.ts

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -189,10 +189,11 @@ export class ModelVariant implements IModel {
189189

190190
/**
191191
* Creates a ResponsesClient for interacting with the model via the Responses API.
192-
* @param baseUrl - The base URL of the Foundry Local web service.
192+
* Uses native FFI for create/createStreaming and falls back to HTTP when baseUrl is provided.
193+
* @param baseUrl - Optional base URL of the Foundry Local web service for HTTP fallback and server-backed operations.
193194
* @returns A ResponsesClient instance.
194195
*/
195-
public createResponsesClient(baseUrl: string): ResponsesClient {
196-
return new ResponsesClient(baseUrl, this._modelInfo.id);
196+
public createResponsesClient(baseUrl?: string): ResponsesClient {
197+
return ResponsesClient.createWithCoreInterop(baseUrl, this._modelInfo.id, this.coreInterop);
197198
}
198199
}

sdk/js/src/foundryLocalManager.ts

Lines changed: 3 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -214,17 +214,12 @@ export class FoundryLocalManager {
214214

215215
/**
216216
* Creates a ResponsesClient for interacting with the Responses API.
217-
* The web service must be started first via `startWebService()`.
217+
* Uses native FFI for create/createStreaming and falls back to HTTP if the web
218+
* service has been started via `startWebService()`.
218219
* @param modelId - Optional default model ID for requests.
219220
* @returns A ResponsesClient instance.
220-
* @throws Error - If the web service is not running.
221221
*/
222222
public createResponsesClient(modelId?: string): ResponsesClient {
223-
if (this._urls.length === 0) {
224-
throw new Error(
225-
'Web service is not running. Call startWebService() before creating a ResponsesClient.'
226-
);
227-
}
228-
return new ResponsesClient(this._urls[0], modelId);
223+
return ResponsesClient.createWithCoreInterop(this._urls[0], modelId, this.coreInterop);
229224
}
230225
}

sdk/js/src/imodel.ts

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -36,11 +36,10 @@ export interface IModel {
3636
createLiveTranscriptionSession(): LiveAudioTranscriptionSession;
3737
/**
3838
* Creates a ResponsesClient for interacting with the model via the Responses API.
39-
* Unlike createChatClient/createAudioClient (which use FFI), the Responses API
40-
* is HTTP-based, so the web service base URL must be provided.
41-
* @param baseUrl - The base URL of the Foundry Local web service.
39+
* Uses native FFI for create/createStreaming and falls back to HTTP when baseUrl is provided.
40+
* @param baseUrl - Optional base URL of the Foundry Local web service for HTTP fallback and server-backed operations.
4241
*/
43-
createResponsesClient(baseUrl: string): ResponsesClient;
42+
createResponsesClient(baseUrl?: string): ResponsesClient;
4443

4544
/**
4645
* Variants of the model that are available. Variants of the model are optimized for different devices.

0 commit comments

Comments
 (0)