Skip to content

Commit d9f6437

Browse files
Veselin-Vasilev-IBM-1Veselin-Vasilev-IBM-1
committed
fix(serve): port OpenAI server loopback default, streaming, and sequence fixes to TS
Mirrors the Python-side security/correctness fixes (beb08b3) in the TypeScript OpenAI server adapter and addresses remaining code-review comments. Security (#1516): - Default OpenAIServerConfig.host to 127.0.0.1 (loopback); binding all interfaces (0.0.0.0) is now opt-in. - Warn when serving on a non-loopback host with no apiKey set. - Drop the hard-coded host: '0.0.0.0' from examples/serve/openai_responses.ts. Correctness (streaming): - ChatCompletionAPI and ResponsesAPI now emit text deltas from the events the agent actually produces. Previously both handlers filtered for event.name === 'update', but ToolCallingAgent only emits start/success, so stream: true against the documented example yielded an SSE envelope with zero text deltas. Now 'success' (state.result.text) is also emitted, keeping streaming working for both ReActAgent and ToolCallingAgent. Sequence number: - sendEvent() now increments sequence_number before assigning it to the event payload, instead of pre-building events with a stale/duplicated sequence_number, violating the monotonic-sequence contract. - Removed redundant sequence_number fields from event objects; the type is now optional and populated centrally by sendEvent. Docs: - Replace the 'coming soon' TypeScript placeholder in integrations/openai-api.mdx with the real openai_responses.ts example. Tests: - Add unit tests mirroring Python's test_openai_server_config.py: default host is loopback; warning fires for non-loopback without apiKey; no warning on loopback or when apiKey is set. Expose the module-level logger so tests can spy on warn(). Signed-off-by: Veselin-Vasilev-IBM-1 <veselin.vasilev1@ibm.com>
1 parent 80652c3 commit d9f6437

7 files changed

Lines changed: 184 additions & 51 deletions

File tree

‎docs/src/content/docs/integrations/openai-api.mdx‎

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,8 +51,28 @@ Key benefits
5151

5252
```
5353

54+
{/* <!-- embedme typescript/examples/serve/openai_responses.ts --> */}
5455
```ts TypeScript [expandable]
55-
// coming soon
56+
import "dotenv/config.js";
57+
58+
import { OpenMeteoTool } from "beeai-framework/tools/weather/openMeteo";
59+
import { OllamaChatModel } from "beeai-framework/adapters/ollama/backend/chat";
60+
import { ToolCallingAgent } from "beeai-framework/agents/toolCalling/agent";
61+
import { UnconstrainedMemory } from "beeai-framework/memory/unconstrainedMemory";
62+
import { OpenAIServer } from "beeai-framework/adapters/openai/serve/server";
63+
64+
// ensure the model is pulled before running
65+
const llm = new OllamaChatModel("granite4:micro");
66+
67+
const agent = new ToolCallingAgent({
68+
llm,
69+
memory: new UnconstrainedMemory(),
70+
tools: [
71+
new OpenMeteoTool(), // weather tool
72+
],
73+
});
74+
75+
await new OpenAIServer({ api: "responses", port: 9999 }).register(agent).serve();
5676
```
5777

5878
</CodeGroup>

‎typescript/examples/serve/openai_responses.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,4 +17,4 @@ const agent = new ToolCallingAgent({
1717
],
1818
});
1919

20-
await new OpenAIServer({ api: "responses", host: "0.0.0.0", port: 9999 }).register(agent).serve();
20+
await new OpenAIServer({ api: "responses", port: 9999 }).register(agent).serve();

‎typescript/src/adapters/openai/serve/api.ts‎

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -66,14 +66,19 @@ export class ChatCompletionAPI {
6666
await clonedAgent
6767
.run({ prompt: null }, { signal: controller.signal })
6868
.observe((emitter) => {
69-
// Match all events and filter for "update" — required because the
70-
// emitter is typed as Emitter<unknown> at the AnyAgent abstraction.
69+
// Match all events — required because the emitter is typed as
70+
// Emitter<unknown> at the AnyAgent abstraction. We emit text deltas
71+
// from "update" (ReActAgent incremental tokens) and from "success"
72+
// (ToolCallingAgent emits only start/success, with the final answer
73+
// in state.result). This keeps streaming working for both agents.
7174
emitter.match("*.*", async (eventData: any, event) => {
72-
if (event.name !== "update") {
73-
return;
75+
let delta: string | undefined;
76+
if (event.name === "update") {
77+
delta = eventData?.update?.value;
78+
} else if (event.name === "success") {
79+
delta = eventData?.state?.result?.text;
7480
}
75-
const { update } = eventData as { update: { value: string } };
76-
if (res.writableEnded || res.destroyed) {
81+
if (!delta || res.writableEnded || res.destroyed) {
7782
return;
7883
}
7984
const data = {
@@ -86,7 +91,7 @@ export class ChatCompletionAPI {
8691
index: 0,
8792
delta: {
8893
role: "assistant",
89-
content: update.value,
94+
content: delta,
9095
},
9196
finish_reason: null,
9297
},

‎typescript/src/adapters/openai/serve/responses_api.ts‎

Lines changed: 13 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -94,8 +94,9 @@ export class ResponsesAPI {
9494
const outputItemId = `msg_${uuidv4()}`;
9595
let accumulatedText = "";
9696

97-
const sendEvent = (eventType: string, data: unknown) => {
97+
const sendEvent = (eventType: string, data: { sequence_number?: number }) => {
9898
sequenceNumber++;
99+
data.sequence_number = sequenceNumber;
99100
if (res.writableEnded || res.destroyed) {
100101
return;
101102
}
@@ -118,15 +119,13 @@ export class ResponsesAPI {
118119
const createdEvent: ResponsesStreamResponseCreated = {
119120
type: "response.created",
120121
response: createdResponse,
121-
sequence_number: sequenceNumber,
122122
};
123123
sendEvent("response.created", createdEvent);
124124

125125
// response.in_progress
126126
const inProgressEvent: ResponsesStreamResponseInProgress = {
127127
type: "response.in_progress",
128128
response: { ...createdResponse },
129-
sequence_number: sequenceNumber,
130129
};
131130
sendEvent("response.in_progress", inProgressEvent);
132131

@@ -142,7 +141,6 @@ export class ResponsesAPI {
142141
type: "response.output_item.added",
143142
item: outputItem,
144143
output_index: 0,
145-
sequence_number: sequenceNumber,
146144
};
147145
sendEvent("response.output_item.added", outputItemAddedEvent);
148146

@@ -158,7 +156,6 @@ export class ResponsesAPI {
158156
item_id: outputItemId,
159157
output_index: 0,
160158
part: emptyPart,
161-
sequence_number: sequenceNumber,
162159
};
163160
sendEvent("response.content_part.added", contentPartAddedEvent);
164161

@@ -174,17 +171,21 @@ export class ResponsesAPI {
174171
await clonedAgent
175172
.run({ prompt: null }, { signal: controller.signal })
176173
.observe((emitter) => {
177-
// Match all events and filter for "update" — required because the
178-
// emitter is typed as Emitter<unknown> at the AnyAgent abstraction.
174+
// Match all events — required because the emitter is typed as
175+
// Emitter<unknown> at the AnyAgent abstraction. We emit text deltas
176+
// from "update" (ReActAgent incremental tokens) and from "success"
177+
// (ToolCallingAgent emits only start/success, with the final answer
178+
// in state.result). This keeps streaming working for both agents.
179179
emitter.match("*.*", async (data: any, event) => {
180-
if (event.name !== "update") {
181-
return;
180+
let delta: string | undefined;
181+
if (event.name === "update") {
182+
delta = data?.update?.value;
183+
} else if (event.name === "success") {
184+
delta = data?.state?.result?.text;
182185
}
183-
const { update } = data as { update: { value: string } };
184-
if (res.writableEnded || res.destroyed) {
186+
if (!delta || res.writableEnded || res.destroyed) {
185187
return;
186188
}
187-
const delta = update.value;
188189
accumulatedText += delta;
189190

190191
const deltaEvent: ResponsesStreamOutputTextDelta = {
@@ -193,7 +194,6 @@ export class ResponsesAPI {
193194
delta,
194195
item_id: outputItemId,
195196
output_index: 0,
196-
sequence_number: sequenceNumber,
197197
};
198198
sendEvent("response.output_text.delta", deltaEvent);
199199
});
@@ -206,7 +206,6 @@ export class ResponsesAPI {
206206
text: accumulatedText,
207207
item_id: outputItemId,
208208
output_index: 0,
209-
sequence_number: sequenceNumber,
210209
};
211210
sendEvent("response.output_text.done", textDoneEvent);
212211

@@ -222,7 +221,6 @@ export class ResponsesAPI {
222221
item_id: outputItemId,
223222
output_index: 0,
224223
part: finalPart,
225-
sequence_number: sequenceNumber,
226224
};
227225
sendEvent("response.content_part.done", contentPartDoneEvent);
228226

@@ -238,7 +236,6 @@ export class ResponsesAPI {
238236
type: "response.output_item.done",
239237
item: completedOutputItem,
240238
output_index: 0,
241-
sequence_number: sequenceNumber,
242239
};
243240
sendEvent("response.output_item.done", outputItemDoneEvent);
244241

@@ -254,7 +251,6 @@ export class ResponsesAPI {
254251
const completedEvent: ResponsesStreamResponseCompleted = {
255252
type: "response.completed",
256253
response: completedResponse,
257-
sequence_number: sequenceNumber,
258254
};
259255
sendEvent("response.completed", completedEvent);
260256
} catch (error) {
@@ -263,7 +259,6 @@ export class ResponsesAPI {
263259
code: "500",
264260
message: String(error),
265261
param: "",
266-
sequence_number: sequenceNumber,
267262
};
268263
sendEvent("error", errorEvent);
269264
}

‎typescript/src/adapters/openai/serve/responses_types.ts‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -92,33 +92,33 @@ export interface ResponsesResponse {
9292
export interface ResponsesStreamResponseCreated {
9393
type: "response.created";
9494
response: ResponsesResponse;
95-
sequence_number: number;
95+
sequence_number?: number;
9696
}
9797

9898
export interface ResponsesStreamResponseInProgress {
9999
type: "response.in_progress";
100100
response: ResponsesResponse;
101-
sequence_number: number;
101+
sequence_number?: number;
102102
}
103103

104104
export interface ResponsesStreamResponseCompleted {
105105
type: "response.completed";
106106
response: ResponsesResponse;
107-
sequence_number: number;
107+
sequence_number?: number;
108108
}
109109

110110
export interface ResponsesStreamOutputItemAdded {
111111
type: "response.output_item.added";
112112
item: ResponsesResponseOutput;
113113
output_index: number;
114-
sequence_number: number;
114+
sequence_number?: number;
115115
}
116116

117117
export interface ResponsesStreamOutputItemDone {
118118
type: "response.output_item.done";
119119
item: ResponsesResponseOutput;
120120
output_index: number;
121-
sequence_number: number;
121+
sequence_number?: number;
122122
}
123123

124124
export interface ResponsesStreamPartOutputText {
@@ -133,7 +133,7 @@ export interface ResponsesStreamContentPartAdded {
133133
item_id: string;
134134
output_index: number;
135135
part: ResponsesStreamPartOutputText;
136-
sequence_number: number;
136+
sequence_number?: number;
137137
}
138138

139139
export interface ResponsesStreamContentPartDone {
@@ -142,7 +142,7 @@ export interface ResponsesStreamContentPartDone {
142142
item_id: string;
143143
output_index: number;
144144
part: ResponsesStreamPartOutputText;
145-
sequence_number: number;
145+
sequence_number?: number;
146146
}
147147

148148
export interface ResponsesStreamOutputTextDelta {
@@ -151,7 +151,7 @@ export interface ResponsesStreamOutputTextDelta {
151151
delta: string;
152152
item_id: string;
153153
output_index: number;
154-
sequence_number: number;
154+
sequence_number?: number;
155155
}
156156

157157
export interface ResponsesStreamOutputTextDone {
@@ -160,13 +160,13 @@ export interface ResponsesStreamOutputTextDone {
160160
text: string;
161161
item_id: string;
162162
output_index: number;
163-
sequence_number: number;
163+
sequence_number?: number;
164164
}
165165

166166
export interface ResponsesStreamError {
167167
type: "error";
168168
code: string;
169169
message: string;
170170
param: string;
171-
sequence_number: number;
171+
sequence_number?: number;
172172
}

‎typescript/src/adapters/openai/serve/server.ts‎

Lines changed: 23 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -14,15 +14,26 @@ import { ValueError } from "@/errors.js";
1414

1515
export type OpenAIAPIType = "chat-completion" | "responses";
1616

17-
const logger = Logger.root.child({
17+
export const logger = Logger.root.child({
1818
name: "OpenAI server",
1919
});
2020

21+
const LOOPBACK_HOSTS = ["127.0.0.1", "localhost", "::1"];
22+
2123
export class OpenAIServerConfig {
22-
host = "0.0.0.0";
24+
// Bind to loopback by default. The endpoints are unauthenticated unless `apiKey`
25+
// is set, so binding to all interfaces ("0.0.0.0") is opt-in to avoid exposing an
26+
// unauthenticated server to the network by default.
27+
host = "127.0.0.1";
2328
port = 9999;
2429
api: OpenAIAPIType = "chat-completion";
2530
apiKey?: string;
31+
32+
constructor(partial?: Partial<OpenAIServerConfig>) {
33+
if (partial) {
34+
Object.assign(this, partial);
35+
}
36+
}
2637
}
2738

2839
export interface OpenAIServerMetadata {
@@ -38,8 +49,8 @@ export class OpenAIServer extends Server<
3849
> {
3950
private ready = false;
4051

41-
constructor(config: OpenAIServerConfig = new OpenAIServerConfig()) {
42-
super(config);
52+
constructor(config?: Partial<OpenAIServerConfig>) {
53+
super(config ? new OpenAIServerConfig(config) : new OpenAIServerConfig());
4354
}
4455

4556
public register(input: AnyAgent, metadata?: OpenAIServerMetadata) {
@@ -54,6 +65,14 @@ export class OpenAIServer extends Server<
5465

5566
const app = express();
5667

68+
if (this.config.apiKey === undefined && !LOOPBACK_HOSTS.includes(this.config.host)) {
69+
logger.warn(
70+
`OpenAIServer is binding to a non-loopback host (${this.config.host}) with no \`apiKey\` set: ` +
71+
`the API will be reachable from the network without authentication. ` +
72+
`Set \`apiKey\` in the config, or bind to 127.0.0.1, to avoid exposing it.`,
73+
);
74+
}
75+
5776
const modelFactory = async (modelId: string): Promise<AnyAgent> => {
5877
// Find the first agent that matches the requested model ID by name.
5978
// If the client doesn't pass a specific name or it doesn't match, we fallback to the first member.

0 commit comments

Comments
 (0)