Skip to content

Commit a9db780

Browse files
mouzhimouzhi
committed
address remaining review for deepseek responses repair
Budget responseIdMap and reasoning text, emit [DONE] for failed/incomplete, preserve plaintext reasoning for DeepSeek replay, alias rewritten response ids into local previous_response_id state, document rewriteNonCanonicalIds, and cover eager trailer plus budget regressions.
1 parent 3c32ac8 commit a9db780

8 files changed

Lines changed: 162 additions & 40 deletions

File tree

‎docs-site/src/content/docs/ja/reference/configuration/providers.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ description: プロバイダー エントリ、認証、エンドポイント、
7777
| `noTopPModels?` | `string[]` |発信者指定の`top_p`を拒否するモデル。 |
7878
| `noPenaltyModels?` | `string[]` |存在/周波数ペナルティを拒否するモデル。 |
7979
| `parallelToolCalls?` | `boolean` |並列ツール呼び出しを切り替えます。 OpenAI Chat はデフォルトでオンになっています。非チャット アダプターは明示的な `true` でのみアドバタイズします。 |
80-
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean }` |正確なプレースホルダー ID および欠落している端末 ID に対するダウンストリーム SSE 修復はデフォルトで無効になっています。関数呼び出し ID は決して書き換えられません。 |
80+
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean; rewriteNonCanonicalIds?: boolean }` |正確なプレースホルダー ID、欠落している端末 ID、および非標準の UUID 形式 response/item ID（`rewriteNonCanonicalIds`）に対するダウンストリーム SSE 修復はデフォルトで無効です。関数呼び出し ID は決して書き換えられません。 |
8181
| `autoToolChoiceOnlyModels?` | `string[]` | `tool_choice` が `auto` または `none` のみを受け入れるモデル。強制的な選択は格下げされます。 |
8282
| `preserveReasoningContentModels?` | `string[]` |チャット履歴に以前のアシスタント `reasoning_content` が必要なモデル。 |
8383
| `thinkingToggleModels?` | `string[]` |エフォート ラダーではなく `thinking.enabled` を使用してモデルをチャットします。 |
@@ -190,7 +190,8 @@ Anthropic アカウント ポリシーのリスクを理解していない限り
190190
"responsesItemIdRepair": {
191191
"reasoning": ["rs_0"],
192192
"message": ["msg_0"],
193-
"repairMissingTerminalIds": true
193+
"repairMissingTerminalIds": true,
194+
"rewriteNonCanonicalIds": true
194195
}
195196
}
196197
}

‎docs-site/src/content/docs/ko/reference/configuration/providers.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ description: 공급자 항목, 인증, 엔드포인트, 모델 카탈로그, 할
7777
| `noTopPModels?` | `string[]` | 호출자가 지정한 `top_p`를 거부하는 모델입니다. |
7878
| `noPenaltyModels?` | `string[]` | presence/frequency penalty를 허용하지 않는 모델입니다. |
7979
| `parallelToolCalls?` | `boolean` | 병렬 도구 호출을 켜거나 끕니다. OpenAI Chat은 기본으로 켜져 있고, 비-chat 어댑터는 명시적으로 `true`일 때만 이를 노출합니다. |
80-
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean }` | 기본값이 꺼진 downstream SSE 복구입니다. 정확한 자리표시자 id와 누락된 종료 id를 복구합니다. function-call id는 다시 쓰지 않습니다. |
80+
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean; rewriteNonCanonicalIds?: boolean }` | 기본값이 꺼진 downstream SSE 복구입니다. 정확한 자리표시자 id, 누락된 종료 id, 비표준 UUID 스타일 response/item id(`rewriteNonCanonicalIds`)를 복구합니다. function-call id는 다시 쓰지 않습니다. |
8181
| `autoToolChoiceOnlyModels?` | `string[]` | `tool_choice`가 `auto` 또는 `none`만 받는 모델입니다. 강제 선택은 낮은 수준으로 바뀝니다. |
8282
| `preserveReasoningContentModels?` | `string[]` | chat 기록에서 이전 assistant `reasoning_content`가 필요한 모델입니다. |
8383
| `thinkingToggleModels?` | `string[]` | effort 계층 대신 `thinking.enabled`를 쓰는 chat 모델입니다. |
@@ -193,7 +193,8 @@ Anthropic 계정 정책 위험을 이해하지 못한다면 이 기능은 꺼두
193193
"responsesItemIdRepair": {
194194
"reasoning": ["rs_0"],
195195
"message": ["msg_0"],
196-
"repairMissingTerminalIds": true
196+
"repairMissingTerminalIds": true,
197+
"rewriteNonCanonicalIds": true
197198
}
198199
}
199200
}

‎docs-site/src/content/docs/reference/configuration/providers.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,7 @@ differing backup and rewrites known legacy namespaced selected ids to bare ids.
8888
| `noTopPModels?` | `string[]` | Models that reject caller-specified `top_p`. |
8989
| `noPenaltyModels?` | `string[]` | Models that reject presence/frequency penalties. |
9090
| `parallelToolCalls?` | `boolean` | Toggle parallel tool calls. OpenAI Chat defaults on; non-chat adapters advertise only on explicit `true`. |
91-
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean }` | Disabled-by-default downstream SSE repair for exact placeholder ids and missing terminal ids. Function-call ids are never rewritten. |
91+
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean; rewriteNonCanonicalIds?: boolean }` | Disabled-by-default downstream SSE repair for exact placeholder ids, missing terminal ids, and non-canonical UUID-style response/item ids (`rewriteNonCanonicalIds`). Function-call ids are never rewritten. |
9292
| `autoToolChoiceOnlyModels?` | `string[]` | Models whose `tool_choice` accepts only `auto` or `none`; forced choices are downgraded. |
9393
| `preserveReasoningContentModels?` | `string[]` | Models requiring prior assistant `reasoning_content` in chat history. |
9494
| `thinkingToggleModels?` | `string[]` | Chat models using `thinking.enabled` rather than an effort ladder. |
@@ -234,15 +234,15 @@ For a broken `openai-responses` gateway, repair belongs on the provider object:
234234
"responsesItemIdRepair": {
235235
"reasoning": ["rs_0"],
236236
"message": ["msg_0"],
237-
"repairMissingTerminalIds": true
237+
"repairMissingTerminalIds": true,
238+
"rewriteNonCanonicalIds": true
238239
}
239240
}
240241
}
241242
}
242243
```
243244

244-
Placeholder lists are exact matches. Leave the field unset for normal/stateful Responses providers
245-
so passthrough stays byte-for-byte identical.
245+
Placeholder lists are exact matches. Set `rewriteNonCanonicalIds` for gateways such as DeepSeek Responses that emit UUID item/response ids and raw `reasoning_text` streams; OpenCodex rewrites those ids to Codex-friendly prefixes, folds reasoning into `encrypted_content` (while keeping plaintext `content` for replay), and ensures a terminal `[DONE]`. Leave the field unset for normal/stateful Responses providers so passthrough stays byte-for-byte identical.
246246

247247
## Cursor provider (`adapter: "cursor"`)
248248

‎docs-site/src/content/docs/ru/reference/configuration/providers.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,7 @@ cross-route credential fallback не существует. Строки API GPT-
9393
| `noTopPModels?` | `string[]` | Модели, отвергающие переданный вызывающей стороной `top_p`. |
9494
| `noPenaltyModels?` | `string[]` | Модели, отвергающие penalty presence/frequency. |
9595
| `parallelToolCalls?` | `boolean` | Переключатель parallel tool call'ов. Для OpenAI Chat по умолчанию включено; не-chat adapter'ы рекламируют это только при явном `true`. |
96-
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean }` | По умолчанию выключенная downstream SSE-repair для exact placeholder-id и отсутствующих terminal-id. Function-call id никогда не переписываются. |
96+
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean; rewriteNonCanonicalIds?: boolean }` | По умолчанию выключенная downstream SSE-repair для exact placeholder-id, отсутствующих terminal-id и неканонических UUID-style response/item id (`rewriteNonCanonicalIds`). Function-call id никогда не переписываются. |
9797
| `autoToolChoiceOnlyModels?` | `string[]` | Модели, у которых `tool_choice` принимает только `auto` или `none`; forced choice понижается. |
9898
| `preserveReasoningContentModels?` | `string[]` | Модели, которым нужен предыдущий assistant `reasoning_content` в chat history. |
9999
| `thinkingToggleModels?` | `string[]` | Chat-модели, использующие `thinking.enabled` вместо effort-ladder. |
@@ -244,7 +244,8 @@ Beijing, а `alibaba-token-plan-intl` обслуживает междунаро
244244
"responsesItemIdRepair": {
245245
"reasoning": ["rs_0"],
246246
"message": ["msg_0"],
247-
"repairMissingTerminalIds": true
247+
"repairMissingTerminalIds": true,
248+
"rewriteNonCanonicalIds": true
248249
}
249250
}
250251
}

‎docs-site/src/content/docs/zh-cn/reference/configuration/providers.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ description: 提供者条目、身份验证、端点、模型目录、配额、
7777
| `noTopPModels?` | `string[]` | 会拒绝调用方指定 `top_p` 的模型。 |
7878
| `noPenaltyModels?` | `string[]` | 会拒绝 presence/frequency penalty 的模型。 |
7979
| `parallelToolCalls?` | `boolean` | 切换并行工具调用。OpenAI Chat 默认开启；非 chat 适配器只有显式 `true` 时才会声明支持。 |
80-
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean }` | 默认关闭的下游 SSE 修复，用于精确占位 id 和缺失的终止 id。function-call id 永远不会被重写。 |
80+
| `responsesItemIdRepair?` | `{ message?: string[]; reasoning?: string[]; repairMissingTerminalIds?: boolean; rewriteNonCanonicalIds?: boolean }` | 默认关闭的下游 SSE 修复，用于精确占位 id、缺失的终止 id，以及非标准 UUID 风格的 response/item id（`rewriteNonCanonicalIds`）。function-call id 永远不会被重写。 |
8181
| `autoToolChoiceOnlyModels?` | `string[]` | `tool_choice` 只接受 `auto` 或 `none` 的模型；强制选择会被降级。 |
8282
| `preserveReasoningContentModels?` | `string[]` | 需要在聊天历史中保留先前 assistant `reasoning_content` 的模型。 |
8383
| `thinkingToggleModels?` | `string[]` | 使用 `thinking.enabled` 而不是 effort 阶梯的 chat 模型。 |
@@ -187,14 +187,15 @@ affinity。这些策略不能规避 provider enforcement。
187187
"responsesItemIdRepair": {
188188
"reasoning": ["rs_0"],
189189
"message": ["msg_0"],
190-
"repairMissingTerminalIds": true
190+
"repairMissingTerminalIds": true,
191+
"rewriteNonCanonicalIds": true
191192
}
192193
}
193194
}
194195
}
195196
```
196197

197-
占位列表必须精确匹配。对于正常/有状态的 Responses 提供者，请保持该字段未设置，以便转发能保持逐字节一致。
198+
占位列表必须精确匹配。对 DeepSeek Responses 这类会返回 UUID item/response id 与原始 `reasoning_text` 流的网关，请设置 `rewriteNonCanonicalIds`：OpenCodex 会把 id 改写成 Codex 友好前缀，把 reasoning 折叠进 `encrypted_content`（同时保留明文 `content` 便于回放），并补齐终端 `[DONE]`。对于正常/有状态的 Responses 提供者，请保持该字段未设置，以便转发能保持逐字节一致。
198199

199200
## Cursor 提供者（`adapter: "cursor"`）
200201

‎src/server/responses-item-id-repair.ts‎

Lines changed: 72 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,8 @@ interface ResponsesItemIdRepairState {
1616
readonly responseIdMap: Map<string, string>;
1717
readonly scope: string;
1818
readonly budget?: TranslatorBudget;
19-
sawCompleted: boolean;
19+
/** True once a terminal response event is observed (completed/failed/incomplete). */
20+
sawTerminal: boolean;
2021
sawDoneTrailer: boolean;
2122
}
2223

@@ -41,6 +42,12 @@ const ITEM_ID_EVENT_TYPES: Readonly<Record<string, RepairableItemType>> = {
4142
"response.reasoning_text.done": "reasoning",
4243
};
4344

45+
const TERMINAL_RESPONSE_TYPES = new Set([
46+
"response.completed",
47+
"response.failed",
48+
"response.incomplete",
49+
]);
50+
4451
function isPlainObject(value: unknown): value is Record<string, unknown> {
4552
return !!value && typeof value === "object" && !Array.isArray(value);
4653
}
@@ -124,7 +131,7 @@ function createRepairState(config: ResponsesItemIdRepairConfig, budget?: Transla
124131
responseIdMap: new Map<string, string>(),
125132
scope: randomUUID().replace(/-/g, ""),
126133
budget,
127-
sawCompleted: false,
134+
sawTerminal: false,
128135
sawDoneTrailer: false,
129136
};
130137
budget?.chargeRetained(new TextEncoder().encode(JSON.stringify({
@@ -155,10 +162,53 @@ function mapResponseId(state: ResponsesItemIdRepairState, rawId: string | undefi
155162
if (existing) return existing;
156163
// Keep each distinct upstream response id unique within the request-local scope.
157164
const minted = `resp_ocx_${state.scope}_${state.responseIdMap.size}`;
165+
state.budget?.chargeRetained(
166+
new TextEncoder().encode(JSON.stringify([rawId, minted])).byteLength,
167+
{ kind: "item_ids" },
168+
);
158169
state.responseIdMap.set(rawId, minted);
159170
return minted;
160171
}
161172

173+
function releaseReasoningText(state: ResponsesItemIdRepairState, outputIndex: number): void {
174+
const existing = state.reasoningTextByOutputIndex.get(outputIndex);
175+
if (existing === undefined) return;
176+
state.budget?.releaseRetained(new TextEncoder().encode(existing).byteLength, { kind: "reasoning" });
177+
state.reasoningTextByOutputIndex.delete(outputIndex);
178+
}
179+
180+
function setReasoningText(
181+
state: ResponsesItemIdRepairState,
182+
outputIndex: number,
183+
text: string,
184+
): void {
185+
const existing = state.reasoningTextByOutputIndex.get(outputIndex);
186+
if (existing === text) return;
187+
if (existing !== undefined) {
188+
state.budget?.releaseRetained(new TextEncoder().encode(existing).byteLength, { kind: "reasoning" });
189+
}
190+
if (!text) {
191+
state.reasoningTextByOutputIndex.delete(outputIndex);
192+
return;
193+
}
194+
state.budget?.chargeRetained(new TextEncoder().encode(text).byteLength, { kind: "reasoning" });
195+
state.reasoningTextByOutputIndex.set(outputIndex, text);
196+
}
197+
198+
function accumulateReasoningText(
199+
state: ResponsesItemIdRepairState,
200+
outputIndex: number,
201+
delta: string,
202+
): void {
203+
if (!delta) return;
204+
// Charge only the newly retained delta bytes; release the whole entry when the stream ends.
205+
state.budget?.chargeRetained(new TextEncoder().encode(delta).byteLength, { kind: "reasoning" });
206+
state.reasoningTextByOutputIndex.set(
207+
outputIndex,
208+
(state.reasoningTextByOutputIndex.get(outputIndex) ?? "") + delta,
209+
);
210+
}
211+
162212
function normalizeReasoningItem(
163213
state: ResponsesItemIdRepairState,
164214
outputIndex: number,
@@ -173,15 +223,24 @@ function normalizeReasoningItem(
173223
.map(part => String(part.text))
174224
.join("")
175225
: "");
176-
if (text) state.reasoningTextByOutputIndex.set(outputIndex, text);
226+
if (text && !state.reasoningTextByOutputIndex.has(outputIndex)) {
227+
// Snapshot content-derived text into the budgeted map only when we still need it later.
228+
if (!terminal) setReasoningText(state, outputIndex, text);
229+
}
177230
const next: Record<string, unknown> = {
178231
type: "reasoning",
179232
id: mapped,
180233
summary: Array.isArray(item.summary) ? item.summary : [],
181234
};
182235
if (terminal || text) {
236+
// Codex consumes encrypted_content; DeepSeek tool-call continuations need plaintext
237+
// reasoning_text after sanitizeReasoningInputContent strips the proxy-only ocxr1 envelope.
183238
next.encrypted_content = encodeReasoningEnvelope({ txt: text || " " });
239+
if (text) {
240+
next.content = [{ type: "reasoning_text", text }];
241+
}
184242
}
243+
if (terminal) releaseReasoningText(state, outputIndex);
185244
return next;
186245
}
187246

@@ -286,17 +345,6 @@ function rewriteResponseSnapshot(
286345
return changed ? { response: { ...next, output }, changed: true } : { response: next, changed };
287346
}
288347

289-
function accumulateReasoningText(
290-
state: ResponsesItemIdRepairState,
291-
outputIndex: number,
292-
delta: string,
293-
): void {
294-
state.reasoningTextByOutputIndex.set(
295-
outputIndex,
296-
(state.reasoningTextByOutputIndex.get(outputIndex) ?? "") + delta,
297-
);
298-
}
299-
300348
function repairEventPayload(
301349
payload: string,
302350
state: ResponsesItemIdRepairState,
@@ -327,7 +375,7 @@ function repairEventPayload(
327375
}
328376
if (type === "response.reasoning_text.done") {
329377
if (outputIndex !== null && typeof event.text === "string") {
330-
state.reasoningTextByOutputIndex.set(outputIndex, event.text);
378+
setReasoningText(state, outputIndex, event.text);
331379
}
332380
return null;
333381
}
@@ -337,7 +385,7 @@ function repairEventPayload(
337385
&& event.part.type === "reasoning_text"
338386
) {
339387
if (outputIndex !== null && typeof event.part.text === "string" && event.part.text) {
340-
state.reasoningTextByOutputIndex.set(outputIndex, event.part.text);
388+
setReasoningText(state, outputIndex, event.part.text);
341389
}
342390
return null;
343391
}
@@ -369,7 +417,7 @@ function repairEventPayload(
369417
}
370418
}
371419

372-
if (type === "response.completed") state.sawCompleted = true;
420+
if (type && TERMINAL_RESPONSE_TYPES.has(type)) state.sawTerminal = true;
373421

374422
if (state.rewriteNonCanonicalIds && isPlainObject(nextEvent.part) && nextEvent.part.type === "output_text") {
375423
nextEvent = {
@@ -380,10 +428,6 @@ function repairEventPayload(
380428
annotations: Array.isArray(nextEvent.part.annotations) ? nextEvent.part.annotations : [],
381429
},
382430
};
383-
if ("logprobs" in nextEvent) {
384-
const { logprobs: _lp, ...rest } = nextEvent;
385-
nextEvent = rest;
386-
}
387431
changed = true;
388432
}
389433
if (state.rewriteNonCanonicalIds && "logprobs" in nextEvent) {
@@ -413,6 +457,11 @@ function repairEventPayload(
413457
export interface ResponsesItemIdRepairHandlers {
414458
rewrite: SsePayloadRewrite;
415459
trailer: () => string | undefined;
460+
/**
461+
* Map a provider-raw response id to the client-visible id emitted on the repaired
462+
* stream. Used to alias local previous_response_id continuation state.
463+
*/
464+
clientResponseId: (rawId: string | undefined) => string | undefined;
416465
}
417466

418467
export function createResponsesItemIdRepairHandlers(
@@ -423,9 +472,10 @@ export function createResponsesItemIdRepairHandlers(
423472
return {
424473
rewrite: (payload) => repairEventPayload(payload, state),
425474
trailer: () => {
426-
if (state.sawCompleted && !state.sawDoneTrailer) return "data: [DONE]\n\n";
475+
if (state.sawTerminal && !state.sawDoneTrailer) return "data: [DONE]\n\n";
427476
return undefined;
428477
},
478+
clientResponseId: (rawId) => mapResponseId(state, rawId),
429479
};
430480
}
431481

‎src/server/responses/core.ts‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1612,9 +1612,19 @@ async function handleResponsesInner(
16121612
// recording it would let a later expansion rehydrate the chain Codex just replaced.
16131613
const passthroughRecordEligible = parsed._compactionRequest !== true
16141614
&& (!parsed.previousResponseId || parsed._previousResponseInputExpanded === true);
1615+
// When item-id repair rewrites response ids, the inspector still sees the raw
1616+
// upstream UUID while Codex chains previous_response_id with the client-visible
1617+
// resp_ocx_* id. Alias both keys into the local continuation cache.
1618+
let mapClientResponseId: ((rawId: string | undefined) => string | undefined) | undefined;
16151619
const rememberPassthroughResponse = passthroughRecordEligible
1616-
? (response: { id?: unknown; output?: unknown; status?: unknown }) =>
1617-
rememberResponseState(parsed._rawBody, response, undefined, { force: true })
1620+
? (response: { id?: unknown; output?: unknown; status?: unknown }) => {
1621+
rememberResponseState(parsed._rawBody, response, undefined, { force: true });
1622+
if (typeof response.id !== "string" || !mapClientResponseId) return;
1623+
const clientId = mapClientResponseId(response.id);
1624+
if (clientId && clientId !== response.id) {
1625+
rememberResponseState(parsed._rawBody, { ...response, id: clientId }, undefined, { force: true });
1626+
}
1627+
}
16181628
: undefined;
16191629
if (parsed.previousResponseId && !parsed._previousResponseInputExpanded) {
16201630
console.warn(
@@ -1821,6 +1831,7 @@ async function handleResponsesInner(
18211831
const itemIdRepair = hasResponsesItemIdRepair(repairConfig)
18221832
? createResponsesItemIdRepairHandlers(repairConfig!, translatorBudget)
18231833
: undefined;
1834+
if (itemIdRepair) mapClientResponseId = itemIdRepair.clientResponseId;
18241835
// Compose opt-in payload rewrites into one parse/stringify pass (image-gen restore first).
18251836
const payloadRewrites = [
18261837
createImageGenCallRestoreRewrite(imageGenCallAliases),

0 commit comments

Comments
 (0)