-
Notifications
You must be signed in to change notification settings - Fork 531
fix(deepseek): preserve Responses reasoning replay on continuations (#875) #906
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -1,6 +1,6 @@ | ||||||||||||||||||||||||||||||||
| import { createHash } from "node:crypto"; | ||||||||||||||||||||||||||||||||
| import type { IncomingMeta, ProviderAdapter } from "./base"; | ||||||||||||||||||||||||||||||||
| import { namespacedToolName, type AdapterEvent, type OcxParsedRequest, type OcxProviderConfig, type OcxUsage } from "../types"; | ||||||||||||||||||||||||||||||||
| import { modelInList, namespacedToolName, type AdapterEvent, type OcxParsedRequest, type OcxProviderConfig, type OcxUsage } from "../types"; | ||||||||||||||||||||||||||||||||
| import { catalogModelSupportsReasoningSummaries } from "../codex/catalog"; | ||||||||||||||||||||||||||||||||
| import { COMPACT_PROMPT, decodeCompactionSummary, SUMMARY_PREFIX } from "../responses/compaction"; | ||||||||||||||||||||||||||||||||
| import { collectResponsesToolGroups } from "../responses/tool-groups"; | ||||||||||||||||||||||||||||||||
|
|
@@ -32,27 +32,42 @@ export const FORWARD_HEADERS = [ | |||||||||||||||||||||||||||||||
| "x-responsesapi-include-timing-metrics", | ||||||||||||||||||||||||||||||||
| ]; | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| export function sanitizeReasoningInputContent(body: unknown): unknown { | ||||||||||||||||||||||||||||||||
| export function sanitizeReasoningInputContent( | ||||||||||||||||||||||||||||||||
| body: unknown, | ||||||||||||||||||||||||||||||||
| opts?: { preserveContentModels?: string[]; modelId?: string }, | ||||||||||||||||||||||||||||||||
| ): unknown { | ||||||||||||||||||||||||||||||||
| if (!body || typeof body !== "object" || Array.isArray(body)) return body; | ||||||||||||||||||||||||||||||||
| const raw = body as Record<string, unknown>; | ||||||||||||||||||||||||||||||||
| if (!Array.isArray(raw.input)) return body; | ||||||||||||||||||||||||||||||||
| const preserveContent = modelInList(opts?.preserveContentModels, opts?.modelId ?? ""); | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| let changed = false; | ||||||||||||||||||||||||||||||||
| const input = raw.input.map(item => { | ||||||||||||||||||||||||||||||||
| if (!item || typeof item !== "object" || Array.isArray(item)) return item; | ||||||||||||||||||||||||||||||||
| const rec = item as Record<string, unknown>; | ||||||||||||||||||||||||||||||||
| if (rec.type !== "reasoning") return item; | ||||||||||||||||||||||||||||||||
| const hasRawContent = Array.isArray(rec.content) && rec.content.length > 0; | ||||||||||||||||||||||||||||||||
| // ocxr1 envelopes are proxy-minted (Anthropic signatures), not OpenAI encryption — the native | ||||||||||||||||||||||||||||||||
| // backend cannot decrypt them and would reject the request. Strip regardless of content shape. | ||||||||||||||||||||||||||||||||
| // ocxr1 envelopes are proxy-minted (Anthropic signatures), not OpenAI encryption — no | ||||||||||||||||||||||||||||||||
| // backend can decrypt them. Strip regardless of content shape or preserve setting. | ||||||||||||||||||||||||||||||||
| const hasOcxEnvelope = typeof rec.encrypted_content === "string" && rec.encrypted_content.startsWith(OCX_REASONING_PREFIX); | ||||||||||||||||||||||||||||||||
| if (!hasRawContent && !hasOcxEnvelope) return item; | ||||||||||||||||||||||||||||||||
| const next: Record<string, unknown> = { ...rec }; | ||||||||||||||||||||||||||||||||
| let mutated = false; | ||||||||||||||||||||||||||||||||
| if (hasOcxEnvelope) { | ||||||||||||||||||||||||||||||||
| delete next.encrypted_content; | ||||||||||||||||||||||||||||||||
| mutated = true; | ||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||
| // Routed models can produce raw `reasoning_text` output items. ChatGPT's Responses backend | ||||||||||||||||||||||||||||||||
| // accepts reasoning input only with empty `content`, so native passthrough keeps summaries/ids | ||||||||||||||||||||||||||||||||
| // and drops the raw content to avoid a 400. Stateless Responses backends such as DeepSeek | ||||||||||||||||||||||||||||||||
| // instead require the caller to replay the reasoning text on every continuation, so models in | ||||||||||||||||||||||||||||||||
| // `preserveReasoningContentModels` keep it (issue #875). | ||||||||||||||||||||||||||||||||
| if (hasRawContent && !preserveContent) { | ||||||||||||||||||||||||||||||||
| next.content = []; | ||||||||||||||||||||||||||||||||
| mutated = true; | ||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||
| if (!mutated) return item; | ||||||||||||||||||||||||||||||||
| changed = true; | ||||||||||||||||||||||||||||||||
| // Routed models can produce raw `reasoning_text` output items. Codex echoes those in later | ||||||||||||||||||||||||||||||||
| // native GPT requests, but ChatGPT's Responses backend accepts reasoning input only with empty | ||||||||||||||||||||||||||||||||
| // `content`; keep summaries/ids and drop the raw content so native passthrough does not 400. | ||||||||||||||||||||||||||||||||
| const next: Record<string, unknown> = { ...rec, content: [] }; | ||||||||||||||||||||||||||||||||
| if (hasOcxEnvelope) delete next.encrypted_content; | ||||||||||||||||||||||||||||||||
| return next; | ||||||||||||||||||||||||||||||||
| }); | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
|
|
@@ -1024,7 +1039,10 @@ export function createResponsesPassthroughAdapter(provider: OcxProviderConfig): | |||||||||||||||||||||||||||||||
| if (parsed._compactionRequest === true && !isCanonicalOpenAiForwardProvider(provider)) { | ||||||||||||||||||||||||||||||||
| outBody = buildRoutedCompactionBody(outBody); | ||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||
| const sanitizedBody = normalizeToolSchemas(stripSparkCompatibility(stripUnsupportedReasoningParams(stripItemIdsWhenUnstored(stripInvalidItemIds(stripUnsupportedHostedTools(sanitizeReasoningInputContent(scrubOcxCompactionItems(outBody)))))))); | ||||||||||||||||||||||||||||||||
| const sanitizedBody = normalizeToolSchemas(stripSparkCompatibility(stripUnsupportedReasoningParams(stripItemIdsWhenUnstored(stripInvalidItemIds(stripUnsupportedHostedTools(sanitizeReasoningInputContent(scrubOcxCompactionItems(outBody), { | ||||||||||||||||||||||||||||||||
| preserveContentModels: provider.preserveReasoningContentModels, | ||||||||||||||||||||||||||||||||
| modelId: parsed.modelId, | ||||||||||||||||||||||||||||||||
| }))))))); | ||||||||||||||||||||||||||||||||
|
Comment on lines
+1042
to
+1045
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win Deeply nested call chain gets harder to read with each new parameter. This line already chains eight function calls ( ♻️ Proposed refactor: sequential pipeline instead of nested calls- const sanitizedBody = normalizeToolSchemas(stripSparkCompatibility(stripUnsupportedReasoningParams(stripItemIdsWhenUnstored(stripInvalidItemIds(stripUnsupportedHostedTools(sanitizeReasoningInputContent(scrubOcxCompactionItems(outBody), {
- preserveContentModels: provider.preserveReasoningContentModels,
- modelId: parsed.modelId,
- })))))));
+ let sanitizedBody = scrubOcxCompactionItems(outBody);
+ sanitizedBody = sanitizeReasoningInputContent(sanitizedBody, {
+ preserveContentModels: provider.preserveReasoningContentModels,
+ modelId: parsed.modelId,
+ });
+ sanitizedBody = stripUnsupportedHostedTools(sanitizedBody);
+ sanitizedBody = stripInvalidItemIds(sanitizedBody);
+ sanitizedBody = stripItemIdsWhenUnstored(sanitizedBody);
+ sanitizedBody = stripUnsupportedReasoningParams(sanitizedBody);
+ sanitizedBody = stripSparkCompatibility(sanitizedBody);
+ sanitizedBody = normalizeToolSchemas(sanitizedBody);📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||||||||||||
| const body = JSON.stringify(stripDisabledReasoningSummaries( | ||||||||||||||||||||||||||||||||
| normalizeConfiguredReasoningSummaryDelivery(sanitizedBody, provider, parsed.modelId), | ||||||||||||||||||||||||||||||||
| provider, | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| /** | ||
| * DeepSeek V4 is stateless on the Responses API and requires the caller to replay | ||
| * prior assistant reasoning text on every continuation (issue #875). The passthrough | ||
| * sanitizer must therefore keep `reasoning.content` for models listed in | ||
| * `preserveReasoningContentModels` (seeded for DeepSeek thinking models) while still | ||
| * blanking it for ChatGPT-native passthrough and still stripping proxy-minted ocxr1 | ||
| * envelopes. A routable DeepSeek entry also has to be usable as a sub-agent model. | ||
| */ | ||
| import { afterEach, describe, expect, test } from "bun:test"; | ||
| import { sanitizeReasoningInputContent } from "../src/adapters/openai-responses"; | ||
| import { buildSubagentModelChain, isSubagentModelUnavailable } from "../src/codex/subagent-model-fallback"; | ||
| import { providerConfigSeed } from "../src/providers/derive"; | ||
| import { getProviderRegistryEntry } from "../src/providers/registry"; | ||
| import { OCX_REASONING_PREFIX } from "../src/responses/reasoning-envelope"; | ||
| import { handleResponses } from "../src/server/responses/core"; | ||
| import type { OcxConfig, OcxProviderConfig } from "../src/types"; | ||
| const MODEL = "deepseek-v4-flash"; | ||
|
|
||
| const RAW_REASONING = { | ||
| type: "reasoning", | ||
| id: "rs_1", | ||
| summary: [], | ||
| content: [{ type: "reasoning_text", text: "inspect the failing build" }], | ||
| }; | ||
|
|
||
| function deepseekProvider(): OcxProviderConfig { | ||
| return { ...providerConfigSeed(getProviderRegistryEntry("deepseek")!), apiKey: "sk-test" }; | ||
| } | ||
|
|
||
| describe("sanitizeReasoningInputContent respects replay-capable models", () => { | ||
| test("keeps raw reasoning content for a preserve-listed model", () => { | ||
| const out = sanitizeReasoningInputContent( | ||
| { input: [RAW_REASONING] }, | ||
| { preserveContentModels: ["deepseek-v4-flash"], modelId: MODEL }, | ||
| ) as { input: Array<Record<string, unknown>> }; | ||
| expect(out.input[0].content).toEqual(RAW_REASONING.content); | ||
| }); | ||
|
|
||
| test("still blanks raw reasoning content for non-preserving passthrough", () => { | ||
| const out = sanitizeReasoningInputContent( | ||
| { input: [RAW_REASONING] }, | ||
| { modelId: "gpt-5.6" }, | ||
| ) as { input: Array<Record<string, unknown>> }; | ||
| expect(out.input[0].content).toEqual([]); | ||
| }); | ||
|
|
||
| test("still strips ocxr1 envelopes while preserving content for DeepSeek", () => { | ||
| const enveloped = { | ||
| ...RAW_REASONING, | ||
| id: "rs_2", | ||
| encrypted_content: `${OCX_REASONING_PREFIX}abc`, | ||
| }; | ||
| const out = sanitizeReasoningInputContent( | ||
| { input: [enveloped] }, | ||
| { preserveContentModels: ["deepseek-v4-flash"], modelId: MODEL }, | ||
| ) as { input: Array<Record<string, unknown>> }; | ||
| expect(out.input[0].content).toEqual(RAW_REASONING.content); | ||
| expect(out.input[0].encrypted_content).toBeUndefined(); | ||
| }); | ||
| }); | ||
|
|
||
| describe("a DeepSeek Responses continuation keeps its reasoning replay", () => { | ||
| const originalFetch = globalThis.fetch; | ||
| afterEach(() => { globalThis.fetch = originalFetch; }); | ||
|
|
||
| test("a tool-call follow-up carries reasoning content and tool output upstream", async () => { | ||
| let capturedBody: Record<string, unknown> | undefined; | ||
| globalThis.fetch = (async (_input: RequestInfo | URL, init?: RequestInit) => { | ||
| capturedBody = JSON.parse(String(init?.body ?? "{}")) as Record<string, unknown>; | ||
| return Response.json({ | ||
| id: "resp_deepseek_2", | ||
| object: "response", | ||
| status: "completed", | ||
| output: [], | ||
| }); | ||
| }) as typeof fetch; | ||
|
|
||
| const config = { providers: { deepseek: deepseekProvider() } } as unknown as OcxConfig; | ||
| await handleResponses( | ||
| new Request("http://localhost/v1/responses", { | ||
| method: "POST", | ||
| headers: { "content-type": "application/json" }, | ||
| body: JSON.stringify({ | ||
| model: MODEL, | ||
| stream: false, | ||
| store: false, | ||
| input: [ | ||
| RAW_REASONING, | ||
| { type: "function_call", id: "fc_1", call_id: "call_1", name: "exec_command", arguments: "{}" }, | ||
| { type: "function_call_output", call_id: "call_1", output: "build failed" }, | ||
| ], | ||
| }), | ||
| }), | ||
| config, | ||
| { model: "", provider: "" }, | ||
| ); | ||
|
|
||
| const input = capturedBody!.input as unknown[]; | ||
| expect(input).toEqual(expect.arrayContaining([ | ||
| expect.objectContaining({ type: "reasoning", content: RAW_REASONING.content }), | ||
| expect.objectContaining({ type: "function_call_output", call_id: "call_1" }), | ||
| ])); | ||
| }); | ||
| }); | ||
|
|
||
| describe("DeepSeek V4 Flash is a viable sub-agent candidate", () => { | ||
| test("it is routable and sits first in a sub-agent fallback chain", () => { | ||
| const config = { | ||
| defaultProvider: "deepseek", | ||
| providers: { deepseek: deepseekProvider() }, | ||
| } as unknown as OcxConfig; | ||
| const chain = buildSubagentModelChain("deepseek/deepseek-v4-flash", config); | ||
| expect(chain[0]).toBe("deepseek/deepseek-v4-flash"); | ||
| expect(isSubagentModelUnavailable("deepseek/deepseek-v4-flash", config)).toBe(false); | ||
| }); | ||
| }); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When
reasoning.summaryis absent ornone,src/bridge.ts:380-397stores the prior raw reasoning only in a txt-onlyocxr1envelope, with nocontentarray. On the next DeepSeek continuation, this branch deletes that sole copy, whilepreserveContentcannot help becausehasRawContentis false; the stateless upstream therefore still receives no reasoning replay and tool-call loops can stall in the default hidden-summary mode. Decode the envelope'stxtintocontent: [{ type: "reasoning_text", text }]for preserve-listed models before removingencrypted_content, and cover a bridge-output-to-passthrough round trip.AGENTS.md reference: src/AGENTS.md:L19-L19
Useful? React with 👍 / 👎.