fix(llm): OpenAI provider compatibility for caching, temperature, and tool calls - #98
Merged
Conversation
… tool calls Three request-shape bugs surfaced when running against the real OpenAI API (DeepSeek tolerates all three, which is why they went unnoticed): 1. Anthropic prompt-caching shape sent to every provider. With prompt_caching enabled, the loop moved the system prompt into the Anthropic-only top-level "system" field and added cache_control markers; OpenAI rejects this with 400 unknown_parameter. Markers are now only applied when the endpoint is Anthropic (new Client.IsAnthropic). OpenAI and DeepSeek cache automatically. 2. temperature:0 sent to reasoning models. odek defaults to temperature 0 for determinism, but OpenAI reasoning models (o1/o3/o4, gpt-5 series) only accept the default (1) and answer 400 unsupported_value. The field is now omitted for those models; behavior for all other providers/models is unchanged. 3. Function tools + reasoning_effort rejected by gpt-5.6 models. gpt-5.6-luna/sol reject tools combined with any reasoning_effort other than "none" — and their default effort is not "none", so omitting the field still 400s. The client now learns the constraint from the 400 (atomic flag), retries once with effort "none", and pins it for subsequent tool-bearing calls; tool-less calls keep the configured effort. Verified live against api.openai.com (gpt-4o-mini, gpt-5-nano, gpt-5.6-luna, gpt-5.6-sol — basic runs, tool round-trips, automatic prompt caching) and regression-checked against DeepSeek v4 Flash (native cache metrics and temperature:0 default intact). Regression tests: TestClient_IsAnthropic, TestCall_OmitsTemperatureForReasoningModels, TestCall_LearnsReasoningEffortNoneWithTools, TestEngine_PromptCaching_NonAnthropicSkipsMarkers.
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
odek | b94f9e4 | Commit Preview URL Branch Preview URL |
Jul 25 2026, 12:23 PM |
…s sending the Anthropic object
The thinking object ({"type":"enabled"/"disabled"}) was sent
unconditionally; OpenAI rejects unknown top-level parameters with a
400, so thinking: disabled (and enabled) was broken against OpenAI —
same family as the system-field bug fixed earlier in this branch.
The shaping logic now lives in an extracted, directly testable
buildCallParams method:
- Anthropic/DeepSeek (new sendsThinkingObject gate): unchanged,
thinking object sent as before.
- OpenAI reasoning models (o1/o3/o4, gpt-5 series): enabled maps to
reasoning_effort "high", disabled to "none" (verified accepted on
the gpt-5 series); temperature stays omitted per the forbidlist.
- Non-reasoning models (e.g. gpt-4o): no thinking capability exists,
so both fields are omitted and temperature is honored as configured.
The two existing thinking tests asserted the old unconditional
behavior through an httptest server whose URL is never a recognized
provider; they now exercise buildCallParams directly. New table test
TestBuildCallParams_ThinkingMapping covers all provider/model/thinking
combinations.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Request-shape fixes in the LLM client/loop, found by running the agent against the real OpenAI API. DeepSeek tolerates all of these shapes, which is why they went unnoticed until switching providers.
The bugs
1. Anthropic prompt-caching shape sent to every provider — with
prompt_cachingenabled, the loop moved the system prompt into the Anthropic-only top-levelsystemfield and addedcache_controlmarkers. OpenAI rejects this:400 Unknown parameter: 'system'.2.
temperature: 0sent to reasoning models — odek defaults to temperature 0 for determinism, but OpenAI reasoning models (o1/o3/o4, gpt-5 series) only accept the default (1):400 'temperature' does not support 0 with this model.3. Function tools +
reasoning_effortrejected by gpt-5.6 models — gpt-5.6-luna/sol reject tools combined with any effort other than"none", and their default effort is not"none", so omitting the field still 400s (verified with raw curl).4.
thinking: enabled/disabledsent the Anthropic object unconditionally — OpenAI rejects the unknown top-levelthinkingparameter with a 400, so both values were broken against OpenAI.The fixes
internal/loop/loop.go:ApplyCacheMarkersnow only runs whenClient.IsAnthropic()(new). OpenAI/DeepSeek cache automatically — no markers needed.internal/llm/client.go:modelForbidsTemperature(prefix match ono1/o3/o4/gpt-5) omits the temperature field for reasoning models only; every other provider/model keeps the deterministictemperature: 0default.internal/llm/client.go: learned fallback — when a 400 namesreasoning_effortas the offending parameter on a tool-bearing request, an atomicforceNoneEffortflag is set, the call retries once withreasoning_effort: "none", and subsequent tool calls send it directly (no more failed round-trips). Tool-less calls keep the configured effort.internal/llm/client.go: thinking shaping extracted intobuildCallParamsand gated per provider viasendsThinkingObject(Anthropic/DeepSeek unchanged). OpenAI reasoning models mapenabled→reasoning_effort: "high"anddisabled→"none"; non-reasoning models omit both fields.Verified live (api.openai.com + api.deepseek.com)
math_eval→ 391)DeepSeek regression notes: native cache metrics (
cache_creation/cache_read) still flow, and the deterministictemperature: 0default is still sent for non-reasoning models.Tests
TestClient_IsAnthropic,TestCall_OmitsTemperatureForReasoningModels,TestCall_LearnsReasoningEffortNoneWithTools,TestBuildCallParams_ThinkingMapping(client),TestEngine_PromptCaching_NonAnthropicSkipsMarkers(loop)buildCallParamsdirectlygo test ./internal/llm/ ./internal/loop/ ./cmd/odek/ -count=1— all pass