Skip to content

fix(llm_judge): OpenAI base URL, max_completion_tokens for reasoning models, PIL image support#1381

Open
Luodian wants to merge 2 commits into
mainfrom
fix/llm-judge-openai-providers
Open

fix(llm_judge): OpenAI base URL, max_completion_tokens for reasoning models, PIL image support#1381
Luodian wants to merge 2 commits into
mainfrom
fix/llm-judge-openai-providers

Conversation

@Luodian

@Luodian Luodian commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Fix a malformed OPENAI_API_URL default (.../v1/chat/completions/v1) that was being passed straight into the OpenAI(base_url=...) client; now stores a clean base URL (default https://api.openai.com/v1) and appends /chat/completions only for the raw-requests fallback path.
  • Apply the same base-URL fix to the async provider, and additionally pass base_url into the AsyncOpenAI(...) client construction -- previously a custom OPENAI_API_URL override was silently ignored by the async client (only the aiohttp fallback respected it).
  • Use max_completion_tokens instead of max_tokens for gpt-5*/o1*/o3*/o4* models in both the sync and async providers, via a shared build_sampling_kwargs helper (single source of truth so the two can't drift). For those reasoning models the helper also omits temperature, which they reject at non-default values.
  • Accept PIL.Image.Image objects directly in OpenAIProvider._add_images_to_messages (in addition to file-path strings and pre-encoded bytes), reusing the existing encode_image_to_base64 helper which already supports Union[Image.Image, str].

Compatibility note

OPENAI_API_URL now means a base URL (e.g. https://api.openai.com/v1 or your proxy's base). If you previously set it to a full .../chat/completions endpoint, drop that suffix — the SDK client appends the route itself.

Test plan

  • python3 -m py_compile on changed files
  • uvx ruff check -- clean
  • Ephemeral env import + runtime check: default/custom OPENAI_API_URL resolves correctly for both sync and async clients and fallback paths; PIL image branch produces a correct data:image/jpeg;base64,... URL; existing bytes-branch behavior unchanged
  • Payload matrix: gpt-4o -> max_tokens + temperature; gpt-5-mini / o3 -> max_completion_tokens only (no temperature, no max_tokens); sync and async providers verified to share the same helper object

Review pass (2026-07-06)

Self-review caught that the original commit applied the max_completion_tokens switch to the sync provider only — the async provider (the default concurrent path) still sent max_tokens, and both providers always sent temperature. Follow-up commit centralizes the logic in build_sampling_kwargs (used by both) and fixes stale images type hints.

Follow-up candidate (out of scope here): azure_openai.py / async_azure_openai.py have the same latent issue (always send temperature + max_tokens); the shared helper is import-ready for them.

@Luodian Luodian marked this pull request as ready for review July 6, 2026 13:56
@Luodian Luodian requested review from kcz358 and pufanyi July 6, 2026 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant