Skip to content

Normalize OpenAI token params and expose provider finish reasons#709

Open
losingle wants to merge 5 commits intocrmne:mainfrom
losingle:main
Open

Normalize OpenAI token params and expose provider finish reasons#709
losingle wants to merge 5 commits intocrmne:mainfrom
losingle:main

Conversation

@losingle
Copy link
Copy Markdown

This PR improves provider normalization in two areas:

  1. OpenAI chat-completions compatibility
    When max_tokens is provided and max_completion_tokens is not, the OpenAI provider now mirrors the value to max_completion_tokens. This keeps existing callers working while improving compatibility with newer OpenAI-compatible endpoints that expect max_completion_tokens.

  2. Finish reason propagation
    Assistant messages and streaming chunks now expose a normalized finish_reason across OpenAI, Anthropic, and Gemini. The stream accumulator also preserves the final non-nil finish reason on the assembled message.

What changed

  • Added OpenAI-specific param normalization in the provider layer.
  • Preserved explicitly provided max_completion_tokens values.
  • Added finish_reason support to message/chunk objects.
  • Propagated finish_reason through streaming aggregation.
  • Normalized provider-specific stop reasons:
    • OpenAI: tool_calls and function_call -> tool_use
    • Anthropic: end_turn and stop_sequence -> stop, max_tokens -> length
    • Gemini: STOP -> stop, MAX_TOKENS -> length, SAFETY and RECITATION -> content_filter
  • Added focused specs for:
    • OpenAI max_tokens / max_completion_tokens normalization
    • Stream accumulator finish_reason propagation
    • OpenAI, Anthropic, and Gemini finish_reason parsing in sync and streaming paths
  • Cleaned up related spec lint offenses.

Validation

  • Targeted RSpec coverage added and passing
  • RuboCop offenses introduced by the changes were fixed
  • Changes were pushed in three commits:
    • Normalize OpenAI max token params
    • Expose provider finish reasons
    • Fix spec lint offenses

Copilot AI review requested due to automatic review settings March 31, 2026 00:24
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves cross-provider response normalization in RubyLLM by (1) making OpenAI chat-completions token parameters compatible with newer OpenAI-style endpoints and (2) propagating a normalized finish_reason through both sync and streaming flows (including stream accumulation).

Changes:

  • OpenAI provider now mirrors max_tokens into max_completion_tokens when the latter is not explicitly provided.
  • Added finish_reason to RubyLLM::Message/Chunk, normalized provider-specific stop reasons (OpenAI/Anthropic/Gemini), and preserved the final non-nil finish reason in StreamAccumulator.
  • Added/updated focused RSpec coverage across providers and streaming paths; cleaned up spec lint/quoting issues.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
spec/ruby_llm/stream_accumulator_spec.rb Adds coverage that the accumulator keeps the last non-nil finish_reason.
spec/ruby_llm/providers/open_ai/streaming_spec.rb Verifies OpenAI streaming chunks expose normalized finish_reason.
spec/ruby_llm/providers/open_ai/chat_spec.rb Verifies OpenAI sync message parsing normalizes finish_reason (e.g., tool_callstool_use).
spec/ruby_llm/providers/open_ai_provider_spec.rb Verifies OpenAI provider param normalization for max_tokens/max_completion_tokens.
spec/ruby_llm/providers/gemini/streaming_spec.rb Verifies Gemini streaming chunk finish_reason normalization (e.g., MAX_TOKENSlength).
spec/ruby_llm/providers/gemini/chat_spec.rb Verifies Gemini sync message finish_reason normalization (e.g., SAFETYcontent_filter).
spec/ruby_llm/providers/anthropic/streaming_spec.rb Verifies Anthropic streaming chunk finish_reason normalization (e.g., max_tokenslength).
spec/ruby_llm/providers/anthropic/chat_spec.rb Verifies Anthropic sync message finish_reason normalization.
spec/ruby_llm/generators/chat_ui_generator_spec.rb Adjusts string expectations to resolve spec lint/quoting issues.
lib/ruby_llm/stream_accumulator.rb Tracks and emits the final non-nil finish_reason when assembling a message from chunks.
lib/ruby_llm/providers/openai/streaming.rb Adds normalized finish_reason to OpenAI streaming chunks.
lib/ruby_llm/providers/openai/chat.rb Adds normalized finish_reason to OpenAI parsed messages + normalization helper.
lib/ruby_llm/providers/openai.rb Adds OpenAI-specific request param normalization for max_completion_tokens.
lib/ruby_llm/providers/gemini/streaming.rb Adds normalized finish_reason to Gemini streaming chunks + mapping helper.
lib/ruby_llm/providers/gemini/chat.rb Adds normalized finish_reason to Gemini parsed messages + mapping helper.
lib/ruby_llm/providers/anthropic/streaming.rb Adds normalized finish_reason to Anthropic streaming chunks + mapping helper.
lib/ruby_llm/providers/anthropic/chat.rb Adds normalized finish_reason to Anthropic parsed messages + mapping helper.
lib/ruby_llm/message.rb Adds finish_reason to Message API and serialization (to_h).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +131 to +141
def normalize_finish_reason(reason)
case reason
when 'STOP'
'stop'
when 'MAX_TOKENS'
'length'
when 'SAFETY', 'RECITATION'
'content_filter'
else
reason
end
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

normalize_finish_reason duplicates the same mapping logic that also exists in the Gemini streaming implementation. Consider extracting the mapping into a shared helper/constant (e.g., a single module method used by both Chat and Streaming) to prevent the two paths from diverging over time.

Suggested change
def normalize_finish_reason(reason)
case reason
when 'STOP'
'stop'
when 'MAX_TOKENS'
'length'
when 'SAFETY', 'RECITATION'
'content_filter'
else
reason
end
FINISH_REASON_MAP = {
'STOP' => 'stop',
'MAX_TOKENS' => 'length',
'SAFETY' => 'content_filter',
'RECITATION' => 'content_filter'
}.freeze
def normalize_finish_reason(reason)
FINISH_REASON_MAP.fetch(reason, reason)

Copilot uses AI. Check for mistakes.
Comment on lines +142 to +150
def normalize_finish_reason(reason)
case reason
when 'end_turn', 'stop_sequence'
'stop'
when 'max_tokens'
'length'
else
reason
end
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

normalize_finish_reason is duplicated between Anthropic chat and streaming code paths. Consider centralizing the mapping (shared helper/constant) so any future stop_reason additions/changes stay consistent across sync vs streaming parsing.

Suggested change
def normalize_finish_reason(reason)
case reason
when 'end_turn', 'stop_sequence'
'stop'
when 'max_tokens'
'length'
else
reason
end
FINISH_REASON_MAP = {
'end_turn' => 'stop',
'stop_sequence' => 'stop',
'max_tokens' => 'length'
}.freeze
def normalize_finish_reason(reason)
FINISH_REASON_MAP.fetch(reason, reason)

Copilot uses AI. Check for mistakes.
losingle and others added 2 commits March 31, 2026 21:43
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants