fix(toolcalling): synthesize tool_calls finish_reason when stream lacks one

[MatejKosec]

Overview:

A streamed OpenAI chat completion can contain a complete tool call and then end without any finish_reason chunk. Strict clients wait for a non-null finish reason before treating the call as complete, which produced the 30-second timeout reported in DGH-967. This PR makes both chat-completions tool-call paths emit finish_reason: "tool_calls" when tool calls were emitted but the upstream stream omitted its terminal reason. Text-only streams do not receive an invented finish reason.

This PR follows merged PR #10988. Its diff now contains only the DGH-967 chat-completions follow-up.

Details:

1. JailedStream::fix_finish_reason now emits a synthetic ToolCalls chunk before an empty-choices usage chunk. An end-of-stream path covers streams without a usage chunk. Synthetic terminal chunks do not copy usage or internal LLM metrics from their template.
2. tool_parser_v2::apply_stream now emits a finish-only terminal chunk before a usage-only response for every unterminated choice that previously emitted tool calls. End of stream remains the fallback when no usage response arrives, and synthetic chunks clear copied usage and metrics.
3. Regression coverage exercises missing finish reasons, text-only streams, usage-chunk ordering, both ToolCalls and legacy FunctionCall finish reasons, and the affected HTTP replay paths.

Where should the reviewer start?

Start with JailedStream::fix_finish_reason in lib/llm/src/protocols/openai/chat_completions/jail.rs, then compare the v2 terminal-finalization path in lib/llm/src/protocols/openai/chat_completions/tool_parser_v2.rs. The focused regressions are adjacent to those implementations and in lib/llm/tests/test_jail.rs.

Related Issues

🚫 This PR is NOT linked to a GitHub issue:

• Confirmed — no related issue
• Internal tracker: DGH-967
• Follow-on to fix(toolcalling): publish fragmented tool calls on finish instead of end-of-stream #10988

Validation

The required GitHub checks on the latest commit are the authoritative validation record. The affected coverage is GPU-free and runs within the existing dynamo-llm Rust test jobs.

[github-actions]

🌿 Fern Docs Preview: https://nvidia-preview-69b37bb1-5afa-4e2a-bad0-aa9e6072d69a.docs.buildwithfern.com/dynamo/dev

[coderabbitai]

Walkthrough

Three stream converters (Anthropic, OpenAI chat-completions jail/tool_parser_v2, Responses) are refactored to buffer tool-call blocks and emit them only when a terminal finish_reason is observed, with EOF fallbacks. Synthetic finish_reason=ToolCalls backstops are added in jail.rs and tool_parser_v2.rs. A shared HTTP test harness, scripted chat engine, SSE fixtures, and end-to-end HTTP replay tests are introduced.

Changes

Tool-call finish-reason buffering across protocol converters

Layer / File(s)	Summary
AnthropicStreamConverter: ToolCallState and buffering helpers lib/llm/src/protocols/anthropic/stream_converter.rs	ToolCallState is restructured with argument_fragments and is_emit_ready(); record_tool_call, drain_buffered_tool_events, and append_buffered_tool_events are added to accumulate and flush tool blocks. tool_blocks_flushed field added to converter state.
AnthropicStreamConverter: finish-reason flush and EOF fallback lib/llm/src/protocols/anthropic/stream_converter.rs	append_chunk_events detects terminal finish_reason to set should_flush_tool_blocks and records tool calls; append_end_events uses drain_buffered_tool_events as EOF fallback; test-only paths mirror production logic.
AnthropicStreamConverter: updated unit tests lib/llm/src/protocols/anthropic/stream_converter.rs	finish_chunk helper nulls tool_calls in delta; all test assertions updated for buffered-flush semantics covering text/tool ordering, fragmented args, incomplete identity, deduplication, and parallel flushing.
jail.rs: synthetic ToolCalls finish-reason backstop lib/llm/src/protocols/openai/chat_completions/jail.rs	fix_finish_reason tracks terminated choices and stores a template response; at stream end it synthesizes a trailing chunk with finish_reason=ToolCalls for choices that emitted tool calls without a terminal finish reason. Tests cover synthesis and text-only absence.
tool_parser_v2.rs: per-choice finish_reason in backstop flush lib/llm/src/protocols/openai/chat_completions/tool_parser_v2.rs	The end-of-stream backstop now sets finish_reason=ToolCalls per choice when tool_emitted contains that index. Two new tests cover synthesized and absent finish_reason.
ResponseStreamConverter: FunctionCallState and identity/delta queuing lib/llm/src/protocols/openai/responses/stream_converter.rs	FunctionCallState gains pending_arg_deltas and optional output_index; has_identity() gates output_item.added and deferred delta emission; should_finish_function_calls controls when done events fire.
ResponseStreamConverter: done-event helpers and end-of-stream path lib/llm/src/protocols/openai/responses/stream_converter.rs	append_pending_function_call_done_events finalizes tool calls sorted by output_index; completed_output builds the ordered output list; append_end_events delegates to the shared helper and emits response.completed.
ResponseStreamConverter: updated unit tests lib/llm/src/protocols/openai/responses/stream_converter.rs	finish_chunk helper added; tests assert done events appear only after finish_reason, identity-only and deferred-delta cases, EOF fallback, multiple tool calls, ordering, and no duplicate done events.
Shared HTTP test harness: HarnessService, ScriptedChatEngine, SSE utils lib/llm/tests/common/http_harness.rs, lib/llm/tests/common/scripted_chat_engine.rs	HarnessService starts/stops a local HttpService with a ScriptedChatEngine; ScriptedChatEngine queues immediate and gated-tail scripts; IncrementalSseParser, parse_json_sse, and canonicalize utilities support HTTP replay tests.
SSE replay fixtures and README lib/llm/tests/data/replays/agent-harness/*	Four agent-harness SSE fixtures (text, fragmented-tool, parallel-tools, thinking-tool) and a README documenting their construction and coverage are added.
Anthropic HTTP replay integration tests lib/llm/tests/anthropic_http_replay.rs	End-to-end tests covering unary and streaming text, fragmented tool arguments, finish-signal ordering (gated tail), parallel tools, tool-result round trip, and count_tokens.
Responses API HTTP replay integration tests lib/llm/tests/responses_http_replay.rs	End-to-end tests covering unary and streaming text, fragmented tool arguments with early-completion guard, gated-tail finish-signal ordering, parallel function calls, and round-trip tool output.

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly matches the main change: synthesizing ToolCalls finish_reason when streams end without one.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	The description includes all required sections, a clear summary, reviewer guidance, and a completed Related Issues choice.

---

_{Comment @coderabbitai help to get the list of available commands.}

[MatejKosec]

Addressed the non-threaded CodeRabbit findings on the current head:

1. The v2 EOF path now emits a finish-only ToolCalls chunk when a choice emitted tool calls earlier and parser.finish() has no additional output. It determines the finish reason after recording calls emitted by the EOF flush. Commit 34850187f6b also clears usage and LLM metrics copied from a usage-only template.
2. ResponseStreamConverter now has mirrored coverage for FinishReason::FunctionCall as well as FinishReason::ToolCalls.
3. The Responses round-trip replay now derives the follow-up call_id from the first response.
4. I did not add duplicate Anthropic HTTP fixtures for EOF without a finish reason. The exact converter cases are already covered by test_tool_only_response_flushes_at_eof_without_finish_reason and test_text_only_response_stop_in_end_events; the DGH-967 chat-completions behavior is covered separately in the v2 and jail regressions. The existing HTTP replays retain their narrower role of checking cross-protocol conversion and finish-signal ordering.

The PR description now follows the repository template and identifies the stacked #10988 scope. All five inline review threads are resolved.