Describe your environment
OS: macOS (also reproduced on Linux)
Python version: Python 3.12+
Package version: opentelemetry-instrumentation-openai-v2 v2.3b0
What happened?
ChoiceBuffer.append_tool_call() unconditionally appends tool_call.function.arguments to its internal buffer list. During BaseStreamWrapper.cleanup(), the buffer is serialized with "".join(...). If any provider sends arguments=None on a tool-call delta chunk (instead of arguments=""), this crashes with TypeError: sequence item 0: expected str instance, NoneType found.
The OpenAI API sends arguments="" on the first tool-call delta, but many OpenAI-compatible providers (vLLM, TGI, etc.) send arguments=None. This is a known pattern across the ecosystem — see vllm#9693, pydantic-ai#1654, gptel#1283.
In llama-stack, this kills the stream mid-flight and causes silent failures (no conversation storage, no assistant message). We've added a workaround normalizing arguments=None → "" before yielding chunks (see llama-stack#5200), but the fix belongs here.
Steps to Reproduce
from opentelemetry.instrumentation.openai_v2.patch import ChoiceBuffer
from openai.types.chat.chat_completion_chunk import ChoiceDeltaToolCall, ChoiceDeltaToolCallFunction
buf = ChoiceBuffer(0)
buf.append_tool_call(ChoiceDeltaToolCall(
index=0,
id="call_1",
type="function",
function=ChoiceDeltaToolCallFunction(name="get_weather", arguments=None),
))
buf.append_tool_call(ChoiceDeltaToolCall(
index=0,
function=ChoiceDeltaToolCallFunction(arguments='{"city": "NYC"}'),
))
# This crashes:
"".join(buf.tool_call_buffers[0].arguments)Expected Result
ChoiceBuffer should handle arguments=None gracefully — either skip it or coerce it to "" before appending. The stream should complete without error.
Actual Result
TypeError: sequence item 0: expected str instance, NoneType found
The stream is killed, and any downstream consumer (e.g., llama-stack's StreamingResponseOrchestrator) receives an unexpected exception instead of StopAsyncIteration.
Additional context
A suggested fix is a one-line null check in ChoiceBuffer.append_tool_call():
# Before:
self.tool_call_buffers[index].arguments.append(tool_call.function.arguments)
# After:
if tool_call.function.arguments is not None:
self.tool_call_buffers[index].arguments.append(tool_call.function.arguments)This also affects the Responses API instrumentation work (#3436, #4166, #4280, #4337) since it uses the same BaseStreamWrapper code path.
Would you like to implement a fix?
No
Describe your environment
OS: macOS (also reproduced on Linux)
Python version: Python 3.12+
Package version: opentelemetry-instrumentation-openai-v2 v2.3b0
What happened?
ChoiceBuffer.append_tool_call()unconditionally appendstool_call.function.argumentsto its internal buffer list. DuringBaseStreamWrapper.cleanup(), the buffer is serialized with"".join(...). If any provider sendsarguments=Noneon a tool-call delta chunk (instead ofarguments=""), this crashes withTypeError: sequence item 0: expected str instance, NoneType found.The OpenAI API sends
arguments=""on the first tool-call delta, but many OpenAI-compatible providers (vLLM, TGI, etc.) sendarguments=None. This is a known pattern across the ecosystem — see vllm#9693, pydantic-ai#1654, gptel#1283.In llama-stack, this kills the stream mid-flight and causes silent failures (no conversation storage, no assistant message). We've added a workaround normalizing
arguments=None→""before yielding chunks (see llama-stack#5200), but the fix belongs here.Steps to Reproduce
Expected Result
ChoiceBuffershould handlearguments=Nonegracefully — either skip it or coerce it to""before appending. The stream should complete without error.Actual Result
The stream is killed, and any downstream consumer (e.g., llama-stack's
StreamingResponseOrchestrator) receives an unexpected exception instead ofStopAsyncIteration.Additional context
A suggested fix is a one-line null check in
ChoiceBuffer.append_tool_call():This also affects the Responses API instrumentation work (#3436, #4166, #4280, #4337) since it uses the same
BaseStreamWrappercode path.Would you like to implement a fix?
No