Skip to content

fix: guard against None chunks in OpenAI streaming client#7903

Open
jahanzaib-iqbal-dev wants to merge 1 commit into
microsoft:mainfrom
jahanzaib-iqbal-dev:fix/openai-client-streaming-none-chunk
Open

fix: guard against None chunks in OpenAI streaming client#7903
jahanzaib-iqbal-dev wants to merge 1 commit into
microsoft:mainfrom
jahanzaib-iqbal-dev:fix/openai-client-streaming-none-chunk

Conversation

@jahanzaib-iqbal-dev

Copy link
Copy Markdown

Summary

Fixes AttributeError: 'NoneType' object has no attribute 'model' that occurs when the OpenAI streaming response yields a None chunk.

  • Adds a None guard at the top of the streaming loop (before any attribute access) so None chunks are silently skipped.
  • Removes a redundant duplicate maybe_model = chunk.model assignment that appeared later in the same loop body (the assignment already happens earlier in the loop).

Root Cause

In _openai_client.py, chunk.model was accessed before checking if chunk is None. When the API yields a None sentinel chunk (observed in compiled binaries and under heavy endpoint load), this caused an unhandled AttributeError that crashed the streaming session:

# Line 908: Iterate over stream chunks
async for chunk in chunks:
    if first_chunk:
        ...
    # Line 919: chunk.model accessed with NO None guard — crashes if chunk is None
    maybe_model = chunk.model

    # Line 923: Empty chunk check comes AFTER — too late
    if len(chunk.choices) == 0:
        ...

The type annotation on line 893 (chunk: ChatCompletionChunk | None = None) already documents that chunk can be None, but the guard was missing.

Fix

async for chunk in chunks:
    # Guard against None chunks which can be yielded by some endpoints.
    # https://github.com/microsoft/autogen/issues/7130
    if chunk is None:
        continue
    ...
    # Set the model from the latest chunk (only runs after None guard)
    maybe_model = chunk.model

Testing

  • Added test_openai_chat_completion_client_create_stream_none_chunk in test_openai_model_client.py which injects None chunks at both the start and in the middle of the stream and verifies the stream completes successfully without raising AttributeError.
  • All existing streaming tests continue to pass.

Fixes #7130

Fixes AttributeError when the streaming response yields a None chunk
before the None check is performed. Added early continue guard to
skip None chunks in the streaming loop, placed before any attribute
access on the chunk object.

Also removes a redundant `maybe_model = chunk.model` assignment on
line 949 (original) that duplicated the assignment already made
earlier in the loop body.

Fixes microsoft#7130
@jahanzaib-iqbal-dev

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] AttributeError in streaming due to missing None check on chunk.model (lines 919, 949)

1 participant