Fix crash when scorer LLM response is blocked by content filter#2072
Fix crash when scorer LLM response is blocked by content filter#2072hannahwestra25 wants to merge 7 commits into
Conversation
When the scorer's own LLM call is blocked by Azure content filtering, the response contains only an error piece with no text piece. The next() call in _score_value_with_llm_async raises StopIteration, which becomes a RuntimeError inside the async coroutine. Add an explicit check for all-blocked response pieces before the next() call and raise a descriptive BadRequestException instead. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…orer-content-filter-crash
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| # Require every piece to be blocked (all) rather than any: a partially blocked | ||
| # response may still carry a usable text piece, so we only bail out when there | ||
| # is no salvageable content to parse. | ||
| if all(piece.is_blocked() for piece in response[0].message_pieces): |
There was a problem hiding this comment.
When a piece is blocked, every piece is marked blocked. The piece's converted_value has the error JSON. There is partial content in the metadata, but no non-blocked text piece.
So this should be updated because it will never get partial content.
There was a problem hiding this comment.
Good catch, you're right. A content-filter block always comes back as a single fully-blocked error piece (partial content lives in prompt_metadata, not a separate text piece), so the old comment's salvageable text piece reasoning was wrong. Fixed the comment in 0aa44b9 to just say a block yields a single error piece with no parseable text. The all() check is functionally equivalent to checking the first piece here; kept it as a harmless defensive check rather than changing behavior.
A content-filter block always yields a single fully-blocked error piece, so the previous comment's claim about a salvageable text piece was inaccurate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When the scorer's own LLM call is blocked by Azure content filtering (e.g., because it's grading harmful content), the response contains only an error piece with no text piece. The next() generator in _score_value_with_llm_async raises StopIteration, which becomes a RuntimeError inside the async coroutine.
This adds an explicit check before the next() call and raises a descriptive BadRequestException suggesting the user configure a scorer endpoint without content filtering.