Skip to content

Commit 4975a18

Browse files
Guard subset filter against non-mapping expected_usage
Address PR review feedback: the conformance fixture's typed model allows usage: null. When that happens, expected_usage is None and the subset filter's k in expected_usage raises TypeError instead of producing a clean assertion failure. Wrap the subset filter in an isinstance(dict) guard. The mapping path keeps the same subset semantics; the non-mapping path falls back to direct comparison so the assert fires with a clear shape mismatch.
1 parent a7d3cae commit 4975a18

1 file changed

Lines changed: 17 additions & 10 deletions

File tree

tests/conformance/test_llm_provider.py

Lines changed: 17 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -430,17 +430,24 @@ def _assert_response_matches(actual: Response, expected: Mapping[str, Any]) -> N
430430
assert actual.finish_reason == expected["finish_reason"]
431431
if "usage" in expected:
432432
expected_usage = expected["usage"]
433-
# Subset comparison: spec fixtures assert which usage fields
434-
# MUST be present with what values. Impl-extension fields
435-
# outside the fixture's expected set (e.g., the 0047
436-
# cache-stat fields on impls that have adopted them but
437-
# against fixtures that pre-date the proposal) are ignored
438-
# when the fixture doesn't assert about them. A fixture key
439-
# that's absent from actual surfaces as a missing key in the
440-
# filtered dict and fails the comparison; the impl can't
441-
# silently drop a field the spec requires.
442433
actual_usage_full = actual.usage.model_dump()
443-
actual_usage = {k: v for k, v in actual_usage_full.items() if k in expected_usage}
434+
# Subset comparison when the fixture asserts about specific
435+
# usage fields: spec fixtures pin which fields MUST be present
436+
# with what values. Impl-extension fields outside the fixture's
437+
# expected set (e.g., the 0047 cache-stat fields on impls that
438+
# have adopted them but against fixtures that pre-date the
439+
# proposal) are ignored when the fixture doesn't assert about
440+
# them. A fixture key that's absent from actual surfaces as a
441+
# missing key in the filtered dict and fails the comparison;
442+
# the impl can't silently drop a field the spec requires.
443+
#
444+
# Non-mapping expected_usage (e.g., a fixture sets usage: null)
445+
# falls back to direct comparison so the assertion fires with a
446+
# clean shape mismatch rather than crashing on the subset filter.
447+
if isinstance(expected_usage, dict):
448+
actual_usage = {k: v for k, v in actual_usage_full.items() if k in expected_usage}
449+
else:
450+
actual_usage = actual_usage_full
444451
assert actual_usage == expected_usage, (
445452
f"usage mismatch: actual={actual_usage}, expected={expected_usage}"
446453
)

0 commit comments

Comments
 (0)