Skip to content

test litellm

ff9c5ec
Select commit
Loading
Failed to load commit list.
Draft

feat: Send GenAI spans as V2 envelope items #6079

test litellm
ff9c5ec
Select commit
Loading
Failed to load commit list.
@sentry/warden / warden: find-bugs completed Apr 17, 2026 in 21m 30s

5 issues

find-bugs: Found 5 issues (1 high, 4 medium)

High

Wrong variable `event` passed to `_serialized_v1_span_to_serialized_v2_span` instead of `event_opt` - `sentry_sdk/client.py:1124`

On line 1124, event (the original input parameter) is passed instead of event_opt (the prepared event). The _prepare_event function populates critical fields like release, environment, and sdk info (lines 795-801 in _prepare_event). The _serialized_v1_span_to_serialized_v2_span function extracts these fields to populate span attributes (lines 224-241). Using event means GenAI spans will have missing or incorrect values for sentry.release, sentry.environment, sentry.sdk.name, and sentry.sdk.version attributes.

Also found at:

  • sentry_sdk/client.py:124

Medium

Sorting key uses 'name' twice instead of 'name' and 'description' - `tests/integrations/google_genai/test_google_genai.py:330`

The sorting lambda in test_generate_content_with_tools was changed from key=lambda t: (t.get("name", ""), t.get("description", "")) to key=lambda t: (t.get("name", ""), t.get("name", "")). This appears to be an accidental duplication error. While this may not break the test currently (since tool names are unique in this test), it defeats the purpose of the secondary sort key and could cause non-deterministic test ordering if tools have the same name but different descriptions.

Test uses incorrect 'attributes' key instead of 'data' for Google GenAI inline_data format - `tests/integrations/google_genai/test_google_genai.py:2152-2153`

The test change replaces the correct Google GenAI inline_data key 'data' with 'attributes', which doesn't match the actual Google GenAI API format. The transform_google_content_part function in sentry_sdk/ai/utils.py expects inline_data.get('data', ''), so this test is validating behavior with malformed input rather than the actual API format. While the test may pass because _extract_part_content substitutes content with BLOB_DATA_SUBSTITUTE regardless, it's testing an invalid input structure.

test_multiple_providers fails to capture spans, making span assertions ineffective - `tests/integrations/litellm/test_litellm.py:945`

The test captures only "transaction" items via capture_items("transaction") at line 945, but later at lines 1020-1023 tries to filter for "span" items from the same collection. Since spans were never captured, the assertion loop for span in spans: iterates over an empty list, causing the test to silently pass without actually verifying span data. This defeats the purpose of the test which is supposed to verify that SPANDATA.GEN_AI_SYSTEM is present in spans.

Also found at:

  • tests/integrations/litellm/test_litellm.py:1020-1023
Orphaned _meta after GenAI spans are split from transaction - `tests/integrations/openai/test_openai.py:3758-3760`

In test_openai_message_truncation, the test accesses event["_meta"]["spans"]["0"] to verify truncation metadata for the GenAI span. However, with the V2 envelope changes, GenAI spans are now split out of the transaction via _split_gen_ai_spans() in client.py and sent as separate envelope items. The _meta is generated during serialization (line 848) before the span split occurs (line 1104), leaving orphaned metadata that references a span no longer present in the transaction. The test may pass but validates stale metadata that doesn't correspond to any span in the actual transaction payload.


Duration: 21m 15s · Tokens: 9.6M in / 93.9k out · Cost: $14.50 (+extraction: $0.01, +merge: $0.00, +fix_gate: $0.01)

Annotations

Check failure on line 1124 in sentry_sdk/client.py

See this annotation in the file changed.

@sentry-warden sentry-warden / warden: find-bugs

Wrong variable `event` passed to `_serialized_v1_span_to_serialized_v2_span` instead of `event_opt`

On line 1124, `event` (the original input parameter) is passed instead of `event_opt` (the prepared event). The `_prepare_event` function populates critical fields like `release`, `environment`, and `sdk` info (lines 795-801 in `_prepare_event`). The `_serialized_v1_span_to_serialized_v2_span` function extracts these fields to populate span attributes (lines 224-241). Using `event` means GenAI spans will have missing or incorrect values for `sentry.release`, `sentry.environment`, `sentry.sdk.name`, and `sentry.sdk.version` attributes.

Check failure on line 124 in sentry_sdk/client.py

See this annotation in the file changed.

@sentry-warden sentry-warden / warden: find-bugs

[SD9-MAA] Wrong variable `event` passed to `_serialized_v1_span_to_serialized_v2_span` instead of `event_opt` (additional location)

On line 1124, `event` (the original input parameter) is passed instead of `event_opt` (the prepared event). The `_prepare_event` function populates critical fields like `release`, `environment`, and `sdk` info (lines 795-801 in `_prepare_event`). The `_serialized_v1_span_to_serialized_v2_span` function extracts these fields to populate span attributes (lines 224-241). Using `event` means GenAI spans will have missing or incorrect values for `sentry.release`, `sentry.environment`, `sentry.sdk.name`, and `sentry.sdk.version` attributes.

Check warning on line 330 in tests/integrations/google_genai/test_google_genai.py

See this annotation in the file changed.

@sentry-warden sentry-warden / warden: find-bugs

Sorting key uses 'name' twice instead of 'name' and 'description'

The sorting lambda in `test_generate_content_with_tools` was changed from `key=lambda t: (t.get("name", ""), t.get("description", ""))` to `key=lambda t: (t.get("name", ""), t.get("name", ""))`. This appears to be an accidental duplication error. While this may not break the test currently (since tool names are unique in this test), it defeats the purpose of the secondary sort key and could cause non-deterministic test ordering if tools have the same name but different descriptions.

Check warning on line 2153 in tests/integrations/google_genai/test_google_genai.py

See this annotation in the file changed.

@sentry-warden sentry-warden / warden: find-bugs

Test uses incorrect 'attributes' key instead of 'data' for Google GenAI inline_data format

The test change replaces the correct Google GenAI inline_data key 'data' with 'attributes', which doesn't match the actual Google GenAI API format. The `transform_google_content_part` function in `sentry_sdk/ai/utils.py` expects `inline_data.get('data', '')`, so this test is validating behavior with malformed input rather than the actual API format. While the test may pass because `_extract_part_content` substitutes content with `BLOB_DATA_SUBSTITUTE` regardless, it's testing an invalid input structure.

Check warning on line 945 in tests/integrations/litellm/test_litellm.py

See this annotation in the file changed.

@sentry-warden sentry-warden / warden: find-bugs

test_multiple_providers fails to capture spans, making span assertions ineffective

The test captures only "transaction" items via `capture_items("transaction")` at line 945, but later at lines 1020-1023 tries to filter for "span" items from the same collection. Since spans were never captured, the assertion loop `for span in spans:` iterates over an empty list, causing the test to silently pass without actually verifying span data. This defeats the purpose of the test which is supposed to verify that `SPANDATA.GEN_AI_SYSTEM` is present in spans.

Check warning on line 1023 in tests/integrations/litellm/test_litellm.py

See this annotation in the file changed.

@sentry-warden sentry-warden / warden: find-bugs

[658-SZM] test_multiple_providers fails to capture spans, making span assertions ineffective (additional location)

The test captures only "transaction" items via `capture_items("transaction")` at line 945, but later at lines 1020-1023 tries to filter for "span" items from the same collection. Since spans were never captured, the assertion loop `for span in spans:` iterates over an empty list, causing the test to silently pass without actually verifying span data. This defeats the purpose of the test which is supposed to verify that `SPANDATA.GEN_AI_SYSTEM` is present in spans.

Check warning on line 3760 in tests/integrations/openai/test_openai.py

See this annotation in the file changed.

@sentry-warden sentry-warden / warden: find-bugs

Orphaned _meta after GenAI spans are split from transaction

In `test_openai_message_truncation`, the test accesses `event["_meta"]["spans"]["0"]` to verify truncation metadata for the GenAI span. However, with the V2 envelope changes, GenAI spans are now split out of the transaction via `_split_gen_ai_spans()` in `client.py` and sent as separate envelope items. The `_meta` is generated during serialization (line 848) before the span split occurs (line 1104), leaving orphaned metadata that references a span no longer present in the transaction. The test may pass but validates stale metadata that doesn't correspond to any span in the actual transaction payload.