-
-
Notifications
You must be signed in to change notification settings - Fork 1.8k
feat(core): Send gen_ai spans as v2 envelope items #20342
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
andreiborza
wants to merge
22
commits into
develop
Choose a base branch
from
ab/gen-ai-span-v2-poc
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 3 commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
d0cb9d0
feat(core): Send gen_ai spans as v2 envelope items
andreiborza 69b5cf8
Remove SDK-side enrichment and redundant op/origin backfill
andreiborza 257c9fb
Pre-check wether transactions have gen_ai spans when we construct tra…
andreiborza eeb7812
Inline span conversion
andreiborza fd67edd
Merge remote-tracking branch 'origin/develop' into ab/gen-ai-span-v2-poc
andreiborza dc21b49
Stringify array attributes in Vercel AI integration for v2 serializat…
andreiborza d97c7f2
Update Vercel AI Node tests
andreiborza d51f4f2
Update Vercel AI E2E tests
andreiborza bed75f7
Update Anthropic Node tests
andreiborza 0474787
Update Anthropic Cloudflare tests
andreiborza 2630a24
Update OpenAI Node tests
andreiborza 7eea494
Update OpenAI Cloudflare tests
andreiborza 5d2ec5a
Update LangChain Node tests
andreiborza 9968bc8
Update LangChain Cloudflare tests
andreiborza 9e8767b
Update LangGraph Node tests
andreiborza 6a62f0b
Update LangGraph Cloudflare tests
andreiborza 0667e9d
Update Google GenAI Node tests
andreiborza 26f9157
Update Google GenAI Cloudflare tests
andreiborza 76d835c
Update size limits
andreiborza 5f5d49e
Flip `enableTruncation` default to false
andreiborza 7fc4b17
Merge remote-tracking branch 'origin/develop' into ab/gen-ai-span-v2-poc
andreiborza e6ecd23
Bump size limit
andreiborza File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| import type { Client } from '../../client'; | ||
| import type { SpanContainerItem } from '../../types-hoist/envelope'; | ||
| import type { Event } from '../../types-hoist/event'; | ||
| import { hasSpanStreamingEnabled } from './hasSpanStreamingEnabled'; | ||
| import { spanJsonToSerializedStreamedSpan } from './spanJsonToStreamedSpan'; | ||
|
|
||
| /** | ||
| * Extracts gen_ai spans from a transaction event, converts them to span v2 format, | ||
| * and returns them as a SpanContainerItem. | ||
| * | ||
| * Only applies to static mode (non-streaming) transactions. | ||
| * | ||
| * WARNING: This function mutates `event.spans` by removing the extracted gen_ai spans | ||
| * from the array. Call this before creating the event envelope so the transaction | ||
| * item does not include the extracted spans. | ||
| */ | ||
| export function extractGenAiSpansFromEvent(event: Event, client: Client): SpanContainerItem | undefined { | ||
| if ( | ||
| event.type !== 'transaction' || | ||
| !event.spans?.length || | ||
| !event.sdkProcessingMetadata?.hasGenAiSpans || | ||
| hasSpanStreamingEnabled(client) | ||
| ) { | ||
| return undefined; | ||
| } | ||
|
|
||
| const genAiSpans = []; | ||
| const remainingSpans = []; | ||
|
|
||
| for (const span of event.spans) { | ||
| if (span.op?.startsWith('gen_ai.')) { | ||
| genAiSpans.push(span); | ||
| } else { | ||
| remainingSpans.push(span); | ||
| } | ||
| } | ||
|
|
||
| if (genAiSpans.length === 0) { | ||
| return undefined; | ||
| } | ||
|
|
||
| const serializedSpans = genAiSpans.map(span => spanJsonToSerializedStreamedSpan(span)); | ||
|
|
||
| // Remove gen_ai spans from the legacy transaction | ||
| event.spans = remainingSpans; | ||
|
|
||
| return [ | ||
| { type: 'span', item_count: serializedSpans.length, content_type: 'application/vnd.sentry.items.span.v2+json' }, | ||
| { items: serializedSpans }, | ||
| ]; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| import type { RawAttributes } from '../../attributes'; | ||
| import type { SerializedStreamedSpan, SpanJSON, StreamedSpanJSON } from '../../types-hoist/span'; | ||
| import { streamedSpanJsonToSerializedSpan } from '../../utils/spanUtils'; | ||
|
|
||
| /** | ||
| * Converts a v1 SpanJSON (from a legacy transaction) to a serialized v2 StreamedSpan. | ||
| */ | ||
| export function spanJsonToSerializedStreamedSpan(span: SpanJSON): SerializedStreamedSpan { | ||
| const streamedSpan: StreamedSpanJSON = { | ||
| trace_id: span.trace_id, | ||
| span_id: span.span_id, | ||
| parent_span_id: span.parent_span_id, | ||
| name: span.description || '', | ||
|
alexander-alderman-webb marked this conversation as resolved.
|
||
| start_timestamp: span.start_timestamp, | ||
| end_timestamp: span.timestamp || span.start_timestamp, | ||
| status: !span.status || span.status === 'ok' || span.status === 'cancelled' ? 'ok' : 'error', | ||
|
andreiborza marked this conversation as resolved.
|
||
| is_segment: false, | ||
| attributes: { ...(span.data as RawAttributes<Record<string, unknown>>) }, | ||
| links: span.links, | ||
| }; | ||
|
|
||
| return streamedSpanJsonToSerializedSpan(streamedSpan); | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
144 changes: 144 additions & 0 deletions
144
packages/core/test/lib/tracing/spans/extractGenAiSpans.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,144 @@ | ||
| import { describe, expect, it } from 'vitest'; | ||
| import type { Event } from '../../../../src/types-hoist/event'; | ||
| import type { SpanJSON } from '../../../../src/types-hoist/span'; | ||
| import { extractGenAiSpansFromEvent } from '../../../../src/tracing/spans/extractGenAiSpans'; | ||
| import { getDefaultTestClientOptions, TestClient } from '../../../mocks/client'; | ||
|
|
||
| function makeSpanJSON(overrides: Partial<SpanJSON> = {}): SpanJSON { | ||
| return { | ||
| span_id: 'abc123def456789a', | ||
| trace_id: '00112233445566778899aabbccddeeff', | ||
| start_timestamp: 1000, | ||
| data: {}, | ||
| ...overrides, | ||
| }; | ||
| } | ||
|
|
||
| function makeTransactionEvent(spans: SpanJSON[], hasGenAiSpans = false): Event { | ||
| return { | ||
| type: 'transaction', | ||
| transaction: 'GET /api/chat', | ||
| release: '1.0.0', | ||
| environment: 'production', | ||
| contexts: { | ||
| trace: { | ||
| span_id: 'root0000deadbeef', | ||
| trace_id: '00112233445566778899aabbccddeeff', | ||
| }, | ||
| }, | ||
| sdkProcessingMetadata: { | ||
| ...(hasGenAiSpans && { hasGenAiSpans: true }), | ||
| }, | ||
| spans, | ||
| }; | ||
| } | ||
|
|
||
| function makeClient(options: Partial<Parameters<typeof getDefaultTestClientOptions>[0]> = {}): TestClient { | ||
| return new TestClient( | ||
| getDefaultTestClientOptions({ | ||
| dsn: 'https://dsn@ingest.f00.f00/1', | ||
| ...options, | ||
| }), | ||
| ); | ||
| } | ||
|
|
||
| describe('extractGenAiSpansFromEvent', () => { | ||
| it('extracts gen_ai spans and removes them from the event', () => { | ||
| const genAiSpan = makeSpanJSON({ | ||
| span_id: 'genai001', | ||
| op: 'gen_ai.chat', | ||
| description: 'chat gpt-4', | ||
| timestamp: 1005, | ||
| }); | ||
| const httpSpan = makeSpanJSON({ | ||
| span_id: 'http001', | ||
| op: 'http.client', | ||
| description: 'GET /api', | ||
| timestamp: 1002, | ||
| }); | ||
|
|
||
| const event = makeTransactionEvent([genAiSpan, httpSpan], true); | ||
| const result = extractGenAiSpansFromEvent(event, makeClient()); | ||
|
|
||
| expect(result).toBeDefined(); | ||
| const [headers, payload] = result!; | ||
| expect(headers.type).toBe('span'); | ||
| expect(headers.item_count).toBe(1); | ||
| expect(headers.content_type).toBe('application/vnd.sentry.items.span.v2+json'); | ||
| expect(payload.items).toHaveLength(1); | ||
| expect(payload.items[0]!.span_id).toBe('genai001'); | ||
| expect(payload.items[0]!.name).toBe('chat gpt-4'); | ||
|
|
||
| expect(event.spans).toHaveLength(1); | ||
| expect(event.spans![0]!.span_id).toBe('http001'); | ||
| }); | ||
|
|
||
| it('extracts multiple gen_ai spans', () => { | ||
| const chatSpan = makeSpanJSON({ span_id: 'chat001', op: 'gen_ai.chat', description: 'chat' }); | ||
| const embeddingsSpan = makeSpanJSON({ span_id: 'embed001', op: 'gen_ai.embeddings', description: 'embed' }); | ||
| const agentSpan = makeSpanJSON({ span_id: 'agent001', op: 'gen_ai.invoke_agent', description: 'agent' }); | ||
| const dbSpan = makeSpanJSON({ span_id: 'db001', op: 'db.query', description: 'SELECT *' }); | ||
|
|
||
| const event = makeTransactionEvent([chatSpan, embeddingsSpan, dbSpan, agentSpan], true); | ||
| const result = extractGenAiSpansFromEvent(event, makeClient()); | ||
|
|
||
| expect(result).toBeDefined(); | ||
| expect(result![0].item_count).toBe(3); | ||
| expect(result![1].items).toHaveLength(3); | ||
| expect(result![1].items.map(s => s.span_id)).toEqual(['chat001', 'embed001', 'agent001']); | ||
|
|
||
| expect(event.spans).toHaveLength(1); | ||
| expect(event.spans![0]!.span_id).toBe('db001'); | ||
| }); | ||
|
|
||
| it('returns undefined when hasGenAiSpans flag is not set', () => { | ||
| const event = makeTransactionEvent([makeSpanJSON({ op: 'gen_ai.chat' })], false); | ||
|
|
||
| expect(extractGenAiSpansFromEvent(event, makeClient())).toBeUndefined(); | ||
| expect(event.spans).toHaveLength(1); | ||
| }); | ||
|
|
||
| it('returns undefined when there are no gen_ai spans', () => { | ||
| const event = makeTransactionEvent([makeSpanJSON({ op: 'http.client' }), makeSpanJSON({ op: 'db.query' })], true); | ||
|
|
||
| expect(extractGenAiSpansFromEvent(event, makeClient())).toBeUndefined(); | ||
| expect(event.spans).toHaveLength(2); | ||
| }); | ||
|
|
||
| it('returns undefined when event has no spans', () => { | ||
| const event = makeTransactionEvent([]); | ||
| expect(extractGenAiSpansFromEvent(event, makeClient())).toBeUndefined(); | ||
| }); | ||
|
|
||
| it('returns undefined when event is not a transaction', () => { | ||
| const event: Event = { type: undefined, spans: [makeSpanJSON({ op: 'gen_ai.chat' })] }; | ||
| expect(extractGenAiSpansFromEvent(event, makeClient())).toBeUndefined(); | ||
| }); | ||
|
|
||
| it('returns undefined when span streaming is enabled', () => { | ||
| const event = makeTransactionEvent([makeSpanJSON({ op: 'gen_ai.chat' })], true); | ||
| const client = makeClient({ traceLifecycle: 'stream' }); | ||
|
|
||
| expect(extractGenAiSpansFromEvent(event, client)).toBeUndefined(); | ||
| expect(event.spans).toHaveLength(1); | ||
| }); | ||
|
|
||
| it('preserves parent_span_id pointing to v1 spans', () => { | ||
| const genAiSpan = makeSpanJSON({ | ||
| span_id: 'genai001', | ||
| parent_span_id: 'http001', | ||
| op: 'gen_ai.chat', | ||
| }); | ||
| const httpSpan = makeSpanJSON({ | ||
| span_id: 'http001', | ||
| op: 'http.client', | ||
| }); | ||
|
|
||
| const event = makeTransactionEvent([httpSpan, genAiSpan], true); | ||
| const result = extractGenAiSpansFromEvent(event, makeClient()); | ||
|
|
||
| expect(result![1].items[0]!.parent_span_id).toBe('http001'); | ||
| expect(event.spans).toHaveLength(1); | ||
| expect(event.spans![0]!.span_id).toBe('http001'); | ||
| }); | ||
| }); |
93 changes: 93 additions & 0 deletions
93
packages/core/test/lib/tracing/spans/spanJsonToStreamedSpan.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| import { describe, expect, it } from 'vitest'; | ||
| import type { SpanJSON } from '../../../../src/types-hoist/span'; | ||
| import { spanJsonToSerializedStreamedSpan } from '../../../../src/tracing/spans/spanJsonToStreamedSpan'; | ||
|
|
||
| function makeSpanJSON(overrides: Partial<SpanJSON> = {}): SpanJSON { | ||
| return { | ||
| span_id: 'abc123def456789a', | ||
| trace_id: '00112233445566778899aabbccddeeff', | ||
| start_timestamp: 1000, | ||
| data: {}, | ||
| ...overrides, | ||
| }; | ||
| } | ||
|
|
||
| describe('spanJsonToSerializedStreamedSpan', () => { | ||
| it('maps basic SpanJSON fields to StreamedSpan fields', () => { | ||
| const span = makeSpanJSON({ | ||
| description: 'chat gpt-4', | ||
| timestamp: 1005, | ||
| status: 'ok', | ||
| op: 'gen_ai.chat', | ||
| origin: 'auto.ai.openai', | ||
| parent_span_id: 'parent00deadbeef', | ||
| }); | ||
|
|
||
| const result = spanJsonToSerializedStreamedSpan(span); | ||
|
|
||
| expect(result.name).toBe('chat gpt-4'); | ||
| expect(result.start_timestamp).toBe(1000); | ||
| expect(result.end_timestamp).toBe(1005); | ||
| expect(result.status).toBe('ok'); | ||
| expect(result.is_segment).toBe(false); | ||
| expect(result.span_id).toBe('abc123def456789a'); | ||
| expect(result.trace_id).toBe('00112233445566778899aabbccddeeff'); | ||
| expect(result.parent_span_id).toBe('parent00deadbeef'); | ||
| }); | ||
|
|
||
| it('uses empty string for name when description is undefined', () => { | ||
| const result = spanJsonToSerializedStreamedSpan(makeSpanJSON({ description: undefined })); | ||
| expect(result.name).toBe(''); | ||
| }); | ||
|
|
||
| it('uses start_timestamp as end_timestamp when timestamp is undefined', () => { | ||
| const result = spanJsonToSerializedStreamedSpan(makeSpanJSON({ timestamp: undefined })); | ||
| expect(result.end_timestamp).toBe(1000); | ||
| }); | ||
|
|
||
| it('maps v1 status strings to v2 ok/error', () => { | ||
| const cases: Array<[string | undefined, 'ok' | 'error']> = [ | ||
| [undefined, 'ok'], | ||
| ['ok', 'ok'], | ||
| ['cancelled', 'ok'], | ||
| ['internal_error', 'error'], | ||
| ['not_found', 'error'], | ||
| ['unknown_error', 'error'], | ||
| ]; | ||
|
|
||
| for (const [v1Status, expected] of cases) { | ||
| const result = spanJsonToSerializedStreamedSpan(makeSpanJSON({ status: v1Status })); | ||
| expect(result.status).toBe(expected); | ||
| } | ||
| }); | ||
|
|
||
| it('preserves existing span data attributes', () => { | ||
| const span = makeSpanJSON({ | ||
| data: { | ||
| 'gen_ai.system': 'openai', | ||
| 'gen_ai.request.model': 'gpt-4', | ||
| 'gen_ai.usage.input_tokens': 100, | ||
| 'gen_ai.usage.output_tokens': 50, | ||
| }, | ||
| }); | ||
|
|
||
| const result = spanJsonToSerializedStreamedSpan(span); | ||
|
|
||
| expect(result.attributes?.['gen_ai.system']).toEqual({ type: 'string', value: 'openai' }); | ||
| expect(result.attributes?.['gen_ai.request.model']).toEqual({ type: 'string', value: 'gpt-4' }); | ||
| expect(result.attributes?.['gen_ai.usage.input_tokens']).toEqual({ type: 'integer', value: 100 }); | ||
| expect(result.attributes?.['gen_ai.usage.output_tokens']).toEqual({ type: 'integer', value: 50 }); | ||
| }); | ||
|
|
||
| it('carries over links', () => { | ||
| const span = makeSpanJSON({ | ||
| links: [{ trace_id: 'aabb', span_id: 'ccdd', sampled: true, attributes: { foo: 'bar' } }], | ||
| }); | ||
|
|
||
| const result = spanJsonToSerializedStreamedSpan(span); | ||
|
|
||
| expect(result.links).toEqual([ | ||
| { trace_id: 'aabb', span_id: 'ccdd', sampled: true, attributes: { foo: { type: 'string', value: 'bar' } } }, | ||
| ]); | ||
| }); | ||
| }); |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
l: not quite sure, but maybe we can just use
splicehere instead of creating a new array?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Splicing becomes awkward, you'd have to backward splice, or do some pre-collecting of indices iteration... not much gain there.
If we splice backwards, we'd need to reverse gen_ai spans array, I think the code complexity and minimal performance gain from that (still have to reverse) is not worth it.
If we can send the ai spans in reverse order, we'd save some time but that's bound to create some confusion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I at least got rid of the extra
.map