Skip to content

fix(core): Improve Vercel AI SDK instrumentation attributes #19717

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
RulaKhaled merged 9 commits into from
Mar 11, 2026
Merged

fix(core): Improve Vercel AI SDK instrumentation attributes #19717

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
fbe9b90
add description attribute to tool calls
RulaKhaled Mar 9, 2026
8aec90b
update with GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE
RulaKhaled Mar 9, 2026
2945993
check for vercel media types
RulaKhaled Mar 9, 2026
0500875
fix media type
RulaKhaled Mar 9, 2026
f286530
deprecate other attributes
RulaKhaled Mar 10, 2026
3b1e354
bump size
RulaKhaled Mar 10, 2026
0ead8e1
fix nextjs test
RulaKhaled Mar 10, 2026
190d760
resolve pr review comments
RulaKhaled Mar 10, 2026
8e02c00
remove truncation
RulaKhaled Mar 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .size-limit.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -326,7 +326,7 @@ module.exports = [
import: createImport('init'),
ignore: [...builtinModules, ...nodePrefixedBuiltinModules],
gzip: true,
limit: '175 KB',
limit: '176 KB',
},
{
name: '@sentry/node - without tracing',
Expand Down
4 changes: 2 additions & 2 deletions dev-packages/e2e-tests/test-applications/nextjs-15/tests/ai-test.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,14 @@ test('should create AI spans with correct attributes', async ({ page }) => {
expect(firstPipelineSpan?.data?.['vercel.ai.model.id']).toBe('mock-model-id');
expect(firstPipelineSpan?.data?.['vercel.ai.model.provider']).toBe('mock-provider');
expect(firstPipelineSpan?.data?.['vercel.ai.prompt']).toContain('Where is the first span?');
expect(firstPipelineSpan?.data?.['gen_ai.response.text']).toBe('First span here!');
expect(firstPipelineSpan?.data?.['gen_ai.output.messages']).toContain('First span here!');
expect(firstPipelineSpan?.data?.['gen_ai.usage.input_tokens']).toBe(10);
expect(firstPipelineSpan?.data?.['gen_ai.usage.output_tokens']).toBe(20); */

// Second AI call - explicitly enabled telemetry
const secondPipelineSpan = aiPipelineSpans[0];
expect(secondPipelineSpan?.data?.['vercel.ai.prompt']).toContain('Where is the second span?');
expect(secondPipelineSpan?.data?.['gen_ai.response.text']).toContain('Second span here!');
expect(secondPipelineSpan?.data?.['gen_ai.output.messages']).toContain('Second span here!');

// Third AI call - with tool calls
/* const thirdPipelineSpan = aiPipelineSpans[2];
Expand Down
4 changes: 2 additions & 2 deletions dev-packages/e2e-tests/test-applications/nextjs-16/tests/ai-test.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,14 @@ test('should create AI spans with correct attributes', async ({ page }) => {
expect(firstPipelineSpan?.data?.['vercel.ai.model.id']).toBe('mock-model-id');
expect(firstPipelineSpan?.data?.['vercel.ai.model.provider']).toBe('mock-provider');
expect(firstPipelineSpan?.data?.['vercel.ai.prompt']).toContain('Where is the first span?');
expect(firstPipelineSpan?.data?.['gen_ai.response.text']).toBe('First span here!');
expect(firstPipelineSpan?.data?.['gen_ai.output.messages']).toContain('First span here!');
expect(firstPipelineSpan?.data?.['gen_ai.usage.input_tokens']).toBe(10);
expect(firstPipelineSpan?.data?.['gen_ai.usage.output_tokens']).toBe(20); */

// Second AI call - explicitly enabled telemetry
const secondPipelineSpan = aiPipelineSpans[0];
expect(secondPipelineSpan?.data?.['vercel.ai.prompt']).toContain('Where is the second span?');
expect(secondPipelineSpan?.data?.['gen_ai.response.text']).toContain('Second span here!');
expect(secondPipelineSpan?.data?.['gen_ai.output.messages']).toContain('Second span here!');

// Third AI call - with tool calls
/* const thirdPipelineSpan = aiPipelineSpans[2];
Expand Down
1 change: 1 addition & 0 deletions dev-packages/node-integration-tests/suites/tracing/vercelai/scenario.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ async function run() {
}),
tools: {
getWeather: {
description: 'Get the current weather for a location',
parameters: z.object({ location: z.string() }),
execute: async args => {
return `Weather in ${args.location}: Sunny, 72°F`;
Expand Down
36 changes: 21 additions & 15 deletions dev-packages/node-integration-tests/suites/tracing/vercelai/test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,16 @@ import {
GEN_AI_INPUT_MESSAGES_ATTRIBUTE,
GEN_AI_INPUT_MESSAGES_ORIGINAL_LENGTH_ATTRIBUTE,
GEN_AI_OPERATION_NAME_ATTRIBUTE,
GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE,
GEN_AI_REQUEST_AVAILABLE_TOOLS_ATTRIBUTE,
GEN_AI_REQUEST_MODEL_ATTRIBUTE,
GEN_AI_RESPONSE_FINISH_REASONS_ATTRIBUTE,
GEN_AI_RESPONSE_ID_ATTRIBUTE,
GEN_AI_RESPONSE_MODEL_ATTRIBUTE,
GEN_AI_RESPONSE_TEXT_ATTRIBUTE,
GEN_AI_RESPONSE_TOOL_CALLS_ATTRIBUTE,
GEN_AI_SYSTEM_ATTRIBUTE,
GEN_AI_SYSTEM_INSTRUCTIONS_ATTRIBUTE,
GEN_AI_TOOL_CALL_ID_ATTRIBUTE,
GEN_AI_TOOL_DESCRIPTION_ATTRIBUTE,
GEN_AI_TOOL_INPUT_ATTRIBUTE,
GEN_AI_TOOL_NAME_ATTRIBUTE,
GEN_AI_TOOL_OUTPUT_ATTRIBUTE,
Expand Down Expand Up @@ -91,9 +91,10 @@ describe('Vercel AI integration', () => {
data: {
[GEN_AI_INPUT_MESSAGES_ORIGINAL_LENGTH_ATTRIBUTE]: 1,
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]: '[{"role":"user","content":"Where is the second span?"}]',
[GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE]:
'[{"role":"assistant","parts":[{"type":"text","content":"Second span here!"}],"finish_reason":"stop"}]',
[GEN_AI_REQUEST_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_RESPONSE_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_RESPONSE_TEXT_ATTRIBUTE]: expect.any(String),
[GEN_AI_USAGE_INPUT_TOKENS_ATTRIBUTE]: 10,
[GEN_AI_USAGE_OUTPUT_TOKENS_ATTRIBUTE]: 20,
[GEN_AI_USAGE_TOTAL_TOKENS_ATTRIBUTE]: 30,
Expand All @@ -119,11 +120,12 @@ describe('Vercel AI integration', () => {
data: {
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]: expect.any(String),
[GEN_AI_INPUT_MESSAGES_ORIGINAL_LENGTH_ATTRIBUTE]: 1,
[GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE]:
'[{"role":"assistant","parts":[{"type":"text","content":"Second span here!"}],"finish_reason":"stop"}]',
[GEN_AI_REQUEST_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_RESPONSE_FINISH_REASONS_ATTRIBUTE]: ['stop'],
[GEN_AI_RESPONSE_ID_ATTRIBUTE]: expect.any(String),
[GEN_AI_RESPONSE_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_RESPONSE_TEXT_ATTRIBUTE]: expect.any(String),
[GEN_AI_SYSTEM_ATTRIBUTE]: 'mock-provider',
[GEN_AI_USAGE_INPUT_TOKENS_ATTRIBUTE]: 10,
[GEN_AI_USAGE_OUTPUT_TOKENS_ATTRIBUTE]: 20,
Expand Down Expand Up @@ -201,6 +203,7 @@ describe('Vercel AI integration', () => {
status: 'ok',
}),
// Seventh span - tool call execution span
// Note: gen_ai.tool.description is NOT present when sendDefaultPii: false because ai.prompt.tools is not recorded
expect.objectContaining({
data: {
[GEN_AI_TOOL_CALL_ID_ATTRIBUTE]: 'call-1',
Expand All @@ -220,7 +223,7 @@ describe('Vercel AI integration', () => {
};

const EXPECTED_AVAILABLE_TOOLS_JSON =
'[{"type":"function","name":"getWeather","parameters":{"type":"object","properties":{"location":{"type":"string"}},"required":["location"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}}]';
'[{"type":"function","name":"getWeather","description":"Get the current weather for a location","parameters":{"type":"object","properties":{"location":{"type":"string"}},"required":["location"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}}]';

const EXPECTED_TRANSACTION_DEFAULT_PII_TRUE = {
transaction: 'main',
Expand All @@ -230,9 +233,10 @@ describe('Vercel AI integration', () => {
data: {
[GEN_AI_INPUT_MESSAGES_ORIGINAL_LENGTH_ATTRIBUTE]: 1,
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]: '[{"role":"user","content":"Where is the first span?"}]',
[GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE]:
'[{"role":"assistant","parts":[{"type":"text","content":"First span here!"}],"finish_reason":"stop"}]',
[GEN_AI_REQUEST_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_RESPONSE_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_RESPONSE_TEXT_ATTRIBUTE]: 'First span here!',
[GEN_AI_USAGE_INPUT_TOKENS_ATTRIBUTE]: 10,
[GEN_AI_USAGE_OUTPUT_TOKENS_ATTRIBUTE]: 20,
[GEN_AI_USAGE_TOTAL_TOKENS_ATTRIBUTE]: 30,
Expand Down Expand Up @@ -264,11 +268,12 @@ describe('Vercel AI integration', () => {
[GEN_AI_INPUT_MESSAGES_ORIGINAL_LENGTH_ATTRIBUTE]: 1,
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]:
'[{"role":"user","content":[{"type":"text","text":"Where is the first span?"}]}]',
[GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE]:
'[{"role":"assistant","parts":[{"type":"text","content":"First span here!"}],"finish_reason":"stop"}]',
[GEN_AI_REQUEST_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_RESPONSE_FINISH_REASONS_ATTRIBUTE]: ['stop'],
[GEN_AI_RESPONSE_ID_ATTRIBUTE]: expect.any(String),
[GEN_AI_RESPONSE_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_RESPONSE_TEXT_ATTRIBUTE]: 'First span here!',
[GEN_AI_SYSTEM_ATTRIBUTE]: 'mock-provider',
[GEN_AI_USAGE_INPUT_TOKENS_ATTRIBUTE]: 10,
[GEN_AI_USAGE_OUTPUT_TOKENS_ATTRIBUTE]: 20,
Expand Down Expand Up @@ -302,9 +307,10 @@ describe('Vercel AI integration', () => {
data: {
[GEN_AI_INPUT_MESSAGES_ORIGINAL_LENGTH_ATTRIBUTE]: 1,
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]: '[{"role":"user","content":"Where is the second span?"}]',
[GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE]:
'[{"role":"assistant","parts":[{"type":"text","content":"Second span here!"}],"finish_reason":"stop"}]',
[GEN_AI_REQUEST_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_RESPONSE_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_RESPONSE_TEXT_ATTRIBUTE]: expect.any(String),
[GEN_AI_USAGE_INPUT_TOKENS_ATTRIBUTE]: 10,
[GEN_AI_USAGE_OUTPUT_TOKENS_ATTRIBUTE]: 20,
[GEN_AI_USAGE_TOTAL_TOKENS_ATTRIBUTE]: 30,
Expand Down Expand Up @@ -335,11 +341,12 @@ describe('Vercel AI integration', () => {
data: {
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]: expect.any(String),
[GEN_AI_INPUT_MESSAGES_ORIGINAL_LENGTH_ATTRIBUTE]: 1,
[GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE]:
'[{"role":"assistant","parts":[{"type":"text","content":"Second span here!"}],"finish_reason":"stop"}]',
[GEN_AI_REQUEST_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_RESPONSE_FINISH_REASONS_ATTRIBUTE]: ['stop'],
[GEN_AI_RESPONSE_ID_ATTRIBUTE]: expect.any(String),
[GEN_AI_RESPONSE_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_RESPONSE_TEXT_ATTRIBUTE]: expect.any(String),
[GEN_AI_SYSTEM_ATTRIBUTE]: 'mock-provider',
[GEN_AI_USAGE_INPUT_TOKENS_ATTRIBUTE]: 10,
[GEN_AI_USAGE_OUTPUT_TOKENS_ATTRIBUTE]: 20,
Expand Down Expand Up @@ -373,10 +380,10 @@ describe('Vercel AI integration', () => {
data: {
[GEN_AI_INPUT_MESSAGES_ORIGINAL_LENGTH_ATTRIBUTE]: 1,
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]: '[{"role":"user","content":"What is the weather in San Francisco?"}]',
[GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE]:
'[{"role":"assistant","parts":[{"type":"text","content":"Tool call completed!"},{"type":"tool_call","id":"call-1","name":"getWeather","arguments":"{ \\"location\\": \\"San Francisco\\" }"}],"finish_reason":"tool_call"}]',
[GEN_AI_REQUEST_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_RESPONSE_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_RESPONSE_TEXT_ATTRIBUTE]: 'Tool call completed!',
[GEN_AI_RESPONSE_TOOL_CALLS_ATTRIBUTE]: expect.any(String),
[GEN_AI_USAGE_INPUT_TOKENS_ATTRIBUTE]: 15,
[GEN_AI_USAGE_OUTPUT_TOKENS_ATTRIBUTE]: 25,
[GEN_AI_USAGE_TOTAL_TOKENS_ATTRIBUTE]: 40,
Expand Down Expand Up @@ -408,12 +415,12 @@ describe('Vercel AI integration', () => {
[GEN_AI_REQUEST_AVAILABLE_TOOLS_ATTRIBUTE]: EXPECTED_AVAILABLE_TOOLS_JSON,
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]: expect.any(String),
[GEN_AI_INPUT_MESSAGES_ORIGINAL_LENGTH_ATTRIBUTE]: 1,
[GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE]:
'[{"role":"assistant","parts":[{"type":"text","content":"Tool call completed!"},{"type":"tool_call","id":"call-1","name":"getWeather","arguments":"{ \\"location\\": \\"San Francisco\\" }"}],"finish_reason":"tool_call"}]',
[GEN_AI_REQUEST_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_RESPONSE_FINISH_REASONS_ATTRIBUTE]: ['tool-calls'],
[GEN_AI_RESPONSE_ID_ATTRIBUTE]: expect.any(String),
[GEN_AI_RESPONSE_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_RESPONSE_TEXT_ATTRIBUTE]: 'Tool call completed!',
[GEN_AI_RESPONSE_TOOL_CALLS_ATTRIBUTE]: expect.any(String),
[GEN_AI_SYSTEM_ATTRIBUTE]: 'mock-provider',
[GEN_AI_USAGE_INPUT_TOKENS_ATTRIBUTE]: 15,
[GEN_AI_USAGE_OUTPUT_TOKENS_ATTRIBUTE]: 25,
Expand Down Expand Up @@ -447,6 +454,7 @@ describe('Vercel AI integration', () => {
expect.objectContaining({
data: {
[GEN_AI_TOOL_CALL_ID_ATTRIBUTE]: 'call-1',
[GEN_AI_TOOL_DESCRIPTION_ATTRIBUTE]: 'Get the current weather for a location',
[GEN_AI_TOOL_INPUT_ATTRIBUTE]: expect.any(String),
[GEN_AI_TOOL_NAME_ATTRIBUTE]: 'getWeather',
[GEN_AI_TOOL_OUTPUT_ATTRIBUTE]: expect.any(String),
Expand Down Expand Up @@ -809,7 +817,6 @@ describe('Vercel AI integration', () => {
data: expect.objectContaining({
[GEN_AI_INPUT_MESSAGES_ORIGINAL_LENGTH_ATTRIBUTE]: 3,
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]: expect.stringMatching(/^\[.*"(?:text|content)":"C+".*\]$/),
[GEN_AI_RESPONSE_TEXT_ATTRIBUTE]: 'Response to truncated messages',
}),
}),
// Second call: Last message is small and kept intact
Expand All @@ -819,7 +826,6 @@ describe('Vercel AI integration', () => {
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]: expect.stringContaining(
'This is a small message that fits within the limit',
),
[GEN_AI_RESPONSE_TEXT_ATTRIBUTE]: 'Response to small message',
}),
}),
]),
Expand Down
1 change: 1 addition & 0 deletions dev-packages/node-integration-tests/suites/tracing/vercelai/v5/scenario.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ async function run() {
}),
tools: {
getWeather: tool({
description: 'Get the current weather for a location',
inputSchema: z.object({ location: z.string() }),
execute: async ({ location }) => `Weather in ${location}: Sunny, 72°F`,
}),
Expand Down
Loading
Loading