fix: check ai.operationId attribute for operation name transformation#891
fix: check ai.operationId attribute for operation name transformation#891obs-gh-trevorColby wants to merge 7 commits intotraceloop:mainfrom
Conversation
The transformOperationName function checks the span name to determine the gen_ai.operation.name attribute. However, by the time this function runs in onSpanEnd, the span name has already been transformed (e.g., from "ai.generateText" to "run.ai") by transformAiSdkSpanNames in onSpanStart. This means the check for 'generateText', 'streamText', etc. never matches, and the gen_ai.operation.name attribute is never set to 'chat'. This fix checks the ai.operationId attribute first, which is set by the Vercel AI SDK and contains the original operation ID (e.g., "ai.generateText"). This allows the function to correctly identify the operation type. Fixes traceloop#882
|
|
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughPrefer Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment Tip You can enable review details to help with troubleshooting, context usage and more.Enable the |
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
There was a problem hiding this comment.
Important
Looks good to me! 👍
Reviewed everything up to e9ba33a in 13 seconds. Click for details.
- Reviewed
53lines of code in1files - Skipped
0files when reviewing. - Skipped posting
0draft comments. View those below. - Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
Workflow ID: wflow_RYsirwZNEWXy8dqr
You can customize 
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/traceloop-sdk/src/lib/tracing/ai-sdk-transformations.ts`:
- Around line 508-520: operationId (from AI_OPERATION_ID) may be non-string
which makes nameToCheck.includes(...) throw; guard and coerce before using
includes by ensuring operationId is a string (e.g., check typeof operationId ===
"string" or coerce with String(operationId)) and only set nameToCheck to
operationId when it's a string, otherwise fall back to spanName, then run the
existing includes checks on nameToCheck; update the logic around
AI_OPERATION_ID, operationId, nameToCheck and the includes checks to perform
this type guard.
- Around line 505-508: Add the AI_OPERATION_ID constant to the SpanAttributes
export in SemanticAttributes.ts (e.g., add AI_OPERATION_ID: "ai.operationId" to
the SpanAttributes object/type) and then update ai-sdk-transformations.ts to
import that constant instead of using the hardcoded string; replace the local
const AI_OPERATION_ID = "ai.operationId" with an import of AI_OPERATION_ID from
`@traceloop/ai-semantic-conventions` and use that imported symbol when reading
attributes[AI_OPERATION_ID] in the existing logic.
ℹ️ Review info
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
packages/traceloop-sdk/src/lib/tracing/ai-sdk-transformations.ts
packages/traceloop-sdk/src/lib/tracing/ai-sdk-transformations.ts
Outdated
Show resolved
Hide resolved
packages/traceloop-sdk/src/lib/tracing/ai-sdk-transformations.ts
Outdated
Show resolved
Hide resolved
The ai.operationId attribute may be non-string in some cases, which would cause nameToCheck.includes() to throw a runtime error. This fix adds a proper type guard to ensure operationId is a string before using it, falling back to spanName if it's not a string.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
packages/traceloop-sdk/src/lib/tracing/ai-sdk-transformations.ts (1)
534-535: 🛠️ Refactor suggestion | 🟠 MajorReplace hardcoded
ai.operationIdwith a semantic-convention constant (already raised earlier).
const AI_OPERATION_ID = "ai.operationId"reintroduces a hardcoded semantic attribute key in traceloop-sdk. Please define/export this key inpackages/ai-semantic-conventions/src/SemanticAttributes.tsand import it here instead.#!/bin/bash # Verify whether AI_OPERATION_ID exists in semantic conventions # and where ai.operationId is currently hardcoded. rg -n 'AI_OPERATION_ID|ai\.operationId' packages/ai-semantic-conventions/src/SemanticAttributes.ts packages/traceloop-sdk/src/lib/tracing/ai-sdk-transformations.tsAs per coding guidelines:
packages/{instrumentation-*,traceloop-sdk}/**/*.{ts,tsx}must import AI/LLM semantic attribute constants from@traceloop/ai-semantic-conventionsrather than hardcoding strings.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/traceloop-sdk/src/lib/tracing/ai-sdk-transformations.ts` around lines 534 - 535, The code reintroduces a hardcoded semantic key by declaring const AI_OPERATION_ID = "ai.operationId" in ai-sdk-transformations.ts; remove that hardcoded constant, add and export a named constant AI_OPERATION_ID in packages/ai-semantic-conventions/src/SemanticAttributes.ts (matching existing semantic attribute exports), then import { AI_OPERATION_ID } from '@traceloop/ai-semantic-conventions' into ai-sdk-transformations.ts and use it where operationIdValue is assigned so the module uses the shared semantic-conventions constant instead of a magic string.
🧹 Nitpick comments (1)
packages/traceloop-sdk/src/lib/tracing/ai-sdk-transformations.ts (1)
511-519: Operation classification logic is duplicated in two places; consider centralizing it.The same
generateText/streamText/generateObject/streamObjectchecks are repeated intransformLlmRequestTypeandtransformOperationName. Extract one classifier helper and derive both outputs from it to prevent future drift.Also applies to: 547-550
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/traceloop-sdk/src/lib/tracing/ai-sdk-transformations.ts` around lines 511 - 519, The generateText/streamText/generateObject/streamObject check is duplicated between transformLlmRequestType and transformOperationName; create a single helper (e.g., classifyLlmRequestType or isChatLlmOperation) that takes the operation name (nameToCheck) and returns the LLMRequestTypeValues (or a boolean for CHAT) and replace the inline includes() checks in both transformLlmRequestType and transformOperationName with calls to that helper so both functions derive their result from one classifier; also replace the other duplicate occurrence near the second block of similar checks with the same helper.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/traceloop-sdk/src/lib/tracing/ai-sdk-transformations.ts`:
- Around line 494-502: The JSDoc above transformLlmRequestType is inaccurate: it
claims to transform gen_ai.operation.name but the function actually sets
llm.request.type (and reads ai.operationId/span name); update the comment to
describe transformLlmRequestType’s real behavior — that it derives and sets the
llm.request.type attribute from span name or ai.operationId (handling prior
onSpanStart transformations) and reference the linked issue; ensure the doc
mentions the function name transformLlmRequestType and the attributes
llm.request.type and ai.operationId so readers can find the logic.
---
Duplicate comments:
In `@packages/traceloop-sdk/src/lib/tracing/ai-sdk-transformations.ts`:
- Around line 534-535: The code reintroduces a hardcoded semantic key by
declaring const AI_OPERATION_ID = "ai.operationId" in ai-sdk-transformations.ts;
remove that hardcoded constant, add and export a named constant AI_OPERATION_ID
in packages/ai-semantic-conventions/src/SemanticAttributes.ts (matching existing
semantic attribute exports), then import { AI_OPERATION_ID } from
'@traceloop/ai-semantic-conventions' into ai-sdk-transformations.ts and use it
where operationIdValue is assigned so the module uses the shared
semantic-conventions constant instead of a magic string.
---
Nitpick comments:
In `@packages/traceloop-sdk/src/lib/tracing/ai-sdk-transformations.ts`:
- Around line 511-519: The generateText/streamText/generateObject/streamObject
check is duplicated between transformLlmRequestType and transformOperationName;
create a single helper (e.g., classifyLlmRequestType or isChatLlmOperation) that
takes the operation name (nameToCheck) and returns the LLMRequestTypeValues (or
a boolean for CHAT) and replace the inline includes() checks in both
transformLlmRequestType and transformOperationName with calls to that helper so
both functions derive their result from one classifier; also replace the other
duplicate occurrence near the second block of similar checks with the same
helper.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 82f4d4d2-ddc0-4326-95eb-88f820714ddb
📒 Files selected for processing (1)
packages/traceloop-sdk/src/lib/tracing/ai-sdk-transformations.ts
- Add AI_OPERATION_ID to SpanAttributes in @traceloop/ai-semantic-conventions - Update ai-sdk-transformations.ts to import AI_OPERATION_ID from SpanAttributes - Remove local hardcoded AI_OPERATION_ID constant This centralizes the attribute key definition in the semantic conventions package for better maintainability and consistency across instrumentations.
The JSDoc incorrectly stated the function transforms gen_ai.operation.name, but it actually derives and sets the llm.request.type attribute. Updated the comment to accurately describe: - Function name: transformLlmRequestType - What it does: derives and sets llm.request.type attribute - How it works: examines span name or ai.operationId attribute - Why dual approach: handles prior onSpanStart transformations - References the linked issue traceloop#882
2 participants
Summary
This PR fixes an issue where the
gen_ai.operation.nameattribute was not being set for AI SDK spans due to a timing bug in the span name transformation.Problem
The
transformOperationNamefunction checks the span name to determine thegen_ai.operation.nameattribute (e.g., "chat" for generateText operations). However, by the time this function runs inonSpanEnd, the span name has already been transformed bytransformAiSdkSpanNamesinonSpanStart.Transformation timeline:
onSpanStart→transformAiSdkSpanNamesruns → span name changes from"ai.generateText"to"run.ai"onSpanEnd→transformOperationNameruns → checks span name for"generateText"→ never matchesBecause the span name has already been transformed to
"run.ai", the check for"generateText"fails andgen_ai.operation.nameis never set.Solution
This fix checks the
ai.operationIdattribute first, which is set by the Vercel AI SDK and contains the original operation ID (e.g.,"ai.generateText"). This attribute is not transformed and reliably contains the original operation identifier.Updated logic:
ai.operationIdattribute (set by Vercel AI SDK, contains original operation ID)ai.operationIdis not availableExample
Before (broken)
After (fixed)
Backwards Compatibility
ai.operationIdis not presentnameToCheckandspanNamefor.toolsuffix)Related Issue
Fixes #882
Pull Request opened by Augment Code with guidance from the PR author
Important
Fixes
gen_ai.operation.namesetting by checkingai.operationIdbefore span name intransformOperationNamefunction.transformOperationNameinai-sdk-transformations.tsto checkai.operationIdbefore span name for settinggen_ai.operation.name.gen_ai.operation.nameis set correctly even if span name is transformed.ai.operationIdfirst, falls back to span name if not available.generateText,streamText,generateObject,streamObject, and tool calls.ai-sdk-transformations.tsfor clarity.This description was created by
for e9ba33a. You can customize this summary. It will automatically update as commits are pushed.
Summary by CodeRabbit