fix(llmobs): format AI SDK rich tool results#9399
Conversation
Overall package sizeSelf size: 7.48 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.3.2 | 124.41 kB | 440.65 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
BenchmarksBenchmark execution time: 2026-07-23 15:30:38 Comparing candidate commit 8c60bfb in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 2310 metrics, 48 unstable metrics.
|
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 8c60bfb | Docs | Datadog PR Page | Give us feedback! |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #9399 +/- ##
==========================================
- Coverage 98.45% 98.44% -0.01%
==========================================
Files 943 941 -2
Lines 127336 127048 -288
Branches 10824 11202 +378
==========================================
- Hits 125365 125078 -287
+ Misses 1971 1970 -1 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
aaa217b to
aab8927
Compare
AI SDK tool outputs can contain multipart content and error variants. Preserve text, replace file and image payloads with safe summaries, and fall back without throwing on malformed values. Fixes: #9395
AI SDK 7 emits generic file and provider-reference parts that were reported as unparsable. Model the supported output shapes locally and exercise content outputs through both telemetry paths so SDK shape changes fail on the user path.
Adding toModelOutput changed the second provider request without a matching cassette, so every AI SDK v5-v7 matrix forwarded to OpenAI with the fake key. Keep the recorded workflow payload stable while formatter coverage pins the rich result shapes and schema precedence.
aab8927 to
d91d6d8
Compare
Direct utility tests pinned hand-built shapes and missed the SDK transformations that feed LLMObs. Drive supported results through generateText across all model protocol versions, while keeping direct safety assertions only for values the SDK rejects before model invocation.
Summary
AI SDK multipart and error tool outputs were treated as unparsable. This preserves text, emits metadata-free summaries for rich file and image parts, and keeps malformed or circular telemetry inputs from throwing.
Fixes: #9395