diff --git a/.changeset/billed-usage-unit.md b/.changeset/billed-usage-unit.md new file mode 100644 index 000000000..649c3fa1a --- /dev/null +++ b/.changeset/billed-usage-unit.md @@ -0,0 +1,9 @@ +--- +'@tanstack/ai-event-client': minor +'@tanstack/ai': minor +'@tanstack/ai-fal': minor +'@tanstack/ai-grok': minor +'@tanstack/ai-openai': minor +--- + +Add a self-describing `billed` field to `TokenUsage` so non-token billed quantities carry the unit they are counted in (#816). `usage.billed` is `{ quantity, unit }` with a `BillingUnit` union (`'seconds'`, `'units'`, `'images'`, ... open-ended), replacing the guesswork previously needed to interpret the bare `unitsBilled` / `durationSeconds` counts — those two fields are now deprecated but still populated for backward compatibility. The fal adapters report `{ quantity, unit: 'units' }`, Grok video `{ quantity, unit: 'seconds' }`, and the OpenAI/Grok duration-billed transcription paths `{ quantity, unit: 'seconds' }`. `otelMiddleware` emits the pair as `tanstack.ai.usage.billed_quantity` / `tanstack.ai.usage.billed_unit` span attributes. diff --git a/docs/adapters/grok.md b/docs/adapters/grok.md index 832ac9986..a31756cb2 100644 --- a/docs/adapters/grok.md +++ b/docs/adapters/grok.md @@ -292,7 +292,7 @@ const { jobId } = await generateVideo({ Like the Grok Imagine image models, sizing is aspect-ratio based: the `size` option takes an `aspectRatio_resolution` template. Supported aspect ratios are `1:1`, `16:9`, `9:16`, `4:3`, `3:4`, `3:2`, and `2:3`; supported resolutions are `480p`, `720p`, and `1080p` (e.g. `"9:16_1080p"`). The resolution suffix is optional. -When the job completes, the adapter reports usage on the result: `usage.unitsBilled` carries the billed seconds of video and `usage.cost` the exact cost in USD, both as returned by the xAI API. +When the job completes, the adapter reports usage on the result: `usage.billed` carries the billed seconds of video (`{ quantity, unit: 'seconds' }`) and `usage.cost` the exact cost in USD, both as returned by the xAI API. See [Video Generation](../media/video-generation) for the full jobs/polling flow, streaming mode, and the `useGenerateVideo` hook. diff --git a/docs/advanced/otel.md b/docs/advanced/otel.md index 9f6b78ee4..c4a981e62 100644 --- a/docs/advanced/otel.md +++ b/docs/advanced/otel.md @@ -77,7 +77,10 @@ Iteration spans are numbered (`#0`, `#1`, ...) so distinct iterations of the sam | root / iteration | `gen_ai.usage.cache_read.input_tokens` | cached prompt tokens, when reported | | root / iteration | `gen_ai.usage.cache_creation.input_tokens` | cache-write prompt tokens, when reported | | root / iteration | `gen_ai.usage.reasoning.output_tokens` | reasoning/thinking tokens, when reported | -| root / iteration | `tanstack.ai.usage.duration_seconds` | duration-based billing (e.g. transcription), when reported | +| root / iteration | `tanstack.ai.usage.billed_quantity` | non-token billed quantity, when reported | +| root / iteration | `tanstack.ai.usage.billed_unit` | unit of the billed quantity (`seconds`, `units`, ...) | +| root / iteration | `tanstack.ai.usage.duration_seconds` | deprecated duration count; read `billed_quantity`/`billed_unit` instead | +| root / iteration | `tanstack.ai.usage.units_billed` | deprecated bare unit count; read `billed_quantity`/`billed_unit` instead | | root / iteration | `tanstack.ai.usage.upstream_cost` | gateway upstream cost (e.g. OpenRouter), when reported | | root / iteration | `tanstack.ai.usage.upstream_input_cost` | upstream input cost split, when reported | | root / iteration | `tanstack.ai.usage.upstream_output_cost` | upstream output cost split, when reported | @@ -90,7 +93,9 @@ Iteration spans are numbered (`#0`, `#1`, ...) so distinct iterations of the sam | tool | `gen_ai.tool.type` | `function` | | tool | `tanstack.ai.tool.outcome` | `success` / `error` | -Usage attributes beyond input/output tokens are emitted only when the provider reports them, so spans stay clean otherwise. Cache and reasoning breakdowns use the official GenAI semconv names; `gen_ai.usage.cost` and `gen_ai.usage.total_tokens` are de-facto extensions consumed directly by backends like PostHog — without them, backends re-derive cost from their own price tables and lose cache discounts and gateway markup. Fields with no established convention (duration-based billing, the upstream cost split) are TanStack-namespaced. +Usage attributes beyond input/output tokens are emitted only when the provider reports them, so spans stay clean otherwise. Cache and reasoning breakdowns use the official GenAI semconv names; `gen_ai.usage.cost` and `gen_ai.usage.total_tokens` are de-facto extensions consumed directly by backends like PostHog — without them, backends re-derive cost from their own price tables and lose cache discounts and gateway markup. Fields with no established convention (the billed quantity/unit pair, the upstream cost split, and the deprecated bare counts) are TanStack-namespaced. + +For non-token billing (seconds of video or transcription, fal's endpoint units, ...), `tanstack.ai.usage.billed_quantity` and `tanstack.ai.usage.billed_unit` are emitted as a pair from `usage.billed`, so backends can label and aggregate media usage without knowing the provider. The deprecated `duration_seconds` / `units_billed` attributes carry the same quantities without the unit and remain emitted for backward compatibility. ### Metrics @@ -231,7 +236,7 @@ Each media call produces one `CLIENT` span tagged with the activity's `gen_ai.op | `generateSpeech` | `text_to_speech` | | `generateTranscription` | `transcription` | -The span carries `gen_ai.system` and `gen_ai.request.model` at start and, on finish, the same `gen_ai.usage.*` / `tanstack.ai.usage.*` attributes documented above — including `tanstack.ai.usage.units_billed` for unit-billed media. When a `Meter` is supplied it records the `gen_ai.client.operation.duration` histogram, tagged per activity. For streaming video the span covers the full create → poll → complete lifecycle; for non-streaming `generateVideo` it covers job submission. If a streaming video consumer abandons the stream before completion, the span is ended via `onAbort` (status `ERROR`, `tanstack.ai.completion.reason = cancelled`) rather than leaked. +The span carries `gen_ai.system` and `gen_ai.request.model` at start and, on finish, the same `gen_ai.usage.*` / `tanstack.ai.usage.*` attributes documented above — including the `tanstack.ai.usage.billed_quantity` / `tanstack.ai.usage.billed_unit` pair for unit-billed media. When a `Meter` is supplied it records the `gen_ai.client.operation.duration` histogram, tagged per activity. For streaming video the span covers the full create → poll → complete lifecycle; for non-streaming `generateVideo` it covers job submission. If a streaming video consumer abandons the stream before completion, the span is ended via `onAbort` (status `ERROR`, `tanstack.ai.completion.reason = cancelled`) rather than leaked. `otelMiddleware` applies the same `spanNameFormatter`, `attributeEnricher`, `onBeforeSpanStart`, and `onSpanEnd` extension points to media spans — the span info is discriminated by `kind`, where media spans report `kind: 'generation'`. For a custom backend, implement the base `GenerationMiddleware` contract directly; its hooks (`onStart` / `onUsage` / `onFinish` / `onAbort` / `onError`) receive the `GenerationMiddlewareContext` and fire for every activity, chat included. The `GenerationMiddleware` types are exported from the package root, while the `otelMiddleware` value lives on the `@tanstack/ai/middlewares/otel` subpath so importing `@tanstack/ai` never requires the optional `@opentelemetry/api` peer. diff --git a/docs/config.json b/docs/config.json index 8336c1778..e6bf03949 100644 --- a/docs/config.json +++ b/docs/config.json @@ -271,19 +271,19 @@ "label": "Audio Generation", "to": "media/audio-generation", "addedAt": "2026-04-23", - "updatedAt": "2026-06-08" + "updatedAt": "2026-07-04" }, { "label": "Image Generation", "to": "media/image-generation", "addedAt": "2026-04-15", - "updatedAt": "2026-07-01" + "updatedAt": "2026-07-04" }, { "label": "Video Generation", "to": "media/video-generation", "addedAt": "2026-04-15", - "updatedAt": "2026-07-01" + "updatedAt": "2026-07-04" }, { "label": "Generation Hooks", @@ -310,7 +310,7 @@ "label": "OpenTelemetry", "to": "advanced/otel", "addedAt": "2026-05-08", - "updatedAt": "2026-06-17" + "updatedAt": "2026-07-04" } ] }, @@ -527,7 +527,7 @@ "label": "Grok (xAI)", "to": "adapters/grok", "addedAt": "2026-04-15", - "updatedAt": "2026-06-24" + "updatedAt": "2026-07-04" }, { "label": "Groq", diff --git a/docs/media/audio-generation.md b/docs/media/audio-generation.md index 2fbb97651..4c460026a 100644 --- a/docs/media/audio-generation.md +++ b/docs/media/audio-generation.md @@ -130,9 +130,10 @@ interface AudioGenerationResult { } // Canonical TokenUsage (same shape as chat), present when the provider // reports it (e.g. Gemini Lyria via generateContent). Usage-billed providers - // (fal) instead surface `usage.unitsBilled` — the real billed quantity read - // from fal's `x-fal-billable-units` result header. Multiply by the endpoint's - // unit price (fal pricing API) for the exact cost. + // (fal) instead surface `usage.billed` ({ quantity, unit: 'units' }) — the + // real billed quantity read from fal's `x-fal-billable-units` result header. + // Multiply the quantity by the endpoint's unit price (fal pricing API) for + // the exact cost. usage?: TokenUsage } ``` diff --git a/docs/media/image-generation.md b/docs/media/image-generation.md index 8fffa84a2..36fc46b89 100644 --- a/docs/media/image-generation.md +++ b/docs/media/image-generation.md @@ -396,7 +396,7 @@ interface ImageGenerationResult { // Canonical TokenUsage (same shape as chat). Token-billed models also surface // a per-modality breakdown on `promptTokensDetails` (e.g. text vs image input // tokens for gpt-image-1). Usage-billed providers (fal) instead surface - // `usage.unitsBilled` — see the note below. + // `usage.billed` ({ quantity, unit }) — see the note below. usage?: TokenUsage } @@ -408,9 +408,9 @@ interface GeneratedImage { ``` > **Cost tracking (fal):** fal bills by usage-based units rather than tokens. The -> fal image adapter surfaces the real billed quantity as `usage.unitsBilled` -> (read from fal's `x-fal-billable-units` result header). Multiply it by the -> endpoint's unit price from +> fal image adapter surfaces the real billed quantity as `usage.billed` — +> `{ quantity, unit: 'units' }`, read from fal's `x-fal-billable-units` result +> header. Multiply the quantity by the endpoint's unit price from > `GET https://api.fal.ai/v1/models/pricing?endpoint_id=…` for the exact cost — > no `fetch` interceptor needed. @@ -424,9 +424,10 @@ const result = await generateImage({ prompt: 'a serene mountain lake', }) -if (result.usage?.unitsBilled != null) { - const cost = result.usage.unitsBilled * unitPrice // unitPrice from fal pricing API - console.log(`Billed ${result.usage.unitsBilled} units (~$${cost})`) +if (result.usage?.billed) { + const { quantity, unit } = result.usage.billed + const cost = quantity * unitPrice // unitPrice from fal pricing API + console.log(`Billed ${quantity} ${unit} (~$${cost})`) } ``` diff --git a/docs/media/video-generation.md b/docs/media/video-generation.md index 408bae527..d02318114 100644 --- a/docs/media/video-generation.md +++ b/docs/media/video-generation.md @@ -625,7 +625,7 @@ adapter.snapDuration(2.5) // 3 — clamped/rounded into range adapter.snapDuration(99) // 15 ``` -Generated clips include an audio track. When the job completes, the adapter reports `usage.unitsBilled` (billed seconds of video) and `usage.cost` (exact USD cost as returned by the API) on the result. +Generated clips include an audio track. When the job completes, the adapter reports `usage.billed` (`{ quantity, unit: 'seconds' }` — billed seconds of video) and `usage.cost` (exact USD cost as returned by the API) on the result. ## Response Types @@ -660,19 +660,24 @@ interface VideoUrlResult { jobId: string url: string // URL to download/stream the video expiresAt?: Date // When the URL expires - // Usage for the completed generation, when the adapter reports it. fal - // populates `usage.unitsBilled` from its `x-fal-billable-units` header. + // Usage for the completed generation, when the adapter reports it. The + // billed quantity is self-describing: fal reports + // `usage.billed = { quantity, unit: 'units' }` (from its + // `x-fal-billable-units` header), Grok Imagine reports + // `{ quantity, unit: 'seconds' }`. usage?: TokenUsage } ``` > **Cost tracking (fal):** fal bills media generation by usage-based units > rather than tokens. The fal adapters surface the real billed quantity as -> `usage.unitsBilled` (denominated in the endpoint's priced unit). Combine it -> with the endpoint's unit price from -> `GET https://api.fal.ai/v1/models/pricing?endpoint_id=…` to compute the exact -> cost (`unitsBilled * unitPrice`). The same `usage.unitsBilled` is surfaced -> on image, audio, speech, and transcription results. +> `usage.billed` — `{ quantity, unit: 'units' }`, where `'units'` marks fal's +> endpoint-defined priced unit. Combine the quantity with the endpoint's unit +> price from `GET https://api.fal.ai/v1/models/pricing?endpoint_id=…` to +> compute the exact cost (`billed.quantity * unitPrice`). The same +> `usage.billed` is surfaced on image, audio, speech, and transcription +> results. (The deprecated bare count `usage.unitsBilled` is still populated +> for backward compatibility.) ## Model Variants diff --git a/examples/ts-react-media/src/components/ImageGenerator.tsx b/examples/ts-react-media/src/components/ImageGenerator.tsx index 9b4d5fd29..de8153c02 100644 --- a/examples/ts-react-media/src/components/ImageGenerator.tsx +++ b/examples/ts-react-media/src/components/ImageGenerator.tsx @@ -304,12 +304,12 @@ export default function ImageGenerator({ className="w-full h-auto" /> - {modelResult.result.usage?.unitsBilled != null && ( + {modelResult.result.usage?.billed && (

- Billed {modelResult.result.usage.unitsBilled} fal unit - {modelResult.result.usage.unitsBilled === 1 - ? '' - : 's'}{' '} + Billed {modelResult.result.usage.billed.quantity}{' '} + {modelResult.result.usage.billed.unit === 'units' + ? `fal unit${modelResult.result.usage.billed.quantity === 1 ? '' : 's'}` + : modelResult.result.usage.billed.unit}{' '} — multiply by the endpoint unit price for USD cost

)} diff --git a/examples/ts-react-media/src/components/VideoGenerator.tsx b/examples/ts-react-media/src/components/VideoGenerator.tsx index f31a8078e..08b957462 100644 --- a/examples/ts-react-media/src/components/VideoGenerator.tsx +++ b/examples/ts-react-media/src/components/VideoGenerator.tsx @@ -1,5 +1,6 @@ import { useEffect, useRef, useState } from 'react' import { Film, Loader2, Shuffle, Upload, X } from 'lucide-react' +import type { BilledUsage } from '@tanstack/ai' import type { VideoMode } from '@/lib/models' import { @@ -21,9 +22,25 @@ type JobState = model: string progress?: number | undefined } - | { status: 'completed'; url: string; unitsBilled?: number; cost?: number } + | { status: 'completed'; url: string; billed?: BilledUsage; cost?: number } | { status: 'error'; message: string } +/** + * Human label for a billed quantity, driven by the unit the adapter reported — + * no guessing from provider identity or cost presence. + */ +function describeBilled({ quantity, unit }: BilledUsage): string { + const plural = quantity === 1 ? '' : 's' + switch (unit) { + case 'seconds': + return `${quantity} second${plural} of video` + case 'units': + return `${quantity} fal unit${plural}` + default: + return `${quantity} ${unit}` + } +} + interface VideoGeneratorProps { initialImageUrl?: string | null } @@ -98,7 +115,7 @@ export default function VideoGenerator({ [model]: { status: 'completed', url: url, - unitsBilled: urlResult.usage?.unitsBilled, + billed: urlResult.usage?.billed, cost: urlResult.usage?.cost, }, })) @@ -424,16 +441,17 @@ export default function VideoGenerator({ {state.cost != null ? (

Billed ${state.cost.toFixed(3)} - {state.unitsBilled != null - ? ` for ${state.unitsBilled} second${state.unitsBilled === 1 ? '' : 's'} of video` + {state.billed + ? ` for ${describeBilled(state.billed)}` : ''}

) : ( - state.unitsBilled != null && ( + state.billed && (

- Billed {state.unitsBilled} fal unit - {state.unitsBilled === 1 ? '' : 's'} — multiply by the - endpoint unit price for USD cost + Billed {describeBilled(state.billed)} + {state.billed.unit === 'units' + ? ' — multiply by the endpoint unit price for USD cost' + : ''}

) )} diff --git a/examples/ts-react-media/src/lib/server-functions.ts b/examples/ts-react-media/src/lib/server-functions.ts index 1b3b52639..76c58786a 100644 --- a/examples/ts-react-media/src/lib/server-functions.ts +++ b/examples/ts-react-media/src/lib/server-functions.ts @@ -252,8 +252,8 @@ export const createVideoJobFn = createServerFn({ method: 'POST' }) case 'grok-imagine-video': { // Direct xAI Imagine API (XAI_API_KEY) — no fal in between. The base // grok-imagine-video (v1.0) supports text-to-video; durations are - // 1-15 integer seconds. Completed jobs report usage.unitsBilled - // (billed seconds) and usage.cost (exact USD). + // 1-15 integer seconds. Completed jobs report usage.billed + // ({ quantity, unit: 'seconds' }) and usage.cost (exact USD). return generateVideo({ adapter: grokVideo('grok-imagine-video'), prompt: asTextPrompt(data.prompt), diff --git a/packages/ai-event-client/src/index.ts b/packages/ai-event-client/src/index.ts index 0bd1fdc12..0ccd6f18e 100644 --- a/packages/ai-event-client/src/index.ts +++ b/packages/ai-event-client/src/index.ts @@ -185,6 +185,36 @@ export interface UsageCostBreakdown { upstreamOutputCost?: number } +/** + * Unit a billed quantity is counted in. The named members cover the units + * TanStack AI adapters bill in today; the `(string & {})` member keeps the + * union open for genuinely provider-specific units while preserving + * autocompletion for the common ones. + */ +export type BillingUnit = + | 'tokens' + | 'seconds' + | 'characters' + | 'images' + | 'videos' + | 'megapixels' + | 'requests' + | 'units' + | (string & {}) + +/** + * A billed quantity paired with the unit it is counted in, so consumers can + * label and aggregate usage without out-of-band knowledge of the provider or + * activity. `unit: 'units'` marks an opaque provider-defined unit (e.g. fal's + * "fal units") whose price is only knowable from the provider's pricing page. + */ +export interface BilledUsage { + /** Number of units billed. */ + quantity: number + /** The unit `quantity` is counted in. */ + unit: BillingUnit +} + /** * Default value type for {@link TokenUsage.providerUsageDetails} when an adapter * does not supply a specific shape. Values are constrained to non-nullish @@ -221,18 +251,27 @@ export interface TokenUsage { promptTokensDetails?: PromptTokensDetails /** Detailed breakdown of completion tokens by category */ completionTokensDetails?: CompletionTokensDetails - /** Duration in seconds for duration-based billing (e.g., Whisper transcription) */ + /** + * The primary non-token billed quantity, self-describing via its unit — + * e.g. `{ quantity: 8, unit: 'seconds' }` for a video generation or + * `{ quantity: 3, unit: 'units' }` for fal's opaque endpoint units. Absent + * when the activity bills purely in tokens (the token fields above are + * already self-describing). When a provider bills tokens *on top of* a media + * unit, the tokens stay in the token fields and `billed` carries the media + * unit. A quantity, distinct from the monetary `cost` / `costDetails`. + */ + billed?: BilledUsage + /** + * @deprecated Read {@link TokenUsage.billed} instead, which pairs the same + * duration with an explicit `unit: 'seconds'`. Still populated alongside + * `billed` for backward compatibility; will be removed in a future release. + */ durationSeconds?: number /** - * Number of priced units actually billed, for usage-based (non-token) billing. - * This is a bare count, not a cost and not a unit name — the unit itself - * (megapixels, seconds, images, …) is provider-defined and not carried here; - * providers typically expose it via a separate pricing API. Surfaced for media - * generation, where there are no tokens: fal returns this count in its - * `x-fal-billable-units` response header. Multiply by the unit price to get the - * exact cost (`unitsBilled * unitPrice`). The unit-priced analogue of - * `durationSeconds` (the time-priced case); both are quantities, distinct from - * the monetary `cost` / `costDetails`. + * @deprecated Read {@link TokenUsage.billed} instead, which pairs the same + * count with the unit it is denominated in (`seconds`, `units`, …) — this + * bare count is ambiguous across providers. Still populated alongside + * `billed` for backward compatibility; will be removed in a future release. */ unitsBilled?: number /** Provider-specific usage details not covered by standard fields */ diff --git a/packages/ai-fal/src/utils/billing.ts b/packages/ai-fal/src/utils/billing.ts index 8df762baf..4b5335d01 100644 --- a/packages/ai-fal/src/utils/billing.ts +++ b/packages/ai-fal/src/utils/billing.ts @@ -74,9 +74,10 @@ export function takeBillableUnits( /** * Build a {@link TokenUsage} carrying fal's billed quantity. Media generation has * no tokens, so the token fields are zero and the real billing signal rides on - * `unitsBilled` — mirroring how the duration-billed transcription adapters - * surface `durationSeconds`. Returns `undefined` when no units were captured so - * callers can omit `usage` entirely. + * `billed`, denominated in `'units'` — fal's priced unit is endpoint-defined + * (its pricing page maps each endpoint to a unit price), so the count is opaque + * by design. Returns `undefined` when no units were captured so callers can + * omit `usage` entirely. */ export function buildFalUsage( unitsBilled: number | undefined, @@ -86,6 +87,7 @@ export function buildFalUsage( promptTokens: 0, completionTokens: 0, totalTokens: 0, + billed: { quantity: unitsBilled, unit: 'units' }, unitsBilled, } } diff --git a/packages/ai-fal/tests/audio-adapter.test.ts b/packages/ai-fal/tests/audio-adapter.test.ts index 02e6d8ca7..b19a770d1 100644 --- a/packages/ai-fal/tests/audio-adapter.test.ts +++ b/packages/ai-fal/tests/audio-adapter.test.ts @@ -366,6 +366,7 @@ describe('Fal Audio Adapter', () => { promptTokens: 0, completionTokens: 0, totalTokens: 0, + billed: { quantity: 2.5, unit: 'units' }, unitsBilled: 2.5, }) }) diff --git a/packages/ai-fal/tests/billing.test.ts b/packages/ai-fal/tests/billing.test.ts index d35c834dd..7b922393a 100644 --- a/packages/ai-fal/tests/billing.test.ts +++ b/packages/ai-fal/tests/billing.test.ts @@ -85,6 +85,7 @@ describe('buildFalUsage', () => { promptTokens: 0, completionTokens: 0, totalTokens: 0, + billed: { quantity: 4, unit: 'units' }, unitsBilled: 4, }) }) @@ -94,6 +95,7 @@ describe('buildFalUsage', () => { promptTokens: 0, completionTokens: 0, totalTokens: 0, + billed: { quantity: 0, unit: 'units' }, unitsBilled: 0, }) }) diff --git a/packages/ai-fal/tests/image-adapter.test.ts b/packages/ai-fal/tests/image-adapter.test.ts index b429f1000..d2d2f22d5 100644 --- a/packages/ai-fal/tests/image-adapter.test.ts +++ b/packages/ai-fal/tests/image-adapter.test.ts @@ -335,6 +335,7 @@ describe('Fal Image Adapter', () => { promptTokens: 0, completionTokens: 0, totalTokens: 0, + billed: { quantity: 4, unit: 'units' }, unitsBilled: 4, }) }) diff --git a/packages/ai-fal/tests/speech-adapter.test.ts b/packages/ai-fal/tests/speech-adapter.test.ts index 2edafe4f9..090c7e95d 100644 --- a/packages/ai-fal/tests/speech-adapter.test.ts +++ b/packages/ai-fal/tests/speech-adapter.test.ts @@ -338,6 +338,7 @@ describe('Fal Speech Adapter', () => { promptTokens: 0, completionTokens: 0, totalTokens: 0, + billed: { quantity: 3, unit: 'units' }, unitsBilled: 3, }) }) diff --git a/packages/ai-fal/tests/transcription-adapter.test.ts b/packages/ai-fal/tests/transcription-adapter.test.ts index 7b969b2af..3e72b6485 100644 --- a/packages/ai-fal/tests/transcription-adapter.test.ts +++ b/packages/ai-fal/tests/transcription-adapter.test.ts @@ -321,6 +321,7 @@ describe('Fal Transcription Adapter', () => { promptTokens: 0, completionTokens: 0, totalTokens: 0, + billed: { quantity: 1.5, unit: 'units' }, unitsBilled: 1.5, }) }) diff --git a/packages/ai-fal/tests/video-adapter.test.ts b/packages/ai-fal/tests/video-adapter.test.ts index b3778406f..79e496928 100644 --- a/packages/ai-fal/tests/video-adapter.test.ts +++ b/packages/ai-fal/tests/video-adapter.test.ts @@ -404,6 +404,7 @@ describe('Fal Video Adapter', () => { promptTokens: 0, completionTokens: 0, totalTokens: 0, + billed: { quantity: 12, unit: 'units' }, unitsBilled: 12, }) }) diff --git a/packages/ai-grok/src/adapters/transcription.ts b/packages/ai-grok/src/adapters/transcription.ts index cefdd17a7..e3048192c 100644 --- a/packages/ai-grok/src/adapters/transcription.ts +++ b/packages/ai-grok/src/adapters/transcription.ts @@ -137,7 +137,7 @@ export class GrokTranscriptionAdapter< const resolvedLanguage = data.language ?? language // xAI's /v1/stt response carries no token counts — STT is duration-billed — - // so surface the audio duration as `durationSeconds`, mirroring the + // so surface the audio duration as the billed quantity, mirroring the // whisper-1 path in the OpenAI transcription adapter. const usage: TokenUsage | undefined = data.duration !== undefined && data.duration > 0 @@ -145,6 +145,7 @@ export class GrokTranscriptionAdapter< promptTokens: 0, completionTokens: 0, totalTokens: 0, + billed: { quantity: data.duration, unit: 'seconds' }, durationSeconds: data.duration, } : undefined diff --git a/packages/ai-grok/src/adapters/video.ts b/packages/ai-grok/src/adapters/video.ts index a59c45230..19d397b20 100644 --- a/packages/ai-grok/src/adapters/video.ts +++ b/packages/ai-grok/src/adapters/video.ts @@ -81,7 +81,10 @@ function buildGrokVideoUsage( promptTokens: 0, completionTokens: 0, totalTokens: 0, - ...(seconds !== undefined && { unitsBilled: seconds }), + ...(seconds !== undefined && { + billed: { quantity: seconds, unit: 'seconds' }, + unitsBilled: seconds, + }), ...(ticks !== undefined && { cost: ticks / USD_TICKS_PER_DOLLAR }), } } @@ -109,7 +112,7 @@ function buildGrokVideoUsage( * - Aspect-ratio sizing via the "aspectRatio_resolution" size template * (e.g. '16:9_720p'), consistent with the grok-imagine image models * - Image-to-video via an `image` prompt part (starting frame URL or data URI) - * - Usage reporting: billed seconds (`unitsBilled`) and exact cost + * - Usage reporting: billed seconds (`usage.billed`) and exact cost */ export class GrokVideoAdapter< TModel extends GrokVideoModel, diff --git a/packages/ai-grok/tests/audio-adapters.test.ts b/packages/ai-grok/tests/audio-adapters.test.ts index e0255f610..42602af67 100644 --- a/packages/ai-grok/tests/audio-adapters.test.ts +++ b/packages/ai-grok/tests/audio-adapters.test.ts @@ -300,6 +300,15 @@ describe('GrokTranscriptionAdapter', () => { expect(result.text).toBe('hello world') expect(result.language).toBe('en') expect(result.duration).toBe(1.23) + // STT is duration-billed: the audio duration doubles as the billed + // quantity, self-described in seconds. + expect(result.usage).toEqual({ + promptTokens: 0, + completionTokens: 0, + totalTokens: 0, + billed: { quantity: 1.23, unit: 'seconds' }, + durationSeconds: 1.23, + }) // Grok returns `confidence` per word when the model provides one; we // surface it under `GrokTranscriptionWord` so callers that know they're // using Grok can narrow the result via `as Array`. diff --git a/packages/ai-grok/tests/video-adapter.test.ts b/packages/ai-grok/tests/video-adapter.test.ts index a6239adbc..ae14e45a3 100644 --- a/packages/ai-grok/tests/video-adapter.test.ts +++ b/packages/ai-grok/tests/video-adapter.test.ts @@ -535,6 +535,7 @@ describe('Grok Video Adapter', () => { promptTokens: 0, completionTokens: 0, totalTokens: 0, + billed: { quantity: 5, unit: 'seconds' }, unitsBilled: 5, cost: 0.25, }, diff --git a/packages/ai-openai/src/adapters/transcription.ts b/packages/ai-openai/src/adapters/transcription.ts index b5fcd23e9..be3b9f42c 100644 --- a/packages/ai-openai/src/adapters/transcription.ts +++ b/packages/ai-openai/src/adapters/transcription.ts @@ -52,6 +52,22 @@ function mapDiarizedSegmentId(id: string, index: number): number { * Build TokenUsage from transcription response. * Whisper-1 uses duration-based billing, GPT-4o models use token-based billing. */ +/** + * Duration-billed usage: zeroed token fields with the billed seconds carried + * on `billed` and, for backward compatibility, the deprecated + * `durationSeconds`. Shared by the gpt-4o duration branch and the whisper-1 + * path so the two fields can't drift apart. + */ +function durationUsage(seconds: number): TokenUsage { + return { + promptTokens: 0, + completionTokens: 0, + totalTokens: 0, + billed: { quantity: seconds, unit: 'seconds' }, + durationSeconds: seconds, + } +} + function buildTranscriptionUsage( model: string, duration?: number, @@ -72,12 +88,7 @@ function buildTranscriptionUsage( // gpt-4o-transcribe-diarize responses may report duration-based usage; // surface it rather than discarding billing data the API returned. if (usage.type === 'duration') { - return { - promptTokens: 0, - completionTokens: 0, - totalTokens: 0, - durationSeconds: usage.seconds, - } + return durationUsage(usage.seconds) } const result: TokenUsage = { @@ -111,12 +122,7 @@ function buildTranscriptionUsage( // Whisper-1 uses duration-based billing if (duration !== undefined && duration > 0) { - return { - promptTokens: 0, - completionTokens: 0, - totalTokens: 0, - durationSeconds: duration, - } + return durationUsage(duration) } return undefined diff --git a/packages/ai-openai/tests/transcription-adapter.test.ts b/packages/ai-openai/tests/transcription-adapter.test.ts index df022a196..f04d7476a 100644 --- a/packages/ai-openai/tests/transcription-adapter.test.ts +++ b/packages/ai-openai/tests/transcription-adapter.test.ts @@ -629,6 +629,7 @@ describe('OpenAI transcription adapter', () => { promptTokens: 0, completionTokens: 0, totalTokens: 0, + billed: { quantity: 2.5, unit: 'seconds' }, durationSeconds: 2.5, }) }) diff --git a/packages/ai-openai/tests/transcription-usage.test.ts b/packages/ai-openai/tests/transcription-usage.test.ts index 48e691796..e084a79d1 100644 --- a/packages/ai-openai/tests/transcription-usage.test.ts +++ b/packages/ai-openai/tests/transcription-usage.test.ts @@ -97,6 +97,7 @@ describe('OpenAI transcription usage', () => { promptTokens: 0, completionTokens: 0, totalTokens: 0, + billed: { quantity: 12.5, unit: 'seconds' }, durationSeconds: 12.5, }) }) diff --git a/packages/ai/skills/ai-core/media-generation/SKILL.md b/packages/ai/skills/ai-core/media-generation/SKILL.md index b55638111..1c4035a1b 100644 --- a/packages/ai/skills/ai-core/media-generation/SKILL.md +++ b/packages/ai/skills/ai-core/media-generation/SKILL.md @@ -477,7 +477,7 @@ durations 4/8/12s, single `input_reference` image prompt part), `grokVideo(...)` (`grok-imagine-video` does text-to-video + image-to-video; `grok-imagine-video-1.5` is image-to-video only — needs an `image` prompt part as the starting frame, text-only throws; aspect-ratio size template like `'16:9_720p'`, integer durations 1-15s, reports -`usage.unitsBilled` seconds and exact `usage.cost`), and `falVideo(...)` (hosted models, see cost tracking below). +`usage.billed` seconds ({ quantity, unit: 'seconds' }) and exact `usage.cost`), and `falVideo(...)` (hosted models, see cost tracking below). Client hook with job tracking: @@ -499,10 +499,11 @@ const { generate, result, jobId, videoStatus, isLoading } = useGenerateVideo({ fal bills media generation by usage-based units, not tokens. Every fal media adapter (`falImage`, `falAudio`, `falSpeech`, `falTranscription`, `falVideo`) -surfaces the real billed quantity on the result as `usage.unitsBilled`, read -from fal's `x-fal-billable-units` response header — no `fetch` interceptor -needed. It rides on the canonical `TokenUsage` shape (token fields are `0` for -media), mirroring how duration-billed transcription surfaces `durationSeconds`. +surfaces the real billed quantity on the result as `usage.billed` +({ quantity, unit: 'units' }), read from fal's `x-fal-billable-units` response +header — no `fetch` interceptor needed. It rides on the canonical `TokenUsage` +shape (token fields are `0` for media), mirroring how duration-billed +transcription reports { quantity, unit: 'seconds' }. ```typescript import { generateImage } from '@tanstack/ai' @@ -513,10 +514,10 @@ const result = await generateImage({ prompt: 'a serene mountain lake', }) -// usage.unitsBilled is the priced quantity. Multiply by the endpoint unit +// usage.billed.quantity is the priced quantity. Multiply by the endpoint unit // price (GET https://api.fal.ai/v1/models/pricing?endpoint_id=…) for exact cost. -if (result.usage?.unitsBilled != null) { - const cost = result.usage.unitsBilled * unitPrice +if (result.usage?.billed) { + const cost = result.usage.billed.quantity * unitPrice } ``` diff --git a/packages/ai/src/middlewares/usage-attributes.ts b/packages/ai/src/middlewares/usage-attributes.ts index ef17f43ee..1e669116b 100644 --- a/packages/ai/src/middlewares/usage-attributes.ts +++ b/packages/ai/src/middlewares/usage-attributes.ts @@ -12,8 +12,8 @@ import type { TokenUsage } from '../types' * `gen_ai.usage.cost` and `gen_ai.usage.total_tokens` are de-facto extensions * consumed by backends like PostHog (which otherwise re-derive cost from their * own price tables, losing cache discounts and gateway markup). Fields with no - * semconv or de-facto convention (`costDetails`, `durationSeconds`, - * `unitsBilled`) are TanStack-namespaced. + * semconv or de-facto convention (`billed`, `costDetails`, and the deprecated + * `durationSeconds`/`unitsBilled`) are TanStack-namespaced. * * Shared by `otelMiddleware` across every activity (chat and the media * activities) so usage lands identically whichever activity produced the span. @@ -30,6 +30,16 @@ export function usageAttributes( 'gen_ai.usage.input_tokens': usage.promptTokens, 'gen_ai.usage.output_tokens': usage.completionTokens, } + // The self-describing billed quantity: the unit rides along as a string + // attribute so backends can label/aggregate non-token usage without + // out-of-band knowledge of the provider. + if (usage.billed !== undefined) { + const quantity = firstNumber(usage.billed.quantity) + if (quantity !== undefined) { + attrs['tanstack.ai.usage.billed_quantity'] = quantity + attrs['tanstack.ai.usage.billed_unit'] = usage.billed.unit + } + } const optional: Array<[key: string, value: unknown]> = [ ['gen_ai.usage.total_tokens', usage.totalTokens], ['gen_ai.usage.cost', usage.cost], diff --git a/packages/ai/src/types.ts b/packages/ai/src/types.ts index e497090e2..6cb6a4ab8 100644 --- a/packages/ai/src/types.ts +++ b/packages/ai/src/types.ts @@ -9,6 +9,8 @@ import type { CapabilityContext } from './activities/chat/middleware/capabilitie // package (which `@tanstack/ai` already depends on) so there is a single source // of truth without a dependency cycle. They are re-exported below. import type { + BilledUsage, + BillingUnit, CompletionTokensDetails, PromptTokensDetails, ProviderUsageDetails, @@ -1045,6 +1047,8 @@ export interface RunStartedEvent extends AGUIRunStartedEvent { // Re-export the canonical usage types (defined in `@tanstack/ai-event-client`) // so `@tanstack/ai` consumers keep importing them from here unchanged. export type { + BilledUsage, + BillingUnit, CompletionTokensDetails, PromptTokensDetails, ProviderUsageDetails, @@ -1867,8 +1871,8 @@ export interface VideoUrlResult { expiresAt?: Date /** * Usage information for the completed generation, when the adapter can report - * it. For usage-based providers (e.g. fal) this carries `unitsBilled` — the - * real billed quantity — so consumers can compute exact cost. + * it. For usage-based providers (e.g. fal) this carries `billed` — the real + * billed quantity paired with its unit — so consumers can compute exact cost. */ usage?: TokenUsage } diff --git a/packages/ai/tests/middlewares/otel.test.ts b/packages/ai/tests/middlewares/otel.test.ts index f7fa553ac..5b3a720bd 100644 --- a/packages/ai/tests/middlewares/otel.test.ts +++ b/packages/ai/tests/middlewares/otel.test.ts @@ -1335,4 +1335,37 @@ describe('usageAttributes', () => { // No cost reported → key absent. expect('gen_ai.usage.cost' in attrs).toBe(false) }) + + it('emits the self-describing billed quantity with its unit', () => { + const usage: TokenUsage = { + promptTokens: 0, + completionTokens: 0, + totalTokens: 0, + billed: { quantity: 8, unit: 'seconds' }, + } + const attrs = usageAttributes(usage) + + expect(attrs['tanstack.ai.usage.billed_quantity']).toBe(8) + expect(attrs['tanstack.ai.usage.billed_unit']).toBe('seconds') + }) + + it('omits both billed attributes when billed is absent or non-numeric', () => { + const attrs = usageAttributes({ + promptTokens: 1, + completionTokens: 2, + totalTokens: 3, + }) + expect('tanstack.ai.usage.billed_quantity' in attrs).toBe(false) + expect('tanstack.ai.usage.billed_unit' in attrs).toBe(false) + + // A NaN quantity (bad provider data) must not emit a dangling unit. + const bad = usageAttributes({ + promptTokens: 1, + completionTokens: 2, + totalTokens: 3, + billed: { quantity: Number.NaN, unit: 'units' }, + }) + expect('tanstack.ai.usage.billed_quantity' in bad).toBe(false) + expect('tanstack.ai.usage.billed_unit' in bad).toBe(false) + }) }) diff --git a/testing/e2e/global-setup.ts b/testing/e2e/global-setup.ts index 42011f43f..4dafbc7aa 100644 --- a/testing/e2e/global-setup.ts +++ b/testing/e2e/global-setup.ts @@ -89,10 +89,14 @@ export default async function globalSetup() { } function registerMediaFixtures(mock: LLMock) { - // Transcription: onTranscription sets match.endpoint = "transcription" + // Transcription: onTranscription sets match.endpoint = "transcription". + // `duration` is only served on verbose_json responses (whisper-1's default + // mode) — the otel middleware spec asserts it surfaces as the + // self-describing `billed` usage on the transcription span. mock.onTranscription({ transcription: { text: 'I would like to buy a Fender Stratocaster please', + duration: 2.4, }, }) diff --git a/testing/e2e/src/lib/otel-local-tracer.ts b/testing/e2e/src/lib/otel-local-tracer.ts new file mode 100644 index 000000000..cfd57e289 --- /dev/null +++ b/testing/e2e/src/lib/otel-local-tracer.ts @@ -0,0 +1,102 @@ +import type { + AttributeValue, + Context, + Span, + SpanContext, + SpanStatus, + Tracer, +} from '@opentelemetry/api' + +export interface LocalCapturedSpan { + name: string + kind?: number + attributes: Record + status: SpanStatus + ended: boolean +} + +/** + * Single-request in-memory tracer shared by the `api.otel-*` routes. Unlike + * the per-testId capture in `otel-capture.ts` (used by + * `api.middleware-test.ts`), everything in those routes happens inside one + * POST, so spans collect into a local array returned directly in the response + * body. + */ +export function createLocalCaptureTracer(): { + tracer: Tracer + spans: Array +} { + const spans: Array = [] + let spanSeq = 0 + const tracer: Tracer = { + startSpan(name, options = {}, _ctx?: Context): Span { + const id = `span-${spanSeq++}` + const attributes: Record = {} + for (const [k, v] of Object.entries(options.attributes ?? {})) { + if (v !== undefined) attributes[k] = v + } + const captured: LocalCapturedSpan = { + name, + kind: options.kind, + attributes, + status: { code: 0 }, + ended: false, + } + spans.push(captured) + const span: Span = { + spanContext(): SpanContext { + return { traceId: 'otel-local-trace', spanId: id, traceFlags: 1 } + }, + setAttribute(key, value) { + captured.attributes[key] = value + return span + }, + setAttributes(next) { + for (const [k, v] of Object.entries(next)) { + captured.attributes[k] = v as AttributeValue + } + return span + }, + addEvent() { + return span + }, + addLink() { + return span + }, + addLinks() { + return span + }, + setStatus(status) { + captured.status = status + return span + }, + updateName(next) { + captured.name = next + return span + }, + end() { + captured.ended = true + }, + isRecording() { + return !captured.ended + }, + recordException() {}, + } + return span + }, + + // Minimal implementation — otelMiddleware never calls startActiveSpan. + // eslint-disable-next-line @typescript-eslint/no-explicit-any + startActiveSpan(...args: Array) { + const fn = args[args.length - 1] as (span: Span) => unknown + const name = args[0] as string + const span = tracer.startSpan(name, {}) + try { + return fn(span) + } finally { + span.end() + } + }, + } + return { tracer, spans } +} diff --git a/testing/e2e/src/lib/request-body.ts b/testing/e2e/src/lib/request-body.ts new file mode 100644 index 000000000..dc15ed4e2 --- /dev/null +++ b/testing/e2e/src/lib/request-body.ts @@ -0,0 +1,21 @@ +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null +} + +/** + * Extract the payload record from an `api.otel-*` route's POST body, + * unwrapping the `forwardedProps` / `data` envelopes the test harness may + * nest it in. Throws on any non-object shape. + */ +export function recordFromBody(body: unknown): Record { + if (!isRecord(body)) { + throw new Error('Invalid request body') + } + + const data = body.forwardedProps ?? body.data ?? body + if (!isRecord(data)) { + throw new Error('Invalid request body') + } + + return data +} diff --git a/testing/e2e/src/routeTree.gen.ts b/testing/e2e/src/routeTree.gen.ts index 14ffbf330..eae8266cc 100644 --- a/testing/e2e/src/routeTree.gen.ts +++ b/testing/e2e/src/routeTree.gen.ts @@ -28,6 +28,7 @@ import { Route as ApiToolsTestRouteImport } from './routes/api.tools-test' import { Route as ApiToolCallLifecycleWireRouteImport } from './routes/api.tool-call-lifecycle-wire' import { Route as ApiSummarizeRouteImport } from './routes/api.summarize' import { Route as ApiOtelUsageRouteImport } from './routes/api.otel-usage' +import { Route as ApiOtelTranscriptionRouteImport } from './routes/api.otel-transcription' import { Route as ApiOtelMediaRouteImport } from './routes/api.otel-media' import { Route as ApiOpenrouterWebToolsWireRouteImport } from './routes/api.openrouter-web-tools-wire' import { Route as ApiOpenrouterCostRouteImport } from './routes/api.openrouter-cost' @@ -154,6 +155,11 @@ const ApiOtelUsageRoute = ApiOtelUsageRouteImport.update({ path: '/api/otel-usage', getParentRoute: () => rootRouteImport, } as any) +const ApiOtelTranscriptionRoute = ApiOtelTranscriptionRouteImport.update({ + id: '/api/otel-transcription', + path: '/api/otel-transcription', + getParentRoute: () => rootRouteImport, +} as any) const ApiOtelMediaRoute = ApiOtelMediaRouteImport.update({ id: '/api/otel-media', path: '/api/otel-media', @@ -340,6 +346,7 @@ export interface FileRoutesByFullPath { '/api/openrouter-cost': typeof ApiOpenrouterCostRoute '/api/openrouter-web-tools-wire': typeof ApiOpenrouterWebToolsWireRoute '/api/otel-media': typeof ApiOtelMediaRoute + '/api/otel-transcription': typeof ApiOtelTranscriptionRoute '/api/otel-usage': typeof ApiOtelUsageRoute '/api/summarize': typeof ApiSummarizeRoute '/api/tool-call-lifecycle-wire': typeof ApiToolCallLifecycleWireRoute @@ -390,6 +397,7 @@ export interface FileRoutesByTo { '/api/openrouter-cost': typeof ApiOpenrouterCostRoute '/api/openrouter-web-tools-wire': typeof ApiOpenrouterWebToolsWireRoute '/api/otel-media': typeof ApiOtelMediaRoute + '/api/otel-transcription': typeof ApiOtelTranscriptionRoute '/api/otel-usage': typeof ApiOtelUsageRoute '/api/summarize': typeof ApiSummarizeRoute '/api/tool-call-lifecycle-wire': typeof ApiToolCallLifecycleWireRoute @@ -441,6 +449,7 @@ export interface FileRoutesById { '/api/openrouter-cost': typeof ApiOpenrouterCostRoute '/api/openrouter-web-tools-wire': typeof ApiOpenrouterWebToolsWireRoute '/api/otel-media': typeof ApiOtelMediaRoute + '/api/otel-transcription': typeof ApiOtelTranscriptionRoute '/api/otel-usage': typeof ApiOtelUsageRoute '/api/summarize': typeof ApiSummarizeRoute '/api/tool-call-lifecycle-wire': typeof ApiToolCallLifecycleWireRoute @@ -493,6 +502,7 @@ export interface FileRouteTypes { | '/api/openrouter-cost' | '/api/openrouter-web-tools-wire' | '/api/otel-media' + | '/api/otel-transcription' | '/api/otel-usage' | '/api/summarize' | '/api/tool-call-lifecycle-wire' @@ -543,6 +553,7 @@ export interface FileRouteTypes { | '/api/openrouter-cost' | '/api/openrouter-web-tools-wire' | '/api/otel-media' + | '/api/otel-transcription' | '/api/otel-usage' | '/api/summarize' | '/api/tool-call-lifecycle-wire' @@ -593,6 +604,7 @@ export interface FileRouteTypes { | '/api/openrouter-cost' | '/api/openrouter-web-tools-wire' | '/api/otel-media' + | '/api/otel-transcription' | '/api/otel-usage' | '/api/summarize' | '/api/tool-call-lifecycle-wire' @@ -644,6 +656,7 @@ export interface RootRouteChildren { ApiOpenrouterCostRoute: typeof ApiOpenrouterCostRoute ApiOpenrouterWebToolsWireRoute: typeof ApiOpenrouterWebToolsWireRoute ApiOtelMediaRoute: typeof ApiOtelMediaRoute + ApiOtelTranscriptionRoute: typeof ApiOtelTranscriptionRoute ApiOtelUsageRoute: typeof ApiOtelUsageRoute ApiSummarizeRoute: typeof ApiSummarizeRoute ApiToolCallLifecycleWireRoute: typeof ApiToolCallLifecycleWireRoute @@ -789,6 +802,13 @@ declare module '@tanstack/react-router' { preLoaderRoute: typeof ApiOtelUsageRouteImport parentRoute: typeof rootRouteImport } + '/api/otel-transcription': { + id: '/api/otel-transcription' + path: '/api/otel-transcription' + fullPath: '/api/otel-transcription' + preLoaderRoute: typeof ApiOtelTranscriptionRouteImport + parentRoute: typeof rootRouteImport + } '/api/otel-media': { id: '/api/otel-media' path: '/api/otel-media' @@ -1089,6 +1109,7 @@ const rootRouteChildren: RootRouteChildren = { ApiOpenrouterCostRoute: ApiOpenrouterCostRoute, ApiOpenrouterWebToolsWireRoute: ApiOpenrouterWebToolsWireRoute, ApiOtelMediaRoute: ApiOtelMediaRoute, + ApiOtelTranscriptionRoute: ApiOtelTranscriptionRoute, ApiOtelUsageRoute: ApiOtelUsageRoute, ApiSummarizeRoute: ApiSummarizeRoute, ApiToolCallLifecycleWireRoute: ApiToolCallLifecycleWireRoute, diff --git a/testing/e2e/src/routes/api.otel-media.ts b/testing/e2e/src/routes/api.otel-media.ts index a749a973c..049ff4cca 100644 --- a/testing/e2e/src/routes/api.otel-media.ts +++ b/testing/e2e/src/routes/api.otel-media.ts @@ -2,122 +2,9 @@ import { createFileRoute } from '@tanstack/react-router' import { generateImage } from '@tanstack/ai' import { otelMiddleware } from '@tanstack/ai/middlewares/otel' import type { Provider } from '@/lib/types' -import type { - AttributeValue, - Context, - Span, - SpanContext, - SpanStatus, - Tracer, -} from '@opentelemetry/api' import { createImageAdapter } from '@/lib/media-providers' - -interface CapturedSpan { - name: string - kind?: number - attributes: Record - status: SpanStatus - ended: boolean -} - -/** - * Single-request in-memory tracer (mirrors `api.otel-usage.ts`). Everything - * happens inside one POST, so spans collect into a local array returned in the - * response body. - */ -function createLocalCaptureTracer(): { - tracer: Tracer - spans: Array -} { - const spans: Array = [] - let spanSeq = 0 - const tracer: Tracer = { - startSpan(name, options = {}, _ctx?: Context): Span { - const id = `span-${spanSeq++}` - const attributes: Record = {} - for (const [k, v] of Object.entries(options.attributes ?? {})) { - if (v !== undefined) attributes[k] = v - } - const captured: CapturedSpan = { - name, - kind: options.kind, - attributes, - status: { code: 0 }, - ended: false, - } - spans.push(captured) - const span: Span = { - spanContext(): SpanContext { - return { traceId: 'otel-media-trace', spanId: id, traceFlags: 1 } - }, - setAttribute(key, value) { - captured.attributes[key] = value - return span - }, - setAttributes(next) { - for (const [k, v] of Object.entries(next)) { - captured.attributes[k] = v as AttributeValue - } - return span - }, - addEvent() { - return span - }, - addLink() { - return span - }, - addLinks() { - return span - }, - setStatus(status) { - captured.status = status - return span - }, - updateName(next) { - captured.name = next - return span - }, - end() { - captured.ended = true - }, - isRecording() { - return !captured.ended - }, - recordException() {}, - } - return span - }, - - startActiveSpan(...args: Array) { - const fn = args[args.length - 1] as (span: Span) => unknown - const name = args[0] as string - const span = tracer.startSpan(name, {}) - try { - return fn(span) - } finally { - span.end() - } - }, - } - return { tracer, spans } -} - -function isRecord(value: unknown): value is Record { - return typeof value === 'object' && value !== null -} - -function recordFromBody(body: unknown): Record { - if (!isRecord(body)) { - throw new Error('Invalid request body') - } - - const data = body.forwardedProps ?? body.data ?? body - if (!isRecord(data)) { - throw new Error('Invalid request body') - } - - return data -} +import { createLocalCaptureTracer } from '@/lib/otel-local-tracer' +import { recordFromBody } from '@/lib/request-body' /** * Drives `generateImage` with `otelMiddleware` against the same aimock mount diff --git a/testing/e2e/src/routes/api.otel-transcription.ts b/testing/e2e/src/routes/api.otel-transcription.ts new file mode 100644 index 000000000..635026107 --- /dev/null +++ b/testing/e2e/src/routes/api.otel-transcription.ts @@ -0,0 +1,64 @@ +import { createFileRoute } from '@tanstack/react-router' +import { generateTranscription } from '@tanstack/ai' +import { otelMiddleware } from '@tanstack/ai/middlewares/otel' +import type { Provider } from '@/lib/types' +import { createTranscriptionAdapter } from '@/lib/media-providers' +import { createLocalCaptureTracer } from '@/lib/otel-local-tracer' +import { recordFromBody } from '@/lib/request-body' + +/** + * Drives `generateTranscription` with `otelMiddleware` against the whisper + * aimock fixture (which reports an audio `duration`), and returns the captured + * spans. End-to-end proof that a duration-billed activity surfaces the + * self-describing billed quantity on its span: + * `tanstack.ai.usage.billed_quantity` + `tanstack.ai.usage.billed_unit`. + */ +export const Route = createFileRoute('/api/otel-transcription')({ + server: { + handlers: { + POST: async ({ request }) => { + await import('@/lib/llmock-server').then((m) => m.ensureLLMock()) + + try { + const body: unknown = await request.json() + const data = recordFromBody(body) + const audio = data.audio + const provider = data.provider + if (typeof audio !== 'string' || typeof provider !== 'string') { + throw new Error('Missing required fields: audio/provider') + } + + const testId = + typeof data.testId === 'string' ? data.testId : undefined + const aimockPort = + typeof data.aimockPort === 'number' ? data.aimockPort : undefined + const adapter = createTranscriptionAdapter( + provider as Provider, + aimockPort, + testId, + ) + const { tracer, spans } = createLocalCaptureTracer() + + await generateTranscription({ + adapter, + audio, + middleware: [otelMiddleware({ tracer })], + }) + + return new Response(JSON.stringify({ ok: true, spans }), { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }) + } catch (error) { + return new Response( + JSON.stringify({ + ok: false, + error: error instanceof Error ? error.message : String(error), + }), + { status: 200, headers: { 'Content-Type': 'application/json' } }, + ) + } + }, + }, + }, +}) diff --git a/testing/e2e/src/routes/api.otel-usage.ts b/testing/e2e/src/routes/api.otel-usage.ts index 0171fdd2b..629f44b0b 100644 --- a/testing/e2e/src/routes/api.otel-usage.ts +++ b/testing/e2e/src/routes/api.otel-usage.ts @@ -3,105 +3,11 @@ import { chat, createChatOptions } from '@tanstack/ai' import { otelMiddleware } from '@tanstack/ai/middlewares/otel' import { createOpenaiChatCompletions } from '@tanstack/ai-openai' import { createOpenRouterText } from '@tanstack/ai-openrouter' -import type { - AttributeValue, - Context, - Span, - SpanContext, - Tracer, -} from '@opentelemetry/api' +import { createLocalCaptureTracer } from '@/lib/otel-local-tracer' const LLMOCK_DEFAULT_BASE = process.env.LLMOCK_URL || 'http://127.0.0.1:4010' const DUMMY_KEY = 'sk-e2e-test-dummy-key' -interface CapturedSpan { - name: string - kind?: number - attributes: Record - ended: boolean -} - -/** - * Single-request in-memory tracer. Unlike the per-testId capture in - * `api.middleware-test.ts`, everything here happens inside one POST, so spans - * collect into a local array returned directly in the response body. - */ -function createLocalCaptureTracer(): { - tracer: Tracer - spans: Array -} { - const spans: Array = [] - let spanSeq = 0 - const tracer: Tracer = { - startSpan(name, options = {}, _ctx?: Context): Span { - const id = `span-${spanSeq++}` - const attributes: Record = {} - for (const [k, v] of Object.entries(options.attributes ?? {})) { - if (v !== undefined) attributes[k] = v as AttributeValue - } - const captured: CapturedSpan = { - name, - kind: options.kind, - attributes, - ended: false, - } - spans.push(captured) - const span: Span = { - spanContext(): SpanContext { - return { traceId: 'otel-usage-trace', spanId: id, traceFlags: 1 } - }, - setAttribute(key, value) { - captured.attributes[key] = value as AttributeValue - return span - }, - setAttributes(next) { - for (const [k, v] of Object.entries(next)) { - captured.attributes[k] = v as AttributeValue - } - return span - }, - addEvent() { - return span - }, - addLink() { - return span - }, - addLinks() { - return span - }, - setStatus() { - return span - }, - updateName(next) { - captured.name = next - return span - }, - end() { - captured.ended = true - }, - isRecording() { - return !captured.ended - }, - recordException() {}, - } - return span - }, - // Minimal implementation — otelMiddleware never calls startActiveSpan. - // eslint-disable-next-line @typescript-eslint/no-explicit-any - startActiveSpan(...args: Array) { - const fn = args[args.length - 1] as (span: Span) => unknown - const name = args[0] as string - const span = tracer.startSpan(name, {}) - try { - return fn(span) - } finally { - span.end() - } - }, - } - return { tracer, spans } -} - /** * Drives a chat adapter with `otelMiddleware` against the existing * hand-crafted aimock mounts that report rich usage, and returns the captured diff --git a/testing/e2e/tests/middleware.spec.ts b/testing/e2e/tests/middleware.spec.ts index 6b15d6759..964ce0a4c 100644 --- a/testing/e2e/tests/middleware.spec.ts +++ b/testing/e2e/tests/middleware.spec.ts @@ -335,6 +335,43 @@ test.describe('Middleware Lifecycle', () => { }) }) + test('otel middleware emits the self-describing billed quantity for a duration-billed activity', async ({ + request, + testId, + aimockPort, + }) => { + // `/api/otel-transcription` drives whisper-1 (duration-billed) against the + // transcription aimock fixture, whose response reports `duration: 2.4`. + // The adapter surfaces that as `usage.billed = { quantity, unit }`, and the + // middleware must emit it as the paired billed_quantity/billed_unit + // attributes — the machine-readable unit that #816 adds. + const res = await request.post('/api/otel-transcription', { + data: { + audio: 'data:audio/mpeg;base64,SGVsbG8=', + provider: 'openai', + testId, + aimockPort, + }, + }) + expect(res.ok()).toBe(true) + const { ok, error, spans } = await res.json() + expect(error ?? null).toBeNull() + expect(ok).toBe(true) + + const mediaSpans = spans.filter( + (s: any) => s.attributes['gen_ai.operation.name'] === 'transcription', + ) + expect(mediaSpans).toHaveLength(1) + expect(mediaSpans[0].ended).toBe(true) + expect(mediaSpans[0].attributes).toMatchObject({ + 'gen_ai.request.model': 'whisper-1', + 'tanstack.ai.usage.billed_quantity': 2.4, + 'tanstack.ai.usage.billed_unit': 'seconds', + // Deprecated bare count still emitted for backward compatibility. + 'tanstack.ai.usage.duration_seconds': 2.4, + }) + }) + test('no middleware passes content through unchanged', async ({ page, testId,