Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .changeset/billed-usage-unit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
'@tanstack/ai-event-client': minor
'@tanstack/ai': minor
'@tanstack/ai-fal': minor
'@tanstack/ai-grok': minor
'@tanstack/ai-openai': minor
---

Add a self-describing `billed` field to `TokenUsage` so non-token billed quantities carry the unit they are counted in (#816). `usage.billed` is `{ quantity, unit }` with a `BillingUnit` union (`'seconds'`, `'units'`, `'images'`, ... open-ended), replacing the guesswork previously needed to interpret the bare `unitsBilled` / `durationSeconds` counts — those two fields are now deprecated but still populated for backward compatibility. The fal adapters report `{ quantity, unit: 'units' }`, Grok video `{ quantity, unit: 'seconds' }`, and the OpenAI/Grok duration-billed transcription paths `{ quantity, unit: 'seconds' }`. `otelMiddleware` emits the pair as `tanstack.ai.usage.billed_quantity` / `tanstack.ai.usage.billed_unit` span attributes.
2 changes: 1 addition & 1 deletion docs/adapters/grok.md
Original file line number Diff line number Diff line change
Expand Up @@ -292,7 +292,7 @@ const { jobId } = await generateVideo({

Like the Grok Imagine image models, sizing is aspect-ratio based: the `size` option takes an `aspectRatio_resolution` template. Supported aspect ratios are `1:1`, `16:9`, `9:16`, `4:3`, `3:4`, `3:2`, and `2:3`; supported resolutions are `480p`, `720p`, and `1080p` (e.g. `"9:16_1080p"`). The resolution suffix is optional.

When the job completes, the adapter reports usage on the result: `usage.unitsBilled` carries the billed seconds of video and `usage.cost` the exact cost in USD, both as returned by the xAI API.
When the job completes, the adapter reports usage on the result: `usage.billed` carries the billed seconds of video (`{ quantity, unit: 'seconds' }`) and `usage.cost` the exact cost in USD, both as returned by the xAI API.

See [Video Generation](../media/video-generation) for the full jobs/polling flow, streaming mode, and the `useGenerateVideo` hook.

Expand Down
11 changes: 8 additions & 3 deletions docs/advanced/otel.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,10 @@ Iteration spans are numbered (`#0`, `#1`, ...) so distinct iterations of the sam
| root / iteration | `gen_ai.usage.cache_read.input_tokens` | cached prompt tokens, when reported |
| root / iteration | `gen_ai.usage.cache_creation.input_tokens` | cache-write prompt tokens, when reported |
| root / iteration | `gen_ai.usage.reasoning.output_tokens` | reasoning/thinking tokens, when reported |
| root / iteration | `tanstack.ai.usage.duration_seconds` | duration-based billing (e.g. transcription), when reported |
| root / iteration | `tanstack.ai.usage.billed_quantity` | non-token billed quantity, when reported |
| root / iteration | `tanstack.ai.usage.billed_unit` | unit of the billed quantity (`seconds`, `units`, ...) |
| root / iteration | `tanstack.ai.usage.duration_seconds` | deprecated duration count; read `billed_quantity`/`billed_unit` instead |
| root / iteration | `tanstack.ai.usage.units_billed` | deprecated bare unit count; read `billed_quantity`/`billed_unit` instead |
| root / iteration | `tanstack.ai.usage.upstream_cost` | gateway upstream cost (e.g. OpenRouter), when reported |
| root / iteration | `tanstack.ai.usage.upstream_input_cost` | upstream input cost split, when reported |
| root / iteration | `tanstack.ai.usage.upstream_output_cost` | upstream output cost split, when reported |
Expand All @@ -90,7 +93,9 @@ Iteration spans are numbered (`#0`, `#1`, ...) so distinct iterations of the sam
| tool | `gen_ai.tool.type` | `function` |
| tool | `tanstack.ai.tool.outcome` | `success` / `error` |

Usage attributes beyond input/output tokens are emitted only when the provider reports them, so spans stay clean otherwise. Cache and reasoning breakdowns use the official GenAI semconv names; `gen_ai.usage.cost` and `gen_ai.usage.total_tokens` are de-facto extensions consumed directly by backends like PostHog — without them, backends re-derive cost from their own price tables and lose cache discounts and gateway markup. Fields with no established convention (duration-based billing, the upstream cost split) are TanStack-namespaced.
Usage attributes beyond input/output tokens are emitted only when the provider reports them, so spans stay clean otherwise. Cache and reasoning breakdowns use the official GenAI semconv names; `gen_ai.usage.cost` and `gen_ai.usage.total_tokens` are de-facto extensions consumed directly by backends like PostHog — without them, backends re-derive cost from their own price tables and lose cache discounts and gateway markup. Fields with no established convention (the billed quantity/unit pair, the upstream cost split, and the deprecated bare counts) are TanStack-namespaced.

For non-token billing (seconds of video or transcription, fal's endpoint units, ...), `tanstack.ai.usage.billed_quantity` and `tanstack.ai.usage.billed_unit` are emitted as a pair from `usage.billed`, so backends can label and aggregate media usage without knowing the provider. The deprecated `duration_seconds` / `units_billed` attributes carry the same quantities without the unit and remain emitted for backward compatibility.

### Metrics

Expand Down Expand Up @@ -231,7 +236,7 @@ Each media call produces one `CLIENT` span tagged with the activity's `gen_ai.op
| `generateSpeech` | `text_to_speech` |
| `generateTranscription` | `transcription` |

The span carries `gen_ai.system` and `gen_ai.request.model` at start and, on finish, the same `gen_ai.usage.*` / `tanstack.ai.usage.*` attributes documented above — including `tanstack.ai.usage.units_billed` for unit-billed media. When a `Meter` is supplied it records the `gen_ai.client.operation.duration` histogram, tagged per activity. For streaming video the span covers the full create → poll → complete lifecycle; for non-streaming `generateVideo` it covers job submission. If a streaming video consumer abandons the stream before completion, the span is ended via `onAbort` (status `ERROR`, `tanstack.ai.completion.reason = cancelled`) rather than leaked.
The span carries `gen_ai.system` and `gen_ai.request.model` at start and, on finish, the same `gen_ai.usage.*` / `tanstack.ai.usage.*` attributes documented above — including the `tanstack.ai.usage.billed_quantity` / `tanstack.ai.usage.billed_unit` pair for unit-billed media. When a `Meter` is supplied it records the `gen_ai.client.operation.duration` histogram, tagged per activity. For streaming video the span covers the full create → poll → complete lifecycle; for non-streaming `generateVideo` it covers job submission. If a streaming video consumer abandons the stream before completion, the span is ended via `onAbort` (status `ERROR`, `tanstack.ai.completion.reason = cancelled`) rather than leaked.

`otelMiddleware` applies the same `spanNameFormatter`, `attributeEnricher`, `onBeforeSpanStart`, and `onSpanEnd` extension points to media spans — the span info is discriminated by `kind`, where media spans report `kind: 'generation'`. For a custom backend, implement the base `GenerationMiddleware` contract directly; its hooks (`onStart` / `onUsage` / `onFinish` / `onAbort` / `onError`) receive the `GenerationMiddlewareContext` and fire for every activity, chat included. The `GenerationMiddleware` types are exported from the package root, while the `otelMiddleware` value lives on the `@tanstack/ai/middlewares/otel` subpath so importing `@tanstack/ai` never requires the optional `@opentelemetry/api` peer.

Expand Down
10 changes: 5 additions & 5 deletions docs/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -271,19 +271,19 @@
"label": "Audio Generation",
"to": "media/audio-generation",
"addedAt": "2026-04-23",
"updatedAt": "2026-06-08"
"updatedAt": "2026-07-04"
},
{
"label": "Image Generation",
"to": "media/image-generation",
"addedAt": "2026-04-15",
"updatedAt": "2026-07-01"
"updatedAt": "2026-07-04"
},
{
"label": "Video Generation",
"to": "media/video-generation",
"addedAt": "2026-04-15",
"updatedAt": "2026-07-01"
"updatedAt": "2026-07-04"
},
{
"label": "Generation Hooks",
Expand All @@ -310,7 +310,7 @@
"label": "OpenTelemetry",
"to": "advanced/otel",
"addedAt": "2026-05-08",
"updatedAt": "2026-06-17"
"updatedAt": "2026-07-04"
}
]
},
Expand Down Expand Up @@ -527,7 +527,7 @@
"label": "Grok (xAI)",
"to": "adapters/grok",
"addedAt": "2026-04-15",
"updatedAt": "2026-06-24"
"updatedAt": "2026-07-04"
},
{
"label": "Groq",
Expand Down
7 changes: 4 additions & 3 deletions docs/media/audio-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,9 +130,10 @@ interface AudioGenerationResult {
}
// Canonical TokenUsage (same shape as chat), present when the provider
// reports it (e.g. Gemini Lyria via generateContent). Usage-billed providers
// (fal) instead surface `usage.unitsBilled` — the real billed quantity read
// from fal's `x-fal-billable-units` result header. Multiply by the endpoint's
// unit price (fal pricing API) for the exact cost.
// (fal) instead surface `usage.billed` ({ quantity, unit: 'units' }) — the
// real billed quantity read from fal's `x-fal-billable-units` result header.
// Multiply the quantity by the endpoint's unit price (fal pricing API) for
// the exact cost.
usage?: TokenUsage
}
```
Expand Down
15 changes: 8 additions & 7 deletions docs/media/image-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -396,7 +396,7 @@ interface ImageGenerationResult {
// Canonical TokenUsage (same shape as chat). Token-billed models also surface
// a per-modality breakdown on `promptTokensDetails` (e.g. text vs image input
// tokens for gpt-image-1). Usage-billed providers (fal) instead surface
// `usage.unitsBilled` — see the note below.
// `usage.billed` ({ quantity, unit }) — see the note below.
usage?: TokenUsage
}

Expand All @@ -408,9 +408,9 @@ interface GeneratedImage {
```

> **Cost tracking (fal):** fal bills by usage-based units rather than tokens. The
> fal image adapter surfaces the real billed quantity as `usage.unitsBilled`
> (read from fal's `x-fal-billable-units` result header). Multiply it by the
> endpoint's unit price from
> fal image adapter surfaces the real billed quantity as `usage.billed` —
> `{ quantity, unit: 'units' }`, read from fal's `x-fal-billable-units` result
> header. Multiply the quantity by the endpoint's unit price from
> `GET https://api.fal.ai/v1/models/pricing?endpoint_id=…` for the exact cost —
> no `fetch` interceptor needed.

Expand All @@ -424,9 +424,10 @@ const result = await generateImage({
prompt: 'a serene mountain lake',
})

if (result.usage?.unitsBilled != null) {
const cost = result.usage.unitsBilled * unitPrice // unitPrice from fal pricing API
console.log(`Billed ${result.usage.unitsBilled} units (~$${cost})`)
if (result.usage?.billed) {
const { quantity, unit } = result.usage.billed
const cost = quantity * unitPrice // unitPrice from fal pricing API
console.log(`Billed ${quantity} ${unit} (~$${cost})`)
}
```

Expand Down
21 changes: 13 additions & 8 deletions docs/media/video-generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -625,7 +625,7 @@ adapter.snapDuration(2.5) // 3 — clamped/rounded into range
adapter.snapDuration(99) // 15
```

Generated clips include an audio track. When the job completes, the adapter reports `usage.unitsBilled` (billed seconds of video) and `usage.cost` (exact USD cost as returned by the API) on the result.
Generated clips include an audio track. When the job completes, the adapter reports `usage.billed` (`{ quantity, unit: 'seconds' }` — billed seconds of video) and `usage.cost` (exact USD cost as returned by the API) on the result.

## Response Types

Expand Down Expand Up @@ -660,19 +660,24 @@ interface VideoUrlResult {
jobId: string
url: string // URL to download/stream the video
expiresAt?: Date // When the URL expires
// Usage for the completed generation, when the adapter reports it. fal
// populates `usage.unitsBilled` from its `x-fal-billable-units` header.
// Usage for the completed generation, when the adapter reports it. The
// billed quantity is self-describing: fal reports
// `usage.billed = { quantity, unit: 'units' }` (from its
// `x-fal-billable-units` header), Grok Imagine reports
// `{ quantity, unit: 'seconds' }`.
usage?: TokenUsage
}
```

> **Cost tracking (fal):** fal bills media generation by usage-based units
> rather than tokens. The fal adapters surface the real billed quantity as
> `usage.unitsBilled` (denominated in the endpoint's priced unit). Combine it
> with the endpoint's unit price from
> `GET https://api.fal.ai/v1/models/pricing?endpoint_id=…` to compute the exact
> cost (`unitsBilled * unitPrice`). The same `usage.unitsBilled` is surfaced
> on image, audio, speech, and transcription results.
> `usage.billed` — `{ quantity, unit: 'units' }`, where `'units'` marks fal's
> endpoint-defined priced unit. Combine the quantity with the endpoint's unit
> price from `GET https://api.fal.ai/v1/models/pricing?endpoint_id=…` to
> compute the exact cost (`billed.quantity * unitPrice`). The same
> `usage.billed` is surfaced on image, audio, speech, and transcription
> results. (The deprecated bare count `usage.unitsBilled` is still populated
> for backward compatibility.)

## Model Variants

Expand Down
10 changes: 5 additions & 5 deletions examples/ts-react-media/src/components/ImageGenerator.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -304,12 +304,12 @@ export default function ImageGenerator({
className="w-full h-auto"
/>
</div>
{modelResult.result.usage?.unitsBilled != null && (
{modelResult.result.usage?.billed && (
<p className="text-xs text-gray-500">
Billed {modelResult.result.usage.unitsBilled} fal unit
{modelResult.result.usage.unitsBilled === 1
? ''
: 's'}{' '}
Billed {modelResult.result.usage.billed.quantity}{' '}
{modelResult.result.usage.billed.unit === 'units'
? `fal unit${modelResult.result.usage.billed.quantity === 1 ? '' : 's'}`
: modelResult.result.usage.billed.unit}{' '}
— multiply by the endpoint unit price for USD cost
</p>
)}
Expand Down
34 changes: 26 additions & 8 deletions examples/ts-react-media/src/components/VideoGenerator.tsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { useEffect, useRef, useState } from 'react'
import { Film, Loader2, Shuffle, Upload, X } from 'lucide-react'
import type { BilledUsage } from '@tanstack/ai'
import type { VideoMode } from '@/lib/models'

import {
Expand All @@ -21,9 +22,25 @@ type JobState =
model: string
progress?: number | undefined
}
| { status: 'completed'; url: string; unitsBilled?: number; cost?: number }
| { status: 'completed'; url: string; billed?: BilledUsage; cost?: number }
| { status: 'error'; message: string }

/**
* Human label for a billed quantity, driven by the unit the adapter reported —
* no guessing from provider identity or cost presence.
*/
function describeBilled({ quantity, unit }: BilledUsage): string {
const plural = quantity === 1 ? '' : 's'
switch (unit) {
case 'seconds':
return `${quantity} second${plural} of video`
case 'units':
return `${quantity} fal unit${plural}`
default:
return `${quantity} ${unit}`
}
}

interface VideoGeneratorProps {
initialImageUrl?: string | null
}
Expand Down Expand Up @@ -98,7 +115,7 @@ export default function VideoGenerator({
[model]: {
status: 'completed',
url: url,
unitsBilled: urlResult.usage?.unitsBilled,
billed: urlResult.usage?.billed,
cost: urlResult.usage?.cost,
},
}))
Expand Down Expand Up @@ -424,16 +441,17 @@ export default function VideoGenerator({
{state.cost != null ? (
<p className="text-xs text-gray-500">
Billed ${state.cost.toFixed(3)}
{state.unitsBilled != null
? ` for ${state.unitsBilled} second${state.unitsBilled === 1 ? '' : 's'} of video`
{state.billed
? ` for ${describeBilled(state.billed)}`
: ''}
</p>
) : (
state.unitsBilled != null && (
state.billed && (
<p className="text-xs text-gray-500">
Billed {state.unitsBilled} fal unit
{state.unitsBilled === 1 ? '' : 's'} — multiply by the
endpoint unit price for USD cost
Billed {describeBilled(state.billed)}
{state.billed.unit === 'units'
? ' — multiply by the endpoint unit price for USD cost'
: ''}
</p>
)
)}
Expand Down
4 changes: 2 additions & 2 deletions examples/ts-react-media/src/lib/server-functions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -252,8 +252,8 @@ export const createVideoJobFn = createServerFn({ method: 'POST' })
case 'grok-imagine-video': {
// Direct xAI Imagine API (XAI_API_KEY) — no fal in between. The base
// grok-imagine-video (v1.0) supports text-to-video; durations are
// 1-15 integer seconds. Completed jobs report usage.unitsBilled
// (billed seconds) and usage.cost (exact USD).
// 1-15 integer seconds. Completed jobs report usage.billed
// ({ quantity, unit: 'seconds' }) and usage.cost (exact USD).
return generateVideo({
adapter: grokVideo('grok-imagine-video'),
prompt: asTextPrompt(data.prompt),
Expand Down
Loading