feat(ai-openrouter): video generation adapter (/api/v1/videos) + image activity follow-ups#740
Draft
tombeckenham wants to merge 7 commits into
Conversation
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughAdds OpenRouter video generation support with typed model metadata, async job polling, server-side media download, and example wiring. It also tightens OpenRouter image generation validation and updates related docs, tests, exports, and type constraints. ChangesOpenRouter Video Adapter & Image Adapter Fixes
Sequence Diagram(s)sequenceDiagram
participant App
participant OpenRouterVideoAdapter
participant OpenRouterAPI
participant ContentURL
App->>OpenRouterVideoAdapter: createVideoJob
OpenRouterVideoAdapter->>OpenRouterVideoAdapter: validate size and duration
OpenRouterVideoAdapter->>OpenRouterVideoAdapter: map image roles to video fields
OpenRouterVideoAdapter->>OpenRouterAPI: submit generation request
OpenRouterAPI-->>OpenRouterVideoAdapter: return job id
App->>OpenRouterVideoAdapter: getVideoStatus
OpenRouterVideoAdapter->>OpenRouterAPI: fetch generation status
OpenRouterAPI-->>OpenRouterVideoAdapter: return mapped status
App->>OpenRouterVideoAdapter: getVideoUrl
OpenRouterVideoAdapter->>OpenRouterAPI: fetch completed job
OpenRouterAPI-->>OpenRouterVideoAdapter: return content URL and usage
OpenRouterVideoAdapter->>ContentURL: download video bytes with auth
ContentURL-->>OpenRouterVideoAdapter: return media bytes
OpenRouterVideoAdapter-->>App: return data URL and usage
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related issues
Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Contributor
🚀 Changeset Version Preview3 package(s) bumped directly, 19 bumped as dependents. 🟨 Minor bumps
🟩 Patch bumps
|
|
View your CI Pipeline Execution ↗ for commit e813487
☁️ Nx Cloud last updated this comment at |
@tanstack/ai
@tanstack/ai-angular
@tanstack/ai-anthropic
@tanstack/ai-client
@tanstack/ai-code-mode
@tanstack/ai-code-mode-skills
@tanstack/ai-devtools-core
@tanstack/ai-elevenlabs
@tanstack/ai-event-client
@tanstack/ai-fal
@tanstack/ai-gemini
@tanstack/ai-grok
@tanstack/ai-groq
@tanstack/ai-isolate-cloudflare
@tanstack/ai-isolate-node
@tanstack/ai-isolate-quickjs
@tanstack/ai-mcp
@tanstack/ai-ollama
@tanstack/ai-openai
@tanstack/ai-openrouter
@tanstack/ai-preact
@tanstack/ai-react
@tanstack/ai-react-ui
@tanstack/ai-solid
@tanstack/ai-solid-ui
@tanstack/ai-svelte
@tanstack/ai-utils
@tanstack/ai-vue
@tanstack/ai-vue-ui
@tanstack/openai-base
@tanstack/preact-ai-devtools
@tanstack/react-ai-devtools
@tanstack/solid-ai-devtools
commit: |
tombeckenham
force-pushed
the
618-image-to-image-and-image-to-video-support
branch
from
0c65cc7 to
acd7319
Compare
tombeckenham
force-pushed
the
707-featai-openrouter-video-generation-adapter-apiv1videos-+-image-activity-follow-ups
branch
from
413f0a7 to
7bb9066
Compare
Base automatically changed from
618-image-to-image-and-image-to-video-support
to
main
June 17, 2026 13:18
tombeckenham
force-pushed
the
707-featai-openrouter-video-generation-adapter-apiv1videos-+-image-activity-follow-ups
branch
from
7bb9066 to
c27d0b0
Compare
Open
7 tasks
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (3)
packages/ai-openrouter/tests/video-adapter.test.ts (1)
1-1: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winMove this unit test alongside the source module.
This test is under
packages/ai-openrouter/tests/, but the repo guideline requires colocated*.test.tsfiles next to source.
As per coding guidelines, "**/*.test.ts: Place unit tests alongside source code in*.test.tsfiles".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-openrouter/tests/video-adapter.test.ts` at line 1, The video adapter unit test is in the wrong location and should be moved to sit next to the source module to match the repository’s colocated test convention. Relocate the test for the video adapter so it lives alongside the module it exercises, and keep the existing test contents and imports intact; use the video adapter test file itself and the related source module name to find the correct colocated `*.test.ts` location.Source: Coding guidelines
packages/ai-openrouter/tests/video-per-model-type-safety.test.ts (1)
1-1: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winPlace this unit test next to the source it validates.
This file is in
packages/ai-openrouter/tests/, but unit tests should be colocated as*.test.tsbeside source modules.
As per coding guidelines, "**/*.test.ts: Place unit tests alongside source code in*.test.tsfiles".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-openrouter/tests/video-per-model-type-safety.test.ts` at line 1, The unit test is in the wrong location and should be colocated with the source module it validates. Move the video-per-model-type-safety test to the same directory as the related source file and keep its name as a nearby *.test.ts file so it follows the testing convention used across the codebase. Use the existing test name to find the related module and place the test beside it.Source: Coding guidelines
examples/ts-react-media/src/lib/server-functions.ts (1)
72-91: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueSingle source of truth for OpenRouter model ids.
OPENROUTER_VIDEO_MODEL_IDSduplicates the OpenRouter ids that also appear ascreateVideoJobFnswitch cases and inVIDEO_MODELS(models.ts). For example code this is acceptable, but deriving the set fromVIDEO_MODELSwould prevent the two lists from drifting.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@examples/ts-react-media/src/lib/server-functions.ts` around lines 72 - 91, The OpenRouter model id list is duplicated across `OPENROUTER_VIDEO_MODEL_IDS`, `createVideoJobFn`, and `VIDEO_MODELS`, which risks drift. Update `videoAdapterForModel` to derive its OpenRouter membership from the shared `VIDEO_MODELS` source of truth instead of hardcoding ids locally, and make sure the routing logic still correctly selects `openRouterVideo` versus `falVideo` for each model.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/config.json`:
- Around line 259-260: The docs freshness metadata for the
media/image-generation page is stale because the config entry still uses an
older updatedAt value even though docs/media/image-generation.md was modified in
this PR. Update the corresponding entry in docs/config.json for
media/image-generation so its updatedAt matches the PR date, using the existing
docs metadata entry structure as the anchor.
In `@packages/ai-openrouter/src/adapters/video.ts`:
- Around line 45-52: The authenticated video download in getVideoUrl() can stall
indefinitely, so add a configurable timeout or abort signal around the direct
fetch used there and ensure generateVideo() uses that bounded path. Extend
OpenRouterVideoConfig and the getVideoUrl / generateVideo flow in video.ts so
callers can control the fetch deadline, and make the fetch cleanup handle
cancellation consistently when the gateway/CDN is slow.
- Around line 118-147: The video adapter in the OpenRouter path is validating
only start/end frame images, but it still forwards reference/character images
without checking whether the model supports reference conditioning. Update the
input classification logic in the same function that handles `starts`, `ends`,
and `references` to validate `reference` / `character` images against the model
metadata before building `inputReferences`, and throw a local error when the
model does not support them, consistent with the existing
`getVideoModelMeta(model)?.frameImages` checks.
- Around line 281-310: Add runtime validation for the model-specific
`resolution` and `aspectRatio` options before assigning them in the video
request builder. In the video adapter’s request construction flow, after
`validateVideoSize` and `validateVideoDuration`, validate
`modelOptions.resolution` and `modelOptions.aspectRatio` against the allowed
values for `this.model` (using the existing video metadata/helpers if available)
instead of only casting them in the `request` assembly. Keep the current
`request` shape in the video generation path, but ensure unsupported values are
rejected before `VideoGenerationRequest` is sent.
In `@packages/ai/skills/ai-core/media-generation/SKILL.md`:
- Around line 271-283: The role/matrix section has stale Veo availability text
that still says support is “planned” or “no Veo adapter yet,” which no longer
matches current behavior. Update the `'character'`, `'start_frame'`, and
Gemini/Veo rows in SKILL.md to describe the actual Veo handling instead of
deferred support. Keep the wording aligned with the existing mapping table so
the `generateImage`/`generateVideo` provider support matrix and the role mapping
stay consistent.
---
Nitpick comments:
In `@examples/ts-react-media/src/lib/server-functions.ts`:
- Around line 72-91: The OpenRouter model id list is duplicated across
`OPENROUTER_VIDEO_MODEL_IDS`, `createVideoJobFn`, and `VIDEO_MODELS`, which
risks drift. Update `videoAdapterForModel` to derive its OpenRouter membership
from the shared `VIDEO_MODELS` source of truth instead of hardcoding ids
locally, and make sure the routing logic still correctly selects
`openRouterVideo` versus `falVideo` for each model.
In `@packages/ai-openrouter/tests/video-adapter.test.ts`:
- Line 1: The video adapter unit test is in the wrong location and should be
moved to sit next to the source module to match the repository’s colocated test
convention. Relocate the test for the video adapter so it lives alongside the
module it exercises, and keep the existing test contents and imports intact; use
the video adapter test file itself and the related source module name to find
the correct colocated `*.test.ts` location.
In `@packages/ai-openrouter/tests/video-per-model-type-safety.test.ts`:
- Line 1: The unit test is in the wrong location and should be colocated with
the source module it validates. Move the video-per-model-type-safety test to the
same directory as the related source file and keep its name as a nearby
*.test.ts file so it follows the testing convention used across the codebase.
Use the existing test name to find the related module and place the test beside
it.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 4ca3dcb1-4c47-4a38-b8f8-1b1bed471759
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (27)
.changeset/openrouter-video-adapter.md.changeset/video-adapter-duration-constraint.mddocs/adapters/openrouter.mddocs/config.jsondocs/media/image-generation.mddocs/media/video-generation.mdexamples/ts-react-media/package.jsonexamples/ts-react-media/src/lib/models.tsexamples/ts-react-media/src/lib/server-functions.tspackages/ai-openrouter/package.jsonpackages/ai-openrouter/src/adapters/image.tspackages/ai-openrouter/src/adapters/video.tspackages/ai-openrouter/src/image/image-provider-options.tspackages/ai-openrouter/src/index.tspackages/ai-openrouter/src/model-meta.tspackages/ai-openrouter/src/video/video-provider-options.tspackages/ai-openrouter/tests/image-adapter.test.tspackages/ai-openrouter/tests/video-adapter.test.tspackages/ai-openrouter/tests/video-per-model-type-safety.test.tspackages/ai/skills/ai-core/media-generation/SKILL.mdpackages/ai/src/activities/generateVideo/index.tsscripts/convert-openrouter-models.tsscripts/fetch-openrouter-models.tsscripts/openrouter.video-models.jsonscripts/openrouter.video-models.tstesting/e2e/package.jsontesting/e2e/src/lib/feature-support.ts
Comment on lines
+259
to
260
| "updatedAt": "2026-06-10" | ||
| }, |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Refresh media/image-generation updatedAt to this PR date.
docs/media/image-generation.md is changed in this PR, but its entry still shows "updatedAt": "2026-06-10" instead of today (2026-06-24), so docs freshness metadata is inconsistent.
As per coding guidelines, “Update updatedAt timestamp in docs/config.json when making content changes to a documentation page.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/config.json` around lines 259 - 260, The docs freshness metadata for the
media/image-generation page is stale because the config entry still uses an
older updatedAt value even though docs/media/image-generation.md was modified in
this PR. Update the corresponding entry in docs/config.json for
media/image-generation so its updatedAt matches the PR date, using the existing
docs metadata entry structure as the anchor.
Source: Coding guidelines
Comment on lines
+45
to
+52
| export interface OpenRouterVideoConfig extends OpenRouterClientConfig { | ||
| /** | ||
| * Injectable fetch implementation used for the authenticated video | ||
| * content download (tests, custom runtimes). Defaults to the global | ||
| * fetch. | ||
| */ | ||
| fetch?: typeof globalThis.fetch | ||
| } |
Contributor
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Bound the authenticated video download.
getVideoUrl() can hang indefinitely if the gateway/CDN stalls, and downstream generateVideo() waits on this call. Add a configurable timeout/abort signal around the direct content fetch.
Suggested direction
export interface OpenRouterVideoConfig extends OpenRouterClientConfig {
@@
fetch?: typeof globalThis.fetch
+ downloadTimeoutMs?: number
}
+
+const DEFAULT_VIDEO_DOWNLOAD_TIMEOUT_MS = 120_000
@@
const doFetch = this.clientConfig.fetch ?? globalThis.fetch
- const contentResponse = await doFetch(contentUrl, {
- headers: { Authorization: `Bearer ${this.clientConfig.apiKey}` },
- })
+ const controller = new AbortController()
+ const timeout = setTimeout(
+ () => controller.abort(),
+ this.clientConfig.downloadTimeoutMs ?? DEFAULT_VIDEO_DOWNLOAD_TIMEOUT_MS,
+ )
+ let contentResponse: Response
+ try {
+ contentResponse = await doFetch(contentUrl, {
+ headers: { Authorization: `Bearer ${this.clientConfig.apiKey}` },
+ signal: controller.signal,
+ })
+ } finally {
+ clearTimeout(timeout)
+ }Also applies to: 373-382
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ai-openrouter/src/adapters/video.ts` around lines 45 - 52, The
authenticated video download in getVideoUrl() can stall indefinitely, so add a
configurable timeout or abort signal around the direct fetch used there and
ensure generateVideo() uses that bounded path. Extend OpenRouterVideoConfig and
the getVideoUrl / generateVideo flow in video.ts so callers can control the
fetch deadline, and make the fetch cleanup handle cancellation consistently when
the gateway/CDN is slow.
Comment on lines
+118
to
+147
| if (role === 'end_frame') ends.push(url) | ||
| else if (role === 'reference' || role === 'character') references.push(url) | ||
| // Unroled parts default to the start frame (image-to-video). | ||
| else starts.push(url) | ||
| } | ||
|
|
||
| if (starts.length > 1) { | ||
| throw new Error( | ||
| `openrouter: at most one start-frame image is supported per request (received ${starts.length}). Mark additional images with metadata.role 'reference' or 'end_frame'.`, | ||
| ) | ||
| } | ||
| if (ends.length > 1) { | ||
| throw new Error( | ||
| `openrouter: at most one input with metadata.role === 'end_frame' is supported per request (received ${ends.length}).`, | ||
| ) | ||
| } | ||
|
|
||
| const supportedFrames = getVideoModelMeta(model)?.frameImages | ||
| if (supportedFrames) { | ||
| if (starts.length > 0 && !supportedFrames.includes('first_frame')) { | ||
| throw new Error( | ||
| `openrouter: model ${model} does not accept a start-frame image (supported frame images: ${supportedFrames.join(', ') || 'none'}).`, | ||
| ) | ||
| } | ||
| if (ends.length > 0 && !supportedFrames.includes('last_frame')) { | ||
| throw new Error( | ||
| `openrouter: model ${model} does not accept an end-frame image (supported frame images: ${supportedFrames.join(', ') || 'none'}).`, | ||
| ) | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Validate reference-image support before forwarding inputReferences.
Frame images are checked against model metadata, but reference / character inputs are always accepted. For models that do not support reference/image conditioning, this submits an invalid provider request instead of failing locally as the adapter promises.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ai-openrouter/src/adapters/video.ts` around lines 118 - 147, The
video adapter in the OpenRouter path is validating only start/end frame images,
but it still forwards reference/character images without checking whether the
model supports reference conditioning. Update the input classification logic in
the same function that handles `starts`, `ends`, and `references` to validate
`reference` / `character` images against the model metadata before building
`inputReferences`, and throw a local error when the model does not support them,
consistent with the existing `getVideoModelMeta(model)?.frameImages` checks.
Comment on lines
+281
to
+310
| validateVideoSize(this.model, size) | ||
| validateVideoDuration(this.model, duration) | ||
|
|
||
| const imageFields = mapImagePartsToVideoFields(this.model, resolved.images) | ||
|
|
||
| const request: VideoGenerationRequest = { | ||
| model: this.model, | ||
| prompt: resolved.text, | ||
| ...imageFields, | ||
| ...(size ? { size } : {}), | ||
| ...(duration !== undefined ? { duration } : {}), | ||
| ...(modelOptions?.seed !== undefined ? { seed: modelOptions.seed } : {}), | ||
| ...(modelOptions?.generateAudio !== undefined | ||
| ? { generateAudio: modelOptions.generateAudio } | ||
| : {}), | ||
| ...(modelOptions?.callbackUrl | ||
| ? { callbackUrl: modelOptions.callbackUrl } | ||
| : {}), | ||
| ...(modelOptions?.provider ? { provider: modelOptions.provider } : {}), | ||
| } | ||
| // The SDK types these as branded open enums; the per-model literal | ||
| // unions derived from OPENROUTER_VIDEO_MODEL_META can be broader than | ||
| // the SDK's enum members (e.g. grok-imagine-video's '3:2'), so narrow at | ||
| // the boundary — the wire format is a plain string either way. | ||
| if (modelOptions?.resolution) { | ||
| request.resolution = modelOptions.resolution as Resolution | ||
| } | ||
| if (modelOptions?.aspectRatio) { | ||
| request.aspectRatio = modelOptions.aspectRatio as AspectRatio | ||
| } |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Add runtime validation for resolution and aspectRatio.
size and duration are metadata-validated, but modelOptions.resolution and modelOptions.aspectRatio are cast and forwarded without runtime checks. JavaScript callers or escaped TS values can still send unsupported per-model options.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ai-openrouter/src/adapters/video.ts` around lines 281 - 310, Add
runtime validation for the model-specific `resolution` and `aspectRatio` options
before assigning them in the video request builder. In the video adapter’s
request construction flow, after `validateVideoSize` and
`validateVideoDuration`, validate `modelOptions.resolution` and
`modelOptions.aspectRatio` against the allowed values for `this.model` (using
the existing video metadata/helpers if available) instead of only casting them
in the `request` assembly. Keep the current `request` shape in the video
generation path, but ensure unsupported values are rejected before
`VideoGenerationRequest` is sent.
Comment on lines
+271
to
+283
| | `'character'` | Same as `'reference'`; Veo `referenceImages` slot (planned — no Veo adapter yet) | | ||
| | `'mask'` | OpenAI `mask` (gpt-image-2, gpt-image-1, dall-e-2); fal `mask_url` | | ||
| | `'control'` | fal `control_image_url` (ControlNet / depth / pose) | | ||
| | `'start_frame'` | fal `start_image_url` (or the endpoint's field, e.g. `image_url` on Kling i2v); OpenRouter `frame_images[]` `first_frame`; Veo `image` (planned) | | ||
| | `'end_frame'` | fal `end_image_url` (or e.g. `tail_image_url` / `last_frame_url`); OpenRouter `frame_images[]` `last_frame`; Veo `lastFrame` (planned) | | ||
|
|
||
| **Provider support matrix:** | ||
|
|
||
| | Provider | `generateImage` image parts | `generateVideo` image parts | | ||
| | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| | OpenAI | gpt-image-2 / gpt-image-1 / -mini → `images.edit()` (up to 16). dall-e-2 → edit (1). dall-e-3 throws. | Sora-2 / -pro → `input_reference` (single). Throws if >1. | | ||
| | Gemini | Native (gemini-\*-flash-image, "nano-banana") → multimodal `contents`. Imagen throws. | No native Veo adapter yet — deferred to a follow-up. | | ||
| | fal | Per-endpoint field names from a generated map (`pnpm generate:fal-image-fields`). Defaults: 1 input → `image_url`; >1 → `image_urls`; roles → `mask_url` / `control_image_url` / `reference_image_urls`. | Per-endpoint map (e.g. Kling i2v start frame → `image_url`). Defaults: 1 input → `image_url`; `start_frame`/`end_frame` → `start_image_url`/`end_image_url`; `reference` → `reference_image_urls`. | | ||
| | Grok | grok-imagine models → `/v1/images/edits` JSON endpoint (≤3 sources, addressed by xAI in request order; prompt sent verbatim; mask/control throw). grok-2-image-1212 throws. | n/a | | ||
| | OpenRouter | Prompt parts map 1:1 onto multimodal `text` / `image_url` content parts, preserving interleaved order. | n/a | | ||
| | Anthropic | n/a (no image generation API). | n/a | | ||
| | Provider | `generateImage` image parts | `generateVideo` image parts | | ||
| | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| | OpenAI | gpt-image-2 / gpt-image-1 / -mini → `images.edit()` (up to 16). dall-e-2 → edit (1). dall-e-3 throws. | Sora-2 / -pro → `input_reference` (single). Throws if >1. | | ||
| | Gemini | Native (gemini-\*-flash-image, "nano-banana") → multimodal `contents`. Imagen throws. | No native Veo adapter yet — deferred to a follow-up. | | ||
| | fal | Per-endpoint field names from a generated map (`pnpm generate:fal-image-fields`). Defaults: 1 input → `image_url`; >1 → `image_urls`; roles → `mask_url` / `control_image_url` / `reference_image_urls`. | Per-endpoint map (e.g. Kling i2v start frame → `image_url`). Defaults: 1 input → `image_url`; `start_frame`/`end_frame` → `start_image_url`/`end_image_url`; `reference` → `reference_image_urls`. | |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Fix stale Veo availability wording in the role/matrix section.
This section still says Veo support is “planned” / “no Veo adapter yet,” which conflicts with the current Gemini/Veo usage shown in this PR context. Please update these rows to reflect current behavior.
🧰 Tools
🪛 SkillSpector (2.2.3)
[warning] 521: [E1] External Transmission: Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.
Remediation: Verify the destination URL is trusted and necessary. Remove or replace with documented APIs. Ensure no secrets, tokens, or PII are transmitted.
(Data Exfiltration (E1))
[error] 544: [MP3] Memory Manipulation: Skill manipulates agent memory, state, or stored context. Memory corruption can alter personality, override safety rules, or cause unpredictable behavior.
Remediation: Protect agent memory and state from modification by untrusted content. Use read-only memory for critical instructions and validate all state changes.
(Memory Poisoning (MP3))
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ai/skills/ai-core/media-generation/SKILL.md` around lines 271 - 283,
The role/matrix section has stale Veo availability text that still says support
is “planned” or “no Veo adapter yet,” which no longer matches current behavior.
Update the `'character'`, `'start_frame'`, and Gemini/Veo rows in SKILL.md to
describe the actual Veo handling instead of deferred support. Keep the wording
aligned with the existing mapping table so the `generateImage`/`generateVideo`
provider support matrix and the role mapping stay consistent.
…e activity follow-ups Closes #707. - Add openRouterVideo: async jobs adapter for OpenRouter's dedicated video API (submit -> poll -> download). Per-model size/duration/option types are generated from GET /api/v1/videos/models; frame roles map onto frame_images[] / input_references[] per the MediaInputRole taxonomy. - Teach the model-meta sync scripts the videos/models endpoint (openrouter.video-models.json + OPENROUTER_VIDEO_MODEL_META). - Image adapter follow-ups from the #624 review: throw on unmapped sizes (the size union used a Unicode multiplication sign so every non-square size silently dropped its aspect ratio), throw on numberOfImages > 1 (live-verified: the gateway ignores all count keys), expose image_config.strength. - Completed videos are returned as data: URLs (unsigned_urls 401 without the API key header) with gateway-reported cost on usage.cost. The SDK's getVideoContent is bypassed: its matcher only accepts application/octet-stream while the endpoint serves video/mp4. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The getVideoContent response-matcher bug is still present in 0.12.79 (the stream matcher only accepts application/octet-stream while the endpoint serves video/mp4), so the direct unsigned-URL download stays. Link the aimock feature request (CopilotKit/aimock#261) from the e2e matrix exclusion. Submit/poll/download lifecycle re-verified live on the new SDK. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire openRouterVideo onto the shared typed-duration contract (the same one geminiVideo uses): add the sixth BaseVideoAdapter generic (OpenRouterVideoModelDurationByName), narrow `duration` per model from the published `/api/v1/videos/models` metadata, and override availableDurations() / snapDuration() (backed by snapToDurationOption). `duration` is now a compile-time per-model union; the runtime validateVideoDuration backstop stays for JS callers and unknown-meta models. - video-provider-options.ts: OpenRouterVideoModelDurationByName + getVideoDurationOptions (discrete from meta, none when unknown/empty) - adapter: 6th generic, createVideoJob narrowed to per-model size/duration, availableDurations()/snapDuration() overrides - export OpenRouterVideoModelDurationByName from index - tests: 4 introspection cases; existing negative size/duration tests now use @ts-expect-error (proves the union rejects them) while still asserting the runtime throw - docs/media/video-generation.md, docs/adapters/openrouter.md, media-generation SKILL.md: document snapDuration/availableDurations for OpenRouter; bump updatedAt; changeset Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…o typed-duration constraint ts-react-media now demos openRouterVideo alongside fal: - add @tanstack/ai-openrouter dep - Seedance 2.0 (text-to-video) and Veo 3.1 (image-to-video) model entries - switch cases showcasing adapter.snapDuration() to coerce raw UI seconds to the model's nearest supported duration (Seedance 7→7, Veo 3.1 7→6) - videoAdapterForModel() resolver routes the poll/status path to the right adapter (the helpers were hardcoded to falVideo) Building the example surfaced a pre-existing bug in the #624 typed-duration contract: generateVideo()'s `TAdapter extends VideoAdapter<string, any, any, any>` bound let the sixth (duration) generic fall back to its `Record<string, number>` default. Because VideoAdapter.createVideoJob is a contravariant function-valued property, no adapter whose `duration` is a per-model literal union (Veo `4|6|8`, Seedance `4..15`) satisfied the bound — so even the documented `generateVideo({ adapter: geminiVideo('veo-3.1-...') })` failed to type-check. Widen the bound to leave size/duration unpinned at all 10 activity sites; per-model types are still recovered via inference (VideoSizeForAdapter / VideoDurationForAdapter). Adds a compile-only regression test and an @tanstack/ai changeset. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…doc type-check The new kiira doc-typecheck (#805) type-checks fenced TS samples. The OpenRouter snapDuration snippet referenced generateVideo/openRouterVideo/sliderSeconds without imports or a declaration; add them so the snippet type-checks standalone. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tombeckenham
force-pushed
the
707-featai-openrouter-video-generation-adapter-apiv1videos-+-image-activity-follow-ups
branch
from
e55fac2 to
fc9f720
Compare
Contributor
There was a problem hiding this comment.
♻️ Duplicate comments (1)
packages/ai/skills/ai-core/media-generation/SKILL.md (1)
271-283: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winUpdate stale Veo wording in role/matrix to match current support.
Line 271 and Line 282 still say Veo is “planned/deferred,” but Line 439+ documents active Veo support via
geminiVideo. Please align these rows with the current behavior to avoid contradictory guidance.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai/skills/ai-core/media-generation/SKILL.md` around lines 271 - 283, Update the stale Veo entries in the media-generation skill docs so they match the actual `geminiVideo` support. In the role mapping and the provider support matrix, remove the “planned/deferred/no adapter yet” wording for Veo and replace it with the current supported behavior, keeping the descriptions consistent with the existing `geminiVideo` section. Use the nearby symbols `'character'`, `'start_frame'`, `'end_frame'`, `generateImage`, and `generateVideo` to locate the affected rows.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@packages/ai/skills/ai-core/media-generation/SKILL.md`:
- Around line 271-283: Update the stale Veo entries in the media-generation
skill docs so they match the actual `geminiVideo` support. In the role mapping
and the provider support matrix, remove the “planned/deferred/no adapter yet”
wording for Veo and replace it with the current supported behavior, keeping the
descriptions consistent with the existing `geminiVideo` section. Use the nearby
symbols `'character'`, `'start_frame'`, `'end_frame'`, `generateImage`, and
`generateVideo` to locate the affected rows.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 403b8554-7d30-449a-b560-614b7de96d5b
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (27)
.changeset/openrouter-video-adapter.md.changeset/video-adapter-duration-constraint.mddocs/adapters/openrouter.mddocs/config.jsondocs/media/image-generation.mddocs/media/video-generation.mdexamples/ts-react-media/package.jsonexamples/ts-react-media/src/lib/models.tsexamples/ts-react-media/src/lib/server-functions.tspackages/ai-openrouter/package.jsonpackages/ai-openrouter/src/adapters/image.tspackages/ai-openrouter/src/adapters/video.tspackages/ai-openrouter/src/image/image-provider-options.tspackages/ai-openrouter/src/index.tspackages/ai-openrouter/src/model-meta.tspackages/ai-openrouter/src/video/video-provider-options.tspackages/ai-openrouter/tests/image-adapter.test.tspackages/ai-openrouter/tests/video-adapter.test.tspackages/ai-openrouter/tests/video-per-model-type-safety.test.tspackages/ai/skills/ai-core/media-generation/SKILL.mdpackages/ai/src/activities/generateVideo/index.tsscripts/convert-openrouter-models.tsscripts/fetch-openrouter-models.tsscripts/openrouter.video-models.jsonscripts/openrouter.video-models.tstesting/e2e/package.jsontesting/e2e/src/lib/feature-support.ts
✅ Files skipped from review due to trivial changes (8)
- examples/ts-react-media/src/lib/models.ts
- docs/media/image-generation.md
- packages/ai-openrouter/tests/video-per-model-type-safety.test.ts
- .changeset/openrouter-video-adapter.md
- testing/e2e/src/lib/feature-support.ts
- docs/config.json
- .changeset/video-adapter-duration-constraint.md
- testing/e2e/package.json
🚧 Files skipped from review as they are similar to previous changes (18)
- packages/ai-openrouter/package.json
- scripts/openrouter.video-models.ts
- packages/ai-openrouter/src/image/image-provider-options.ts
- examples/ts-react-media/package.json
- packages/ai-openrouter/tests/image-adapter.test.ts
- docs/adapters/openrouter.md
- packages/ai-openrouter/src/index.ts
- scripts/convert-openrouter-models.ts
- packages/ai-openrouter/src/adapters/image.ts
- packages/ai-openrouter/src/model-meta.ts
- docs/media/video-generation.md
- scripts/openrouter.video-models.json
- packages/ai-openrouter/src/adapters/video.ts
- examples/ts-react-media/src/lib/server-functions.ts
- scripts/fetch-openrouter-models.ts
- packages/ai/src/activities/generateVideo/index.ts
- packages/ai-openrouter/src/video/video-provider-options.ts
- packages/ai-openrouter/tests/video-adapter.test.ts
Contributor
Author
|
Video generated using Seedance 2.0 through openrouter in the example app. download.mp4 |
Contributor
Author
|
Hold off a minute. I just spotted something horible Claude did |
…ass comment
Live-verified against @openrouter/sdk 0.12.79: getVideoContent still rejects the
real 'video/mp4' response ('Unexpected Status or Content-Type') even though the
body is a valid MP4, so the manual authenticated download (and its injectable
fetch seam) stays. Only the comment's stale 0.12.35 reference was wrong.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tombeckenham
commented
Jun 24, 2026
| * content download (tests, custom runtimes). Defaults to the global | ||
| * fetch. | ||
| */ | ||
| fetch?: typeof globalThis.fetch |
Contributor
Author
There was a problem hiding this comment.
This is a horrible workaround that claude added. It's needed as the sdk isn't downloading video properly. don't merge like this
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 Changes
Implements #707 (follow-up to #618 / #624, both merged). Rebased onto
main.openRouterVideoadapterPOST /api/v1/videos→ pollGET /api/v1/videos/{jobId}→ download) — Seedance 2.0, Veo 3.1, Wan, Kling, Sora 2 Pro through one key, on the same jobs/polling architecture as the Sora adapter.imageInputsrole mapping:start_frame/end_frame→frame_images[](first_frame/last_frame),reference/character→input_references[],mask/control→ throw, unroled image → start frame. Frame roles validated against each model'ssupported_frame_images.videoInputs/audioInputsthrow (unsupported by the API).size/duration/resolution/aspectRatiotypes and runtime validation generated fromGET /api/v1/videos/models(OPENROUTER_VIDEO_MODEL_META);seed/generateAudiodropped from the type for models whose metadata reports them unsupported. Sync scripts now fetch/convert the second endpoint.durationis narrowed to each model's union, and the adapter implementsavailableDurations()/snapDuration(seconds)to enumerate the valid set and coerce raw UI seconds to the nearest supported value.data:URLs —unsigned_urls401 without the API key header (verified live), so they can't go straight into a<video>tag. Downloads >10 MiB log an OOM warning. Gateway-reported cost surfaced asusage.cost.Image activity follow-ups (#624 review)
sizenow throws with the supported list. Root cause:OpenRouterImageModelSizeByNameused the Unicode×(U+00D7) while the lookup used ASCIIx, so every typed size except1024x1024silently dropped its aspect ratio. Union fixed to ASCII;×still normalized at runtime.numberOfImages > 1now throws (live-verified: the chat-completions pathway ignores every count key and returns one image).image_configcasing confirmed live (snake_case applies, camelCase ignored).image_config.strength(0.0–1.0 i2i influence) exposed viamodelOptions.strength.Core fix:
generateVideotyped-duration constraint (@tanstack/ai)Adding the OpenRouter example surfaced a pre-existing #624 bug:
generateVideo()'sTAdapter extends VideoAdapter<string, any, any, any>bound let the duration generic default toRecord<string, number>. BecausecreateVideoJobis a contravariant function-valued property, no adapter with per-model literal durations satisfied the bound — even the documentedgenerateVideo({ adapter: geminiVideo('veo-3.1-…') })failed to type-check (CI missed it because the e2e passes a type-erased adapter). Widened the bound to leave size/duration unpinned at all 10 activity sites; per-model types are still recovered via inference. Fixes the Gemini path too.Example, tests, docs
examples/ts-react-media: demosopenRouterVideoalongside fal — Seedance 2.0 (text-to-video) and Veo 3.1 (image-to-video), showcasingadapter.snapDuration(); poll/status path now resolves the adapter by model.generateVideoconstraint regression test. Full submit→poll→download lifecycle verified live against OpenRouter.feature-support.ts— aimock 1.29 only mocks the OpenAI-shaped/v1/videos, not OpenRouter's job shape (same constraint as feat: multimodal prompt for generateImage/generateVideo (image-to-image, image-to-video) #624).media/video-generation.md,media/image-generation.md,adapters/openrouter.md(+config.jsondates), media-generationSKILL.md. Changesets:@tanstack/ai-openrouter(minor),@tanstack/ai(patch).Follow-up
falVideoonto the same typed-duration contract (it currently sits on base defaults; the implementation in draft feat(ai,ai-fal): per-model typed durations for video generation #641 is stuck behind feat(schemas): @tanstack/ai-schemas with nightly OpenAPI sync (closes #619) #622).✅ Checklist
pnpm run test:pr.🚀 Release Impact
🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Bug Fixes
Documentation & Tests