feat: add AI image generation tool#11826
Conversation
Port the legacy generate_image tool to the opencode-based CLI as a Kilo-owned tool gated by experimental.image_generation config flag. - New generate_image tool with prompt/path/image/model params - Routes through Kilo Gateway (zero-config) or BYO OpenRouter key - Supports text-to-image generation and image editing - Dynamic model discovery via GET /kilo/models/images endpoint - VS Code settings toggle + live model dropdown in Experimental tab - Writes image to disk and returns inline FilePart attachment - Fallback model catalog for offline resilience
- Remove unused fetchKiloImageModels import in tool - Normalize jpg→jpeg MIME in parser, input image, and attachment - Replace mismatched extensions in ensureExtension (not just append) - Add assertExternalDirectoryEffect for output path traversal guard - Map unauthorized errors to 401 (not 400) in image models handler - Keep last known model list on fetch failure (don't overwrite with empty) - Fix tool description (remove false web search claim, fix grammar) - Remove duplicated provider resolver tests
…ation # Conflicts: # packages/kilo-gateway/src/index.ts # packages/kilo-vscode/webview-ui/src/App.tsx # packages/kilo-vscode/webview-ui/src/components/settings/ExperimentalTab.tsx # packages/kilo-vscode/webview-ui/src/i18n/it.ts # packages/kilo-vscode/webview-ui/src/types/messages/config.ts # packages/opencode/src/config/config.ts # packages/opencode/src/kilocode/tool/registry.ts # packages/opencode/test/kilocode/tool-registry-indexing-import-failure.test.ts # packages/opencode/test/kilocode/tool-registry-indexing.test.ts # packages/opencode/test/kilocode/tool-registry-semantic-import-failure.test.ts # packages/opencode/test/tool/registry.test.ts # packages/sdk/js/src/v2/gen/types.gen.ts # packages/sdk/openapi.json
There was a problem hiding this comment.
Pull request overview
This PR adds an experimental generate_image tool to the agent runtime and wires it through the Kilo Gateway and local kilo serve HTTP API so clients (notably the VS Code extension) can both use image generation and discover image-capable models dynamically.
Changes:
- Added a Kilo-owned
generate_imagetool (prompt + optional input image + output path) gated behindexperimental.image_generationand configurable viaexperimental.image_generation_model. - Added
/kilo/models/imagesendpoint + gateway plumbing for image-capable model discovery, and regenerated the JS SDK/OpenAPI outputs. - Updated the VS Code extension/webview to expose an experimental toggle + model dropdown backed by a new Image Models provider/context.
Reviewed changes
Copilot reviewed 54 out of 54 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/sdk/openapi.json | Adds /kilo/models/images path + experimental config keys in the OpenAPI spec. |
| packages/sdk/js/src/v2/gen/types.gen.ts | Regenerates SDK types for the new endpoint and config flags. |
| packages/sdk/js/src/v2/gen/sdk.gen.ts | Regenerates the SDK client to expose client.kilo.models.images(). |
| packages/opencode/test/tool/registry.test.ts | Updates test layers to provide Auth.defaultLayer needed by the new tool dependency graph. |
| packages/opencode/test/session/snapshot-tool-race.test.ts | Provides Auth.defaultLayer in session test wiring. |
| packages/opencode/test/session/prompt.test.ts | Provides Auth.defaultLayer in session prompt tests. |
| packages/opencode/test/kilocode/tool/generate-image.test.ts | Adds unit tests for response parsing, provider resolution, and path extension behavior. |
| packages/opencode/test/kilocode/tool-registry-semantic-import-failure.test.ts | Extends tool registry test expectations to include generate_image. |
| packages/opencode/test/kilocode/tool-registry-indexing.test.ts | Extends tool registry indexing + gating tests for image_generation. |
| packages/opencode/test/kilocode/tool-registry-indexing-import-failure.test.ts | Extends tool registry test expectations to include generate_image. |
| packages/opencode/test/kilocode/session-prompt-permission-refresh.test.ts | Provides Auth.defaultLayer in session tests. |
| packages/opencode/test/kilocode/session-prompt-compaction-safety.test.ts | Provides Auth.defaultLayer in session tests. |
| packages/opencode/test/kilocode/session-compaction-cap.test.ts | Provides Auth.defaultLayer in session tests. |
| packages/opencode/src/tool/registry.ts | Adds Auth.Service to the tool registry layer to support generate_image. |
| packages/opencode/src/kilocode/tool/registry.ts | Registers GenerateImageTool and gates it behind experimental.image_generation. |
| packages/opencode/src/kilocode/tool/generate-image.txt | Adds end-user tool description text for generate_image. |
| packages/opencode/src/kilocode/tool/generate-image.ts | Implements generate_image tool (provider resolution, request build, response parse, file write + attachment). |
| packages/opencode/src/kilocode/server/httpapi/handlers/kilo-gateway.ts | Adds handler for /kilo/models/images via fetchKiloImageModels. |
| packages/opencode/src/kilocode/server/httpapi/groups/kilo-gateway.ts | Adds the /kilo/models/images endpoint schema + OpenAPI annotations. |
| packages/opencode/src/config/config.ts | Adds experimental config keys image_generation and image_generation_model. |
| packages/kilo-vscode/webview-ui/src/types/messages/webview-messages.ts | Adds webview→extension message type for requesting image models. |
| packages/kilo-vscode/webview-ui/src/types/messages/extension-messages.ts | Adds extension→webview message type for delivering image models. |
| packages/kilo-vscode/webview-ui/src/types/messages/config.ts | Extends experimental config typing with image generation flags. |
| packages/kilo-vscode/webview-ui/src/i18n/zht.ts | Adds new experimental settings strings for image generation (locale: zht). |
| packages/kilo-vscode/webview-ui/src/i18n/zh.ts | Adds new experimental settings strings for image generation (locale: zh). |
| packages/kilo-vscode/webview-ui/src/i18n/uk.ts | Adds new experimental settings strings for image generation (locale: uk). |
| packages/kilo-vscode/webview-ui/src/i18n/tr.ts | Adds new experimental settings strings for image generation (locale: tr). |
| packages/kilo-vscode/webview-ui/src/i18n/th.ts | Adds new experimental settings strings for image generation (locale: th). |
| packages/kilo-vscode/webview-ui/src/i18n/ru.ts | Adds new experimental settings strings for image generation (locale: ru). |
| packages/kilo-vscode/webview-ui/src/i18n/pl.ts | Adds new experimental settings strings for image generation (locale: pl). |
| packages/kilo-vscode/webview-ui/src/i18n/no.ts | Adds new experimental settings strings for image generation (locale: no). |
| packages/kilo-vscode/webview-ui/src/i18n/nl.ts | Adds new experimental settings strings for image generation (locale: nl). |
| packages/kilo-vscode/webview-ui/src/i18n/ko.ts | Adds new experimental settings strings for image generation (locale: ko). |
| packages/kilo-vscode/webview-ui/src/i18n/ja.ts | Adds new experimental settings strings for image generation (locale: ja). |
| packages/kilo-vscode/webview-ui/src/i18n/it.ts | Adds new experimental settings strings for image generation (locale: it). |
| packages/kilo-vscode/webview-ui/src/i18n/fr.ts | Adds new experimental settings strings for image generation (locale: fr). |
| packages/kilo-vscode/webview-ui/src/i18n/es.ts | Adds new experimental settings strings for image generation (locale: es). |
| packages/kilo-vscode/webview-ui/src/i18n/en.ts | Adds new experimental settings strings for image generation (locale: en). |
| packages/kilo-vscode/webview-ui/src/i18n/de.ts | Adds new experimental settings strings for image generation (locale: de). |
| packages/kilo-vscode/webview-ui/src/i18n/da.ts | Adds new experimental settings strings for image generation (locale: da). |
| packages/kilo-vscode/webview-ui/src/i18n/bs.ts | Adds new experimental settings strings for image generation (locale: bs). |
| packages/kilo-vscode/webview-ui/src/i18n/br.ts | Adds new experimental settings strings for image generation (locale: br). |
| packages/kilo-vscode/webview-ui/src/i18n/ar.ts | Adds new experimental settings strings for image generation (locale: ar). |
| packages/kilo-vscode/webview-ui/src/context/image-models.tsx | Adds webview context/provider that requests + stores the image model list (with retry). |
| packages/kilo-vscode/webview-ui/src/components/settings/ExperimentalTab.tsx | Adds toggle + dropdown UI for experimental image generation and model selection. |
| packages/kilo-vscode/webview-ui/src/App.tsx | Wires ImageModelsProvider into the webview provider tree. |
| packages/kilo-vscode/src/KiloProvider.ts | Handles requestImageModels and posts imageModelsLoaded (with caching). |
| packages/kilo-vscode/src/image-generation/models.ts | Adds extension-side fetch helper for /kilo/models/images. |
| packages/kilo-gateway/src/server/routes.ts | Adds gateway route for /models/images (and related image proxy routing additions). |
| packages/kilo-gateway/src/index.ts | Exports new fetchKiloImageModels API and associated types. |
| packages/kilo-gateway/src/api/models.ts | Implements fetchKiloImageModels() and refactors shared raw model fetch. |
| packages/kilo-docs/source-links.md | Updates extracted source links to include new URLs/endpoints referenced by new code. |
| .kilo/plans/1782130204245-image-generation-tool.md | Adds/updates implementation plan doc for the feature. |
| .changeset/image-generation.md | Adds a minor changeset for the VS Code extension package. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| permission: "write", | ||
| patterns: [path.relative(instance.worktree, absPath)], | ||
| always: ["*"], | ||
| metadata: { filepath: absPath }, | ||
| }) |
| "x-codeSamples": [ | ||
| { | ||
| "lang": "js", | ||
| "source": "import { createKiloClient } from \"@kilocode/sdk\n\nconst client = createKiloClient()\nawait client.kilo.models.images({\n ...\n})" |
| export const ImageModelsProvider: ParentComponent = (props) => { | ||
| const vscode = useVSCode() | ||
| const [models, setModels] = createSignal<ImageModel[]>([]) | ||
|
|
||
| const request = () => vscode.postMessage({ type: "requestImageModels" }) | ||
|
|
||
| const unsubscribe = vscode.onMessage((message: ExtensionMessage) => { | ||
| if (message.type !== "imageModelsLoaded") return | ||
| setModels(message.models) | ||
| }) | ||
|
|
||
| request() | ||
|
|
||
| // Retry once after a delay in case the backend wasn't ready for the initial request. | ||
| const retry = setTimeout(request, 3000) | ||
| onCleanup(() => clearTimeout(retry)) | ||
|
|
||
| onCleanup(unsubscribe) | ||
|
|
||
| return <ImageModelsContext.Provider value={{ models }}>{props.children}</ImageModelsContext.Provider> | ||
| } |
| "settings.experimental.imageGeneration.title": "Image Generation", | ||
| "settings.experimental.imageGeneration.description": "Enable AI image generation", | ||
| "settings.experimental.imageGenerationModel.title": "Image Model", | ||
| "settings.experimental.imageGenerationModel.description": "Image Generation Model", | ||
| "settings.experimental.imageGenerationModel.placeholder": "Default (Auto Router)", |
| test("replaces mismatched extension when format differs", () => { | ||
| expect(ensureExtension("output/photo.jpg", "png")).toBe("output/photo.jpg.png") | ||
| }) |
![kilo-code-bot[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
| const jpg = ["jpg", "jpeg"] | ||
| if (ext === "jpg" && jpg.includes(existing)) return relPath | ||
| if (ext === "png" && png.includes(existing)) return relPath | ||
| return `${relPath}.${ext}` |
There was a problem hiding this comment.
WARNING: Mismatched extensions are still appended instead of replaced
This branch preserves matching png/jpg suffixes, but any other image suffix still falls through to return ${relPath}.${ext}``. Saving PNG output to photo.jpg therefore produces `photo.jpg.png`, which leaves a misleading filename on disk and contradicts the intended behavior of this follow-up fix.
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
There was a problem hiding this comment.
To use Kilo from GitHub you first need to link your GitHub account to Kilo. Link your Kilo account to continue. After linking, mention me again in this issue or pull request.
There was a problem hiding this comment.
Fixed in add1ddc. ensureExtension now replaces mismatched image extensions instead of appending: photo.jpg + PNG → photo.png. Non-image extensions (e.g. .final) are still preserved with appending. Updated tests cover jpg→png, jpeg→png, and png→jpg.
| }) | ||
|
|
||
| if (result.error) { | ||
| const status = result.error.kind === "unauthorized" ? 401 : (result.error.status ?? 500) |
There was a problem hiding this comment.
WARNING: The documented response codes no longer match the handler
fetchKiloImageModels() can surface upstream 429/5xx statuses here, but describeRoute() above only advertises 200 and 401. Any client generated from this route will treat those non-auth failures as impossible, so this should either normalize them to a documented status or add the extra responses to the schema.
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
There was a problem hiding this comment.
To use Kilo from GitHub you first need to link your GitHub account to Kilo. Link your Kilo account to continue. After linking, mention me again in this issue or pull request.
There was a problem hiding this comment.
Fixed in add1ddc. Now uses ...errors(400, 401) — the same pattern as every other gateway route (/profile, /organization, /fim, etc.). Non-auth errors map to 400 instead of leaking arbitrary upstream statuses (429, 503, …).
| } | ||
| } | ||
| }, | ||
| "400": { |
There was a problem hiding this comment.
WARNING: The regenerated schema dropped the 401 branch for this endpoint
The new imageModels handler in packages/opencode/src/kilocode/server/httpapi/handlers/kilo-gateway.ts returns HttpApiError.Unauthorized, but this spec only exposes a 400 error response. That leaves the generated SDK unable to represent auth failures for /kilo/models/images, so callers cannot distinguish re-authentication from generic bad requests.
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
There was a problem hiding this comment.
To use Kilo from GitHub you first need to link your GitHub account to Kilo. Link your Kilo account to continue. After linking, mention me again in this issue or pull request.
There was a problem hiding this comment.
Fixed in add1ddc. Added the 401 response to both openapi.json and types.gen.ts, matching the gateway route's errors(400, 401) declaration. The Effect OpenAPI generator suppresses 401 for all /kilo/* endpoints because the group's Authorization middleware declares HttpApiError.UnauthorizedNoContent (a no-content error the generator skips). Manual addition is consistent with what the gateway declares.
Code Review SummaryStatus: No Issues Found | Recommendation: Merge Incremental review of commits since the previous pass (
Every other file that appears to differ between the two commit ranges (i18n files, Re-verified against current
Files Reviewed (2 files)
Previous Review Summaries (7 snapshots, latest commit 4b241f4)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit 4b241f4)Status: No Issues Found | Recommendation: Merge Incremental review of commits since the previous pass (666560c..4b241f4). The only PR-owned commit in this range (
The accompanying Files Reviewed (20 files)
Previous review (commit 666560c)Status: No Issues Found | Recommendation: Merge Incremental review of commits since the previous pass (9a37270..666560c). The only PR-owned commit in this range is a dependency bump ( Files Reviewed (4 files)
Previous review (commit 9a37270)Status: No Issues Found | Recommendation: Merge Incremental review of commits since the previous pass (d322b6b..9a37270). Only two files changed in this range; both are clean. Files Reviewed (2 files)
Previous review (commit d322b6b)Status: 1 Issue Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
Fix these issues in Kilo Cloud Files Reviewed (5 files)
Previous review (commit a29e3a2)Status: 1 Issue Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
Fix these issues in Kilo Cloud Files Reviewed (5 files)
Previous review (commit aa81d0a)Status: 1 Issue Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
Fix these issues in Kilo Cloud Files Reviewed (5 files)
Previous review (commit 0cf21de)Status: 3 Issues Found | Recommendation: Address before merge Overview
Issue Details (click to expand)WARNING
Fix these issues in Kilo Cloud Files Reviewed (54 files)
Reviewed by claude-sonnet-5 · Input: 114 · Output: 37.7K · Cached: 4.6M Review guidance: REVIEW.md from base branch |
- ensureExtension replaces mismatched image extensions instead of appending (photo.jpg + PNG -> photo.png, not photo.jpg.png) - Gateway /models/images normalizes errors to 400/401 matching every other gateway route (was leaking undeclared upstream statuses) - Add 401 response to openapi.json + SDK types for /kilo/models/images to match the gateway's errors(400, 401) declaration
|
|
||
| if (result.error) { | ||
| if (result.error.kind === "unauthorized") return c.json({ error: "unauthorized" }, 401) | ||
| return c.json({ error: result.error.kind }, 400) |
There was a problem hiding this comment.
WARNING: The new 400 branch still does not match the documented schema
errors(400, 401) now advertises a BadRequest | InvalidRequestError payload for this endpoint, but this line still returns { error: result.error.kind } for network / schema / http failures. The regenerated SDK therefore claims clients can decode structured 400 errors here even though the gateway sends a different shape, so this should either emit the documented error body or declare the actual response schema.
Reply with @kilocode-bot fix it to have Kilo Code address this issue.
vkeerthivikram
force-pushed
the
feat/ai-image-generation
branch
2 times, most recently
from
80c5c0e to
add1ddc
Compare
…ation # Conflicts: # packages/opencode/src/kilocode/tool/registry.ts # packages/opencode/test/kilocode/tool-registry-indexing.test.ts # packages/opencode/test/tool/registry.test.ts
|
This looks great. One note: @openrouter/ai-sdk-provider 2.10.0 switched imageModel() to OpenRouter's dedicated POST /api/v1/images endpoint (proper image usage/billing, image-specific params) — worth bumping from 2.9.0 to pick that up. |
Switches imageModel() to OpenRouter's POST /api/v1/images endpoint for proper image usage/billing and image-specific params.
Done |
|
tui-image-gen.webm |
| <!-- packages/opencode/src/kilocode/server/httpapi/public.ts --> | ||
| - <https://opencode.ai/zen> | ||
| <!-- packages/kilo-vscode/webview-ui/src/i18n/en.ts --> | ||
| - <https://openrouter.ai/api/v1/chat/completions> |
There was a problem hiding this comment.
I think we probably don't want to add support for this first in kilo gateway in the backend. Then this PR could use openrouter's image generation models accordingly. This would also allow moving some of the model specific logic into our backend. I can take a look into this.
There was a problem hiding this comment.
Thanks for offering to look into this! Happy to leave image generation to land in the kilo-gateway backend first and consume it from there. Let me know if you need anything from my end on the client side to support it.
- revert @openrouter/ai-sdk-provider 2.9.0->2.10.0 bump (image tool uses raw HTTP, not the SDK) - translate image generation settings strings across all locales - use central KILO_OPENROUTER_BASE instead of hardcoded URL fallback - remove completed plan file
|
Pairs nicely with #12010 (inline image rendering in the chat view) — once a generated image is read back into context, it will render inline. |
Resolve conflicts in App.tsx and AgentManagerApp.tsx provider trees: keep both ImageModelsProvider (image generation) and MemoryProvider (project memory) wrappers.
Tested one more time with kilo gateway. This works already |
|
@vkeerthivikram Merged the feature! Thanks a lot for the contribution, hope it brings value! |
Summary
Adds a new
generate_imagetool that lets the agent generate or edit images from text prompts, gated behindexperimental.image_generation. Routes through the Kilo Gateway (zero-config) or a BYO OpenRouter API key, with a dynamic model picker in the VS Code Experimental settings tab.Restores a feature the legacy Kilo extension had (
kilocode-legacy/GenerateImageTool.ts); this port improves on it by fetching the model list dynamically fromapi.kilo.aiinstead of using a hardcoded array.What I changed
New files:
packages/opencode/src/kilocode/tool/generate-image.ts— the tool (Tool.define, image generation + editing, gateway-aware provider resolution)packages/opencode/src/kilocode/tool/generate-image.txt— tool descriptionpackages/opencode/test/kilocode/tool/generate-image.test.ts— 23 tests (parser, provider resolver, path extension, MIME normalization, model catalog)packages/kilo-vscode/src/image-generation/models.ts— fetch helper for the settings dropdownpackages/kilo-vscode/webview-ui/src/context/image-models.tsx— SolidJS context provider.changeset/image-generation.md— minor changesetModified files:
packages/opencode/src/kilocode/tool/registry.ts— registered the new tool (gated by flag)packages/opencode/src/kilocode/server/httpapi/groups/kilo-gateway.ts— addedGET /kilo/models/imagesendpointpackages/opencode/src/kilocode/server/httpapi/handlers/kilo-gateway.ts— added the handler callingfetchKiloImageModelspackages/kilo-gateway/src/api/models.ts— addedfetchKiloImageModels()(queriesapi.kilo.ai, filtersoutput_modalities.includes("image"))packages/kilo-gateway/src/index.ts— exported the new functionpackages/kilo-vscode/src/KiloProvider.ts— handler forrequestImageModelsmessage, cached fallback on fetch failurepackages/kilo-vscode/webview-ui/src/App.tsx— registeredImageModelsProviderpackages/kilo-vscode/webview-ui/src/components/settings/ExperimentalTab.tsx— Image Generation toggle + model dropdownpackages/kilo-vscode/webview-ui/src/types/messages/*.ts—RequestImageModelsMessage+ImageModelsLoadedMessagepackages/kilo-vscode/webview-ui/src/i18n/*.ts— 20 localespackages/sdk/js/src/v2/gen/sdk.gen.ts,types.gen.ts,openapi.json— regenerated SDKShared files (with
kilocode_changemarkers):packages/opencode/src/config/config.ts— addedexperimental.image_generation+experimental.image_generation_modelconfig keyspackages/opencode/src/tool/registry.ts— addedAuth.Servicedependency (required by the tool)Test updates: 9 test files updated to wire
Auth.defaultLayerinto the tool registry layer + add the newimagetool field.Testing
cd packages/opencode && bun test ./test/kilocode/tool/generate-image.test.ts— 23 passcd packages/opencode && bun run typecheck— passcd packages/kilo-vscode && bun run typecheck— passcd packages/kilo-gateway && bun run typecheck— passbun run script/check-opencode-annotations.ts --base origin/main— pass📎 References · Issue #11565 · Legacy tool