fix : image input gating by model capability#5272
Conversation
|
Thanks, this looks directionally good, but the branch needs to be rebased before review can continue. I checked the PR against the latest After resolving that conflict locally, the focused checks I ran against the rebased state passed:
Once the branch is rebased and CI stays green, I will continue the final review. |
SivanCola
left a comment
There was a problem hiding this comment.
Fixed the merge conflict by merging latest main-v2 into this branch and resolving desktop/frontend/package.json so the current upstream frontend test list is preserved while keeping composer-image-capability.test.tsx in both frontend test scripts.
Review notes:
- The image input gating is applied at the UI/controller boundary for text-only models, while drafts remain recoverable.
- The control path still drops image payloads for text-only models, so a frontend miss does not send image input.
- This is dynamic image capability behavior only; I did not find a stable prompt-prefix or tool-schema cache risk.
Verified:
go test ./internal/control ./internal/cli -run ... -count=1go test . -run ... -count=1fromdesktop/pnpm --dir desktop/frontend exec tsx src/__tests__/composer-image-capability.test.tsxpnpm --dir desktop/frontend exec tsx src/__tests__/use-controller-meta.test.ts
SivanCola
left a comment
There was a problem hiding this comment.
Follow-up pushed in 7da9b0a.
This keeps the original provider-native image guard while avoiding a product regression: text-only models still do not receive direct image bytes, but pasted/dropped images remain usable as local refs so OCR/image/vision MCP tools can inspect them.
What changed:
- The desktop composer no longer rejects image attach/save/send solely because the selected model lacks native image input. It warns non-blockingly when sending image attachments with a text-only model.
- The CLI no longer blocks pasted image refs for text-only models.
ResolveRefsnow describes both paths clearly: direct model image input for vision-capable models, and tool-readable local paths for text-only models.- Regression tests now cover the text-only + tool-readable image path while keeping the controller guard that suppresses native image payloads for text-only models.
Verified:
go test ./internal/control ./internal/cli -run ... -count=1passedpnpm --dir desktop/frontend exec tsx src/__tests__/composer-image-capability.test.tsxpassedpnpm --dir desktop/frontend exec tsx src/__tests__/use-controller-meta.test.tspassedgit diff --checkpassed
SivanCola
left a comment
There was a problem hiding this comment.
Re-approved after merging the latest main-v2 into this branch in 6b788b4.
The new upstream overlap was in the CLI paste tests from #5347. The conflict resolution keeps both contracts:
- folded paste content is expanded for the turn and memory-compiler source event
- text-only models can still send pasted image refs for OCR/image/vision tool use while native image payloads stay gated by model capability
Verified after the merge:
go test ./internal/cli -run ... -count=1passedgo test ./internal/control -run ... -count=1passedgo test . -run ... -count=1fromdesktop/passedpnpm --dir desktop/frontend exec tsx src/__tests__/composer-image-capability.test.tsxpassedpnpm --dir desktop/frontend exec tsx src/__tests__/use-controller-meta.test.tspassedgit diff --checkpassed
Fixes #5175
Summary
imageInputEnabledthrough desktop tab meta and refresh it after model switches so the composer can warn when the selected model will not receive images directlyinputImages(...)on the resolved provider capability, while preserving normal@reftext handlingResolveRefsguidance, and CLI pasted-image behaviorBehavior
For a text-only model:
image_url/ native image content blocks to the provider.@.reasonix/attachments/...refs.For a vision-capable model:
Verification
Automated checks run during review:
go test ./internal/control ./internal/cli -run ... -count=1go test . -run ... -count=1fromdesktop/pnpm --dir desktop/frontend exec tsx src/__tests__/composer-image-capability.test.tsxpnpm --dir desktop/frontend exec tsx src/__tests__/use-controller-meta.test.tsgit diff --checkManual verification guidance:
vision = trueorvision_models = ["..."]on the vision model.cd desktop && wails dev.Local style snapshots
cli

ui

Cache impact
Cache-impact: none - image-input gating and toast rendering do not change provider-visible stable prompts, tool schemas, cache keys, or transcript compaction paths.
Cache-guard: focused desktop/CLI image-capability regression coverage plus manual end-to-end validation.
System-prompt-review: N/A