Skip to content

Feature: strip image content blocks for text-only upstream models (avoid 404 on vision-incapable providers) #3728

Description

@adminwdsj

Problem

When a client (e.g. Cursor in agent mode) sends a request whose messages contain image content blocks, and the target model's upstream has no vision capability, the upstream rejects the entire request — so an otherwise-valid text request fails only because an image block was attached.

Concrete example with a text-only Anthropic-compatible upstream:

POST /v1/messages   (or /v1/chat/completions)
→ HTTP 404: "No endpoints found that support image input"

Both the Claude (/v1/messages) and OpenAI (/v1/chat/completions) entry paths hit the same upstream rejection, so it is not a protocol-conversion issue — the model simply cannot accept images.

Current behavior

CLIProxyAPI forwards image blocks verbatim. There is no per-model / per-provider way to declare a model as text-only and have image input stripped before forwarding.

Proposed

A config flag to mark specific models (or a provider's models) as text-only, e.g.:

# under a provider / model entry
text-only: true   # strip image input before forwarding

When set, before sending upstream:

  • remove type: image blocks from message content arrays (including blocks nested inside tool_result content);
  • drop any message left empty after stripping (reuse the existing empty-message sanitation).

Notes

I currently solve this with a local source patch that strips image blocks for a hardcoded model-name prefix. Happy to open a PR generalizing it into the config-driven text-only flag above if the maintainers are open to it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions