Render and replay MCP tool image content end-to-end

MCP tools that return `image` content blocks (e.g. an MCP
image-generation/edit server) previously surfaced as the placeholder
text `[Image: <media-type>]` and were never replayed back to the LLM,
so iterative edits ("now make it blue") were impossible.

Client UI:
- `chat/tool_calls.clj :send-toolCalled` now partitions tool outputs
  into text vs image, keeps text in the toolCalled :outputs (which
  the protocol declares text-only) and emits one `ChatImageContent`
  per image so clients render inline.
- `chat.clj` `tool_call_output` history replay mirrors the same
  partitioning so reopened chats render images at the original spot.

Provider replay:
- `openai.clj` (Responses API) `normalize-messages` switches
  `keep` -> `mapcat` and, when the model supports image input,
  appends a synthetic user-role `input_image` after the
  `function_call_output` (the Responses-API tool output is
  text-only, mirroring the existing `image_generation_call` replay).
- `anthropic.clj` carries images natively in `tool_result.content`
  as mixed `[{text}, {image base64}]` blocks when the model is
  multimodal, falling back to the legacy text-only string otherwise.
- `openai_chat.clj` and `ollama.clj` get gating comments only;
  image round-trip is not implemented for those APIs yet.

Docs:
- `docs/protocol.md`: `ChatImageContent` doc broadened to cover
  MCP image tools alongside the existing `image_generation` tool.

Tests: 481 / 2639 / 0.

🤖 Generated with [eca](https://eca.dev)

Co-Authored-By: eca-agent <git@eca.dev>

Author: ericdallo

CHANGELOG.md

@@ -2,6 +2,7 @@
 
 ## Unreleased
 
+- MCP tools that return image content blocks (e.g. an MCP image-generation/edit server) now render those images in the chat UI as `ChatImageContent` and replay them back to the LLM as image inputs on follow-up turns when the model supports vision. Implemented for `openai-responses` (synthetic user-role `input_image` after the `function_call_output`) and `anthropic` (mixed text + image blocks inside `tool_result.content`). `openai-chat` and `ollama` continue to receive a text placeholder until a parallel pattern is implemented there.
 - Bugfix: MCP tools without a `description` (which the MCP spec marks optional) no longer break Anthropic chat requests with `tools.<n>.custom.description: Input should be a valid string`. Missing/empty descriptions now fall back to the tool's `title`, then to a synthesized `MCP tool: <name>` string at the MCP boundary so all providers receive a non-null string.
 
 ## 0.131.0

docs/protocol.md

@@ -793,8 +793,12 @@ interface ChatURLContent {
 }
 
 /**
- * Image content from the assistant, produced by a server-side image
- * generation tool (e.g. OpenAI's `image_generation` Responses-API tool).
+ * Image content from the assistant. Produced either by a server-side
+ * image generation tool (e.g. OpenAI's `image_generation` Responses-API
+ * tool) or by an MCP tool whose result includes an `image` content block
+ * (e.g. an MCP image-generation/edit server). In both cases, ECA emits
+ * one `ChatImageContent` per image so clients can render without
+ * inspecting tool-call outputs.
  *
  * The image bytes are delivered inline as base64 so that web/remote ECA
  * clients (e.g. web.eca.dev) can render without filesystem access.

src/eca/features/chat.clj

@@ -151,38 +151,57 @@
                             :details (:details message-content)
                             :arguments-text ""
                             :id (:id message-content)}}]
-    "tool_call_output" [{:role :assistant
-                         :content (assoc-some
-                                   {:type :toolCallRun
-                                    :id (:id message-content)
-                                    :name (:name message-content)
-                                    :server (:server message-content)
-                                    :origin (:origin message-content)
-                                    :arguments (:arguments message-content)}
-                                   :details (:details message-content)
-                                   :summary (:summary message-content))}
-                        {:role :assistant
-                         :content (assoc-some
-                                   {:type :toolCallRunning
-                                    :id (:id message-content)
-                                    :name (:name message-content)
-                                    :server (:server message-content)
-                                    :origin (:origin message-content)
-                                    :arguments (:arguments message-content)}
-                                   :details (:details message-content)
-                                   :summary (:summary message-content))}
-                        {:role :assistant
-                         :content {:type :toolCalled
-                                   :origin (:origin message-content)
-                                   :name (:name message-content)
-                                   :server (:server message-content)
-                                   :arguments (:arguments message-content)
-                                   :total-time-ms (:total-time-ms message-content)
-                                   :summary (:summary message-content)
-                                   :details (:details message-content)
-                                   :error (:error message-content)
-                                   :id (:id message-content)
-                                   :outputs (:contents (:output message-content))}}]
+    ;; Mirror the live path in tool-calls.clj :send-toolCalled: split image
+    ;; outputs out of the toolCalled :outputs (which is text-only per
+    ;; protocol) and re-emit them as standalone ChatImageContent entries so
+    ;; reopened/resumed chats render MCP-produced images at the same point
+    ;; they appeared live.
+    "tool_call_output" (let [contents (:contents (:output message-content))
+                             image? #(and (map? %) (= :image (:type %)))
+                             ;; Only partition when contents is a sequence of content
+                             ;; maps that includes images; otherwise pass through.
+                             image-outputs (when (sequential? contents) (filter image? contents))
+                             text-outputs (if (seq image-outputs)
+                                            (vec (remove image? contents))
+                                            contents)]
+                         (into [{:role :assistant
+                                 :content (assoc-some
+                                           {:type :toolCallRun
+                                            :id (:id message-content)
+                                            :name (:name message-content)
+                                            :server (:server message-content)
+                                            :origin (:origin message-content)
+                                            :arguments (:arguments message-content)}
+                                           :details (:details message-content)
+                                           :summary (:summary message-content))}
+                                {:role :assistant
+                                 :content (assoc-some
+                                           {:type :toolCallRunning
+                                            :id (:id message-content)
+                                            :name (:name message-content)
+                                            :server (:server message-content)
+                                            :origin (:origin message-content)
+                                            :arguments (:arguments message-content)}
+                                           :details (:details message-content)
+                                           :summary (:summary message-content))}
+                                {:role :assistant
+                                 :content {:type :toolCalled
+                                           :origin (:origin message-content)
+                                           :name (:name message-content)
+                                           :server (:server message-content)
+                                           :arguments (:arguments message-content)
+                                           :total-time-ms (:total-time-ms message-content)
+                                           :summary (:summary message-content)
+                                           :details (:details message-content)
+                                           :error (:error message-content)
+                                           :id (:id message-content)
+                                           :outputs text-outputs}}]
+                               (map (fn [img]
+                                      {:role :assistant
+                                       :content {:type :image
+                                                 :media-type (:media-type img)
+                                                 :base64 (:base64 img)}}))
+                               image-outputs))
     "image_generation_call" [{:role :assistant
                               :content {:type :image
                                         :media-type (:media-type message-content)

src/eca/features/chat/tool_calls.clj

@@ -307,19 +307,41 @@
                               :summary (:summary event-data)))
 
     :send-toolCalled
-    (lifecycle/send-content! chat-ctx :assistant
-                             (assoc-some
-                              {:type :toolCalled
-                               :id tool-call-id
-                               :origin (:origin event-data)
-                               :name (:name event-data)
-                               :server (:server event-data)
-                               :arguments (:arguments event-data)
-                               :error (:error event-data)
-                               :total-time-ms (:total-time-ms event-data)
-                               :outputs (:outputs event-data)}
-                              :details (:details event-data)
-                              :summary (:summary event-data)))
+    ;; Tool results may include image content blocks (e.g. from MCP image
+    ;; tools). The protocol declares toolCalled :outputs as text-only, so we
+    ;; partition: text outputs go into the toolCalled event, images are
+    ;; emitted as separate ChatImageContent events tagged :assistant. This
+    ;; reuses the same wire shape produced by the OpenAI Responses-API
+    ;; built-in image_generation tool (see chat.clj on-message-received
+    ;; :image branch).
+    ;; Only partition when outputs is a sequence containing image content
+    ;; maps; otherwise pass it through unchanged. This preserves
+    ;; pre-existing behaviors where :outputs can be nil or a plain string
+    ;; (used by some non-MCP code paths and tests).
+    (let [outputs (:outputs event-data)
+          image? #(and (map? %) (= :image (:type %)))
+          image-outputs (when (sequential? outputs) (filter image? outputs))
+          text-outputs (if (seq image-outputs)
+                         (vec (remove image? outputs))
+                         outputs)]
+      (lifecycle/send-content! chat-ctx :assistant
+                               (assoc-some
+                                {:type :toolCalled
+                                 :id tool-call-id
+                                 :origin (:origin event-data)
+                                 :name (:name event-data)
+                                 :server (:server event-data)
+                                 :arguments (:arguments event-data)
+                                 :error (:error event-data)
+                                 :total-time-ms (:total-time-ms event-data)
+                                 :outputs text-outputs}
+                                :details (:details event-data)
+                                :summary (:summary event-data)))
+      (doseq [img image-outputs]
+        (lifecycle/send-content! chat-ctx :assistant
+                                 {:type :image
+                                  :media-type (:media-type img)
+                                  :base64 (:base64 img)})))
 
     :send-toolCallRejected
     (let [tool-call-state (get-tool-call-state @db* (:chat-id chat-ctx) tool-call-id)

src/eca/llm_providers/anthropic.clj

@@ -183,10 +183,29 @@
                                     :input (or (:arguments content) {})}]}
 
             "tool_call_output"
-            {:role "user"
-             :content [{:type "tool_result"
-                        :tool_use_id (:id content)
-                        :content (llm-util/stringfy-tool-result content)}]}
+            ;; Anthropic's tool_result `content` field accepts a list of
+            ;; mixed text + image blocks natively, so when the tool returned
+            ;; image content and the model supports image input, emit a
+            ;; single tool_result whose content is the textual portion plus
+            ;; one image block per image. Falls back to the legacy text-only
+            ;; string shape when there are no images or the model is not
+            ;; multimodal (preserves prior behavior).
+            (let [contents (-> content :output :contents)
+                  image-contents (when supports-image?
+                                   (seq (filter #(= :image (:type %)) contents)))
+                  text (llm-util/stringfy-tool-result content)]
+              {:role "user"
+               :content [{:type "tool_result"
+                          :tool_use_id (:id content)
+                          :content (if image-contents
+                                     (into [{:type "text" :text text}]
+                                           (map (fn [img]
+                                                  {:type "image"
+                                                   :source {:type "base64"
+                                                            :media_type (or (:media-type img) "image/png")
+                                                            :data (:base64 img)}}))
+                                           image-contents)
+                                     text)}]})
 
             ;; OpenAI-emitted image_generation_call history entries are
             ;; replayed for Anthropic as user-role image blocks (Anthropic

src/eca/llm_providers/ollama.clj

@@ -107,6 +107,10 @@
                                                          :function (-> content
                                                                        (assoc :name (:full-name content))
                                                                        (dissoc :full-name))}]}
+            ;; NOTE: Image content from MCP tool results is flattened to
+            ;; placeholder text via `stringfy-tool-result`. Image round-trip
+            ;; is implemented for openai-responses and anthropic; see those
+            ;; providers' `tool_call_output` branches for the pattern.
             "tool_call_output" {:role "tool" :content (llm-util/stringfy-tool-result content)}
             "reason" {:role "assistant" :content (:text content)}
             {:role (:role msg)