server : auto-insert media marker in embedding / multimodal prompts

The /embedding (and /embeddings, /v1/embeddings) endpoints failed with
"number of media markers in text (0) does not match number of bitmaps (1)"
when passing multimodal data via the "content" object format.

The server initializes the mtmd context with a randomized media marker
(via get_media_marker()), but process_mtmd_prompt() passed the raw prompt
string to mtmd_tokenize() without ensuring it contained the required
markers. The CLI (mtmd-cli.cpp) already handles this by auto-prepending
markers, but the server did not.

Fix: query the actual marker from the mtmd context via mtmd_get_marker()
and auto-insert one per file if the prompt lacks them.

server: auto-insert missing media markers in process_mtmd_prompt

Fixes the /embedding endpoint when multimodal data is provided without
corresponding media markers in the prompt string. Counts existing markers
and prepends only the missing number so the count matches files.size().

Assisted-by: GitHub Copilot

Potential fix for pull request finding

This just makes the wording more accurate

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

server: auto-insert missing media markers in process_mtmd_prompt

Fixes the /embedding endpoint when multimodal data is provided without
corresponding media markers in the prompt string. Counts existing markers
and prepends only the missing number so the count matches files.size().

Assisted-by: GitHub Copilot

Author: TheOneWhoWill

tools/server/server-common.cpp

@@ -702,10 +702,36 @@ server_tokens process_mtmd_prompt(mtmd_context * mctx, const std::string & promp
         }
     }
     // process prompt
+    std::string prompt_adj;
+
+    const std::string marker = mtmd_get_marker(mctx);
+    const size_t marker_len  = marker.size();
+
+    // count existing media markers in the prompt
+    size_t n_markers = 0;
+    size_t pos = 0;
+    while ((pos = prompt.find(marker, pos)) != std::string::npos) {
+        n_markers++;
+        pos += marker_len;
+    }
+
+    // prepend missing markers so the count matches the number of files
+<<<<<<< HEAD
+    // this mirrors the behavior in mtmd-cli.cpp, but also handles prompts that already contain some markers
+=======
+    // this mirrors the behavior in mtmd-cli.cpp but also handles partial matches
+    if (n_markers < files.size()) {
+>>>>>>> 327c5a188 (server: auto-insert missing media markers in process_mtmd_prompt)
+        for (size_t i = 0; i < files.size() - n_markers; i++) {
+            prompt_adj += marker;
+        }
+    }
+    prompt_adj += prompt;
+
     std::vector<server_tokens> inputs;
     // multimodal
     mtmd_input_text inp_txt = {
-        prompt.c_str(),
+        prompt_adj.c_str(),
         /* add_special */   true,
         /* parse_special */ true,
     };