Skip to content

Commit e7c818c

Browse files
committed
feat(doctor): probe embedding endpoint with clear diagnostics (fixes #29)
The most common embedding-config mistakes (wrong URL shape, missing env var, provider that doesn't implement embeddings) produced silent 401/404 failures at runtime with no hint about what was wrong. The dashboard exposes a 'test connection' button backed by a Rust `reqwest` call, but doctor didn't have anything equivalent and was the natural place to surface these errors during setup. Adds `probeEmbeddingEndpoint` in `src/features/magic-context/memory/embedding-probe.ts` — a small Node/Bun fetch-based probe that sends `{model, input}` to `${endpoint}/embeddings` with a 10s timeout and classifies the outcome: - `ok` — 2xx with `data[].embedding` array (reports dimensions) - `auth_failed` — 401/403 with body preview - `endpoint_unsupported` — 404/405, OR 2xx without embeddings payload (catches routers that accept the URL but don't speak the spec) - `http_error` — other non-2xx - `timeout` — AbortSignal.timeout triggered - `network_error` — fetch threw (DNS, connection refused, etc.) - `invalid_scheme` — endpoint missing http(s):// before any request Doctor gains a `checkEmbeddingConfig` step (run as 7b, between user- memories and plugin-cache checks). It uses the same `substituteConfigVariables` path the runtime uses, so the probe sees the real resolved api_key/endpoint — not the literal `{env:...}` template. If a `{env:VAR}` token survived substitution the doctor warns that the env var is not set in the current shell. For each probe outcome doctor emits a specific recommendation. The `endpoint_unsupported` case lists known non-embedding providers (OpenRouter, Anthropic) since those are the two most likely configurations from user reports, and gives concrete working alternatives (OpenAI, Voyage, Together, LM Studio, local). Issue #29's exact config (`endpoint: https://openrouter.ai/api/v1/chat/completions`, `api_key: "{env:OPENROUTER_API_KEY}"`) produces three distinct failure modes that are each handled: 1. `{env:` residue warning if OPENROUTER_API_KEY is not exported 2. 404/405 from `/chat/completions/embeddings` (wrong URL shape) 3. OpenRouter would still return an unsupported response even if the URL were right — covered by `endpoint_unsupported` Tests: 17 unit tests in `embedding-probe.test.ts` covering every outcome kind, custom timeouts, AbortError and TimeoutError branches, trailing- slash trimming, Bearer-auth header presence/absence/whitespace, and preview truncation. 679 total tests pass. CONFIGURATION.md: embedding example updated to show the `{env:OPENAI_API_KEY}` pattern and calls out that doctor validates the endpoint. Closes #29
1 parent 1582549 commit e7c818c

4 files changed

Lines changed: 695 additions & 1 deletion

File tree

CONFIGURATION.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -333,11 +333,15 @@ When `provider: "off"`:
333333
"provider": "openai-compatible",
334334
"model": "text-embedding-3-small",
335335
"endpoint": "https://api.openai.com/v1",
336-
"api_key": "sk-..."
336+
"api_key": "{env:OPENAI_API_KEY}"
337337
}
338338
}
339339
```
340340

341+
> **Note:** Any string in `magic-context.jsonc` can use `{env:VAR}` to reference an environment variable, or `{file:path}` to inline the contents of an external file (matching OpenCode's own config substitution). Paths are resolved relative to the config file's directory; `~/` expands to the home directory. Use `doctor` after editing — it probes the configured embedding endpoint and reports missing env vars, wrong URLs, auth failures, or providers that don't implement the embeddings API.
342+
343+
> **Not every provider offers embeddings.** OpenRouter and Anthropic's public API do not expose `/embeddings`; use OpenAI, Voyage, Together, LM Studio, or the bundled `"local"` provider instead. `doctor` will flag 404/405 responses and show the actual error.
344+
341345
---
342346

343347
## `memory`

packages/plugin/src/cli/doctor.ts

Lines changed: 185 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,11 @@ import { createRequire } from "node:module";
44
import { homedir, platform, tmpdir } from "node:os";
55
import { join } from "node:path";
66
import { parse, stringify } from "comment-json";
7+
import { substituteConfigVariables } from "../config/variable";
8+
import {
9+
type EmbeddingProbeOutcome,
10+
probeEmbeddingEndpoint,
11+
} from "../features/magic-context/memory/embedding-probe";
712
import { detectConflicts } from "../shared/conflict-detector";
813
import { fixConflicts } from "../shared/conflict-fixer";
914
import { ensureTuiPluginEntry } from "../shared/tui-config";
@@ -200,6 +205,180 @@ async function runIssueFlow(): Promise<number> {
200205
}
201206
}
202207

208+
// ── Embedding configuration check ───────────────────────────────────
209+
210+
/**
211+
* Validate the user's embedding configuration by probing the configured
212+
* endpoint. Runs only for `openai-compatible` providers — `local` needs no
213+
* network check and `off` degrades cleanly by design.
214+
*
215+
* Known footguns we surface specifically:
216+
* - `{env:VAR}` in api_key when VAR is not exported → auth will fail with
217+
* a literal `Bearer {env:VAR}` header.
218+
* - Endpoint pointing at a specific route (e.g. `.../chat/completions`)
219+
* rather than the provider base (e.g. `.../v1`) — gets detected by the
220+
* real probe returning 404/405.
221+
* - Provider that accepts the URL shape but doesn't implement embeddings
222+
* (OpenRouter's /v1 for example) — same detection path.
223+
*/
224+
async function checkEmbeddingConfig(magicContextConfigPath: string): Promise<{ issues: number }> {
225+
if (!existsSync(magicContextConfigPath)) {
226+
// No config → local provider defaults apply, nothing to check.
227+
return { issues: 0 };
228+
}
229+
230+
let rawText: string;
231+
try {
232+
rawText = readFileSync(magicContextConfigPath, "utf-8");
233+
} catch {
234+
log.warn("Could not read magic-context.jsonc for embedding check");
235+
return { issues: 1 };
236+
}
237+
238+
// Substitute {env:} and {file:} before parsing so api_key / endpoint
239+
// reflect the values the runtime will actually see, and so we can report
240+
// unresolved tokens as concrete issues.
241+
const substituted = substituteConfigVariables({
242+
text: rawText,
243+
configPath: magicContextConfigPath,
244+
});
245+
246+
let parsedConfig: Record<string, unknown>;
247+
try {
248+
parsedConfig = parse(substituted.text) as Record<string, unknown>;
249+
} catch (error) {
250+
log.warn(
251+
`Embedding check skipped — could not parse magic-context.jsonc: ${error instanceof Error ? error.message : String(error)}`,
252+
);
253+
return { issues: 1 };
254+
}
255+
256+
const embedding = parsedConfig?.embedding as Record<string, unknown> | undefined;
257+
const provider = embedding?.provider;
258+
259+
if (provider === "off") {
260+
log.info("Embedding provider disabled — semantic memory search is off");
261+
return { issues: 0 };
262+
}
263+
264+
if (provider === undefined || provider === "local") {
265+
log.success("Embedding provider: local (Xenova/all-MiniLM-L6-v2 bundled)");
266+
return { issues: 0 };
267+
}
268+
269+
if (provider !== "openai-compatible") {
270+
log.warn(
271+
`Unknown embedding provider: ${String(provider)} (expected local | openai-compatible | off)`,
272+
);
273+
return { issues: 1 };
274+
}
275+
276+
const endpoint = typeof embedding?.endpoint === "string" ? embedding.endpoint.trim() : "";
277+
const model = typeof embedding?.model === "string" ? embedding.model.trim() : "";
278+
const apiKey = typeof embedding?.api_key === "string" ? embedding.api_key : undefined;
279+
280+
let localIssues = 0;
281+
282+
// Static configuration hygiene checks — raise before the network probe so
283+
// users get the specific guidance even when they're offline.
284+
if (!endpoint) {
285+
log.error("Embedding provider is openai-compatible but 'endpoint' is missing");
286+
return { issues: 1 };
287+
}
288+
if (!model) {
289+
log.error("Embedding provider is openai-compatible but 'model' is missing");
290+
return { issues: 1 };
291+
}
292+
293+
// Flag unresolved {env:} residue — the substitution pass above would have
294+
// replaced resolved tokens, so any leftover {env: here means either the
295+
// env var was missing or the user wrote the literal text.
296+
if (apiKey && /\{env:[^}]+\}/.test(apiKey)) {
297+
log.warn(
298+
"api_key still contains {env:...} after substitution — the referenced environment variable is not set in this shell",
299+
);
300+
log.info(` Raw value: ${apiKey}`);
301+
log.info(
302+
" Export the variable before launching OpenCode (e.g. in ~/.zshrc, ~/.bashrc, or a shell profile)",
303+
);
304+
localIssues++;
305+
}
306+
307+
// Surface any substitution warnings for the *user* config — we can't
308+
// tell which substitutions fed the embedding block specifically, but if
309+
// the block is broken and there are env-var warnings, they're almost
310+
// certainly related.
311+
if (substituted.warnings.length > 0) {
312+
for (const w of substituted.warnings.slice(0, 3)) {
313+
log.info(` ${w}`);
314+
}
315+
if (substituted.warnings.length > 3) {
316+
log.info(` ... and ${substituted.warnings.length - 3} more`);
317+
}
318+
}
319+
320+
// Run the live probe.
321+
const probeSpinner = spinner();
322+
probeSpinner.start(`Testing embedding endpoint ${endpoint} (model: ${model})`);
323+
324+
let outcome: EmbeddingProbeOutcome;
325+
try {
326+
outcome = await probeEmbeddingEndpoint({
327+
endpoint,
328+
model,
329+
apiKey: apiKey,
330+
timeoutMs: 10_000,
331+
});
332+
} catch (error) {
333+
probeSpinner.stop("Embedding probe failed unexpectedly");
334+
log.error(`Probe threw: ${error instanceof Error ? error.message : String(error)}`);
335+
return { issues: localIssues + 1 };
336+
}
337+
338+
probeSpinner.stop("Embedding endpoint probed");
339+
340+
switch (outcome.kind) {
341+
case "ok":
342+
log.success(
343+
`Embedding endpoint OK (${outcome.status}, ${outcome.dimensions ?? "?"}-dim vectors)`,
344+
);
345+
return { issues: localIssues };
346+
case "auth_failed":
347+
log.error(
348+
`Embedding endpoint rejected credentials (${outcome.status}) — check api_key / env var`,
349+
);
350+
if (outcome.preview) log.info(` ${outcome.preview}`);
351+
return { issues: localIssues + 1 };
352+
case "endpoint_unsupported":
353+
log.error(`Embedding endpoint does not support embeddings (${outcome.status})`);
354+
if (outcome.preview) log.info(` ${outcome.preview}`);
355+
log.info(
356+
" Common causes: endpoint points at a chat-completion route (should be the provider base, e.g. '.../v1'), or the provider doesn't offer an embeddings API",
357+
);
358+
log.info(
359+
" Known non-embedding providers: OpenRouter (chat proxy), Anthropic (no embeddings endpoint). Use OpenAI, Voyage, Together, or a local provider instead.",
360+
);
361+
return { issues: localIssues + 1 };
362+
case "http_error":
363+
log.error(`Embedding endpoint returned ${outcome.status}`);
364+
if (outcome.preview) log.info(` ${outcome.preview}`);
365+
return { issues: localIssues + 1 };
366+
case "timeout":
367+
log.warn(
368+
`Embedding endpoint did not respond within ${outcome.timeoutMs}ms — check endpoint URL and network`,
369+
);
370+
return { issues: localIssues + 1 };
371+
case "network_error":
372+
log.error(`Could not reach embedding endpoint: ${outcome.message}`);
373+
return { issues: localIssues + 1 };
374+
case "invalid_scheme":
375+
log.error(
376+
`Embedding endpoint must start with http:// or https://: ${outcome.endpoint}`,
377+
);
378+
return { issues: localIssues + 1 };
379+
}
380+
}
381+
203382
// ── Main doctor entry ───────────────────────────────────────────────
204383

205384
export async function runDoctor(
@@ -437,6 +616,12 @@ export async function runDoctor(
437616
}
438617
}
439618

619+
// 7b. Validate embedding configuration — runs a real probe against the
620+
// configured endpoint so users catch misconfigured URL / missing env var /
621+
// wrong provider issues before relying on semantic memory search.
622+
const embeddingCheck = await checkEmbeddingConfig(paths.magicContextConfig);
623+
issues += embeddingCheck.issues;
624+
440625
// 8. Check plugin npm cache — clear only if outdated
441626
const cacheResult = await clearPluginCache(options.force);
442627
if (cacheResult.action === "cleared") {

0 commit comments

Comments
 (0)