Skip to content

Commit 51d3073

Browse files
fix(embeddings): local embeddings on OpenCode Desktop / Electron (#195)
Two bugs broke /ctx-embed on Electron, both confirmed at source against @huggingface/transformers 4.2.0 and reproduced+verified on plain Node: 1. Device selection. When we inject our own ORT via Symbol.for("onnxruntime") (the Electron WASM path for #78), transformers skips its device-registration branch, leaving supportedDevices=[]. With no device option it defaults to "cpu" under IS_NODE_ENV, fails the supportedDevices.includes("cpu") check, and throws 'Unsupported device: "cpu". Should be one of: .'. Pass device:"auto" on the injected path: deviceToExecutionProviders("auto") returns supportedDevices verbatim WITHOUT the membership check, so ORT-web uses its own default (wasm) EP. Native Node/Bun is untouched (no device opt). The reporter's fix (delete the injection) would regress #78; this keeps it. 2. WASM path resolution. require.resolve("onnxruntime-web/package.json") throws ERR_PACKAGE_PATH_NOT_EXPORTED because the package's exports map doesn't expose ./package.json, so wasmPaths fell back to a CDN/blob URL Node's ESM loader rejects. Resolve the main export ('.') and take its dirname (always exported, lands in dist/). Repro at scripts/experiments/issue-195-repro.mjs (gitignored): no-device throws the exact reported error; device:"auto" + fixed wasmPaths => OK (dims=384). Thanks @Treeed for the source-grounded report. Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>
1 parent 881b831 commit 51d3073

1 file changed

Lines changed: 22 additions & 4 deletions

File tree

packages/plugin/src/features/magic-context/memory/embedding-local.ts

Lines changed: 22 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -171,8 +171,14 @@ async function injectWasmOrtForElectron(): Promise<boolean> {
171171
try {
172172
const { createRequire: createRequireFn } = await import("node:module");
173173
const requireFn = createRequireFn(import.meta.url);
174-
const pkgPath = requireFn.resolve("onnxruntime-web/package.json");
175-
const distDir = join(dirname(pkgPath), "dist");
174+
// Resolve the package's MAIN export ('.') rather than its
175+
// package.json: onnxruntime-web ships an `exports` map that does NOT
176+
// expose './package.json' (resolving it throws ERR_PACKAGE_PATH_NOT_
177+
// EXPORTED), whereas '.' is always exported and lands inside dist/.
178+
// Its dirname is the dist/ dir that holds the .wasm/.mjs assets. See
179+
// issue #195.
180+
const mainEntry = requireFn.resolve("onnxruntime-web");
181+
const distDir = dirname(mainEntry);
176182
const wasmPathsPrefix = `${pathToFileURL(distDir).href}/`;
177183
if (ortWeb.env?.wasm) {
178184
ortWeb.env.wasm.wasmPaths = wasmPathsPrefix;
@@ -226,7 +232,7 @@ type EmbeddingPipeline = {
226232
type CreateEmbeddingPipeline = (
227233
task: "feature-extraction",
228234
model: string,
229-
options: { dtype: string },
235+
options: { dtype: string; device?: string },
230236
) => Promise<EmbeddingPipeline>;
231237

232238
/**
@@ -421,7 +427,7 @@ export class LocalEmbeddingProvider implements EmbeddingProvider {
421427
// evaluation time and uses whatever we provide instead of doing its
422428
// own native-vs-web backend selection. No-op on plain Node/Bun.
423429
// See: https://github.com/cortexkit/magic-context/issues/78
424-
await injectWasmOrtForElectron();
430+
const injectedWasmOrt = await injectWasmOrtForElectron();
425431

426432
// Non-literal import specifier prevents Bun from eagerly resolving
427433
// @huggingface/transformers at plugin load time. Desktop sidecar spawns
@@ -490,9 +496,21 @@ export class LocalEmbeddingProvider implements EmbeddingProvider {
490496
// Passing `dtype: "fp32"` selects the full-precision ONNX
491497
// model; the model file on disk is unchanged (~90MB for
492498
// all-MiniLM-L6-v2).
499+
//
500+
// device: "auto" is REQUIRED when we injected our own ORT
501+
// via Symbol.for("onnxruntime") (the Electron WASM path):
502+
// transformers then skips its device-registration branch, so
503+
// supportedDevices stays []. Any concrete device (incl. the
504+
// "cpu" it defaults to under IS_NODE_ENV) fails the
505+
// supportedDevices.includes(device) check and throws
506+
// `Unsupported device: "cpu"`. "auto" returns supportedDevices
507+
// verbatim ([]) without that check, so onnxruntime-web uses its
508+
// own default (wasm) execution provider. Native Node/Bun keeps
509+
// the default selection (no device option). See issue #195.
493510
const pipeline = await withQuietConsole(() =>
494511
createPipeline("feature-extraction", this.model, {
495512
dtype: "fp32",
513+
...(injectedWasmOrt ? { device: "auto" } : {}),
496514
}),
497515
);
498516
if (this.disposing) {

0 commit comments

Comments
 (0)