feat(embeddings): OpenAI-compatible local endpoint support

AGENTS.md

@@ -0,0 +1,91 @@
+<!-- Generated: 2026-05-13 | Updated: 2026-05-13 -->
+
+# lucid-memory
+
+## Purpose
+Local AI memory MCP server providing persistent, reconstructive memory for AI coding assistants. Combines a high-performance Rust retrieval engine (ACT-R spreading activation + MINERVA 2 reconstructive memory) with a TypeScript MCP server. 2.7ms retrieval, $0/query, no cloud dependency. Packaged as a single install for Claude Code, OpenAI Codex, and OpenCode.
+
+## Key Files
+| File | Description |
+|------|-------------|
+| `package.json` | Bun workspace root; build/test/lint/bench scripts for all packages |
+| `Cargo.toml` | Rust workspace; shared deps and version pinning across all crates |
+| `HOW_IT_WORKS.md` | Architecture and algorithm deep-dive |
+| `ROADMAP.md` | Planned features and milestones |
+| `install.sh` / `install.ps1` | End-user install scripts (macOS/Linux and Windows) |
+| `uninstall.sh` / `uninstall.ps1` | End-user uninstall scripts |
+| `biome.json` | TypeScript lint/format config (Biome) |
+| `rustfmt.toml` / `clippy.toml` | Rust format and lint config |
+| `tsconfig.json` | TypeScript compiler config (shared) |
+
+## Subdirectories
+| Directory | Purpose |
+|-----------|---------|
+| `crates/` | Rust crates: retrieval engine and N-API bindings (see `crates/AGENTS.md`) |
+| `packages/` | TypeScript packages: MCP server and pre-built native binaries (see `packages/AGENTS.md`) |
+| `benchmarks/` | Quality and performance benchmarks (see `benchmarks/AGENTS.md`) |
+| `scripts/` | Build, install, and release automation scripts |
+| `.github/` | CI/CD workflows for build, test, and publish |
+| `.claude/` | Claude Code project configuration |
+
+## For AI Agents
+
+### Working In This Directory
+- Stack: **Bun + Rust + TypeScript** monorepo. Keep Rust build steps and JS tooling separate.
+- Build: `bun run build` (Rust first, then TS). Test: `bun test`. Lint: `bun run lint`.
+- Named exports only — no default exports anywhere in TypeScript.
+- `timeout` unavailable on macOS — use `gtimeout`.
+- Good code is self-explaining — comment only when the WHY is non-obvious.
+- A TODO is COMPLETE only when all 5 gates pass:
+
+| Gate | Requirement | How To Verify |
+|------|-------------|---------------|
+| **1. Implemented** | Code written, compiles, no errors | `bun test` passes |
+| **2. Wired** | Connected to systems that call it (wake cycles, hooks, handlers, prompts) | Grep for function calls from outside the module |
+| **3. Tested** | Unit + integration tests covering happy path and edge cases | Test count documented, all pass |
+| **4. Verified in Production** | Evidence of actual usage in the live database | SQL query showing non-zero rows, event counts, or call logs |
+| **5. No Orphaned Outputs** | Every computed output consumed by at least one downstream system | Grep for each output → confirm a reader exists |
+
+### Testing Requirements
+- `bun run test:rust` — Rust unit tests via `cargo test`
+- `bun run test:ts` — Bun test runner across all TS packages
+- `bun run bench` — full benchmark suite (optional, slow)
+- Both test suites must pass before any change is complete.
+
+### Common Patterns
+- Rust crates handle high-performance computation; TypeScript handles I/O and MCP protocol.
+- N-API bindings (`lucid-napi`, `lucid-perception-napi`) bridge Rust computation to TypeScript.
+- Pre-built `.node` binaries ship per platform in `packages/lucid-native` and `packages/lucid-perception`.
+- Database stored at `~/.lucid/` — supports shared, per-client, and profile modes via `LUCID_CLIENT` env.
+
+## Dependencies
+
+### Internal
+- `crates/lucid-core` → `crates/lucid-napi` → `packages/lucid-server`
+- `crates/lucid-perception` → `crates/lucid-perception-napi` → `packages/lucid-server`
+
+### External
+- Bun (runtime + test runner), TypeScript 5, Biome (lint/format)
+- Rust toolchain, napi-rs CLI (N-API binding generation)
+- `@modelcontextprotocol/sdk` (MCP protocol), `zod` (schema validation)
+- FFmpeg CLI (video frame extraction), Whisper (optional audio transcription)
+
+### Domain Language Maintenance
+
+When editing documentation in this repo, consult these canonical definitions:
+
+| Term | Definition |
+|------|-----------|
+| **Memory** | A stored text trace with embedding, activation level, and access history |
+| **Activation** | Composite score combining base-level (recency/frequency), probe similarity, and spreading |
+| **Base-level activation** | B(m) = ln[Σ(t_k)^(-d)] — recency and frequency of past retrievals |
+| **Probe** | The query embedding used to drive retrieval |
+| **Spreading activation** | Activation flowing through the association graph: A_j = Σ(W_i/n_i) × S_ij |
+| **Gist** | Compressed semantic essence of a verbatim memory trace |
+| **Episode** | A temporally bounded sequence of memories representing a work session |
+| **Consolidation** | Background process: strengthen recent memories, decay stale ones, prune weak associations |
+| **Association** | Weighted edge between two memories in the association graph |
+| **Location** | Spatial/temporal context attached to a memory (project, file, line, timestamp) |
+| **Native module** | Pre-built `.node` binary exposing Rust computation via N-API |
+
+<!-- MANUAL -->

CHANGELOG.md

@@ -7,6 +7,108 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+#### OpenAI-Compatible Local Embedding Endpoints
+
+Lucid Memory now supports any OpenAI-compatible embedding server as a drop-in backend — Ollama, LM Studio, vLLM, and others — without an API key.
+
+**Key features:**
+
+- **`LUCID_EMBEDDING_URL`** — Set to any OpenAI-compatible base URL (e.g. `http://localhost:11434/v1`). Checked first, highest priority in provider detection.
+- **`LUCID_EMBEDDING_MODEL`** — Override the embedding model name (e.g. `nomic-embed-text`).
+- **API key optional** — Local endpoints work without authentication.
+- **Long-context models** — Enables models like `nomic-embed-text` (8192-token context) as an alternative to the built-in BGE model (512-token limit).
+- **Endpoint probing** — `isAvailable()` probes `/models` on custom endpoints to verify availability before committing.
+
+**Usage:**
+
+```bash
+LUCID_EMBEDDING_URL=http://localhost:11434/v1 \
+LUCID_EMBEDDING_MODEL=nomic-embed-text \
+lucid-server
+```
+
+**Provider detection order:** `LUCID_EMBEDDING_URL` → local ONNX (BGE) → OpenAI → none.
+
+## [0.6.5] - 2026-02-20
+
+### Added
+
+- **Linux ARM64 pre-built binary** for `lucid-native` (`lucid-native.linux-arm64-gnu.node`) — previously required building from source on Linux ARM64.
+
+### Fixed
+
+**All `.node` binaries shipped since January 30th were macOS ARM64 Mach-O files regardless of target platform**, causing immediate segfaults on Linux and Windows. Only macOS Apple Silicon users were unaffected.
+
+**Root cause:** Two interacting CI bugs: (1) the upload step used `path: *.node`, uploading all checked-out `.node` files per artifact rather than only the freshly built one; (2) the rename step picked the first `.node` file alphabetically from checkout — the pre-existing `darwin-arm64` binary — and renamed it to the target platform name, overwriting the correct build output.
+
+**Fixes:**
+
+- Clean stale binaries before building so only fresh build output exists
+- Upload only the specific target file (`lucid-native.{platform}.node`) instead of `*.node`
+- Add binary format validation (`file` command checks ELF/Mach-O/PE32+) in both publish and release jobs
+- Add format validation to `install.sh` download fallback — wrong-format downloads are rejected and deleted
+- Fix release job permissions for asset uploads
+
+Six layers now prevent recurrence: pre-build clean → post-build verify → targeted upload → publish validation → release validation → installer validation.
+
+**Platform support after this fix:**
+
+| Platform | lucid-native | lucid-perception |
+|----------|:---:|:---:|
+| macOS ARM64 (Apple Silicon) | pre-built | pre-built |
+| macOS x64 (Intel) | build from source | pre-built |
+| Linux x64 | pre-built | pre-built |
+| Linux ARM64 | pre-built (new) | build from source |
+| Windows x64 | pre-built | pre-built |
+
+## [0.6.4] - 2026-02-20
+
+### Fixed
+
+#### Embedding Provider Diagnostics
+
+Silent embedding failures now surface with actionable diagnostics instead of falling through to `null`.
+
+- **`detectProvider()` returns `ProviderDiagnostics`** — per-provider failure reasons recorded; `lucid status` now shows exactly what was tried and why each provider was skipped (e.g. `Native: model failed to load (ORT error)`, `OpenAI: no OPENAI_API_KEY set`, `Ollama: connection refused`). Previously checked file existence only — a corrupt or incompatible model file would still show as "configured".
+- **`retrieve()` returns `RetrievalResult`** — includes `method` (`"semantic"` | `"recency"`) and `warning` field. `memory_query` responses surface these fields so callers can detect when results are ranked by recency only due to embedding failure.
+
+## [0.6.3] - 2026-02-20
+
+### Fixed
+
+- Switched built-in ONNX embedding model from FP16 (`model_fp16.onnx`) to INT8 quantized (`model_quantized.onnx`). The FP16 model contains graph optimizations (`SimplifiedLayerNormFusion`, `InsertedPrecisionFreeCast`) incompatible with `ort` 2.0.0-rc.11, causing embeddings to silently fail to load and report "No embedding provider configured". The quantized model (110MB vs 218MB) is fully compatible and produces equivalent retrieval quality.
+- Installers now clean up the old FP16 model file on re-install.
+
+## [0.6.2] - 2026-02-15
+
+### Fixed
+
+- Removed dead NAPI bindings that were exported but never called from TypeScript
+- Removed orphaned struct fields causing Clippy warnings
+- Resolved remaining Biome lint issues introduced by 0.6.1
+
+## [0.6.1] - 2026-02-15
+
+### Added
+
+- **`visual_list` MCP tool** — Browse and filter visual memories by project, media type, and recency. Previously `queryVisualMemories()` was unreachable from the MCP surface.
+
+### Fixed
+
+The 0.6.0 5-gate audit incorrectly made 5 storage methods private and deleted them to clear Biome lint errors, breaking wired features. This release restores and wires each:
+
+| Method | Wired to |
+|--------|---------|
+| `queryVisualMemories()` | New `visual_list` MCP tool |
+| `getVisualEmbedding()` | Consolidation visual pruning guard (skip unprocessed visuals) |
+| `getEpisode()` | Public API for single-episode lookup |
+| `updateProjectContext()` | `store()` flow for project metadata accumulation |
+| `hasEmbedding()` | Public API for per-memory embedding existence check |
+
+`memory_consolidation_status` enriched with episode diagnostics (recent episodes with event counts and boundary types) and pending embedding counts (memories awaiting background processing).
+
 ## [0.6.0] - 2026-02-15
 
 ### Added

CLAUDE.md

@@ -0,0 +1 @@
+AGENTS.md

GEMINI.md

@@ -0,0 +1 @@
+AGENTS.md

benchmarks/AGENTS.md

@@ -0,0 +1,21 @@
+<!-- Parent: ../AGENTS.md -->
+<!-- Generated: 2026-05-13 | Updated: 2026-05-13 -->
+
+# benchmarks
+
+## Purpose
+Quality and performance benchmarks for the retrieval and memory systems. Measures recall, token efficiency, and retrieval quality against baselines (Claude-mem, Pinecone RAG, Traditional RAG).
+
+## Subdirectories
+| Directory | Purpose |
+|-----------|---------|
+| `quality/` | TypeScript benchmark scripts for recall, RAG comparison, token efficiency, and episodic retrieval (see `quality/AGENTS.md`) |
+
+## For AI Agents
+
+### Working In This Directory
+- Run via root scripts: `bun run bench:quality`, `bun run bench:rag`, `bun run bench:realistic`, `bun run bench:tokens`.
+- Rust benchmarks live in `crates/lucid-core/benches/`; run with `bun run bench:rust`.
+- Benchmarks are standalone scripts — no test framework, no assertions. Output is for human review.
+
+<!-- MANUAL -->

benchmarks/CLAUDE.md

@@ -0,0 +1 @@
+AGENTS.md

benchmarks/GEMINI.md

@@ -0,0 +1 @@
+AGENTS.md

benchmarks/quality/AGENTS.md

@@ -0,0 +1,27 @@
+<!-- Parent: ../AGENTS.md -->
+<!-- Generated: 2026-05-13 | Updated: 2026-05-13 -->
+
+# benchmarks/quality
+
+## Purpose
+Quality benchmarks measuring retrieval recall, token efficiency, and memory system performance against baseline approaches (Claude-mem, Pinecone RAG, Traditional RAG).
+
+## Key Files
+| File | Description |
+|------|-------------|
+| `index.ts` | Main quality benchmark suite: recall@k, precision, F1 across retrieval strategies |
+| `rag-comparison.ts` | Side-by-side comparison: lucid-memory vs Pinecone RAG vs Traditional RAG |
+| `membrane-comparison.ts` | Comparison against membrane-style memory architectures |
+| `realistic-dev.ts` | Simulates a realistic developer session with code questions and context switching |
+| `episodic-e2e.ts` | End-to-end episodic memory benchmark: episode boundary detection accuracy |
+| `token-efficiency.ts` | Token budget benchmarks: recall at fixed token counts (5x efficiency target) |
+
+## For AI Agents
+
+### Working In This Directory
+- Run individual benchmarks: `bun run benchmarks/quality/<file>.ts`
+- Run all via root scripts: `bun run bench:quality`, `bun run bench:rag`, `bun run bench:realistic`, `bun run bench:tokens`
+- Benchmarks use real retrieval calls — require initialized storage at `~/.lucid/`.
+- Output is human-readable tables for comparison review, not assertions.
+
+<!-- MANUAL -->

benchmarks/quality/CLAUDE.md

@@ -0,0 +1 @@
+AGENTS.md

benchmarks/quality/GEMINI.md

@@ -0,0 +1 @@
+AGENTS.md

crates/AGENTS.md

@@ -0,0 +1,42 @@
+<!-- Parent: ../AGENTS.md -->
+<!-- Generated: 2026-05-13 | Updated: 2026-05-13 -->
+
+# crates
+
+## Purpose
+Rust workspace members providing the high-performance computation layer. Two independent subsystems: the memory retrieval engine (`lucid-core` + `lucid-napi`) and the visual perception pipeline (`lucid-perception` + `lucid-perception-napi`). Each subsystem ships a pure Rust crate and an N-API binding crate that exposes it to TypeScript.
+
+## Subdirectories
+| Directory | Purpose |
+|-----------|---------|
+| `lucid-core/` | ACT-R spreading activation + MINERVA 2 retrieval engine (see `lucid-core/AGENTS.md`) |
+| `lucid-napi/` | N-API bindings exposing lucid-core to Node.js/Bun (see `lucid-napi/AGENTS.md`) |
+| `lucid-perception/` | Video frame extraction, scene detection, audio transcription (see `lucid-perception/AGENTS.md`) |
+| `lucid-perception-napi/` | N-API bindings exposing lucid-perception to Node.js/Bun (see `lucid-perception-napi/AGENTS.md`) |
+
+## For AI Agents
+
+### Working In This Directory
+- All crates are members of the Cargo workspace at the repo root. Never add a separate `[workspace]` block inside a crate.
+- Shared dependencies are declared in root `Cargo.toml` under `[workspace.dependencies]`; crates reference them with `{ workspace = true }`.
+- Lint rules (`[lints]`) are workspace-level; do not override per-crate without strong justification.
+
+### Testing Requirements
+- `cargo test` from repo root runs all crate tests.
+- `cargo bench -p lucid-core` runs lucid-core criterion benchmarks.
+- `cargo clippy --all-targets --all-features` must pass clean.
+
+### Common Patterns
+- Pure computation logic lives in the non-napi crate (`lucid-core`, `lucid-perception`).
+- N-API crates contain only binding glue — no business logic.
+- Optional heavy features (`embedding`, `transcription`, `cuda`) are feature-flagged; default build stays minimal.
+
+## Dependencies
+
+### External
+- `napi` + `napi-derive` (N-API binding framework)
+- `faer`, `nalgebra` (linear algebra), `rayon` (parallelism)
+- `serde` + `rkyv` + `bincode` (serialization)
+- `ort` + `tokenizers` (ONNX Runtime embedding, feature-gated)
+
+<!-- MANUAL -->

crates/CLAUDE.md

@@ -0,0 +1 @@
+AGENTS.md

crates/GEMINI.md

@@ -0,0 +1 @@
+AGENTS.md