Skip to content

feat(embeddings): OpenAI-compatible local endpoint support#10

Open
jonathanhfmills wants to merge 3 commits into
JasonDocton:mainfrom
jonathanhfmills:main
Open

feat(embeddings): OpenAI-compatible local endpoint support#10
jonathanhfmills wants to merge 3 commits into
JasonDocton:mainfrom
jonathanhfmills:main

Conversation

@jonathanhfmills

Copy link
Copy Markdown

Summary

  • Adds LUCID_EMBEDDING_URL and LUCID_EMBEDDING_MODEL env vars for keyless local embedding servers (Ollama, LM Studio, vLLM, etc.)
  • Custom endpoint checked first in provider detection — highest priority
  • API key optional for local endpoints; isAvailable() probes /models to verify before committing
  • Enables long-context models (e.g. nomic-embed-text at 8192 tokens) as an alternative to the built-in BGE model (512-token limit)

Usage

LUCID_EMBEDDING_URL=http://localhost:11434/v1 \
LUCID_EMBEDDING_MODEL=nomic-embed-text \
lucid-server

Provider detection order: LUCID_EMBEDDING_URL → local ONNX (BGE) → OpenAI → none.

Changes

  • EmbeddingConfig: new openaiBaseUrl field
  • embedOpenAIBatch: uses configurable baseUrl, API key optional
  • isAvailable: probes /models when openaiBaseUrl set
  • detectProvider: checks LUCID_EMBEDDING_URL first

Also includes

  • AGENTS.md docs — 24 hierarchical context files (one per directory) providing structured AI-readable documentation: purpose, key files, conventions, testing requirements, and dependency relationships. CLAUDE.md and GEMINI.md symlink to each AGENTS.md so Claude, Gemini, and other AI agents load the right context automatically when working in any subdirectory.
  • CHANGELOG filled in for 0.6.1–0.6.5 (entries were missing for all point releases since the 0.6.0 audit).

Test plan

  • LUCID_EMBEDDING_URL=http://localhost:11434/v1 LUCID_EMBEDDING_MODEL=nomic-embed-text lucid-server — verify embeddings work via Ollama
  • lucid status — verify custom endpoint shown as active provider
  • No env vars set — verify fallback to BGE still works
  • Invalid URL — verify graceful failure and lucid status shows skip reason
  • bun test passes

HellFireAE and others added 3 commits May 13, 2026 08:41
Add openaiBaseUrl to EmbeddingConfig and LUCID_EMBEDDING_URL /
LUCID_EMBEDDING_MODEL env vars for keyless local embedding servers.

- EmbeddingConfig: add openaiBaseUrl field
- embedOpenAIBatch: use configurable baseUrl, API key optional for local endpoints
- isAvailable: probe custom endpoint via /models when openaiBaseUrl set
- detectProvider: check LUCID_EMBEDDING_URL first (highest priority)

Enables Ollama, LM Studio, and vLLM as drop-in embedding backends
with long-context models (e.g. nomic-embed-text at 8192 tokens),
bypassing the 512-token BGE sequence limit without API costs.

Usage:
  LUCID_EMBEDDING_URL=http://localhost:11434/v1 \
  LUCID_EMBEDDING_MODEL=nomic-embed-text \
  lucid-server

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
24 AGENTS.md files covering root, crates (lucid-core, lucid-napi,
lucid-perception, lucid-perception-napi), packages (lucid-server,
lucid-native, lucid-perception), benchmarks, scripts, and all src/
subdirectories. CLAUDE.md and GEMINI.md symlinked to AGENTS.md in
each directory. Domain glossary seeded in root. Original CLAUDE.md
content (5-gate completion gates, conventions, gotchas) preserved in
root AGENTS.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add missing release entries: 0.6.1 (wire 5 unused storage methods),
0.6.2 (dead NAPI cleanup), 0.6.3 (fp16→quantized ONNX model),
0.6.4 (embedding provider diagnostics), 0.6.5 (cross-platform binary
rebuild). Add [Unreleased] entry for OpenAI-compatible local embedding
endpoint support (LUCID_EMBEDDING_URL / LUCID_EMBEDDING_MODEL).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jonathanhfmills jonathanhfmills marked this pull request as ready for review May 13, 2026 12:45
@jonathanhfmills

Copy link
Copy Markdown
Author

PR #8 only touches install.sh, our PR touches embeddings.ts, cli.ts, CHANGELOG.md, and AGENTS.md files. Zero conflicts.

And PR #8 actually solves the exact pain we hit today — having to manually cp cli.ts ~/.lucid/server/src/cli.ts. Once that merges, ./install.sh --local from the repo root handles it 👍

Worth merging PR #8 first, then this one lands clean on top.

I trust your judgment, @JasonDocton. Hope all's well on your end! LMK if you need anything from me 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants