Centralized embedding backend: semantic memory, session_search & skill matching (#28)

* Extract shared internal/embedding package from memory

Phase 1 of bringing sessions and skills to embedding parity with memory.
The text-embedding backends (RandomProjections + OpenAI-compatible HTTP),
their config shape, fingerprinting, and persistence were package-private
inside internal/memory, so sessions and skills could not reuse them.

Move them to a new internal/embedding package with an exported surface:
  - Config (was EmbeddingConfig)
  - TextEmbedder interface (exported methods)
  - New / NewRP constructors (was newTextEmbedder / newRPTextEmbedder)
  - Cosine similarity helper (was memory's cosineVector)
  - the bigram featurization (featurize.go)

Memory keeps package-local aliases (EmbeddingConfig, textEmbedder) and thin
constructors so all its call sites and the on-disk .gob/fingerprint formats
are unchanged — no behavior change, no migration. cosineVector now delegates
to embedding.Cosine to avoid a divergent copy.

The embedder and featurize unit tests move with the code; memory keeps a
local mock embeddings server for its episode-index integration tests.

* Sessions: semantic session_search over shared embedding backend

Phase 2: route the session VectorIndex through internal/embedding so
session_search gains real semantic matching when an HTTP embeddings
backend (Ollama/OpenAI/etc.) is configured, instead of being hardwired to
go-vector RandomProjections.

  - VectorIndex.emb is now embedding.TextEmbedder, selected by an
    embedding.Config. Default (nil) stays RandomProjections — unchanged
    for existing users.
  - Add a vectors_meta.json fingerprint file next to vectors.gob. On init
    the persisted vectors are only reused when the embedder fingerprint
    (provider/model/dims) matches; switching backends forces a one-time
    rebuild in the new space. A missing meta (legacy layout) also rebuilds,
    since the old raw-text RP vectors live in a different space than the
    shared featurized embedder.
  - Rebuild batch-embeds the whole corpus via EmbedAll (one HTTP call) and
    adopts the episode index's 30s failure cool-down so a down backend is
    not hammered on every search or session save.
  - Resilience: Add never fails a session save and Search never surfaces an
    error when the backend is unavailable — both degrade silently to the
    keyword fallback in session_search_tool.go (left untouched).

Wiring: Store.InitVectorIndex takes an *embedding.Config; the continue and
telegram entry points pass memory's embedding config so a single endpoint
powers both subsystems (a dedicated sessions block lands in Phase 4).

Tests mirror memory's: HTTP semantic match (kitten→feline), fingerprint
invalidation forcing a rebuild, and the rebuild backoff under a down backend.

* Skills semantic matching (opt-in) + shared embedding config

Phase 3 — skills:
Route the skill VectorMatcher through internal/embedding. The skill corpus
embeds once at reload (cheap); only the query embeds per turn, so the HTTP
backend is opt-in and time-bounded:
  - NewVectorMatcherWithConfig selects the backend; default stays local RP.
  - For an HTTP config that omits timeout_seconds, a short 2s query timeout is
    applied so the per-turn embed fails fast.
  - SkillManager.MatchLazySkills is the single entry point the agent loop wires:
    when a remote backend is configured it tries semantic matching first and
    falls back to the keyword ScoredMatcher on no-match/timeout/down-backend;
    otherwise it uses the keyword matcher directly (unchanged default). The
    local RP vector matcher stays a fallback, never the primary.

Phase 4 — shared config + docs:
  - New top-level `embedding` block is the shared default. Memory inherits it
    unless memory.embedding overrides (back-compat); sessions read it for
    semantic session_search; skills do NOT inherit it — opt in via
    skills.embedding because they run on every turn.
  - Resolved into ResolvedConfig.Embedding and Skills.Embedding; merge + memory
    inheritance wired in the loader.
  - Session init and the skill managers across all agent entry points (run,
    continue, repl, serve, subagent, mcp, telegram) now take the resolved
    embedding configs.
  - CONFIG.md documents the shared backend, the inherit/opt-in matrix, and the
    per-subsystem resilience (session fingerprint rebuild, skill fail-fast
    fallback, egress warning).

Tests: HTTP-backed semantic skill match, default-RP-not-semantic, keyword
fallback when the skills backend is down, and config resolution for top-level
inheritance + per-subsystem overrides.

* Add unit tests for embedding edge cases, session index, skill matching

Raises coverage on the new code: embedding 91.7%→98.3%, session 84.0%→85.5%,
skills 83.3%→85.0%.

- embedding: Cosine edge cases (mismatched/empty/zero-norm + NaN/Inf clamp),
  NewRP dims default, HTTP apikey/timeout/dims fingerprint, stateless
  SaveState/LoadState no-ops, RP save-to-bad-path safety.
- session: public Save + replace-on-Add + empty-text no-op, Remove/Save on a
  not-ready index.
- skills: semantic-success path of MatchLazySkills, the non-semantic fallback
  chain (scored→vector→trie→empty), HybridMatcher trie+vector merge, and the
  buildEmbedText separate/with-body branches.

* docs: document shared embedding backend across sessions and skills

- SESSIONS.md: session_search now rides the shared embedding backend (RP
  default or opt-in HTTP semantic), with the fingerprint rebuild + 30s
  backoff and DeepSearch fallback. Corrects the stale "64-dim" note (256).
- LEARNING.md: new "How lazy skills are matched" section — keyword-first with
  opt-in, time-bounded semantic matching and keyword fallback; config field.
- MEMORY.md: point the embedder mechanics at internal/embedding (the shared
  package) instead of the now-thin memory/embedder.go aliases.

* Centralize embedding: shared default flows to every subsystem

Make the top-level `embedding` block the default for all three subsystems —
memory, sessions, AND skills — each with an optional override, instead of skills
being opt-in and sessions having no override slot.

  - Skills now inherit the shared default (was opt-in). To keep the per-turn
    hot path safe, an *inherited* skills config has its query timeout capped at
    2s (maxSkillsInheritedTimeout); an explicit skills.embedding is respected
    verbatim. Keyword fallback on miss/timeout/down-backend is unchanged.
  - Add an optional `sessions.embedding` override (new SessionsConfig block) so
    sessions are symmetric with memory/skills; resolved into
    ResolvedConfig.SessionEmbedding. main/telegram wire it through.
  - Memory inheritance unchanged (memory.embedding still wins for back-compat).

So a single top-level `embedding` block turns on semantic memory recall,
session_search, and skill matching at once; each subsystem can still point at a
different model/endpoint via its own block.

Docs: CONFIG.md inherit/override matrix rewritten (all inherit, all optional
override, skills timeout bound documented); SESSIONS.md and LEARNING.md updated
for inheritance + the new sessions override; anchors fixed. Tests updated to
assert every layer inherits, the skills timeout cap, and per-subsystem overrides
(including explicit skills timeout respected as-is).

* docker: point the shared embedding block at the local sidecar

The compose setup already ships a private llama.cpp embeddings sidecar
(`llama-embeddings`). Promote it from `memory.embedding` to the top-level
`embedding` block in both bundled configs so the single in-network endpoint now
powers all three consumers — memory, session_search, and skill matching — via
the centralized config (skills inherit with the bounded 2s per-turn timeout).

Updated docker-compose.yml, config.restricted.json, config.godmode.json,
Dockerfile.embeddings, .env.example, docker/README.md (heading + anchor), and
docs/DOCKER_COMPOSE_USER_GUIDE.md to describe the shared backend and the
fall-back-to-RandomProjections-everywhere disable path.

* docs: verification-protocol fixes — document new config surface

Axis 2.9 (documentation coverage) findings from the verification pass over the
embedding PR:

- CONFIG.md Skills table: add the `skills.embedding` field row (was only in the
  shared-backend section and LEARNING.md).
- CONFIG.md memory `embedding` row: correct the "unset = RandomProjections"
  description — memory now inherits the top-level shared default when unset
  (RP only when neither is set). Added cross-link to the shared section.

---------

Co-authored-by: Claude <noreply@anthropic.com>

Author: jkyberneees

cmd/odek/main.go

@@ -828,9 +828,10 @@ func run(args []string) error {
 	// Skills setup
 	var sm *skills.SkillManager
 	if resolved.Skills.Learn {
-		sm = skills.NewSkillManager(
+		sm = skills.NewSkillManagerWithEmbedding(
 			expandHome("~/.odek/skills"),
 			"./.odek/skills",
+			resolved.Skills.Embedding,
 		)
 	}
 
@@ -1640,8 +1641,6 @@ func continueCmd(args []string) error {
 	if err != nil {
 		return fmt.Errorf("session store: %w", err)
 	}
-	// Initialize semantic search index (non-fatal on failure).
-	_ = store.InitVectorIndex()
 
 	var sess *session.Session
 	if sessionID != "" {
@@ -1659,6 +1658,10 @@ func continueCmd(args []string) error {
 	// Resolve config (no CLI flags for continue — uses session's model)
 	resolved := config.LoadConfig(config.CLIFlags{Model: sess.Model})
 
+	// Initialize semantic search index (non-fatal on failure). Sessions use the
+	// shared embedding backend (or a sessions.embedding override).
+	_ = store.InitVectorIndex(resolved.SessionEmbedding)
+
 	// Auto-apply sandbox if session was sandboxed (even if config changed)
 	if sess.Sandbox && !resolved.Sandbox {
 		resolved.Sandbox = true
@@ -1668,9 +1671,10 @@ func continueCmd(args []string) error {
 	// Build tools
 	var sm *skills.SkillManager
 	if resolved.Skills.Learn {
-		sm = skills.NewSkillManager(
+		sm = skills.NewSkillManagerWithEmbedding(
 			expandHome("~/.odek/skills"),
 			"./.odek/skills",
+			resolved.Skills.Embedding,
 		)
 	}
 	tools := builtinTools(resolved.Dangerous, sm, nil, resolved.MaxConcurrency, resolved.APIKey, toolConfig{Transcription: resolved.Transcription, Vision: resolved.Vision, WebSearch: resolved.WebSearch}, store)

cmd/odek/mcp.go

@@ -66,9 +66,10 @@ Flags:
 	// Build skills manager (for skill tools)
 	var sm *skills.SkillManager
 	if resolved.Skills.Learn {
-		sm = skills.NewSkillManager(
+		sm = skills.NewSkillManagerWithEmbedding(
 			expandHome("~/.odek/skills"),
 			"./.odek/skills",
+			resolved.Skills.Embedding,
 		)
 	}
 

cmd/odek/repl.go

@@ -72,9 +72,10 @@ func replCmd(args []string) error {
 	// Build tools
 	var sm *skills.SkillManager
 	if resolved.Skills.Learn {
-		sm = skills.NewSkillManager(
+		sm = skills.NewSkillManagerWithEmbedding(
 			expandHome("~/.odek/skills"),
 			"./.odek/skills",
+			resolved.Skills.Embedding,
 		)
 	}
 	tools := builtinTools(resolved.Dangerous, sm, nil, resolved.MaxConcurrency, resolved.APIKey, toolConfig{WebSearch: resolved.WebSearch}, nil)

cmd/odek/serve.go

@@ -257,9 +257,10 @@ func serveOnListener(listener net.Listener, mux *http.ServeMux) error {
 func newServeAgent(resolved config.ResolvedConfig, system string, sendFn func(v any) error) (*odek.Agent, func() error, func(), *wsApprover, error) {
 	var sm *skills.SkillManager
 	if resolved.Skills.Learn {
-		sm = skills.NewSkillManager(
+		sm = skills.NewSkillManagerWithEmbedding(
 			expandHome("~/.odek/skills"),
 			"./.odek/skills",
+			resolved.Skills.Embedding,
 		)
 	}
 

cmd/odek/subagent.go

@@ -286,9 +286,10 @@ func subagentCmd(args []string) error {
 	// Build tools
 	var sm *skills.SkillManager
 	if resolved.Skills.Learn {
-		sm = skills.NewSkillManager(
+		sm = skills.NewSkillManagerWithEmbedding(
 			expandHome("~/.odek/skills"),
 			"./.odek/skills",
+			resolved.Skills.Embedding,
 		)
 	}
 	tools := builtinTools(resolved.Dangerous, sm, nil, resolved.MaxConcurrency, resolved.APIKey, toolConfig{WebSearch: resolved.WebSearch}, nil)

cmd/odek/telegram.go

@@ -143,8 +143,9 @@ func telegramCmd(args []string) error {
 		return err
 	}
 
-	// Initialize semantic search index.
-	if err := store.InitVectorIndex(); err != nil {
+	// Initialize semantic search index using the shared embedding backend (or a
+	// sessions.embedding override) so a single endpoint config powers it.
+	if err := store.InitVectorIndex(resolved.SessionEmbedding); err != nil {
 		fmt.Fprintf(os.Stderr, "odek telegram: vector index: %v\n", err)
 		// Non-fatal — search falls back to metadata-only.
 	}

docker/.env.example

@@ -65,15 +65,16 @@ GIT_COMMITTER_EMAIL=you@example.com
 # ODEK_SCHEDULES_CATCHUP=false                # run a missed fire once on startup
 # ODEK_SCHEDULES_ALLOW_TELEGRAM_MANAGEMENT=true  # set false to make /schedule read-only (CLI-manages)
 
-# ── Memory embeddings (llama.cpp sidecar; see docker/README.md) ──────────
+# ── Semantic embeddings (llama.cpp sidecar; see docker/README.md) ────────
 # The compose file runs a private llama.cpp server (the `llama-embeddings`
-# service) that gives the memory system real semantic embeddings. The model
-# (nomic-embed-text-v1.5) is BAKED INTO the image — no key, no first-run
-# download, nothing to set here. To change the model or quantization, rebuild
-# with build args, e.g.:
+# service) that gives odek real semantic embeddings. Both bundled configs set
+# the top-level `embedding` block to it, so memory, session_search, and skill
+# matching all run semantically. The model (nomic-embed-text-v1.5) is BAKED
+# INTO the image — no key, no first-run download, nothing to set here. To
+# change the model or quantization, rebuild with build args, e.g.:
 #   docker compose --profile restricted build \
 #     --build-arg EMBED_QUANT=Q8_0 llama-embeddings
-# Then bump memory.embedding.model in config.*.json if you switch models.
+# Then bump embedding.model in config.*.json if you switch models.
 
 # ── Web search (SearXNG sidecar; see docs/DOCKER_COMPOSE_USER_GUIDE.md) ──
 # The compose file runs a private SearXNG instance backing the `web_search`

docker/Dockerfile.embeddings

@@ -1,6 +1,7 @@
 # syntax=docker/dockerfile:1
 
-# llama.cpp embeddings sidecar — backs odek's memory.embedding (http provider).
+# llama.cpp embeddings sidecar — backs odek's shared `embedding` block (http
+# provider): memory, session_search, and skill matching.
 #
 # Builds llama-server from source pinned to the SAME llama.cpp release as the
 # main image (LLAMA_VERSION), and bundles a pinned nomic-embed-text-v1.5 GGUF

docker/README.md

@@ -188,40 +188,51 @@ sidecar** backing the `web_search` tool — no cloud search API, no keys.
 - To run **without** web search: comment out the `searxng` service (and the
   `depends_on: [searxng]` lines), and remove the `web_search` block from the configs.
 
-## Local memory embeddings (out of the box)
+## Local semantic embeddings (out of the box)
 
 The compose setup runs a **private [llama.cpp](https://github.com/ggerganov/llama.cpp)
-embeddings sidecar** backing odek's memory system — no cloud embeddings API, no keys.
+embeddings sidecar** backing odek's semantic features — no cloud embeddings API, no keys.
 
-Without it, memory similarity (episode recall, dedup, ranking, fact merge-on-write)
-runs on local bag-of-words vectors: fast, but purely lexical — *"fixed the auth bug"*
-and *"repaired login issue"* don't match. The sidecar swaps that for a real embedding
-model, so recall matches by **meaning**. See
-[`../docs/MEMORY.md`](../docs/MEMORY.md) → *Pluggable Embeddings* and
-[`../docs/CONFIG.md`](../docs/CONFIG.md) → `embedding`.
+Without it, similarity runs on local bag-of-words vectors: fast, but purely lexical —
+*"fixed the auth bug"* and *"repaired login issue"* don't match. The sidecar swaps that
+for a real embedding model, so everything matches by **meaning**. Both bundled configs
+set the **top-level `embedding` block** to the sidecar, so one endpoint powers all three
+consumers at once:
+
+- **Memory** — episode recall, dedup, ranking, fact merge-on-write.
+- **Sessions** — the `session_search` tool matches past sessions by meaning.
+- **Skills** — lazy skill matching (inherits the shared default, with the per-turn query
+  timeout bounded to 2s so the hot path stays fast).
+
+See [`../docs/MEMORY.md`](../docs/MEMORY.md) → *Pluggable Embeddings*,
+[`../docs/SESSIONS.md`](../docs/SESSIONS.md) → *Session Search*, and
+[`../docs/CONFIG.md`](../docs/CONFIG.md) → *Shared embedding backend*.
 
 - The `llama-embeddings` service co-starts with every profile and is reachable only by
-  the odek containers at `http://llama-embeddings:8080` (**no host port** — odek's memory
-  is the only consumer). Both bundled configs set `memory.embedding` to it.
+  the odek containers at `http://llama-embeddings:8080` (**no host port** — the odek
+  containers are the only consumers). Both bundled configs set the top-level `embedding`
+  block to it; memory, sessions, and skills inherit it.
 - The image **bundles `llama-server` (built from source, pinned to the same llama.cpp
   release as the main image) and `nomic-embed-text-v1.5`** (768-dim, ~84 MB at Q4_K_M)
   — so there's **no first-run model download** and no volume, mirroring the bundled
   whisper / MiniCPM-V models. The server runs `--embeddings --pooling mean` and exposes
   the OpenAI-compatible `/v1/embeddings` endpoint.
-- **Graceful by design:** if the sidecar is still loading or unreachable, recall just
-  degrades to "no context" and retries with a 30s backoff — the agent loop is never
-  blocked, and a wrong dedup never deletes an episode. Default behavior without the
-  service is local RandomProjections.
+- **Graceful by design:** if the sidecar is still loading or unreachable, each consumer
+  degrades safely — memory recall falls back to "no context", `session_search` to its
+  keyword tier, skill matching to the keyword matcher — all with a 30s/short-timeout
+  backoff, so the agent loop is never blocked and a wrong dedup never deletes an episode.
+  Default behavior without the service is local RandomProjections everywhere.
 - Want a higher-quality quantization? Rebuild with
   `--build-arg EMBED_QUANT=Q8_0` (available: `Q4_K_M` default, `Q5_K_M`, `Q6_K`, `Q8_0`,
   `f16`). To use a different model, override `EMBED_HF_REPO` / `EMBED_HF_REVISION` /
-  `EMBED_FILE` and update `memory.embedding.model` in the configs.
+  `EMBED_FILE` and update `embedding.model` in the configs.
 - To run **without** local embeddings: comment out the `llama-embeddings` service (and
-  the matching `depends_on` entries), and remove the `embedding` block from the configs
-  — memory falls back to RandomProjections automatically.
-- **Point `base_url` only at a server you trust:** every episode summary and fact entry
-  is sent there for embedding. Here it's the in-network sidecar, so nothing leaves the
-  compose network; if you repoint it at a cloud API, that text egresses.
+  the matching `depends_on` entries), and remove the top-level `embedding` block from the
+  configs — every subsystem falls back to RandomProjections automatically.
+- **Point `base_url` only at a server you trust:** session transcripts, episode summaries,
+  fact entries, and skill text are all sent there for embedding. Here it's the in-network
+  sidecar, so nothing leaves the compose network; if you repoint it at a cloud API, that
+  text egresses.
 
 ## Verify the profiles differ
 

docker/config.godmode.json

@@ -21,6 +21,13 @@
     "max_results": 10,
     "timeout_seconds": 15
   },
+  "embedding": {
+    "provider": "http",
+    "base_url": "http://llama-embeddings:8080/v1",
+    "model": "nomic-embed-text-v1.5",
+    "dims": 768,
+    "timeout_seconds": 10
+  },
   "memory": {
     "enabled": true,
     "facts_limit_user": 1500,
@@ -33,14 +40,7 @@
     "llm_extract": true,
     "llm_consolidate": true,
     "merge_threshold": 0.7,
-    "add_threshold": 0.3,
-    "embedding": {
-      "provider": "http",
-      "base_url": "http://llama-embeddings:8080/v1",
-      "model": "nomic-embed-text-v1.5",
-      "dims": 768,
-      "timeout_seconds": 10
-    }
+    "add_threshold": 0.3
   },
   "dangerous": {
     "action": "allow",