feat(docker): llama.cpp embeddings sidecar + local model for memory.embedding

Adds a private llama.cpp embeddings sidecar so the Docker setup uses real
semantic embeddings for memory (recall, dedup, ranking, fact merge-on-write)
out of the box — completing the consumer side of PR #27's pluggable embeddings.

- docker/Dockerfile.embeddings: builds llama-server from source pinned to the
  same llama.cpp release as the main image (b9549, Bookworm base → no ABI
  mismatch) and bundles a pinned nomic-embed-text-v1.5 GGUF (768-dim, ~84 MB
  Q4_K_M) at a fixed HF revision. No first-run download, no volume — mirrors the
  bundled whisper / MiniCPM-V models. Runs --embeddings --pooling mean, non-root.
- docker-compose.yml: llama-embeddings service on all 4 profiles, no host port
  (only the odek containers reach http://llama-embeddings:8080), /health
  healthcheck, wired into each odek service's depends_on.
- config.restricted.json / config.godmode.json: memory.embedding → the sidecar.
- README.md / .env.example / DOCKER_COMPOSE_USER_GUIDE.md: 'local memory
  embeddings out of the box' docs, quant/model override build args, disable path,
  and the egress-trust note.

Verified: image builds; container reaches /health in ~20s; /v1/embeddings
returns 768-dim vectors; cosine('fixed the auth bug','repaired the login
issue')=0.70 vs 0.35 for an unrelated phrase — the semantic match RP cannot do.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Author: jkyberneees

docker/.env.example

@@ -65,6 +65,16 @@ GIT_COMMITTER_EMAIL=you@example.com
 # ODEK_SCHEDULES_CATCHUP=false                # run a missed fire once on startup
 # ODEK_SCHEDULES_ALLOW_TELEGRAM_MANAGEMENT=true  # set false to make /schedule read-only (CLI-manages)
 
+# ── Memory embeddings (llama.cpp sidecar; see docker/README.md) ──────────
+# The compose file runs a private llama.cpp server (the `llama-embeddings`
+# service) that gives the memory system real semantic embeddings. The model
+# (nomic-embed-text-v1.5) is BAKED INTO the image — no key, no first-run
+# download, nothing to set here. To change the model or quantization, rebuild
+# with build args, e.g.:
+#   docker compose --profile restricted build \
+#     --build-arg EMBED_QUANT=Q8_0 llama-embeddings
+# Then bump memory.embedding.model in config.*.json if you switch models.
+
 # ── Web search (SearXNG sidecar; see docs/DOCKER_COMPOSE_USER_GUIDE.md) ──
 # The compose file runs a private SearXNG instance backing the `web_search`
 # tool. SearXNG reads this as server.secret_key at app load and it overrides

docker/Dockerfile.embeddings

@@ -0,0 +1,79 @@
+# syntax=docker/dockerfile:1
+
+# llama.cpp embeddings sidecar — backs odek's memory.embedding (http provider).
+#
+# Builds llama-server from source pinned to the SAME llama.cpp release as the
+# main image (LLAMA_VERSION), and bundles a pinned nomic-embed-text-v1.5 GGUF
+# so the service is fully offline after build — no first-run model download.
+# This mirrors how the main image bundles whisper.cpp + MiniCPM-V.
+#
+# nomic-embed-text-v1.5 is a 768-dim text embedding model (~84 MB at Q4_K_M).
+# odek sends raw summary/query text for both corpus and query, so the embedding
+# space stays symmetric even without nomic's optional task-instruction prefixes.
+#
+# Override the model with build args, e.g. a higher-quality quant:
+#   --build-arg EMBED_QUANT=Q8_0
+# Available quants in this repo: Q4_K_M (default) | Q5_K_M | Q6_K | Q8_0 | f16
+
+# ---- build stage: compile llama-server + fetch the model ----
+# Same Debian Bookworm base as the runtime stage so the binary links against the
+# exact glibc/libstdc++ present at runtime (no ABI mismatch — see the main
+# Dockerfile's llama.cpp stage for the rationale).
+FROM debian:bookworm-slim AS build
+ARG LLAMA_VERSION=b9549
+# Pin the embeddings model + git revision so the image is reproducible.
+ARG EMBED_HF_REPO=nomic-ai/nomic-embed-text-v1.5-GGUF
+ARG EMBED_HF_REVISION=0188c9bf409793f810680a5a431e7b899c46104c
+ARG EMBED_QUANT=Q4_K_M
+ARG EMBED_FILE=nomic-embed-text-v1.5.${EMBED_QUANT}.gguf
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+      git cmake make g++ curl ca-certificates \
+ && rm -rf /var/lib/apt/lists/*
+
+# BUILD_SHARED_LIBS=OFF statically links llama's internal libraries; GGML_OPENMP
+# OFF drops the libgomp dependency; LLAMA_CURL=OFF because the model is bundled
+# (no runtime download). Target llama-server (the OpenAI-compatible HTTP server)
+# instead of the main image's llama-mtmd-cli.
+RUN git clone --depth 1 --branch "${LLAMA_VERSION}" \
+      https://github.com/ggerganov/llama.cpp /llama \
+ && cmake -S /llama -B /llama/build \
+      -DCMAKE_BUILD_TYPE=Release \
+      -DBUILD_SHARED_LIBS=OFF \
+      -DGGML_NATIVE=OFF \
+      -DGGML_OPENMP=OFF \
+      -DLLAMA_CURL=OFF \
+      -DLLAMA_BUILD_TESTS=OFF \
+ && cmake --build /llama/build -j"$(nproc)" --target llama-server \
+ && install -m 755 /llama/build/bin/llama-server /usr/local/bin/llama-server \
+ && rm -rf /llama
+
+# Fetch the pinned GGUF into a fixed image path.
+RUN mkdir -p /models \
+ && curl -fSL --retry 5 --retry-delay 5 --retry-connrefused --retry-all-errors \
+      "https://huggingface.co/${EMBED_HF_REPO}/resolve/${EMBED_HF_REVISION}/${EMBED_FILE}" \
+      -o /models/embed.gguf
+
+# ---- runtime stage ----
+FROM debian:bookworm-slim
+# libstdc++6 for the C++ binary; curl only for the compose healthcheck;
+# ca-certificates is harmless and tiny. No outbound network is needed at runtime.
+RUN apt-get update && apt-get install -y --no-install-recommends \
+      libstdc++6 ca-certificates curl \
+ && rm -rf /var/lib/apt/lists/*
+COPY --from=build /usr/local/bin/llama-server /usr/local/bin/llama-server
+COPY --from=build /models/embed.gguf /models/embed.gguf
+
+# Run non-root (uid 1000), matching the main image's posture.
+RUN useradd -u 1000 -m llama
+USER llama
+EXPOSE 8080
+
+# --embeddings enables the /v1/embeddings endpoint; --pooling mean matches
+# nomic-embed-text; --ctx-size 2048 is the model's training context (episode
+# summaries are capped at ~1 KB, well within it). Bind all interfaces — only
+# the compose network can reach it (no host port is published).
+ENTRYPOINT ["llama-server", \
+            "--host", "0.0.0.0", "--port", "8080", \
+            "--model", "/models/embed.gguf", \
+            "--embeddings", "--pooling", "mean", "--ctx-size", "2048"]

docker/README.md

@@ -21,7 +21,8 @@ For the full walkthrough, threat model, and tuning, see
 ```
 docker/
 ├── Dockerfile               # multi-stage build of the odek binary
-├── docker-compose.yml       # 4 services across 4 profiles
+├── Dockerfile.embeddings    # llama.cpp embeddings sidecar (bundled GGUF)
+├── docker-compose.yml       # odek (4 profiles) + searxng + llama-embeddings
 ├── config.restricted.json   # Restricted permission policy
 ├── config.godmode.json      # Godmode (YOLO) permission policy
 ├── .env.example             # copy to .env, add your API key
@@ -187,6 +188,41 @@ sidecar** backing the `web_search` tool — no cloud search API, no keys.
 - To run **without** web search: comment out the `searxng` service (and the
   `depends_on: [searxng]` lines), and remove the `web_search` block from the configs.
 
+## Local memory embeddings (out of the box)
+
+The compose setup runs a **private [llama.cpp](https://github.com/ggerganov/llama.cpp)
+embeddings sidecar** backing odek's memory system — no cloud embeddings API, no keys.
+
+Without it, memory similarity (episode recall, dedup, ranking, fact merge-on-write)
+runs on local bag-of-words vectors: fast, but purely lexical — *"fixed the auth bug"*
+and *"repaired login issue"* don't match. The sidecar swaps that for a real embedding
+model, so recall matches by **meaning**. See
+[`../docs/MEMORY.md`](../docs/MEMORY.md) → *Pluggable Embeddings* and
+[`../docs/CONFIG.md`](../docs/CONFIG.md) → `embedding`.
+
+- The `llama-embeddings` service co-starts with every profile and is reachable only by
+  the odek containers at `http://llama-embeddings:8080` (**no host port** — odek's memory
+  is the only consumer). Both bundled configs set `memory.embedding` to it.
+- The image **bundles `llama-server` (built from source, pinned to the same llama.cpp
+  release as the main image) and `nomic-embed-text-v1.5`** (768-dim, ~84 MB at Q4_K_M)
+  — so there's **no first-run model download** and no volume, mirroring the bundled
+  whisper / MiniCPM-V models. The server runs `--embeddings --pooling mean` and exposes
+  the OpenAI-compatible `/v1/embeddings` endpoint.
+- **Graceful by design:** if the sidecar is still loading or unreachable, recall just
+  degrades to "no context" and retries with a 30s backoff — the agent loop is never
+  blocked, and a wrong dedup never deletes an episode. Default behavior without the
+  service is local RandomProjections.
+- Want a higher-quality quantization? Rebuild with
+  `--build-arg EMBED_QUANT=Q8_0` (available: `Q4_K_M` default, `Q5_K_M`, `Q6_K`, `Q8_0`,
+  `f16`). To use a different model, override `EMBED_HF_REPO` / `EMBED_HF_REVISION` /
+  `EMBED_FILE` and update `memory.embedding.model` in the configs.
+- To run **without** local embeddings: comment out the `llama-embeddings` service (and
+  the matching `depends_on` entries), and remove the `embedding` block from the configs
+  — memory falls back to RandomProjections automatically.
+- **Point `base_url` only at a server you trust:** every episode summary and fact entry
+  is sent there for embedding. Here it's the in-network sidecar, so nothing leaves the
+  compose network; if you repoint it at a cloud API, that text egresses.
+
 ## Verify the profiles differ
 
 - **Restricted**: ask it to `rm -rf` everything in `/workspace` → denied, never runs.

docker/config.godmode.json

@@ -32,7 +32,12 @@
     "llm_extract": true,
     "llm_consolidate": true,
     "merge_threshold": 0.7,
-    "add_threshold": 0.3
+    "add_threshold": 0.3,
+    "embedding": {
+      "provider": "http",
+      "base_url": "http://llama-embeddings:8080/v1",
+      "model": "nomic-embed-text-v1.5"
+    }
   },
   "dangerous": {
     "action": "allow",

docker/config.restricted.json

@@ -32,7 +32,12 @@
     "llm_extract": true,
     "llm_consolidate": true,
     "merge_threshold": 0.7,
-    "add_threshold": 0.3
+    "add_threshold": 0.3,
+    "embedding": {
+      "provider": "http",
+      "base_url": "http://llama-embeddings:8080/v1",
+      "model": "nomic-embed-text-v1.5"
+    }
   },
   "dangerous": {
     "non_interactive": "deny",

docker/docker-compose.yml

@@ -30,7 +30,7 @@ services:
       - ./workspace:/workspace
       - ./config.restricted.json:/home/odek/.odek/config.json:ro
     restart: "no"
-    depends_on: [searxng]
+    depends_on: [searxng, llama-embeddings]
 
   # ── Godmode (all permissions) — non-interactive, disposable container ──
   odek-godmode:
@@ -50,7 +50,7 @@ services:
       - ./workspace:/workspace
       - ./config.godmode.json:/home/odek/.odek/config.json:ro
     restart: "no"
-    depends_on: [searxng]
+    depends_on: [searxng, llama-embeddings]
 
   # ── Telegram bot — Restricted (approvals via inline keyboards) ──
   # State (sessions, skills, telegram.pid) lives in the local ./.odek folder —
@@ -74,7 +74,7 @@ services:
       - ./.odek:/home/odek/.odek
       - ./config.restricted.json:/home/odek/.odek/config.json:ro
     restart: unless-stopped
-    depends_on: [searxng]
+    depends_on: [searxng, llama-embeddings]
 
   # ── Telegram bot — Godmode (no prompts; unrestricted) ──
   # Same ./.odek state folder; config.godmode.json is layered on top of
@@ -93,7 +93,35 @@ services:
       - ./.odek:/home/odek/.odek
       - ./config.godmode.json:/home/odek/.odek/config.json:ro
     restart: unless-stopped
-    depends_on: [searxng]
+    depends_on: [searxng, llama-embeddings]
+
+  # ── llama.cpp embeddings — local backend for memory.embedding ──
+  # Co-starts with every odek profile so the memory system gets real semantic
+  # embeddings (recall, dedup, ranking, fact merge) out of the box — no cloud
+  # embeddings API, no keys. Reachable only by the odek containers over the
+  # compose network at http://llama-embeddings:8080 — NO host port is published
+  # (odek's memory system is the only consumer). The GGUF is baked into the
+  # image (see docker/Dockerfile.embeddings), so there is no first-run download
+  # and no volume. Both bundled configs point memory.embedding.base_url here.
+  # To run without local embeddings, comment this service and the matching
+  # depends_on entries, and drop the "embedding" block from the configs (memory
+  # then falls back to local RandomProjections — lexical, zero-cost).
+  llama-embeddings:
+    profiles: ["restricted", "godmode", "telegram-restricted", "telegram-godmode"]
+    build:
+      context: ..
+      dockerfile: docker/Dockerfile.embeddings
+    image: odek-embeddings:local
+    # Health is observability only — odek tolerates the server being briefly
+    # unavailable (recall degrades to "no context" and rebuilds back off 30s),
+    # so startup is not gated on it. The model is bundled, so load is quick.
+    healthcheck:
+      test: ["CMD", "curl", "-fsS", "http://127.0.0.1:8080/health"]
+      interval: 10s
+      timeout: 5s
+      retries: 30
+      start_period: 60s
+    restart: unless-stopped
 
   # ── SearXNG — self-hosted metasearch backing the `web_search` tool ──
   # Co-starts with every odek profile so web search works out of the box.

docs/DOCKER_COMPOSE_USER_GUIDE.md

@@ -138,9 +138,17 @@ The compose file also runs a private **SearXNG** metasearch sidecar that backs t
 It co-starts with every profile, is reachable only by the odek containers at
 `http://searxng:8080` (no host port), and needs only `SEARXNG_SECRET` set above —
 no Redis/Valkey. To disable web search, comment the `searxng` service and the
-`depends_on: [searxng]` lines in the compose file and drop the `web_search` block
+`depends_on` entries in the compose file and drop the `web_search` block
 from the config files.
 
+A second sidecar, **`llama-embeddings`** (a llama.cpp server with a bundled
+`nomic-embed-text-v1.5` GGUF), gives the memory system real semantic embeddings —
+see [docker/README.md](../docker/README.md#local-memory-embeddings-out-of-the-box).
+It also co-starts on every profile, is reachable only at `http://llama-embeddings:8080`
+(no host port, no key, no first-run download), and both bundled configs point
+`memory.embedding` at it. Disable the same way: comment the service + its `depends_on`
+entries and drop the `embedding` block (memory falls back to local RandomProjections).
+
 ---
 
 ## 5. Permission policy files