feat: unified Docker workspace mount with supervised daemon (#135)

* feat: unified Docker workspace mount with supervised daemon

Reshape the Docker experience around a single bind mount and a single
named volume. Global settings live on the host under
$HOME/.cocoindex_code/ (visible and editable); index data and the model
cache persist in one cocoindex-data volume; daemon runtime state stays
on the container's native filesystem.

CLI and MCP output now show host-side paths via a bidirectional
COCOINDEX_CODE_HOST_PATH_MAPPING translator. A shell wrapper that
forwards $PWD (COCOINDEX_CODE_HOST_CWD) lets ccc work from any project
subdirectory on the host.

The daemon tolerates a missing global_settings.yml (starts in
no-settings mode) so ccc init's interactive picker works in Docker on
first run. A supervisor restart loop in the entrypoint, driven by a new
COCOINDEX_CODE_DAEMON_SUPERVISED contract, makes settings-change
auto-restart safe — editing global_settings.yml triggers an in-place
daemon respawn without taking the container down.

Linux ownership alignment via PUID/PGID, gosu privilege drop, and a
coco user baked into the image. Release workflow now publishes to both
Docker Hub (cocoindex/cocoindex-code) and GHCR
(ghcr.io/cocoindex-io/cocoindex-code).

Also:
- Merge cocoindex-db and cocoindex-model-cache into a single volume
- find_parent_with_marker requires .cocoindex_code/settings.yml, so a
  workspace-root global-only dir doesn't trigger nested-init warnings
- New pytest marker `docker_e2e` gates the Docker-backed E2E suite
  (excluded from default pytest runs)

* fix: mypy on Windows for POSIX-only os.getuid/getgid calls

Author: georgeh0

.dockerignore

@@ -0,0 +1,31 @@
+# Git + VCS
+.git/
+.gitignore
+
+# Python
+__pycache__/
+*.pyc
+*.pyo
+.mypy_cache/
+.pytest_cache/
+.ruff_cache/
+.venv/
+venv/
+dist/
+build/
+*.egg-info/
+
+# IDE / editor
+.idea/
+.vscode/
+*.swp
+.DS_Store
+
+# Project artifacts
+.cocoindex_code/
+specs/
+tests/e2e_docker_fixtures/  # test fixtures — not needed in image builds
+
+# Docs (the image ships no docs)
+*.md
+!README.md

.github/workflows/release.yml

@@ -105,3 +105,47 @@ jobs:
           gh release upload
           '${{ github.ref_name }}' dist/*
           --repo '${{ github.repository }}'
+
+  publish-docker:
+    name: Build & push Docker image to Docker Hub and GHCR
+    if: github.event_name == 'release'
+    needs:
+      - publish-to-pypi
+    runs-on: ubuntu-latest
+    environment:
+      name: docker-hub
+      url: https://hub.docker.com/r/cocoindex/cocoindex-code
+    permissions:
+      contents: read
+      packages: write
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+
+      - name: Log in to Docker Hub
+        uses: docker/login-action@v3
+        with:
+          registry: docker.io
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+
+      - name: Log in to GHCR
+        uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Build and push to both registries
+        uses: docker/build-push-action@v5
+        with:
+          context: .
+          file: docker/Dockerfile
+          push: true
+          tags: |
+            cocoindex/cocoindex-code:latest
+            cocoindex/cocoindex-code:${{ github.ref_name }}
+            ghcr.io/cocoindex-io/cocoindex-code:latest
+            ghcr.io/cocoindex-io/cocoindex-code:${{ github.ref_name }}

.gitignore

@@ -46,3 +46,7 @@ src/cocoindex_code/_version.py
 
 # CocoIndex Code (ccc)
 /.cocoindex_code/
+
+# Docker E2E fixtures contain a `lib/` dir that the generic Python rule above
+# would ignore — keep it tracked.
+!tests/e2e_docker_fixtures/**

README.md

@@ -198,33 +198,79 @@ The recommended approach is a **persistent container**: start it once, and use
 `docker exec` to run CLI commands or connect MCP sessions to it. The daemon
 inside stays warm across sessions, so the embedding model is loaded only once.
 
-### Step 1 — Start the container
+### Quick start — `docker compose up -d`
+
+Grab [`docker/docker-compose.yml`](./docker/docker-compose.yml) from this repo and run:
+
+```bash
+# macOS / Windows
+docker compose up -d
+
+# Linux (aligns file ownership on bind-mounted paths with your host user)
+PUID=$(id -u) PGID=$(id -g) docker compose up -d
+```
+
+By default your home directory is mounted into the container (set
+`COCOINDEX_HOST_WORKSPACE` to narrow this to a specific code folder). Index
+data and the embedding model cache persist in a Docker volume across
+restarts. Your global settings file at `$HOME/.cocoindex_code/global_settings.yml`
+is visible and editable on the host; edits take effect on your next `ccc` command.
+
+> **GHCR:** to pull from GitHub Container Registry instead of Docker Hub,
+> change the `image:` line in your copy of `docker-compose.yml` to
+> `ghcr.io/cocoindex-io/cocoindex-code:latest`.
+
+### Or: `docker run`
+
+<details>
+<summary>Docker Desktop (macOS / Windows)</summary>
 
 ```bash
 docker run -d --name cocoindex-code \
-  --volume "$(pwd):/workspace" \
-  --volume cocoindex-db:/db \
-  --volume cocoindex-model-cache:/root/.cache \
-  ghcr.io/cocoindex-io/cocoindex-code:latest
+  --volume "$HOME:/workspace" \
+  --volume cocoindex-data:/var/cocoindex \
+  -e COCOINDEX_CODE_HOST_PATH_MAPPING="/workspace=$HOME" \
+  cocoindex/cocoindex-code:latest
 ```
+</details>
 
-- `/workspace` — mount your project root here
-- `cocoindex-db` — index databases live inside the container (fast native I/O, no cross-OS volume issues)
-- `cocoindex-model-cache` — persists the embedding model across image upgrades
+<details>
+<summary>Linux (with <code>PUID</code>/<code>PGID</code>)</summary>
 
-### Step 2 — Index your codebase
+```bash
+docker run -d --name cocoindex-code \
+  -e PUID=$(id -u) -e PGID=$(id -g) \
+  --volume "$HOME:/workspace" \
+  --volume cocoindex-data:/var/cocoindex \
+  -e COCOINDEX_CODE_HOST_PATH_MAPPING="/workspace=$HOME" \
+  cocoindex/cocoindex-code:latest
+```
+</details>
+
+### Shell wrapper for `ccc` commands
+
+Paste this into `~/.bashrc` / `~/.zshrc` so `ccc` feels native on the host
+and picks up the right project based on your current directory:
 
 ```bash
-docker exec -it cocoindex-code ccc index
+ccc() {
+  docker exec -it -e COCOINDEX_CODE_HOST_CWD="$PWD" cocoindex-code ccc "$@"
+}
 ```
 
-### Step 3 — Connect your coding agent
+Now `cd` into any project under your workspace and run `ccc init`, `ccc index`,
+`ccc search ...`, `ccc status`, etc. — it just works.
+
+### Connect your coding agent
 
 <details>
 <summary>Claude Code</summary>
 
+Register MCP from inside the target project so `$PWD` points there:
+
 ```bash
-claude mcp add cocoindex-code -- docker exec -i cocoindex-code ccc mcp
+claude mcp add cocoindex-code -- docker exec -i \
+  -e COCOINDEX_CODE_HOST_CWD="$PWD" cocoindex-code ccc mcp
 ```
 
 Or via `.mcp.json`:
@@ -235,40 +281,50 @@ Or via `.mcp.json`:
     "cocoindex-code": {
       "type": "stdio",
       "command": "docker",
-      "args": ["exec", "-i", "cocoindex-code", "ccc", "mcp"]
+      "args": [
+        "exec",
+        "-i",
+        "-e",
+        "COCOINDEX_CODE_HOST_CWD=${PWD}",
+        "cocoindex-code",
+        "ccc",
+        "mcp"
+      ]
     }
   }
 }
 ```
+
+> Note: use `-i` (not `-it`). The `-t` flag allocates a terminal, which
+> interferes with MCP's JSON messaging over stdin/stdout — only add it for
+> interactive `ccc` commands like `ccc init`.
 </details>
 
 <details>
 <summary>Codex</summary>
 
 ```bash
-codex mcp add cocoindex-code -- docker exec -i cocoindex-code ccc mcp
+codex mcp add cocoindex-code -- docker exec -i \
+  -e COCOINDEX_CODE_HOST_CWD="$PWD" cocoindex-code ccc mcp
 ```
 </details>
 
-### CLI usage inside the container
+### Upgrading from an older image
 
-All `ccc` commands work via `docker exec`:
+Earlier images used separate `cocoindex-db` and `cocoindex-model-cache`
+volumes; the current image consolidates them into a single `cocoindex-data`
+volume. Before pulling the new image, drop the old container and volumes —
+indexes rebuild on your next `ccc index`, and the embedding model is
+re-populated automatically on first start:
 
 ```bash
-docker exec -it cocoindex-code ccc index
-docker exec -it cocoindex-code ccc search "authentication logic"
-docker exec -it cocoindex-code ccc status
-```
-
-Or set an alias on your host so it feels native:
-
-```bash
-alias ccc='docker exec -it cocoindex-code ccc'
+docker rm -f cocoindex-code
+docker volume rm cocoindex-db cocoindex-model-cache
 ```
 
 ### Configuration via environment variables
 
-Pass configuration to `docker run` with `-e`:
+Pass configuration to `docker run` / compose with `-e`:
 
 ```bash
 # Extra extensions (e.g. Typesafe Config, SBT build files)
@@ -281,6 +337,10 @@ Pass configuration to `docker run` with `-e`:
 -e VOYAGE_API_KEY=your-key
 ```
 
+> **Security note:** mounting `$HOME` gives the container read/write access
+> to everything under it. If that's too broad, bind-mount a narrower
+> directory instead (`COCOINDEX_HOST_WORKSPACE=/path/to/code`).
+
 ### Build the image locally
 
 ```bash
@@ -315,6 +375,8 @@ envs:                                                # extra environment variabl
 
 > **Note:** The daemon inherits your shell environment. If an API key (e.g. `OPENAI_API_KEY`) is already set as an environment variable, you don't need to duplicate it in `envs`. The `envs` field is only for values that aren't in your environment.
 
+> **Custom location:** set `COCOINDEX_CODE_DIR` to place `global_settings.yml` somewhere other than `~/.cocoindex_code/` — useful if you want the file to live alongside your projects (e.g. on a synced folder).
+
 ### Project Settings (`<project>/.cocoindex_code/settings.yml`)
 
 Per-project. Controls which files to index.

docker/Dockerfile

@@ -3,59 +3,79 @@
 # Alpine / musl-libc would require building from source.
 FROM python:3.12-slim AS builder
 
-# Install uv via pip — avoids a ghcr.io pull during build
 RUN pip install --quiet uv
 
 WORKDIR /build
 
+# Default: install the released cocoindex-code from PyPI (release flow).
+# Tests/local dev override with:
+#   --build-arg CCC_INSTALL_SPEC=/ccc-src[default]
+# which installs from the copied-in source tree instead. The COPY always runs;
+# with .dockerignore trimming build artifacts it adds ~nothing.
+ARG CCC_INSTALL_SPEC="cocoindex-code[default]"
+COPY . /ccc-src
+
 RUN uv pip install --system --prerelease=allow \
     "cocoindex>=1.0.0a33" \
-    "cocoindex-code[default]"
+    "${CCC_INSTALL_SPEC}"
 
 # ─── Stage 2: pre-bake the default embedding model ────────────────────────────
-# Bakes Snowflake/snowflake-arctic-embed-xs into the image so cold container
-# starts don't trigger a download. Skip this stage (--target builder) if
-# you always supply a global_settings.yml pointing at a non-local model.
+# Bakes Snowflake/snowflake-arctic-embed-xs into the merged data directory at
+# /var/cocoindex/cache/..., so on first run Docker's volume copy-up populates
+# the cocoindex-data volume with the model — no network fetch needed.
 FROM builder AS model_cache
 
-RUN python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('Snowflake/snowflake-arctic-embed-xs'); print('Model cached.')"
+ENV HF_HOME=/var/cocoindex/cache/huggingface \
+    SENTENCE_TRANSFORMERS_HOME=/var/cocoindex/cache/sentence-transformers
+
+RUN mkdir -p /var/cocoindex/cache/huggingface /var/cocoindex/cache/sentence-transformers \
+    && python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('Snowflake/snowflake-arctic-embed-xs'); print('Model cached.')"
 
 # ─── Stage 3: runtime ─────────────────────────────────────────────────────────
 FROM python:3.12-slim AS runtime
 
-# Copy installed packages and cached model from previous stages
+# gosu for privilege-drop (PUID/PGID pattern); create non-root coco user.
+RUN apt-get update \
+    && apt-get install -y --no-install-recommends gosu \
+    && rm -rf /var/lib/apt/lists/* \
+    && groupadd -g 1000 coco \
+    && useradd -u 1000 -g 1000 -m coco
+
+# Copy installed packages + pre-baked model from previous stages.
 COPY --from=model_cache /usr/local/lib/python3.12 /usr/local/lib/python3.12
 COPY --from=model_cache /usr/local/bin/cocoindex-code /usr/local/bin/cocoindex-code
 COPY --from=model_cache /usr/local/bin/ccc /usr/local/bin/ccc
-COPY --from=model_cache /root/.cache /root/.cache
+COPY --from=model_cache /var/cocoindex/cache /var/cocoindex/cache
+
+# Pre-create writable paths so the entrypoint's chown (under PUID) works even on
+# a fresh container, and so the default root-uid path has them in place.
+RUN mkdir -p /var/cocoindex/db /var/run/cocoindex_code \
+    && chown -R coco:coco /var/cocoindex /var/run/cocoindex_code
 
-# The codebase is mounted at runtime — nothing project-specific lives here.
 WORKDIR /workspace
 
 # ── Runtime defaults (all overridable via -e / --env) ─────────────────────────
+#
+# COCOINDEX_CODE_DIR — holds global_settings.yml on the bind mount so users can
+#   edit it directly on the host.
+# COCOINDEX_CODE_RUNTIME_DIR — keeps daemon.sock/pid/log on the container's
+#   native filesystem (AF_UNIX sockets on bind mounts are unreliable on
+#   Docker Desktop, and /var/run is the standard spot for ephemeral runtime
+#   state — wiped on container recreate, no stale-socket risk).
+# COCOINDEX_CODE_DB_PATH_MAPPING — keeps the indexer's LMDB + SQLite databases
+#   on the native filesystem for speed and correctness.
+# HF_HOME / SENTENCE_TRANSFORMERS_HOME — direct the model cache at the path
+#   the cocoindex-data volume mounts over.
+ENV COCOINDEX_CODE_DIR=/workspace/.cocoindex_code \
+    COCOINDEX_CODE_RUNTIME_DIR=/var/run/cocoindex_code \
+    COCOINDEX_CODE_DB_PATH_MAPPING=/workspace=/var/cocoindex/db \
+    COCOINDEX_CODE_DAEMON_SUPERVISED=1 \
+    HF_HOME=/var/cocoindex/cache/huggingface \
+    SENTENCE_TRANSFORMERS_HOME=/var/cocoindex/cache/sentence-transformers
 
-# Map index databases into the container's native filesystem so SQLite avoids
-# the slow/unreliable cross-OS volume layer (especially on macOS / Windows).
-# Format: /source=/target  — databases for projects under /workspace go to /db.
-ENV COCOINDEX_CODE_DB_PATH_MAPPING=/workspace=/db
-
-# Additional extensions: add project-specific extras at runtime, e.g.:
-#   -e COCOINDEX_CODE_EXTRA_EXTENSIONS="conf,sbt"
-# See README for ext:lang syntax to map extensions to existing parsers.
-
-# Exclude patterns: override at runtime for your build system, e.g.:
-#   -e COCOINDEX_CODE_EXCLUDE_PATTERNS='["**/target/**","**/node_modules/**"]'
-
-# Embedding model: defaults to local sentence-transformers
-# (Snowflake/snowflake-arctic-embed-xs, pre-baked above).
-# To use a different model, pre-mount a global_settings.yml into
-# ~/.cocoindex_code/ before the container starts, e.g.
-#   -v /path/to/global_settings.yml:/root/.cocoindex_code/global_settings.yml
+# Set COCOINDEX_CODE_HOST_PATH_MAPPING at run time — it depends on the host path
+# the user bind-mounts to /workspace and can't be baked into the image.
 
-# ── Persistent daemon entrypoint ──────────────────────────────────────────────
-# Initializes user settings on first start, then runs the daemon in the
-# foreground so the container stays alive.
-# Use `docker exec` to invoke the CLI or start an MCP session — see README.
 COPY docker/entrypoint.sh /entrypoint.sh
 RUN chmod +x /entrypoint.sh
 ENTRYPOINT ["/entrypoint.sh"]