feat: support LiteLLM

Author: georgeh0

README.md

@@ -7,58 +7,124 @@ An MCP (Model Context Protocol) server for indexing and querying codebases using
 - **Semantic Code Search**: Find relevant code using natural language queries
 - **Incremental Indexing**: Only re-indexes changed files for fast updates
 - **Multi-Language Support**: Python, JavaScript/TypeScript, Rust, Go
-- **Vector Embeddings**: Uses sentence-transformers for semantic similarity
+- **Flexible Embeddings**: Local SentenceTransformers (default) or 100+ cloud providers via [LiteLLM](https://docs.litellm.ai/docs/embedding/supported_embedding)
 - **SQLite Storage**: Portable, no external database required
 
-## Installation
+## Usage with Claude Code
+
+No installation needed — `uvx` runs it directly.
+
+### Default (Local Embeddings)
+
+Uses a local SentenceTransformers model (`sentence-transformers/all-MiniLM-L6-v2`). No API key required:
 
 ```bash
-pip install cocoindex-code
+claude mcp add cocoindex-code -- uvx cocoindex-code
 ```
 
-Or with uv:
+### OpenAI
 
 ```bash
-uv pip install cocoindex-code
+claude mcp add cocoindex-code \
+  -e COCOINDEX_CODE_EMBEDDING_MODEL=text-embedding-3-small \
+  -e OPENAI_API_KEY=your-api-key \
+  -- uvx cocoindex-code
 ```
 
-## Usage with Claude Code
+### Azure OpenAI
 
-Add to your Claude Code MCP configuration (`.claude/mcp_config.json`):
+```bash
+claude mcp add cocoindex-code \
+  -e COCOINDEX_CODE_EMBEDDING_MODEL=azure/your-deployment-name \
+  -e AZURE_API_KEY=your-api-key \
+  -e AZURE_API_BASE=https://your-resource.openai.azure.com \
+  -e AZURE_API_VERSION=2024-06-01 \
+  -- uvx cocoindex-code
+```
 
-```json
-{
-  "mcpServers": {
-    "cocoindex-code": {
-      "command": "cocoindex-code",
-      "env": {
-        "COCOINDEX_CODE_ROOT_PATH": "/path/to/your/codebase"
-      }
-    }
-  }
-}
+### Gemini
+
+```bash
+claude mcp add cocoindex-code \
+  -e COCOINDEX_CODE_EMBEDDING_MODEL=gemini/text-embedding-004 \
+  -e GEMINI_API_KEY=your-api-key \
+  -- uvx cocoindex-code
 ```
 
-Or without explicit path (auto-discovers from current directory):
+### Mistral
 
-```json
-{
-  "mcpServers": {
-    "cocoindex-code": {
-      "command": "cocoindex-code"
-    }
-  }
-}
+```bash
+claude mcp add cocoindex-code \
+  -e COCOINDEX_CODE_EMBEDDING_MODEL=mistral/mistral-embed \
+  -e MISTRAL_API_KEY=your-api-key \
+  -- uvx cocoindex-code
 ```
 
-## Configuration
+### Voyage (Code-Optimized)
+
+```bash
+claude mcp add cocoindex-code \
+  -e COCOINDEX_CODE_EMBEDDING_MODEL=voyage/voyage-code-3 \
+  -e VOYAGE_API_KEY=your-api-key \
+  -- uvx cocoindex-code
+```
+
+### Cohere
+
+```bash
+claude mcp add cocoindex-code \
+  -e COCOINDEX_CODE_EMBEDDING_MODEL=cohere/embed-english-v3.0 \
+  -e COHERE_API_KEY=your-api-key \
+  -- uvx cocoindex-code
+```
+
+### AWS Bedrock
+
+```bash
+claude mcp add cocoindex-code \
+  -e COCOINDEX_CODE_EMBEDDING_MODEL=bedrock/amazon.titan-embed-text-v2:0 \
+  -e AWS_ACCESS_KEY_ID=your-access-key \
+  -e AWS_SECRET_ACCESS_KEY=your-secret-key \
+  -e AWS_REGION_NAME=us-east-1 \
+  -- uvx cocoindex-code
+```
+
+### Ollama (Local)
+
+```bash
+claude mcp add cocoindex-code \
+  -e COCOINDEX_CODE_EMBEDDING_MODEL=ollama/nomic-embed-text \
+  -- uvx cocoindex-code
+```
+
+Set `OLLAMA_API_BASE` if your Ollama server is not at `http://localhost:11434`.
 
-Environment variables:
+### Nebius
+
+```bash
+claude mcp add cocoindex-code \
+  -e COCOINDEX_CODE_EMBEDDING_MODEL=nebius/BAAI/bge-en-icl \
+  -e NEBIUS_API_KEY=your-api-key \
+  -- uvx cocoindex-code
+```
+
+### Other Providers
+
+Any model supported by LiteLLM works — see the [full list of embedding providers](https://docs.litellm.ai/docs/embedding/supported_embedding).
+
+## Configuration
 
 | Variable | Description | Default |
 |----------|-------------|---------|
 | `COCOINDEX_CODE_ROOT_PATH` | Root path of the codebase | Auto-discovered (see below) |
-| `COCOINDEX_CODE_EMBEDDING_MODEL` | Embedding model to use | `sentence-transformers/all-MiniLM-L6-v2` |
+| `COCOINDEX_CODE_EMBEDDING_MODEL` | Embedding model (see below) | `sbert/sentence-transformers/all-MiniLM-L6-v2` |
+
+### Embedding Model
+
+The `COCOINDEX_CODE_EMBEDDING_MODEL` variable uses a prefix to select the embedding backend:
+
+- **`sbert/`** prefix — uses [SentenceTransformers](https://www.sbert.net/) (runs locally, no API key needed). Example: `sbert/sentence-transformers/all-MiniLM-L6-v2`
+- **Otherwise** — uses [LiteLLM](https://docs.litellm.ai/docs/embedding/supported_embedding) (supports 100+ providers). Example: `text-embedding-3-small`
 
 ### Root Path Discovery
 

pyproject.toml

@@ -23,7 +23,7 @@ classifiers = [
 
 dependencies = [
     "mcp>=1.0.0",
-    "cocoindex==1.0.0a10",
+    "cocoindex[litellm]>=1.0.0a12",
     "sentence-transformers>=2.2.0",
     "sqlite-vec>=0.1.0",
     "pydantic>=2.0.0",

src/cocoindex_code/config.py

@@ -62,9 +62,10 @@ def from_env(cls) -> "Config":
             root = _discover_codebase_root()
 
         # Get embedding model
+        # Prefix "sbert/" for SentenceTransformers models, otherwise LiteLLM.
         embedding_model = os.environ.get(
             "COCOINDEX_CODE_EMBEDDING_MODEL",
-            "sentence-transformers/all-MiniLM-L6-v2",
+            "sbert/sentence-transformers/all-MiniLM-L6-v2",
         )
 
         # Index directory is always under the root

src/cocoindex_code/shared.py

@@ -1,21 +1,36 @@
 """Shared resources for CocoIndex Code."""
 
+from __future__ import annotations
+
 from collections.abc import Iterator
 from dataclasses import dataclass
-from typing import Annotated
+from typing import TYPE_CHECKING, Annotated
 
 import cocoindex as coco
 from cocoindex.connectors import sqlite
-from cocoindex.ops.sentence_transformers import SentenceTransformerEmbedder
 from numpy.typing import NDArray
 
+if TYPE_CHECKING:
+    from cocoindex.ops.litellm import LiteLLMEmbedder
+    from cocoindex.ops.sentence_transformers import SentenceTransformerEmbedder
+
 from .config import Config
 
+SBERT_PREFIX = "sbert/"
+
 # Load configuration at module level
 config = Config.from_env()
 
-# Initialize embedder at module level
-embedder = SentenceTransformerEmbedder(config.embedding_model)
+# Initialize embedder at module level based on model prefix
+embedder: SentenceTransformerEmbedder | LiteLLMEmbedder
+if config.embedding_model.startswith(SBERT_PREFIX):
+    from cocoindex.ops.sentence_transformers import SentenceTransformerEmbedder
+
+    embedder = SentenceTransformerEmbedder(config.embedding_model[len(SBERT_PREFIX) :])
+else:
+    from cocoindex.ops.litellm import LiteLLMEmbedder
+
+    embedder = LiteLLMEmbedder(config.embedding_model)
 
 # Context key for SQLite database (connection managed in lifespan)
 SQLITE_DB = coco.ContextKey[sqlite.SqliteDatabase]("sqlite_db")