Skip to content

Commit 4b57ffd

Browse files
GoodbyePlanetclaude
andcommitted
chore: Add CLAUDE.md for Claude Code guidance
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent cbf5b22 commit 4b57ffd

1 file changed

Lines changed: 85 additions & 0 deletions

File tree

CLAUDE.md

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## What This Project Is
6+
7+
**semcode** is an MCP (Model Context Protocol) server that provides hybrid semantic code search over GitHub repositories. It lets AI clients search codebases using natural language or symbol names by combining dense vector embeddings with BM25 sparse search and Tree-sitter-based code parsing.
8+
9+
## Commands
10+
11+
```bash
12+
# Install dependencies
13+
uv sync # production deps
14+
uv sync --group dev # include test/dev deps
15+
16+
# Run tests
17+
uv run pytest # all tests
18+
uv run pytest tests/parser/ # parser tests only
19+
uv run pytest tests/test_pipeline.py # single test file
20+
21+
# Docker (Qdrant + optional local Jina TEI)
22+
make docker-up # hosted embeddings provider
23+
make docker-up-jina # local Jina TEI embeddings
24+
make docker-logs # all container logs
25+
make docker-logs-semcode # semcode container only
26+
27+
# Trigger indexing
28+
make index-code # POST /reindex
29+
make index-history # POST /reindex-history
30+
31+
# Qdrant utilities
32+
make qdrant-clean # delete all collections
33+
make qdrant-dashboard # open Qdrant UI
34+
```
35+
36+
## Architecture
37+
38+
All server code lives under `server/`. The system has two main phases: **ingestion** and **retrieval**.
39+
40+
### Ingestion pipeline (`server/indexer/`)
41+
42+
1. **Discovery**`github_source.py` lists files in a GitHub repo and applies filters from config
43+
2. **Change detection** — compares Git blob SHAs to skip unchanged files
44+
3. **Parsing**`parser/registry.py` routes each file to the correct language parser; each parser uses Tree-sitter to extract `CodeSymbol` objects (functions, classes, methods, etc.) with framework metadata
45+
4. **Embedding**`embeddings/factory.py` creates two vectors per symbol: dense (Jina/Voyage/OpenAI/Ollama) + BM25 sparse (`embeddings/bm25.py` with camelCase/snake_case tokenization)
46+
5. **Upsert** — both vectors stored in Qdrant collection `code_symbols` with rich payload (language, type, file, line, signature, docstring, annotations, parent class, framework metadata)
47+
6. **Cleanup** — removes stale entries for deleted files/symbols
48+
7. **Git history** (optional) — `git_history.py` indexes commits into a separate `git_commits` collection (dense-only)
49+
50+
### Retrieval (`server/store/`, `server/tools/`)
51+
52+
- `store/qdrant.py` runs hybrid search via Qdrant `FusionQuery(fusion=RRF)` — combines dense and BM25 sparse results using Reciprocal Rank Fusion
53+
- `store/commit_store.py` handles commit search (dense-only)
54+
- `tools/` contains the MCP tool implementations: `search.py`, `index.py`, `history.py`, `admin.py`
55+
56+
### MCP interface (`server/main.py`)
57+
58+
FastMCP serves tools and prompts over stdio, SSE, or HTTP. HTTP indexing endpoints (`POST /reindex`, `POST /reindex-history`) return streaming NDJSON for CI/CD consumption.
59+
60+
### Configuration
61+
62+
`server/config.py` loads `config.yaml` via Pydantic. Services (repos to index), embedding provider, and Qdrant connection are all defined there.
63+
64+
### Parser structure (`server/parser/`)
65+
66+
One file per language (24 languages). Each implements the abstract `CodeParser` from `base.py` and returns a list of `CodeSymbol` dataclasses. When adding a new language: implement the parser, register it in `registry.py`, add fixture files under `tests/fixtures/<language>/`, and add snapshot tests under `tests/parser/`.
67+
68+
### Embedding providers (`server/embeddings/`)
69+
70+
Pluggable via `factory.py` registry. Swap providers in config without reindexing. BM25 sparse vectors are always generated alongside dense vectors.
71+
72+
### Test structure
73+
74+
```
75+
tests/
76+
├── parser/ # snapshot tests per language (24 files)
77+
├── fixtures/ # canonical code samples for parser tests
78+
├── embeddings/ # embedding provider tests
79+
├── test_pipeline.py # indexing pipeline integration
80+
├── test_store.py # Qdrant store operations
81+
├── test_reindex_route.py # HTTP endpoint tests
82+
└── conftest.py # shared fixtures
83+
```
84+
85+
Async tests use `asyncio_mode = "auto"` (pytest-asyncio). HTTP calls are mocked with `respx`.

0 commit comments

Comments
 (0)