Skip to content

Commit 9a52baa

Browse files
prosdevclaude
andcommitted
docs: update all documentation for antfly migration
Part 1.6 of antfly migration: README.md: - Add Docker/Antfly to prerequisites - Add `dev setup` to quick start and install sections - Update tech stack: LanceDB → Antfly, @xenova → Termite CLAUDE.md: - Update mission description with hybrid search - Update monorepo structure (Antfly not LanceDB) - Add `dev setup` to commands and workflow - Update non-negotiables (Termite not @xenova) - Update dev_search description (hybrid search) Doc site: - install.mdx: Add Docker prerequisite, `dev setup` section - quickstart.mdx: Add setup step, update under-the-hood explanation - architecture.mdx: Replace LanceDB with Antfly throughout, update diagrams, data flow, storage, and security sections Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 7992389 commit 9a52baa

5 files changed

Lines changed: 60 additions & 50 deletions

File tree

CLAUDE.md

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,8 @@
44

55
**Local-first repository context for AI tools — no hallucinations.**
66

7-
Semantic search, code analysis, and GitHub integration through MCP.
7+
Hybrid search (BM25 + vector), code analysis, and GitHub integration through MCP.
8+
Powered by [Antfly](https://antfly.io) for search and embeddings.
89
Everything runs on your machine. No data leaves.
910

1011
---
@@ -20,7 +21,7 @@ Everything runs on your machine. No data leaves.
2021

2122
```
2223
packages/
23-
core/ # Scanner (ts-morph, tree-sitter), vector storage (LanceDB), services
24+
core/ # Scanner (ts-morph, tree-sitter), vector storage (Antfly), services
2425
cli/ # Commander.js CLI — dev index, dev mcp install, etc.
2526
mcp-server/ # MCP server with 9 built-in adapters
2627
subagents/ # Coordinator, explorer, planner, PR agents
@@ -59,6 +60,7 @@ pnpm format # Biome format
5960
pnpm dev # Watch mode
6061
pnpm clean # Clean build outputs
6162
pnpm changeset # Document changes for release
63+
dev setup # One-time: start Antfly search backend
6264
```
6365

6466
---
@@ -70,7 +72,7 @@ pnpm changeset # Document changes for release
7072
- **Workspace protocol** for internal deps: `"@prosdevlab/dev-agent-core": "workspace:*"`
7173
- **Tests run from root only:** `pnpm test` — centralized Vitest config.
7274
- **Logger:** Use `@prosdevlab/kero` — never `console.log` in packages.
73-
- **Local-first:** No data sent externally. Embeddings via @xenova/transformers.
75+
- **Local-first:** No data sent externally. Embeddings via Antfly/Termite (local ONNX).
7476
- **Code review before PR.** Always run the `code-reviewer` agent (which
7577
launches security-reviewer, logic-reviewer, and quality-reviewer in
7678
parallel) on the branch diff before creating a pull request. Address
@@ -113,7 +115,7 @@ See `.claude/da-plans/README.md` for status and format details.
113115

114116
| Tool | Purpose |
115117
|------|---------|
116-
| `dev_search` | Semantic code search (use FIRST for conceptual queries) |
118+
| `dev_search` | Hybrid code search — BM25 + vector + RRF (use FIRST for conceptual queries) |
117119
| `dev_refs` | Find callers/callees of functions |
118120
| `dev_map` | Codebase structure with change frequency |
119121
| `dev_history` | Semantic search over git commits |
@@ -142,6 +144,7 @@ See `.claude/da-plans/README.md` for status and format details.
142144
```bash
143145
# Common workflows
144146
pnpm install && pnpm build && pnpm test # Full setup
147+
dev setup # One-time: start Antfly
145148
dev index . # Index repository
146149
dev mcp install # Install for Claude Code
147150
dev mcp install --cursor # Install for Cursor

README.md

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -52,17 +52,17 @@ We benchmarked dev-agent against baseline Claude Code across 5 task types:
5252
# Install globally
5353
npm install -g dev-agent
5454

55-
# Index your repository (initial indexing can take 5-10 minutes for large codebases)
55+
# One-time setup (starts search backend via Docker or native)
56+
dev setup
57+
58+
# Index your repository
5659
cd /path/to/your/repo
5760
dev index .
5861

5962
# Install MCP integration
6063
dev mcp install --cursor # For Cursor IDE
6164
dev mcp install # For Claude Code
6265

63-
# Keep index up to date (fast incremental updates)
64-
dev update
65-
6666
# That's it! AI tools now have access to dev-agent capabilities.
6767
```
6868

@@ -213,15 +213,18 @@ Check MCP server and component health.
213213
### Prerequisites
214214

215215
- [Node.js](https://nodejs.org/) v22 LTS or higher
216-
- [pnpm](https://pnpm.io/) v8.15.4 or higher
216+
- [Docker Desktop](https://docker.com/get-started) (recommended) or [Antfly](https://antfly.io) native binary
217217
- [GitHub CLI](https://cli.github.com/) (for GitHub features)
218218

219219
### Global Install (Recommended)
220220

221221
```bash
222222
npm install -g dev-agent
223+
dev setup # One-time: starts search backend (Docker or native)
223224
```
224225

226+
`dev setup` handles everything — pulls the Docker image, starts the server, and verifies the connection. If Docker isn't available, it falls back to the native Antfly binary and offers to install it.
227+
225228
### From Source
226229

227230
```bash
@@ -363,11 +366,10 @@ dev-agent/
363366
- **TypeScript** (strict mode)
364367
- **ts-morph** / TypeScript Compiler API (TypeScript/JS analysis)
365368
- **tree-sitter** WASM (Go analysis, extensible to Python/Rust)
366-
- **LanceDB** (embedded vector storage)
367-
- **@xenova/transformers** (local embeddings)
369+
- **[Antfly](https://antfly.io)** (hybrid search: BM25 + vector + RRF, local embeddings via Termite)
368370
- **MCP** (Model Context Protocol)
369371
- **Turborepo** (monorepo builds)
370-
- **Vitest** (1500+ tests)
372+
- **Vitest** (1900+ tests)
371373

372374
## Development
373375

website/content/docs/architecture.mdx

Lines changed: 16 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ dev-agent is built as a TypeScript monorepo with specialized packages for differ
3737
│ │ │ │ │
3838
│ ┌──────▼─────────────────▼─────────────────▼───────────────────┐ │
3939
│ │ Vector Storage │ │
40-
│ │ (LanceDB) │ │
40+
│ │ (Antfly — hybrid search) │ │
4141
│ └──────────────────────────────────────────────────────────────┘ │
4242
└─────────────────────────────────────────────────────────────────────┘
4343
```
@@ -52,8 +52,8 @@ The foundation layer providing:
5252
- TypeScript/JavaScript via ts-morph
5353
- Go via tree-sitter WASM
5454
- Markdown via remark
55-
- **Indexer**Converts code components to vector embeddings
56-
- **Vector Storage**LanceDB for fast similarity search
55+
- **Indexer**Sends code components to Antfly for embedding and storage
56+
- **Vector Storage**Antfly for hybrid search (BM25 + vector + RRF)
5757
- **GitHub Integration** — Fetches and indexes issues/PRs via GitHub CLI
5858

5959
### @prosdevlab/dev-agent-cli
@@ -110,9 +110,8 @@ Centralized logging:
110110
| Build | Turborepo | Monorepo orchestration |
111111
| Testing | Vitest | 1500+ tests |
112112
| Linting | Biome | Fast linting/formatting |
113-
| Vector DB | LanceDB | Embedded vector storage |
114-
| Embeddings | @xenova/transformers | Local ML inference |
115-
| Model | all-MiniLM-L6-v2 | Sentence embeddings |
113+
| Search | [Antfly](https://antfly.io) | Hybrid search (BM25 + vector + RRF) |
114+
| Embeddings | Termite (via Antfly) | Local ONNX inference (bge-small-en-v1.5) |
116115
| Protocol | MCP | AI tool integration |
117116
| TS/JS Parser | ts-morph | TypeScript Compiler API |
118117
| Go Parser | tree-sitter WASM | Fast, portable parsing |
@@ -126,14 +125,10 @@ Source Code
126125
127126
128127
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
129-
│ Glob │ ──▶ │ Parse │ ──▶ │ Extract │ ──▶ │ Embed
130-
│ Files │ │ AST │ │Components│ │ Vectors
128+
│ Glob │ ──▶ │ Parse │ ──▶ │ Extract │ ──▶ │ Store
129+
│ Files │ │ AST │ │Components│ │ (Antfly)
131130
└──────────┘ └──────────┘ └──────────┘ └──────────┘
132-
133-
134-
┌──────────┐
135-
│ LanceDB │
136-
└──────────┘
131+
Antfly auto-embeds via Termite
137132
```
138133

139134
### Query Flow
@@ -143,32 +138,24 @@ User Query: "Where is auth handled?"
143138
144139
145140
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
146-
Embed │ ──▶ │ Vector │ ──▶ │ Rank │ ──▶ │ Format │
147-
│ Query │ │ Search │ │ Results │ │ Output │
141+
Hybrid │ ──▶ │BM25+Vec │ ──▶ │ RRF │ ──▶ │ Format │
142+
│ Query │ │ Search │ │ Fusion │ │ Output │
148143
└──────────┘ └──────────┘ └──────────┘ └──────────┘
149144
```
150145

151146
## Storage
152147

153-
All data is stored locally in `~/.dev-agent/`:
148+
Data is stored locally by Antfly (default `~/.antfly/`) and dev-agent config in `~/.dev-agent/`.
154149

155-
```
156-
~/.dev-agent/
157-
├── indexes/
158-
│ └── <repo-hash>/
159-
│ ├── code.lance/ # Code vector index
160-
│ └── metadata.json # Index metadata
161-
├── github/
162-
│ └── <repo-hash>/
163-
│ ├── issues.lance/ # Issues vector index
164-
│ └── prs.lance/ # PRs vector index
165-
└── config.json # Global config (optional)
166-
```
150+
Antfly manages its own storage, indexes, and embeddings. dev-agent creates three tables:
151+
- `dev-agent-{project}-code` — code components
152+
- `dev-agent-{project}-git` — git commit history
153+
- `dev-agent-{project}-github` — GitHub issues/PRs
167154

168155
## Security
169156

170157
- **100% Local** — No data sent to external services
171158
- **No Cloud** — All processing happens on your machine
172-
- **Embeddings** — Generated locally via @xenova/transformers
159+
- **Embeddings** — Generated locally via Antfly's Termite engine (ONNX)
173160
- **GitHub Data** — Fetched via authenticated GitHub CLI
174161

website/content/docs/install.mdx

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
## Requirements
44

55
- **Node.js 22+** (LTS recommended)
6-
- **pnpm**, **npm**, or **yarn**
6+
- **[Docker Desktop](https://docker.com/get-started)** (recommended) or [Antfly](https://antfly.io) native binary
77
- **Cursor** or **Claude Code** (for MCP integration)
88

99
## Install dev-agent
@@ -29,6 +29,23 @@ Verify the installation:
2929
dev --version
3030
```
3131

32+
## One-time setup
33+
34+
Run `dev setup` to start the search backend:
35+
36+
```bash
37+
dev setup
38+
```
39+
40+
This does three things:
41+
1. **Checks for Docker** (preferred) or native Antfly binary
42+
2. **Starts the Antfly server** — handles hybrid search and embeddings locally
43+
3. **Verifies the connection** — confirms everything is working
44+
45+
If Docker isn't available, `dev setup` falls back to the native binary and offers to install it.
46+
47+
> **What is Antfly?** [Antfly](https://antfly.io) is the search engine that powers dev-agent. It runs locally on your machine — your code never leaves. It provides hybrid search (BM25 keyword matching + vector similarity) which is significantly better than pure vector search for code.
48+
3249
## Setup for Cursor
3350

3451
### 1. Index your repository
@@ -40,8 +57,6 @@ cd /path/to/your/project
4057
dev index .
4158
```
4259

43-
This creates a local vector database in `~/.dev-agent/indexes/`.
44-
4560
### 2. Install MCP integration
4661

4762
```bash

website/content/docs/quickstart.mdx

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,15 +5,19 @@ Get from zero to semantic search in 5 minutes.
55
## Prerequisites
66

77
- Node.js 22+ installed
8+
- Docker Desktop (recommended) or Antfly native binary
89
- Cursor IDE (or Claude Code)
910
- A code repository to index
1011

11-
## Step 1: Install dev-agent
12+
## Step 1: Install and setup
1213

1314
```bash
1415
npm install -g @prosdevlab/dev-agent
16+
dev setup
1517
```
1618

19+
`dev setup` starts the search backend (via Docker or native). One-time step.
20+
1721
## Step 2: Index your repository
1822

1923
```bash
@@ -70,15 +74,14 @@ Try these prompts:
7074

7175
When you run `dev index .`:
7276

73-
1. **Scanner** parses your code using the TypeScript Compiler API
77+
1. **Scanner** parses your code using the TypeScript Compiler API (ts-morph) and tree-sitter (Go)
7478
2. **Extractor** identifies functions, classes, interfaces, types, arrow functions, hooks, and exported constants
75-
3. **Embedder** generates vector embeddings locally (MiniLM-L6-v2)
76-
4. **Storage** saves everything to LanceDB (`~/.dev-agent/indexes/`)
79+
3. **Storage** sends documents to Antfly, which generates embeddings locally via Termite (ONNX)
7780

7881
When AI tools call `dev_search`:
7982

80-
1. **Query** gets embedded using the same model
81-
2. **Vector search** finds semantically similar code
83+
1. **Hybrid search** runs BM25 (keyword matching) + vector similarity in parallel
84+
2. **RRF fusion** combines both signals into a single ranked result set
8285
3. **Formatter** returns results optimized for LLM context windows
8386

8487
## Next Steps

0 commit comments

Comments
 (0)