Skip to content

Commit 43ffb03

Browse files
RoyLinRoyLin
authored andcommitted
feat(code): replace vector RAG with Sirchmunk-inspired agentic search
Replace embedding-based vector search with intelligent multi-phase code search inspired by the Sirchmunk paper (https://github.com/modelscope/sirchmunk). ## Added - agentic_search tool with 3 modes: - FAST: 2-level keyword cascade (2-5s) - DEEP: Monte Carlo evidence sampling with Gaussian importance sampling (10-30s) - FILENAME_ONLY: Quick file discovery - IDF-weighted relevance scoring - File type awareness (code 1.2x, config 1.0x, docs 0.9x) - Automatic keyword extraction with stop word filtering - ripgrep_provider: Ripgrep-based context provider - knowledge_cluster: Sirchmunk-inspired clustering (future feature) - agentic-search skill definition ## Removed - embedding.rs: Embedding-based vector search - vector_provider.rs: Vector RAG context provider - vector_store.rs: Vector store implementations - codesearch.rs: Old code search tool ## Changed - README.md: Add agentic_search to tools table, update test count (1505 → 1471) - mod.rs: Register agentic_search tool, remove vector RAG imports - builtin.rs: Add agentic-search skill - registry.rs: Update skill loading ## Tests - 12 new tests for agentic_search (all passing) - 9 new tests for knowledge_cluster (all passing) - Total: 1471 tests passing
1 parent 0636a67 commit 43ffb03

14 files changed

Lines changed: 1901 additions & 1933 deletions

README.md

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
[![Crates.io](https://img.shields.io/crates/v/a3s-code-core.svg)](https://crates.io/crates/a3s-code-core)
66
[![Documentation](https://docs.rs/a3s-code-core/badge.svg)](https://docs.rs/a3s-code-core)
77
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](./LICENSE)
8-
[![Tests](https://img.shields.io/badge/tests-1505%20passing-brightgreen.svg)](./core/tests)
8+
[![Tests](https://img.shields.io/badge/tests-1471%20passing-brightgreen.svg)](./core/tests)
99

1010
---
1111

@@ -85,7 +85,7 @@ print(result.text)
8585
| Category | Tools | Description |
8686
| ---------------------------- | ---------------------------------------- | ----------------------------------------------------------------------------- |
8787
| **File Operations** | `read`, `write`, `edit`, `patch` | Read/write files, apply diffs |
88-
| **Search** | `grep`, `glob`, `ls` | Search content, find files, list directories |
88+
| **Search** | `grep`, `glob`, `ls`, `agentic_search` | Search content, find files, list directories, intelligent multi-phase code search |
8989
| **Execution** | `bash` | Execute shell commands |
9090
| **Web** | `web_fetch`, `web_search` | Fetch URLs, search the web |
9191
| **Git** | `git_worktree` | Create/list/remove/status git worktrees for parallel work |
@@ -666,9 +666,7 @@ All policies are replaceable via traits with working defaults:
666666
| SecurityProvider | Input taint, output sanitization | DefaultSecurityProvider |
667667
| PermissionChecker | Tool access control | PermissionPolicy |
668668
| ConfirmationProvider | Human confirmation | ConfirmationManager |
669-
| ContextProvider | RAG retrieval | FileSystemContextProvider |
670-
| EmbeddingProvider | Vector embeddings for semantic search | OpenAiEmbeddingProvider |
671-
| VectorStore | Embedding storage and similarity search | InMemoryVectorStore |
669+
| ContextProvider | RAG retrieval | RipgrepContextProvider |
672670
| SessionStore | Session persistence | FileSessionStore |
673671
| MemoryStore | Long-term memory backend (from `a3s-memory`) | InMemoryStore |
674672
| Tool | Custom tools | 14 built-in tools |
@@ -708,7 +706,7 @@ Agent (config-driven)
708706
├── HookEngine (11 lifecycle events)
709707
├── Security (PII redaction, injection detection)
710708
├── Skills (instruction injection + tool permissions)
711-
├── Context (RAG providers: filesystem, vector)
709+
├── Context (RAG providers: ripgrep, filesystem)
712710
└── Memory (AgentMemory: working/short-term/long-term via a3s-memory)
713711
714712
AgentTeam (multi-agent coordination)
@@ -1071,7 +1069,7 @@ cargo run --example 02_streaming
10711069
- `test_auto_compact` — Context window auto-compaction
10721070
- `test_security` — Default and custom SecurityProvider
10731071
- `test_batch_tool` — Parallel tool execution via batch
1074-
- `test_vector_rag`Semantic code search with filesystem context
1072+
- `test_ripgrep_context`Fast indexless code search with ripgrep provider
10751073
- `test_hooks` — Lifecycle hook handlers (audit, block, transform)
10761074
- `test_parallel_processing` — Concurrent multi-session workloads
10771075
- `test_git_worktree` — Git worktree tool: create, list, remove, status + LLM-driven

core/skills/agentic-search.md

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
---
2+
name: agentic-search
3+
description: Intelligent multi-phase code search with keyword extraction and relevance ranking
4+
allowed-tools: "agentic_search(*), grep(*), read(*)"
5+
kind: instruction
6+
tags: ["search", "code", "intelligent", "sirchmunk"]
7+
version: "1.0.0"
8+
---
9+
10+
# Agentic Search Skill
11+
12+
You are a specialized code search assistant using intelligent multi-phase retrieval inspired by Sirchmunk.
13+
14+
## When to Use
15+
16+
Use the `agentic_search` tool when the user asks to:
17+
- Find code related to a concept or feature
18+
- Locate implementations of specific functionality
19+
- Search for patterns across the codebase
20+
- Discover files related to a topic
21+
22+
## Search Strategy
23+
24+
The `agentic_search` tool implements a multi-phase pipeline:
25+
26+
1. **Phase 1 - Keyword Extraction**: Automatically extracts meaningful keywords from natural language queries, removing stop words
27+
2. **Phase 2 - Parallel Search**: Searches file contents and structure simultaneously
28+
3. **Phase 3 - Relevance Ranking**: Ranks results using IDF-weighted scoring (files matching more keywords rank higher)
29+
30+
## Mode Selection
31+
32+
### FAST Mode (Default)
33+
Use for most queries. Returns results in 2-5 seconds.
34+
35+
```typescript
36+
agentic_search({
37+
query: "JWT token validation",
38+
max_results: 10
39+
})
40+
```
41+
42+
### DEEP Mode (Monte Carlo Sampling)
43+
Use for comprehensive analysis requiring deep understanding. Returns results in 10-30 seconds.
44+
45+
```typescript
46+
agentic_search({
47+
query: "explain the entire authentication flow from login to session management",
48+
mode: "deep",
49+
max_results: 3
50+
})
51+
```
52+
53+
**When to use DEEP mode:**
54+
- Complex queries requiring comprehensive understanding
55+
- When FAST mode results are insufficient
56+
- Analyzing architectural patterns across multiple files
57+
- Understanding data flow through the system
58+
59+
**How DEEP mode works:**
60+
1. **Act 1 - Fuzzy Anchoring**: Identifies high-scoring code regions
61+
2. **Act 2 - Gaussian Sampling**: Samples lines around matches using Gaussian distribution (sigma adapts 5.0 → 2.0)
62+
3. **Act 3 - Evidence Synthesis**: Combines sampled regions with evidence scores
63+
64+
### FILENAME_ONLY Mode
65+
Use for quick file discovery when you only need to find files by name.
66+
67+
```typescript
68+
agentic_search({
69+
query: "auth",
70+
mode: "filename_only"
71+
})
72+
```
73+
74+
## Usage Examples
75+
76+
### Example 1: Find Authentication Logic
77+
```typescript
78+
agentic_search({
79+
query: "how does authentication work",
80+
max_results: 5
81+
})
82+
```
83+
84+
The tool will:
85+
- Extract keywords: ["authentication", "work"]
86+
- Search for files containing these terms
87+
- Rank by relevance (code files with more keyword matches rank higher)
88+
- Return top 5 results with context
89+
90+
### Example 2: Find Configuration Files
91+
```typescript
92+
agentic_search({
93+
query: "database configuration",
94+
include: "*.{toml,yaml,json}",
95+
context_lines: 3
96+
})
97+
```
98+
99+
### Example 3: Quick File Discovery
100+
```typescript
101+
agentic_search({
102+
query: "test",
103+
mode: "filename_only",
104+
max_results: 20
105+
})
106+
```
107+
108+
## Relevance Scoring
109+
110+
Results are ranked by:
111+
1. **Keyword coverage**: Files matching more unique keywords rank higher
112+
2. **Match density**: More matches relative to file size = higher score
113+
3. **File type boost**: Code files (1.2x) > Config files (1.0x) > Docs (0.9x)
114+
115+
## Best Practices
116+
117+
1. **Use natural language**: "how does authentication work" is better than just "auth"
118+
2. **Be specific**: "JWT token validation" is better than "tokens"
119+
3. **Combine with read**: After finding relevant files, use `read` to examine them in detail
120+
4. **Use include patterns**: Narrow search to specific file types when appropriate
121+
5. **Adjust context_lines**: Use 0-1 for overview, 2-3 for detailed context
122+
123+
## Follow-up Actions
124+
125+
After getting search results:
126+
1. Use `read` to examine the most relevant files
127+
2. Use `grep` for more specific pattern matching within discovered files
128+
3. Use `agentic_search` again with refined keywords if needed
129+
130+
## Comparison to grep
131+
132+
| Feature | agentic_search | grep |
133+
|---------|----------------|------|
134+
| Input | Natural language | Regex pattern |
135+
| Keyword extraction | Automatic | Manual |
136+
| Relevance ranking | Yes (IDF-weighted) | No (chronological) |
137+
| File type awareness | Yes | No |
138+
| Best for | Exploratory search | Precise pattern matching |
139+
140+
Use `agentic_search` for initial discovery, then `grep` for precise follow-up searches.

0 commit comments

Comments
 (0)