Skip to content

Commit 04e49f7

Browse files
committed
docs: update parallelization guide with opt-in API and real benchmarks
- Rewrite in English (was Chinese) - Document enable_parallelization opt-in flag - Document ParallelizationStrategy (min_tool_count, allowed_tools, blocked_tools) - Add all three SDK examples (Rust, Python, Node.js) - Update performance numbers with real benchmarks (9x tool speedup) - Add Query-lane tool classification table
1 parent 48515ed commit 04e49f7

1 file changed

Lines changed: 229 additions & 0 deletions

File tree

docs/parallelization.md

Lines changed: 229 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,229 @@
1+
# Query-Lane Tool Parallelization
2+
3+
A3S Code provides Query-lane tool parallelization, allowing parallel execution when a single LLM turn returns multiple query-type tools. This significantly improves performance for network I/O and bulk query operations.
4+
5+
## Quick Start
6+
7+
### Enabling Parallelization
8+
9+
Parallelization is **opt-in** — you must explicitly enable it via `SessionQueueConfig.enable_parallelization`.
10+
11+
**Rust:**
12+
```rust
13+
use a3s_code_core::{Agent, SessionOptions, SessionQueueConfig};
14+
use a3s_code_core::queue::ParallelizationStrategy;
15+
16+
let mut queue_config = SessionQueueConfig::default();
17+
queue_config.enable_parallelization = true; // OPT-IN
18+
queue_config.query_max_concurrency = 10;
19+
20+
// Optional: customize strategy
21+
let mut strategy = ParallelizationStrategy::default();
22+
strategy.min_tool_count = 3; // Default: 8
23+
strategy.allowed_tools = vec!["web_fetch".into(), "web_search".into()];
24+
queue_config.parallelization_strategy = Some(strategy);
25+
26+
let session = agent.session(
27+
".",
28+
Some(SessionOptions::default().with_queue_config(queue_config))
29+
)?;
30+
```
31+
32+
**Python:**
33+
```python
34+
from a3s_code import Agent, SessionOptions, SessionQueueConfig, ParallelizationStrategy
35+
36+
queue_config = SessionQueueConfig()
37+
queue_config.enable_parallelization = True
38+
queue_config.set_query_concurrency(10)
39+
40+
strategy = ParallelizationStrategy()
41+
strategy.min_tool_count = 3
42+
strategy.allowed_tools = ["web_fetch", "web_search"]
43+
queue_config.parallelization_strategy = strategy
44+
45+
options = SessionOptions()
46+
options.queue_config = queue_config
47+
session = agent.session(".", options)
48+
```
49+
50+
**Node.js:**
51+
```javascript
52+
const { Agent } = require('@a3s-lab/code');
53+
54+
const agent = await Agent.create('config.hcl');
55+
const session = agent.session('.', {
56+
queueConfig: {
57+
enableParallelization: true,
58+
queryConcurrency: 10,
59+
parallelizationStrategy: {
60+
minToolCount: 3,
61+
allowedTools: ['web_fetch', 'web_search'],
62+
},
63+
},
64+
});
65+
```
66+
67+
## How It Works
68+
69+
### Parallelization Trigger Conditions
70+
71+
Parallelization triggers only when **all** conditions are met:
72+
73+
1.`enable_parallelization = true` (opt-in)
74+
2. ✅ Queue is configured (`SessionQueueConfig` provided)
75+
3. ✅ Single LLM turn returns >= `min_tool_count` Query-lane tools (default: 8)
76+
4. ✅ Tools are in the allowed list (if specified) and not in the blocked list
77+
78+
### Query-Lane Tools
79+
80+
| Tool | Lane | Parallelizable |
81+
|------|------|---------------|
82+
| `read` | Query ||
83+
| `glob` | Query ||
84+
| `grep` | Query ||
85+
| `ls` | Query ||
86+
| `search` | Query ||
87+
| `list_files` | Query ||
88+
| `web_fetch` | Query ||
89+
| `web_search` | Query ||
90+
| `bash` | Execute | ❌ (sequential) |
91+
| `write` | Execute | ❌ (sequential) |
92+
| `edit` | Execute | ❌ (sequential) |
93+
94+
### Execution Flow
95+
96+
```
97+
LLM call → returns 10 web_fetch tools
98+
99+
[Parallelization check]
100+
101+
enable_parallelization? ✅
102+
>= min_tool_count? ✅
103+
Query-lane tools? ✅
104+
In allowed_tools? ✅
105+
106+
[Parallel execution]
107+
fetch(url1) ┐
108+
fetch(url2) ├─ concurrent execution
109+
fetch(url3) ┘
110+
...
111+
112+
Collect results → return to LLM
113+
```
114+
115+
## Performance
116+
117+
### Network I/O (recommended)
118+
119+
| Operation | Serial | Parallel | Speedup |
120+
|-----------|--------|----------|---------|
121+
| 10x web_fetch | ~17s | ~1.8s | **~9x** |
122+
| 20x web_search | ~60s | ~8s | **~7.5x** |
123+
| 15x API calls | ~45s | ~6s | **~7.5x** |
124+
125+
### Local File I/O (not recommended)
126+
127+
| Operation | Serial | Parallel | Speedup |
128+
|-----------|--------|----------|---------|
129+
| 10x read | 0.1s | 0.5s | **0.2x**|
130+
| 20x glob | 0.2s | 0.8s | **0.25x**|
131+
132+
**Conclusion: parallelization is best for slow I/O, not fast local operations.**
133+
134+
## ParallelizationStrategy
135+
136+
| Field | Type | Default | Description |
137+
|-------|------|---------|-------------|
138+
| `min_tool_count` | usize | 8 | Minimum Query-lane tools to trigger parallelization |
139+
| `allowed_tools` | Vec<String> | [] (all) | Only parallelize these tools. Empty = all Query-lane tools |
140+
| `blocked_tools` | Vec<String> | [] | Never parallelize these tools. Takes precedence over allowed |
141+
142+
## Use Cases
143+
144+
### ✅ Good for parallelization
145+
146+
**1. Network requests**
147+
```rust
148+
session.send(
149+
"Fetch these 10 web pages and extract titles:
150+
1. https://example.com/page1
151+
2. https://example.com/page2
152+
...",
153+
None
154+
).await?;
155+
```
156+
157+
**2. Bulk document search**
158+
```python
159+
result = session.send(
160+
"Search for 'authentication' in these 15 files:
161+
docs/api.md, docs/security.md, ..."
162+
)
163+
```
164+
165+
**3. Batch API calls**
166+
```javascript
167+
const result = await session.send(
168+
"Fetch user info for IDs: 1, 2, 3, ..., 20\n" +
169+
"Use web_fetch to call /api/users/{id}"
170+
);
171+
```
172+
173+
### ❌ Not recommended
174+
175+
**1. Local file reads** — too fast, parallelization overhead > benefit
176+
**2. Few tool calls** — below `min_tool_count` threshold
177+
**3. Fast operations** — glob/ls complete in milliseconds
178+
179+
## Best Practices
180+
181+
### 1. Choose based on scenario
182+
183+
```rust
184+
// Network requests → enable parallelization
185+
let session_web = agent.session(".", Some(
186+
SessionOptions::default().with_queue_config(queue_config)
187+
))?;
188+
189+
// Local files → skip parallelization
190+
let session_local = agent.session(".", None)?;
191+
```
192+
193+
### 2. Monitor with logs
194+
195+
```bash
196+
RUST_LOG=info cargo run
197+
```
198+
199+
Look for:
200+
```
201+
[INFO] Using parallel execution for Query-lane tools [tool_count=10] ← enabled
202+
[INFO] Parallel execution bypassed: too few tools ← not triggered
203+
[INFO] Parallel execution bypassed: not enabled ← not opt-in
204+
```
205+
206+
## Performance Tests
207+
208+
Run the test examples to see actual performance:
209+
210+
```bash
211+
# Rust
212+
cd crates/code
213+
cargo run --example test_internal_parallel
214+
215+
# Python
216+
cd crates/code/sdk/python
217+
python3 examples/test_internal_parallel.py
218+
219+
# Node.js
220+
cd crates/code/sdk/node
221+
node examples/test_internal_parallel.js
222+
```
223+
224+
**Test scenario:** Fetch 10 web page titles
225+
226+
**Expected results:**
227+
- Serial: ~17s tool execution
228+
- Parallel: ~1.8s tool execution
229+
- Speedup: ~9x for tool execution

0 commit comments

Comments
 (0)