|
| 1 | +# Query-Lane Tool Parallelization |
| 2 | + |
| 3 | +A3S Code provides Query-lane tool parallelization, allowing parallel execution when a single LLM turn returns multiple query-type tools. This significantly improves performance for network I/O and bulk query operations. |
| 4 | + |
| 5 | +## Quick Start |
| 6 | + |
| 7 | +### Enabling Parallelization |
| 8 | + |
| 9 | +Parallelization is **opt-in** — you must explicitly enable it via `SessionQueueConfig.enable_parallelization`. |
| 10 | + |
| 11 | +**Rust:** |
| 12 | +```rust |
| 13 | +use a3s_code_core::{Agent, SessionOptions, SessionQueueConfig}; |
| 14 | +use a3s_code_core::queue::ParallelizationStrategy; |
| 15 | + |
| 16 | +let mut queue_config = SessionQueueConfig::default(); |
| 17 | +queue_config.enable_parallelization = true; // OPT-IN |
| 18 | +queue_config.query_max_concurrency = 10; |
| 19 | + |
| 20 | +// Optional: customize strategy |
| 21 | +let mut strategy = ParallelizationStrategy::default(); |
| 22 | +strategy.min_tool_count = 3; // Default: 8 |
| 23 | +strategy.allowed_tools = vec!["web_fetch".into(), "web_search".into()]; |
| 24 | +queue_config.parallelization_strategy = Some(strategy); |
| 25 | + |
| 26 | +let session = agent.session( |
| 27 | + ".", |
| 28 | + Some(SessionOptions::default().with_queue_config(queue_config)) |
| 29 | +)?; |
| 30 | +``` |
| 31 | + |
| 32 | +**Python:** |
| 33 | +```python |
| 34 | +from a3s_code import Agent, SessionOptions, SessionQueueConfig, ParallelizationStrategy |
| 35 | + |
| 36 | +queue_config = SessionQueueConfig() |
| 37 | +queue_config.enable_parallelization = True |
| 38 | +queue_config.set_query_concurrency(10) |
| 39 | + |
| 40 | +strategy = ParallelizationStrategy() |
| 41 | +strategy.min_tool_count = 3 |
| 42 | +strategy.allowed_tools = ["web_fetch", "web_search"] |
| 43 | +queue_config.parallelization_strategy = strategy |
| 44 | + |
| 45 | +options = SessionOptions() |
| 46 | +options.queue_config = queue_config |
| 47 | +session = agent.session(".", options) |
| 48 | +``` |
| 49 | + |
| 50 | +**Node.js:** |
| 51 | +```javascript |
| 52 | +const { Agent } = require('@a3s-lab/code'); |
| 53 | + |
| 54 | +const agent = await Agent.create('config.hcl'); |
| 55 | +const session = agent.session('.', { |
| 56 | + queueConfig: { |
| 57 | + enableParallelization: true, |
| 58 | + queryConcurrency: 10, |
| 59 | + parallelizationStrategy: { |
| 60 | + minToolCount: 3, |
| 61 | + allowedTools: ['web_fetch', 'web_search'], |
| 62 | + }, |
| 63 | + }, |
| 64 | +}); |
| 65 | +``` |
| 66 | + |
| 67 | +## How It Works |
| 68 | + |
| 69 | +### Parallelization Trigger Conditions |
| 70 | + |
| 71 | +Parallelization triggers only when **all** conditions are met: |
| 72 | + |
| 73 | +1. ✅ `enable_parallelization = true` (opt-in) |
| 74 | +2. ✅ Queue is configured (`SessionQueueConfig` provided) |
| 75 | +3. ✅ Single LLM turn returns >= `min_tool_count` Query-lane tools (default: 8) |
| 76 | +4. ✅ Tools are in the allowed list (if specified) and not in the blocked list |
| 77 | + |
| 78 | +### Query-Lane Tools |
| 79 | + |
| 80 | +| Tool | Lane | Parallelizable | |
| 81 | +|------|------|---------------| |
| 82 | +| `read` | Query | ✅ | |
| 83 | +| `glob` | Query | ✅ | |
| 84 | +| `grep` | Query | ✅ | |
| 85 | +| `ls` | Query | ✅ | |
| 86 | +| `search` | Query | ✅ | |
| 87 | +| `list_files` | Query | ✅ | |
| 88 | +| `web_fetch` | Query | ✅ | |
| 89 | +| `web_search` | Query | ✅ | |
| 90 | +| `bash` | Execute | ❌ (sequential) | |
| 91 | +| `write` | Execute | ❌ (sequential) | |
| 92 | +| `edit` | Execute | ❌ (sequential) | |
| 93 | + |
| 94 | +### Execution Flow |
| 95 | + |
| 96 | +``` |
| 97 | +LLM call → returns 10 web_fetch tools |
| 98 | + ↓ |
| 99 | + [Parallelization check] |
| 100 | + ↓ |
| 101 | + enable_parallelization? ✅ |
| 102 | + >= min_tool_count? ✅ |
| 103 | + Query-lane tools? ✅ |
| 104 | + In allowed_tools? ✅ |
| 105 | + ↓ |
| 106 | + [Parallel execution] |
| 107 | + fetch(url1) ┐ |
| 108 | + fetch(url2) ├─ concurrent execution |
| 109 | + fetch(url3) ┘ |
| 110 | + ... |
| 111 | + ↓ |
| 112 | + Collect results → return to LLM |
| 113 | +``` |
| 114 | + |
| 115 | +## Performance |
| 116 | + |
| 117 | +### Network I/O (recommended) |
| 118 | + |
| 119 | +| Operation | Serial | Parallel | Speedup | |
| 120 | +|-----------|--------|----------|---------| |
| 121 | +| 10x web_fetch | ~17s | ~1.8s | **~9x** | |
| 122 | +| 20x web_search | ~60s | ~8s | **~7.5x** | |
| 123 | +| 15x API calls | ~45s | ~6s | **~7.5x** | |
| 124 | + |
| 125 | +### Local File I/O (not recommended) |
| 126 | + |
| 127 | +| Operation | Serial | Parallel | Speedup | |
| 128 | +|-----------|--------|----------|---------| |
| 129 | +| 10x read | 0.1s | 0.5s | **0.2x** ❌ | |
| 130 | +| 20x glob | 0.2s | 0.8s | **0.25x** ❌ | |
| 131 | + |
| 132 | +**Conclusion: parallelization is best for slow I/O, not fast local operations.** |
| 133 | + |
| 134 | +## ParallelizationStrategy |
| 135 | + |
| 136 | +| Field | Type | Default | Description | |
| 137 | +|-------|------|---------|-------------| |
| 138 | +| `min_tool_count` | usize | 8 | Minimum Query-lane tools to trigger parallelization | |
| 139 | +| `allowed_tools` | Vec<String> | [] (all) | Only parallelize these tools. Empty = all Query-lane tools | |
| 140 | +| `blocked_tools` | Vec<String> | [] | Never parallelize these tools. Takes precedence over allowed | |
| 141 | + |
| 142 | +## Use Cases |
| 143 | + |
| 144 | +### ✅ Good for parallelization |
| 145 | + |
| 146 | +**1. Network requests** |
| 147 | +```rust |
| 148 | +session.send( |
| 149 | + "Fetch these 10 web pages and extract titles: |
| 150 | + 1. https://example.com/page1 |
| 151 | + 2. https://example.com/page2 |
| 152 | + ...", |
| 153 | + None |
| 154 | +).await?; |
| 155 | +``` |
| 156 | + |
| 157 | +**2. Bulk document search** |
| 158 | +```python |
| 159 | +result = session.send( |
| 160 | + "Search for 'authentication' in these 15 files: |
| 161 | + docs/api.md, docs/security.md, ..." |
| 162 | +) |
| 163 | +``` |
| 164 | + |
| 165 | +**3. Batch API calls** |
| 166 | +```javascript |
| 167 | +const result = await session.send( |
| 168 | + "Fetch user info for IDs: 1, 2, 3, ..., 20\n" + |
| 169 | + "Use web_fetch to call /api/users/{id}" |
| 170 | +); |
| 171 | +``` |
| 172 | + |
| 173 | +### ❌ Not recommended |
| 174 | + |
| 175 | +**1. Local file reads** — too fast, parallelization overhead > benefit |
| 176 | +**2. Few tool calls** — below `min_tool_count` threshold |
| 177 | +**3. Fast operations** — glob/ls complete in milliseconds |
| 178 | + |
| 179 | +## Best Practices |
| 180 | + |
| 181 | +### 1. Choose based on scenario |
| 182 | + |
| 183 | +```rust |
| 184 | +// Network requests → enable parallelization |
| 185 | +let session_web = agent.session(".", Some( |
| 186 | + SessionOptions::default().with_queue_config(queue_config) |
| 187 | +))?; |
| 188 | + |
| 189 | +// Local files → skip parallelization |
| 190 | +let session_local = agent.session(".", None)?; |
| 191 | +``` |
| 192 | + |
| 193 | +### 2. Monitor with logs |
| 194 | + |
| 195 | +```bash |
| 196 | +RUST_LOG=info cargo run |
| 197 | +``` |
| 198 | + |
| 199 | +Look for: |
| 200 | +``` |
| 201 | +[INFO] Using parallel execution for Query-lane tools [tool_count=10] ← enabled |
| 202 | +[INFO] Parallel execution bypassed: too few tools ← not triggered |
| 203 | +[INFO] Parallel execution bypassed: not enabled ← not opt-in |
| 204 | +``` |
| 205 | + |
| 206 | +## Performance Tests |
| 207 | + |
| 208 | +Run the test examples to see actual performance: |
| 209 | + |
| 210 | +```bash |
| 211 | +# Rust |
| 212 | +cd crates/code |
| 213 | +cargo run --example test_internal_parallel |
| 214 | + |
| 215 | +# Python |
| 216 | +cd crates/code/sdk/python |
| 217 | +python3 examples/test_internal_parallel.py |
| 218 | + |
| 219 | +# Node.js |
| 220 | +cd crates/code/sdk/node |
| 221 | +node examples/test_internal_parallel.js |
| 222 | +``` |
| 223 | + |
| 224 | +**Test scenario:** Fetch 10 web page titles |
| 225 | + |
| 226 | +**Expected results:** |
| 227 | +- Serial: ~17s tool execution |
| 228 | +- Parallel: ~1.8s tool execution |
| 229 | +- Speedup: ~9x for tool execution |
0 commit comments