Skip to content

Commit f42c5ec

Browse files
committed
gather: deterministic context-pack tool (aft_gather)
One call assembles a bounded context pack — ranked, deduped, budgeted verbatim code evidence — replacing the serial search → outline → zoom → callgraph read chain. - Two modes (XOR): question (seeds via handle_semantic_search, same pipeline as aft_search) or symbol+filePath (callgraph impact depth-1 callers + call_tree depth-1 callees) - 1-hop neighbor expansion; dedupe by canonicalized (file, symbol), seeds win - Hard line budget (default 400, cap 800) via render_symbol_within_budget; over-budget candidates emit as visible stubs — nothing silently dropped - Grep-fallback hits resolve to their containing symbol by line containment; no-symbol hits stay visible stubs - Semantic-index-building degradation flagged in the pack header (semantic_status), suppressed when any symbol-level seed resolves - Unresolved external callees collapse to one summary line; unresolved seeds/callers never suppressed - Follows the tri-state honest-reporting convention (protocol.rs Response doc-comment) - 22 unit tests incl. red-checked regressions; no new dependencies, no LLM calls, no caching
1 parent 0b04f9a commit f42c5ec

9 files changed

Lines changed: 1629 additions & 5 deletions

File tree

ARCHITECTURE.md

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@
5959
**Protocol and command layer:**
6060
- Purpose: Accept NDJSON requests, route tool calls via the unified `tool_call` command, and dispatch them to focused command handlers.
6161
- Location: `crates/aft/src/main.rs`, `crates/aft/src/protocol.rs`, `crates/aft/src/commands/`, `crates/aft/src/run_tool_call.rs`, `crates/aft/src/subc_translate.rs`, `crates/aft/src/subc_format.rs`
62-
- Contains: Request dispatch, response encoding, a unified `tool_call` routing engine, tool-to-command translation mapping, server-rendered agent-facing text formatting, control channel 0 health check responder, and standalone command handlers for read/write/edit/apply_patch/delete_file/move_file/outline/zoom/bash/bash_orchestrate/bash_status/bash_wait_detach/batch/grep/glob/search/imports/refactor/LSP/inspect/conflicts/checkpoints/state
62+
- Contains: Request dispatch, response encoding, a unified `tool_call` routing engine, tool-to-command translation mapping, server-rendered agent-facing text formatting, control channel 0 health check responder, and standalone command handlers for read/write/edit/apply_patch/delete_file/move_file/outline/zoom/bash/bash_orchestrate/bash_status/bash_wait_detach/batch/grep/glob/search/gather/imports/refactor/LSP/inspect/conflicts/checkpoints/state
6363
- Depends on: `crates/aft/src/context.rs`, `crates/aft/src/parser.rs`, `crates/aft/src/callgraph.rs`, `crates/aft/src/callgraph_store/mod.rs`, `crates/aft/src/edit.rs`, `crates/aft/src/semantic_index.rs`, `crates/aft/src/search_index.rs`, `crates/aft/src/compress/`
6464
- Used by: `packages/aft-bridge/src/bridge.ts`
6565

@@ -110,6 +110,14 @@
110110
3. Classify query shape (prose vs code) using the query shape parser -- `crates/aft/src/query_shape.rs`. Identify "type-concept identifier queries" (TitleCase PascalCase types combined with lowercase concepts) to trigger definition semantic priors.
111111
4. Serve `grep` (trigram, full-text) and `aft_search` (semantic + hybrid) queries, delegating to `GrepExecutor` for accelerated path evaluation and enforcing execution safety limits (like `MAX_FALLBACK_WALK_FILES` and `FALLBACK_WALK_BUDGET`) during fallback walks when indexes are building or unavailable -- `crates/aft/src/grep_executor.rs`, `crates/aft/src/commands/grep.rs`, `crates/aft/src/commands/semantic_search.rs`. Interactive query embeddings are bounded by a dedicated `query_timeout_ms` configuration (clamped to 500..15000ms, defaulting to 3000ms) enforced via a `QueryBudget` to keep interactive requests fast without affecting background build/refresh timeouts, falling back to lexical search if query embedding fails or times out. Downrank generated documentation artifacts (e.g. minified CSS/JS, maps, SVGs) in lexical and hybrid search results. For external search requests, resolve and cache external git roots, querying cached read-only search and semantic indexes from the `borrowed_index_cache` (capped at 4 concurrent entries) to avoid redundant git probes and disk parsing.
112112

113+
**Context-pack gather flow:**
114+
115+
1. Accept one of two mutually exclusive modes -- `crates/aft/src/commands/gather.rs`, `crates/aft/src/subc_translate.rs::translate_gather`. In question mode, seed with a natural-language query (semantic search). In symbol mode, seed with a `(symbol, filePath)` pair (callgraph impact: depth-1 callers via `impact` + depth-1 callees via `call_tree`). Mode validation rejects both or neither with `invalid_request`, and the OpenCode plugin enforces the same XOR at the schema boundary in `packages/opencode-plugin/src/tools/gather.ts`.
116+
2. Resolve seeds and 1-hop neighbors from the persisted indexes -- `crates/aft/src/callgraph_store/mod.rs`, `crates/aft/src/commands/semantic_search.rs`. Question mode routes through `handle_semantic_search` (the same pipeline as `aft_search`), so hybrid semantic + lexical scoring and the `query_timeout_ms` budget apply. Symbol mode uses `impact` and `call_tree` against the SQLite store, dropping paths outside the project root via `pending_path_in_roots` containment. Unresolved external callees collapse to a single summary line; seeds and unresolved callers are never suppressed.
117+
3. Dedupe candidates by canonicalized `(file, symbol)` (project-root-prefixed absolute paths merged with relative hits) -- `crates/aft/src/commands/gather.rs::dedup_by_file_and_name`. Seeds win on conflict; the same canonical symbol reached through two routes is rendered once.
118+
4. Render each candidate's symbol body within a hard line budget (default 400, cap 800) via `render_symbol_within_budget` -- `crates/aft/src/commands/gather.rs::render_symbol_section`, `crates/aft/src/commands/symbol_render.rs`. Grep-fallback hits resolve to their containing symbol by line containment (`resolve_containing_symbol`); hits with no containing symbol stay visible one-line stubs so nothing is silently dropped. Over-budget candidates are emitted under `## Beyond budget (zoom to expand)` as stubs rather than truncated, so the agent can see what was excluded.
119+
5. Flag index degradation honestly in the pack header. A `semantic_status: "building"` field surfaces when the semantic index is still building AND no symbol-mode seed resolved -- suppressed the moment any symbol seed provides a real anchor. Resolve relative paths against `project_root` to keep `file:line` headers stable across worktree mounts. Follow the tri-state honest-reporting convention (`crates/aft/src/protocol.rs` `Response` doc-comment) so the agent can distinguish empty scope, partial pack, and complete pack without guessing.
120+
113121
**File read flow:**
114122

115123
1. Map read arguments and validate boundary permissions -- `packages/opencode-plugin/src/tools/reading.ts`, `packages/pi-plugin/src/tools/reading.ts`. Under project-root path restriction, allow restricted reading of files outside the project root if they are session-owned bash artifact outputs (stdout, stderr, exit code, or pty outputs) registered under the requesting session ID (validated via `AppContext::validate_read_path` using `BgTaskRegistry::is_session_owned_artifact_path`), while strictly rejecting any mutations (which continue to enforce project root boundaries via `AppContext::validate_path`). The plugin skips the external-directory permission prompt for session-owned bash task artifacts under the AFT storage root when performing server-validated reads, avoiding hangs in unattended runs.
@@ -191,7 +199,7 @@
191199

192200
**Tool groups (OpenCode):**
193201
- Purpose: Group related OpenCode tool definitions by capability surface.
194-
- Location: `packages/opencode-plugin/src/tools/hoisted.ts`, `packages/opencode-plugin/src/tools/reading.ts`, `packages/opencode-plugin/src/tools/imports.ts`, `packages/opencode-plugin/src/tools/structure.ts`, `packages/opencode-plugin/src/tools/navigation.ts`, `packages/opencode-plugin/src/tools/refactoring.ts`, `packages/opencode-plugin/src/tools/safety.ts`, `packages/opencode-plugin/src/tools/conflicts.ts`, `packages/opencode-plugin/src/tools/lsp.ts`, `packages/opencode-plugin/src/tools/ast.ts`, `packages/opencode-plugin/src/tools/bash.ts`, `packages/opencode-plugin/src/tools/bash_watch.ts`, `packages/opencode-plugin/src/tools/bash_write.ts`, `packages/opencode-plugin/src/tools/inspect.ts`, `packages/opencode-plugin/src/tools/search.ts`, `packages/opencode-plugin/src/tools/semantic.ts`, `packages/opencode-plugin/src/tools/permissions.ts`, `packages/opencode-plugin/src/tools/hoisted-internals.ts`
202+
- Location: `packages/opencode-plugin/src/tools/hoisted.ts`, `packages/opencode-plugin/src/tools/reading.ts`, `packages/opencode-plugin/src/tools/imports.ts`, `packages/opencode-plugin/src/tools/structure.ts`, `packages/opencode-plugin/src/tools/navigation.ts`, `packages/opencode-plugin/src/tools/refactoring.ts`, `packages/opencode-plugin/src/tools/safety.ts`, `packages/opencode-plugin/src/tools/conflicts.ts`, `packages/opencode-plugin/src/tools/lsp.ts`, `packages/opencode-plugin/src/tools/ast.ts`, `packages/opencode-plugin/src/tools/bash.ts`, `packages/opencode-plugin/src/tools/bash_watch.ts`, `packages/opencode-plugin/src/tools/bash_write.ts`, `packages/opencode-plugin/src/tools/inspect.ts`, `packages/opencode-plugin/src/tools/search.ts`, `packages/opencode-plugin/src/tools/semantic.ts`, `packages/opencode-plugin/src/tools/gather.ts`, `packages/opencode-plugin/src/tools/permissions.ts`, `packages/opencode-plugin/src/tools/hoisted-internals.ts`
195203
- Pattern: Thin TypeScript adapters delegating to the unified `tool_call` transport
196204

197205
**Tool groups (Pi):**

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,7 @@ AFT is **1 of the 3 plugins you'll ever need.** It perceives and acts; Magic Con
9494
- **`aft_search`**: find code by *meaning* when grep keywords fall short. Hybrid semantic + lexical retrieval over an indexed codebase, with local, OpenAI-compatible, or Ollama embedding backends.
9595
- **`aft_callgraph`**: follow callers, callees, data flow, impact analysis, and the shortest call path between two symbols across the workspace.
9696
- **`aft_inspect`**: a one-call codebase-health report covering LSP errors and warnings, TODOs, metrics, dead code, unused exports, and duplicates. The Problems and inspections panels an IDE keeps open, on demand.
97+
- **`aft_gather`**: assemble a bounded context pack — ranked, deduped, budgeted verbatim code evidence — in one call instead of a serial `search → outline → zoom → callgraph` chain. Seed from a `question` (semantic) or a `symbol`+`filePath` (callgraph), expand one hop, render within a hard line budget.
9798
- **`grep` / `glob`**: trigram-indexed regex search and file discovery, built in the background, persisted to disk, and kept fresh by a file watcher.
9899

99100
---

STRUCTURE.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ opencode-aft/
5050
**`crates/aft/src/commands/`:**
5151
- Purpose: Add one handler file per protocol command.
5252
- Contains: ~60 command-specific request parsing and response generation modules
53-
- Key files: `crates/aft/src/commands/tool_call.rs`, `crates/aft/src/commands/read.rs`, `crates/aft/src/commands/write.rs`, `crates/aft/src/commands/apply_patch.rs`, `crates/aft/src/commands/bash_orchestrate.rs`, `crates/aft/src/commands/bash_wait_detach.rs`, `crates/aft/src/commands/outline.rs`, `crates/aft/src/commands/zoom.rs`, `crates/aft/src/commands/bash.rs`, `crates/aft/src/commands/grep.rs`, `crates/aft/src/commands/semantic_search.rs`, `crates/aft/src/commands/configure.rs`
53+
- Key files: `crates/aft/src/commands/tool_call.rs`, `crates/aft/src/commands/read.rs`, `crates/aft/src/commands/write.rs`, `crates/aft/src/commands/apply_patch.rs`, `crates/aft/src/commands/bash_orchestrate.rs`, `crates/aft/src/commands/bash_wait_detach.rs`, `crates/aft/src/commands/outline.rs`, `crates/aft/src/commands/zoom.rs`, `crates/aft/src/commands/gather.rs`, `crates/aft/src/commands/bash.rs`, `crates/aft/src/commands/grep.rs`, `crates/aft/src/commands/semantic_search.rs`, `crates/aft/src/commands/configure.rs`
5454

5555
**`crates/aft/src/compress/`:**
5656
- Purpose: Provide tiered output compression for hoisted bash commands.
@@ -114,7 +114,7 @@ opencode-aft/
114114

115115
**`packages/opencode-plugin/src/tools/`:**
116116
- Purpose: Group OpenCode tool definitions by capability area.
117-
- Contains: Thin adapters for hoisted, reading, import, structure, navigation, refactor, safety, bash, conflict, AST, LSP, search, semantic, and inspect tools; permissions and internals helpers
117+
- Contains: Thin adapters for hoisted, reading, import, structure, navigation, refactor, safety, bash, conflict, AST, LSP, search, semantic, gather, and inspect tools; permissions and internals helpers
118118
- Key files: `packages/opencode-plugin/src/tools/_shared.ts`, `packages/opencode-plugin/src/tools/hoisted.ts`, `packages/opencode-plugin/src/tools/reading.ts`, `packages/opencode-plugin/src/tools/refactoring.ts`, `packages/opencode-plugin/src/tools/bash.ts`, `packages/opencode-plugin/src/tools/inspect.ts`, `packages/opencode-plugin/src/tools/search.ts`
119119

120120
**`packages/pi-plugin/`:**

0 commit comments

Comments
 (0)