Skip to content

Commit fb73e0d

Browse files
feat: tiered show/snippet lookup fast paths (#184)
* feat: tiered show lookup fast paths and covering index Route lone name:Token queries to equality lookup; add idx_symbols_name_covering; document fast vs slow tiers in CLI/MCP help. * harden: lift tiered lookup docs across consumer surfaces Fix roadmap anchor, glossary/show recipe cross-refs, mcp-instructions tiers, and consumer-clean changeset wording. * harden: snippet help and rule parity for lookup tiers Mirror show lookup-tier docs on snippet --help; extend served rule and MCP instructions fast-tier row for show/snippet parity. * harden: consumer-surface parity and stricter EXPLAIN test Document optional kind/in on fast-tier show/snippet across MCP, rule, and skill; align snippet CLI help and changeset; assert COVERING INDEX use. * fix: align show/snippet integration tests with lookup tiers Lone name:Token without wildcards is fast-tier equality; substring and multi-field queries use slow-tier LIKE (kind:class name:Auth). * chore: run paired unit tests when staging source files lint-staged relatedTests runs co-located *.test.ts when a src/ source file is staged without its test; document in verify-after-each-step. * harden: lint-staged test globs and verify-after-each-step parity Add *.test.tsx glob for co-staged pairs; align rule table and Rule 4 with lint-staged.config.js (scripts/**/*.test.mjs, relatedTests scope).
1 parent b509e4c commit fb73e0d

22 files changed

Lines changed: 397 additions & 197 deletions

.agents/rules/verify-after-each-step.md

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -19,12 +19,14 @@ AI agents tend to chain many edits across files and only discover breakage at co
1919

2020
## Current Per-File Checks (from `lint-staged.config.js`)
2121

22-
| File pattern | Checks |
23-
| ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
24-
| `*.{js,jsx,ts,tsx,mjs,mts,cjs,cts}` | `bun run format:check`, `bun run lint` |
25-
| `*.{css,json,md,mdc,html,yaml,yml}` | `bun run format:check` |
26-
| `*.{ts,tsx}` | `bun run typecheck` with a temporary `tsconfig.lint-staged.json` that includes only **staged files under `src/`** (project-wide types still interconnect — use `bun run typecheck` if you need full-project certainty) |
27-
| `*.test.ts` | `bun test` (on changed test files) |
22+
| File pattern | Checks |
23+
| ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
24+
| `*.{js,jsx,ts,tsx,mjs,mts,cjs,cts}` | `bun run format:check`, `bun run lint` |
25+
| `*.{css,json,md,mdc,html,yaml,yml}` | `bun run format:check` |
26+
| `*.{ts,tsx}` | `bun run typecheck` with a temporary `tsconfig.lint-staged.json` that includes only **staged files under `src/`** (project-wide types still interconnect — use `bun run typecheck` if you need full-project certainty); **`bun test`** on co-located `*.test.{ts,tsx}` pairs when a staged `src/` source file is present but its test file is not |
27+
| `*.test.ts` | `bun test` (on changed test files) |
28+
| `*.test.tsx` | `bun test` (on changed test files) |
29+
| `scripts/**/*.test.mjs` | `bun test` (on changed test files) |
2830

2931
## What Counts as a Step
3032

@@ -41,6 +43,6 @@ A "step" is any self-contained unit of work where you've finished editing and ar
4143
1. **Verify after every step** — Run the matching checks on every file you touched during that step before moving to the next one.
4244
2. **Fix before moving on** — If any check fails, fix it immediately while context is fresh. Never carry forward known failures.
4345
3. **Use the right scope** — Run `bun run lint` and `bun run format:check` on specific files when possible. Prefer `bun run typecheck` project-wide when types may depend on unstaged files.
44-
4. **Run affected tests** — If you modified or created `*.test.ts` files, run `bun test <file>` on them.
46+
4. **Run affected tests** — If you modified or created `*.test.ts` / `*.test.tsx` / `scripts/**/*.test.mjs` files, run `bun test <file>` on them; if you changed a `src/` source file with a co-located test, run that pair even when the test file itself is unstaged.
4547
5. **Re-index before querying Codemap** — If you changed indexed source and plan to run SQL against the structural index next, run `bun src/index.ts --files <paths>` with paths **relative to the indexed project root** (set `CODEMAP_TEST_BENCH` / `CODEMAP_ROOT` or `--root` so that root is correct — see [docs/benchmark.md § Indexing another project](../../docs/benchmark.md#indexing-another-project)).
4648
6. **Don't duplicate the hook's job** — You don't need to re-verify at commit time; the pre-commit hook (`lint-staged`) handles that automatically when AI/agent env vars trigger it. Your job is to stay green _between_ commits.
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"@stainless-code/codemap": patch
3+
---
4+
5+
`show` and `snippet` now use fast equality lookup for exact `name` and lone `name:Token` queries (no wildcards); substring, multi-field, and FTS paths stay on the broader slow tier. CLI help, MCP tool descriptions, and bundled agent guidance document the two tiers.

docs/architecture.md

Lines changed: 15 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -190,7 +190,7 @@ Three **mutually exclusive** CLI entry shapes; all converge on `applyDiffPayload
190190

191191
**Backlog (not rejected):** `organize-imports` diff-shape recipe; `codemap-to-tsmorph` Path B adapter — tracked in [roadmap.md § Backlog](./roadmap.md#backlog). **Tracked elsewhere:** C.9 entry-point integration — [`plans/c9-plugin-layer.md`](./plans/c9-plugin-layer.md).
192192

193-
**Show / snippet wiring:** **`src/cli/show-snippet-args.ts`** (shared argv parser) + **`src/cli/show-snippet-render.ts`** (shared terminal/JSON error helpers) + **`src/cli/cmd-show.ts`** + **`src/cli/cmd-snippet.ts`** — sibling CLI verbs sharing the same parser shape (`<name>` or **`--query '<field:value …>'`** + **`--with-fts`** + `--kind` + `--in <path>` + `--json`; show adds **`--print-sql`**) and the pure engines **`src/application/show-engine.ts`** (exact lookup + envelope builders), **`src/application/search-query-parser.ts`** + **`src/application/search-engine.ts`** (field-qualified search → parameterized SQL on `symbols`, optional `source_fts` join), and **`src/application/show-search-mode.ts`** (shared parse/normalize + FTS resolution + **`executeShowLookup`** + **`formatShowSearchSqlForQuery`** for CLI/MCP/HTTP). Exact lookup: `findSymbolsByName({db, name, kind?, inPath?})`. Query lookup: `searchSymbols({db, parsed, withFts?})`. Snippet FS read: `readSymbolSource({match, projectRoot, indexedContentHash?})` + `getIndexedContentHash(db, filePath)`. **`buildShowResult`** + **`buildSnippetResult`** envelope builders — same engines the MCP show/snippet tools call. Both verbs return the same `{matches, disambiguation?, warning?}` envelope — single match → `{matches: [{...}]}`; multi-match adds `{n, by_kind, files, hint}`; optional **`warning`** when FTS was requested but `source_fts` is empty. Snippet matches add `source` / `stale` / `missing` fields (additive — no shape divergence). **`--in <path>`** and **`path:`** inside **`--query`** normalize through `toProjectRelative(projectRoot, p)` (from **`src/application/validate-engine.ts`**). Stale-file behavior on `snippet`: `hashContent` (from **`src/hash.ts`**) compares on-disk content against `files.content_hash`; mismatch sets `stale: true` but source IS still returned. MCP tools `show` and `snippet` register parallel to the CLI surface (see [§ MCP wiring](#cli-usage)).
193+
**Show / snippet wiring:** **`src/cli/show-snippet-args.ts`** (shared argv parser) + **`src/cli/show-snippet-render.ts`** (shared terminal/JSON error helpers) + **`src/cli/cmd-show.ts`** + **`src/cli/cmd-snippet.ts`** — sibling CLI verbs sharing the same parser shape (`<name>` or **`--query '<field:value …>'`** + **`--with-fts`** + `--kind` + `--in <path>` + `--json`; show adds **`--print-sql`**) and the pure engines **`src/application/show-engine.ts`** (exact lookup + envelope builders), **`src/application/search-query-parser.ts`** + **`src/application/search-engine.ts`** (field-qualified search → parameterized SQL on `symbols`, optional `source_fts` join), and **`src/application/show-search-mode.ts`** (shared parse/normalize + FTS resolution + tiered routing via **`resolveExactNameFromParsedQuery`** / **`isExactNamePattern`** + **`executeShowLookup`** + **`formatShowSearchSqlForQuery`** for CLI/MCP/HTTP). **Fast tier:** positional `<name>` or lone `name:Token` without `%`/`_` wildcards (no `kind`/`path`/`in`/free text) → `findSymbolsByName` (`name = ?`, **`idx_symbols_name_covering`**). **Slow tier:** `name LIKE` substring, multi-field query, or FTS free-text → `searchSymbols`. Exact lookup with filters: `findSymbolsByName({db, name, kind?, inPath?})`. Snippet FS read: `readSymbolSource({match, projectRoot, indexedContentHash?})` + `getIndexedContentHash(db, filePath)`. **`buildShowResult`** + **`buildSnippetResult`** envelope builders — same engines the MCP show/snippet tools call. Both verbs return the same `{matches, disambiguation?, warning?}` envelope — single match → `{matches: [{...}]}`; multi-match adds `{n, by_kind, files, hint}`; optional **`warning`** when FTS was requested but `source_fts` is empty. Snippet matches add `source` / `stale` / `missing` fields (additive — no shape divergence). **`--in <path>`** and **`path:`** inside **`--query`** normalize through `toProjectRelative(projectRoot, p)` (from **`src/application/validate-engine.ts`**). Stale-file behavior on `snippet`: `hashContent` (from **`src/hash.ts`**) compares on-disk content against `files.content_hash`; mismatch sets `stale: true` but source IS still returned. MCP tools `show` and `snippet` register parallel to the CLI surface (see [§ MCP wiring](#cli-usage)).
194194

195195
**Evidence columns (high-judgment recipes):** Some bundled recipes add optional **`reason`** and **`evidence_json`** TEXT columns on each result row — factual detection path for agents, not pass/fail verdicts. Contract: [golden-queries.md § Evidence columns](./golden-queries.md#evidence-columns-high-judgment-recipes).
196196

@@ -1053,16 +1053,17 @@ A covering index includes all columns needed by a query, so SQLite never touches
10531053

10541054
Key covering indexes:
10551055

1056-
| Index | Columns | Covers |
1057-
| ------------------------ | --------------------------------------------------------------------- | -------------------------- |
1058-
| `idx_symbols_name` | `name, kind, file_path, line_start, line_end, signature, is_exported` | Symbol lookup by name |
1059-
| `idx_imports_source` | `source, file_path` | "Who imports X?" queries |
1060-
| `idx_imports_resolved` | `resolved_path, file_path` | Resolved path lookups |
1061-
| `idx_exports_name` | `name, file_path, kind, is_default` | Export lookup by name |
1062-
| `idx_components_name` | `name, file_path, props_type, hooks_used` | Component search by name |
1063-
| `idx_components_file` | `file_path, name` | Components in a directory |
1064-
| `idx_dependencies_to` | `to_path, from_path` | Reverse dependency lookups |
1065-
| `idx_markers_kind` | `kind, file_path, line_number, content` | Marker listing by kind |
1066-
| `idx_css_variables_name` | `name, value, scope, file_path` | CSS token lookup by name |
1067-
| `idx_css_classes_name` | `name, file_path, is_module` | CSS class lookup |
1068-
| `idx_css_keyframes_name` | `name, file_path` | Keyframe lookup |
1056+
| Index | Columns | Covers |
1057+
| --------------------------- | ---------------------------------------------------------------------------------------------- | ---------------------------------------------------- |
1058+
| `idx_symbols_name` | `name, kind, file_path, line_start, line_end, signature, is_exported` | Symbol lookup by name |
1059+
| `idx_symbols_name_covering` | `name, kind, file_path, line_start, line_end, signature, is_exported, parent_name, visibility` | Full `findSymbolsByName` SELECT without table lookup |
1060+
| `idx_imports_source` | `source, file_path` | "Who imports X?" queries |
1061+
| `idx_imports_resolved` | `resolved_path, file_path` | Resolved path lookups |
1062+
| `idx_exports_name` | `name, file_path, kind, is_default` | Export lookup by name |
1063+
| `idx_components_name` | `name, file_path, props_type, hooks_used` | Component search by name |
1064+
| `idx_components_file` | `file_path, name` | Components in a directory |
1065+
| `idx_dependencies_to` | `to_path, from_path` | Reverse dependency lookups |
1066+
| `idx_markers_kind` | `kind, file_path, line_number, content` | Marker listing by kind |
1067+
| `idx_css_variables_name` | `name, value, scope, file_path` | CSS token lookup by name |
1068+
| `idx_css_classes_name` | `name, file_path, is_module` | CSS class lookup |
1069+
| `idx_css_keyframes_name` | `name, file_path` | Keyframe lookup |

docs/glossary.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -194,7 +194,7 @@ TS shape for the JSON emitted by `codemap context`. Stable contract; agents can
194194

195195
### covering index
196196

197-
A SQLite index that includes every column needed by a query, so SQLite reads everything from the index B-tree without touching the main table. The query plan shows `USING COVERING INDEX`. Used heavily for AI agent query patterns — see [architecture § Covering indexes](./architecture.md#covering-indexes).
197+
A SQLite index that includes every column needed by a query, so SQLite reads everything from the index B-tree without touching the main table. The query plan shows `USING COVERING INDEX`. Used heavily for AI agent query patterns — e.g. `idx_symbols_name_covering` for `codemap show` equality lookups. See [architecture § Covering indexes](./architecture.md#covering-indexes).
198198

199199
### `css_classes` (table)
200200

@@ -543,11 +543,11 @@ Long-running transport shutdown rules in `src/application/session-lifecycle.ts`.
543543

544544
### show
545545

546-
`codemap show <name>` — one-step lookup that returns metadata (`file_path:line_start-line_end` + `signature` + `kind`) for symbol(s). **Exact mode:** `<name>` is case-sensitive; flags `--kind`, `--in`. **Field-qualified mode:** `--query 'kind:… name:… path:… in:…'` with optional free text; `--with-fts` (or `fts5: true` when indexed) searches file bodies via `source_fts` and returns every symbol in matching files; `--print-sql` prints Moat-A equivalent SQL. Output: `{matches, disambiguation?, warning?}` (`warning` when FTS was requested but `source_fts` is empty). MCP: `show` with `{name}` or `{query, with_fts?}`. Distinct from **snippet** (adds source text) and from hand-composed `query` SQL. See [`architecture.md` § Show / snippet wiring](./architecture.md#cli-usage).
546+
`codemap show <name>` — one-step lookup that returns metadata (`file_path:line_start-line_end` + `signature` + `kind`) for symbol(s). **Fast tier (equality index):** positional `<name>` or lone `name:Token` with no `%`/`_` wildcards and no other query fields — same rows as exact `<name>`. **Slow tier:** `name:%pat%` substring LIKE, multi-field `--query`, or free text (`name LIKE` / `source_fts` with `--with-fts` or `fts5: true`). **Exact mode flags:** `--kind`, `--in` (positional only). `--print-sql` prints Moat-A equivalent SQL. Output: `{matches, disambiguation?, warning?}` (`warning` when FTS was requested but `source_fts` is empty). MCP: `show` with `{name}` or `{query, with_fts?}`. Distinct from **snippet** (adds source text) and from hand-composed `query` SQL. See [`architecture.md` § Show / snippet wiring](./architecture.md#cli-usage).
547547

548548
### snippet
549549

550-
`codemap snippet <name>` — same lookup modes as **show** (`<name>` + `--kind` / `--in`, or `--query` + `--with-fts`), but each match also carries `source` (file lines from disk at `line_start..line_end`), `stale` (true when content_hash drifted since last index), and `missing` (true when file is gone). Envelope: `{matches, disambiguation?, warning?}` with the same additive fields on each row. Stale-file behavior: `source` is always returned when the file exists; `stale: true` is metadata the agent reads (no auto-reindex). MCP: `snippet` with `{name}` or `{query, with_fts?}`. See [`architecture.md` § Show / snippet wiring](./architecture.md#cli-usage).
550+
`codemap snippet <name>` — same fast/slow lookup tiers as **show** (`<name>` + `--kind` / `--in`, or `--query` + `--with-fts`), but each match also carries `source` (file lines from disk at `line_start..line_end`), `stale` (true when content_hash drifted since last index), and `missing` (true when file is gone). Envelope: `{matches, disambiguation?, warning?}` with the same additive fields on each row. Stale-file behavior: `source` is always returned when the file exists; `stale: true` is metadata the agent reads (no auto-reindex). MCP: `snippet` with `{name}` or `{query, with_fts?}`. See [`architecture.md` § Show / snippet wiring](./architecture.md#cli-usage).
551551

552552
### `suppressions` (table) / `// codemap-ignore-next-line` / `// codemap-ignore-file`
553553

0 commit comments

Comments
 (0)