Skip to content

Commit f181052

Browse files
SutuSebastianSutuSebastian
committed
docs: fact-check call resolution and bench coverage docs
Align architecture with deletion-only resolve scope and method-call NULL kinds; fix testing-coverage paths and agent-eval probe wording; sync CAPABILITIES.json substrate scenario ids with testing-coverage table.
1 parent 60fe33a commit f181052

6 files changed

Lines changed: 48 additions & 39 deletions

File tree

‎docs/architecture.md‎

Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -245,26 +245,26 @@ All base tables use `STRICT` mode; **`source_fts`** is an FTS5 virtual table (no
245245

246246
### `calls` — Function-scoped call edges, deduped per file (`STRICT`)
247247

248-
| Column | Type | Description |
249-
| ---------------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------- |
250-
| id | INTEGER PK | Auto-increment row id |
251-
| file_path | TEXT FK | References `files(path)` ON DELETE CASCADE |
252-
| caller_name | TEXT | Name of the calling function/method |
253-
| caller_scope | TEXT | Dot-joined scope path (e.g. `UserService.run`). Anonymous scopes encode as `$anon_<localId>` to avoid sibling-callback collisions |
254-
| callee_name | TEXT | Name of the called function, `obj.method` / `obj.foo.bar` for member chains (recursive flatten), `this.method` for self |
255-
| line_start | INTEGER | 1-based line of the callee identifier token (per [R.6]) |
256-
| column_start | INTEGER | 0-based byte column of the callee token |
257-
| column_end | INTEGER | One-past-last column |
258-
| args_count | INTEGER | Argument count; NULL when a spread argument is present |
259-
| is_method_call | INTEGER | 1 when callee is a member expression (`obj.method()`) |
260-
| is_constructor_call | INTEGER | 1 for `new Foo()` (`NewExpression`) |
261-
| is_optional_chain | INTEGER | 1 when the call uses optional chaining (`?.`) |
262-
| callee_symbol_id | INTEGER FK | Resolved callee in `symbols` (NULL when unresolved / external global) |
263-
| callee_resolution_kind | TEXT | `same-file`, `imported`, `re-exported`, `global`, or `unresolved` — filled by `resolveCalls` after bindings |
248+
| Column | Type | Description |
249+
| ---------------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
250+
| id | INTEGER PK | Auto-increment row id |
251+
| file_path | TEXT FK | References `files(path)` ON DELETE CASCADE |
252+
| caller_name | TEXT | Name of the calling function/method |
253+
| caller_scope | TEXT | Dot-joined scope path (e.g. `UserService.run`). Anonymous scopes encode as `$anon_<localId>` to avoid sibling-callback collisions |
254+
| callee_name | TEXT | Name of the called function, `obj.method` / `obj.foo.bar` for member chains (recursive flatten), `this.method` for self |
255+
| line_start | INTEGER | 1-based line of the callee identifier token (per [R.6]) |
256+
| column_start | INTEGER | 0-based byte column of the callee token |
257+
| column_end | INTEGER | One-past-last column |
258+
| args_count | INTEGER | Argument count; NULL when a spread argument is present |
259+
| is_method_call | INTEGER | 1 when callee is a member expression (`obj.method()`) |
260+
| is_constructor_call | INTEGER | 1 for `new Foo()` (`NewExpression`) |
261+
| is_optional_chain | INTEGER | 1 when the call uses optional chaining (`?.`) |
262+
| callee_symbol_id | INTEGER FK | Resolved callee in `symbols` (NULL when unresolved / external global) |
263+
| callee_resolution_kind | TEXT | `same-file`, `imported`, `re-exported`, `global`, or `unresolved` — set by `resolveCalls` after bindings; NULL on method calls (`is_method_call = 1`, deferred) |
264264

265265
Edges are deduped per (caller_scope, callee, call vs constructor) per file: if `foo` calls `bar` three times in the same file, only one row is stored. `foo()` and `new Foo()` with the same callee name remain distinct rows. Same-named methods in different classes get distinct `caller_scope` values. Module-level calls (outside any function) are excluded — only function-scoped calls are tracked.
266266

267-
**Call resolution:** `src/application/call-resolver.ts` runs after bindings on full rebuild and after incremental file updates. Incremental scope is `expandHeritageResolveScope` over all requested/changed paths (includes importers). Method calls (`is_method_call = 1`) are not name-bound yet. Unresolved sites are staged in `unresolved_calls`; `meta.unresolved_calls_residual` is the global queue `COUNT(*)`.
267+
**Call resolution:** `src/application/call-resolver.ts` runs after bindings on full rebuild and after incremental file updates. Incremental scope is `expandHeritageResolveScope` over changed/requested paths **and** deletion-only paths (includes importers). Method calls (`is_method_call = 1`) are not name-bound yet (`callee_*` stay NULL; not queued). Unresolved sites are staged in `unresolved_calls`; `meta.unresolved_calls_residual` is the global queue `COUNT(*)`.
268268

269269
### `unresolved_calls` — Staging queue for unresolved call sites (`STRICT`)
270270

‎docs/plans/in-repo-test-bench.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ fixtures/
4949

5050
- [x] `src/cli/cmd-test-bench-e2e.test.ts` — show, snippet, impact, validate, `shop-symbols`, SARIF (`boundary-violations`)
5151
- [x] `cmd-cli-parity-e2e.test.ts` — trace, explore, node, context, batch, resources
52-
- [x] Expand `test:agent-eval` to one probe per `CAPABILITIES.json` group (18 probes; guarded by `capability-probes.test.mjs`)
52+
- [x] Expand `test:agent-eval` to one probe per `CAPABILITIES.json` capability id (18 probes; 15 golden-backed + 3 `unitTests`/`enforcedBy` groups; `capability-probes.test.mjs`)
5353

5454
### Phase 4 — Scale (optional, replaces Tier B′)
5555

‎docs/roadmap.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ _(none — P1 agent-surface work complete)_
6868
- [ ] **Framework route extraction** — Express / React Router / NestJS `http_routes` substrate. Plan: [`plans/framework-route-extraction.md`](./plans/framework-route-extraction.md). Blocked on C.9 contract. Effort: L.
6969
- [ ] **Callback dispatch synthesis** — heuristic `calls` with `provenance`. Plan: [`plans/callback-dispatch-synthesis.md`](./plans/callback-dispatch-synthesis.md). Effort: L.
7070
- [x] **In-repo test bench depth** — Phases 1–3: `fixtures/README.md`, `CAPABILITIES.json`, substrate goldens, method-call slice, shop-symbols recipe, CLI/MCP e2e + agent-eval probes — all without external `CODEMAP_ROOT`. Shipped [#162](https://github.com/stainless-code/codemap/pull/162). Optional Phase 4 scale: [`plans/in-repo-test-bench.md`](./plans/in-repo-test-bench.md).
71-
- [x] **Unresolved calls staging** — two-phase call resolution queue (`unresolved_calls`, scoped incremental `resolveCalls`). Shipped [#162](https://github.com/stainless-code/codemap/pull/162). Details: [`architecture.md`](./architecture.md) (`calls`, `unresolved_calls`, `call-resolver.ts`).
71+
- [x] **Unresolved calls staging** — two-phase call resolution queue (`unresolved_calls`, scoped incremental `resolveCalls`). Shipped [#162](https://github.com/stainless-code/codemap/pull/162). Details: [`architecture.md`](./architecture.md) (`calls`, `unresolved_calls`, `src/application/call-resolver.ts`).
7272
- [ ] **Cross-project MCP root** — optional `root` on tools + DB cache. Plan: [`plans/cross-project-mcp-root.md`](./plans/cross-project-mcp-root.md). Effort: M.
7373
- [ ] **FTS default-on evaluation** — measure DB size tax; maybe flip default. Plan: [`plans/fts-default-on-evaluation.md`](./plans/fts-default-on-evaluation.md). Effort: S–M.
7474

‎docs/testing-coverage.md‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -8,15 +8,15 @@
88

99
## Harness layers
1010

11-
| Layer | Command | What it proves |
12-
| ---------------------- | ---------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
13-
| **Unit** | `bun test ./src` | Parsers, DB DDL, engines, CLI/MCP handlers in isolation (`:memory:` or mocks). |
14-
| **In-repo test bench** | `bun run test:golden` | Index `fixtures/minimal/` (bench corpus) → compare `fixtures/golden/minimal/*.json`. Map: `fixtures/CAPABILITIES.json`. |
15-
| **Golden guard** | `bun run test:scripts` | `scripts/query-golden-coverage-matrix.test.mjs` — every bundled recipe + substrate table has a scenario. |
16-
| **Agent eval** | `bun run test:agent-eval` | Probe arms vs golden ids (MCP-on vs glob/read). |
17-
| **Integration (git)** | `src/application/run-index.test.ts` | `runCodemapIndex` incremental paths: heritage + calls re-resolution, delete/reindex. |
18-
| **CLI e2e** | `cmd-test-bench-e2e.test.ts`, `cmd-cli-parity-e2e.test.ts` | Spawned CLI on `fixtures/minimal` (bench smoke + resource parity). |
19-
| **Check** | `bun run check` | build + lint + unit + scripts + golden + agent-eval. |
11+
| Layer | Command | What it proves |
12+
| ---------------------- | -------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
13+
| **Unit** | `bun test ./src` | Parsers, DB DDL, engines, CLI/MCP handlers in isolation (`:memory:` or mocks). |
14+
| **In-repo test bench** | `bun run test:golden` | Index `fixtures/minimal/` (bench corpus) → compare `fixtures/golden/minimal/*.json`. Map: `fixtures/CAPABILITIES.json`. |
15+
| **Golden guard** | `bun run test:scripts` | `scripts/query-golden-coverage-matrix.test.mjs` — every bundled recipe + substrate table has a scenario. |
16+
| **Agent eval** | `bun run test:agent-eval` | Probe arms vs golden ids (MCP-on vs glob/read). |
17+
| **Integration (git)** | `src/application/run-index.test.ts` | `runCodemapIndex` incremental paths: heritage + calls re-resolution, delete/reindex. |
18+
| **CLI e2e** | `src/cli/cmd-test-bench-e2e.test.ts`, `src/cli/cmd-cli-parity-e2e.test.ts` | Spawned CLI on `fixtures/minimal` (bench smoke + resource parity). |
19+
| **Check** | `bun run check` | build + lint + unit + scripts + golden + agent-eval. |
2020

2121
Refresh Tier A goldens after intentional fixture or schema changes:
2222

@@ -81,7 +81,7 @@ Codemap development uses **only** the committed bench ([fixtures/README.md](../f
8181

8282
## Agent eval probes
8383

84-
`scripts/agent-eval/scenarios.json` — **18 probes**, one golden id per `CAPABILITIES.json` group (except bundled-recipe guard + CLI-only groups). Enforced by `scripts/agent-eval/capability-probes.test.mjs` in `test:scripts`.
84+
`scripts/agent-eval/scenarios.json` — **18 probes** (one per `CAPABILITIES.json` capability id). Each probe’s `goldenId` must appear in that group’s `goldenScenarios` (or the group uses `unitTests` / `enforcedBy` instead — `recipes.bundled`, `cli.bench-smoke`, `cli.mcp.http`). Some groups list multiple substrate scenarios; probes need only cover one id per group. Enforced by `scripts/agent-eval/capability-probes.test.mjs` in `test:scripts`.
8585

8686
---
8787

‎fixtures/CAPABILITIES.json‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,11 @@
88
"id": "index.schema",
99
"description": "All indexed tables populated on full rebuild",
1010
"fixtureFiles": ["src/consumer.ts", "src/api/client.ts"],
11-
"goldenScenarios": ["index-table-stats", "index-summary"]
11+
"goldenScenarios": [
12+
"index-table-stats",
13+
"index-summary",
14+
"file-metrics-complexity-fixture"
15+
]
1216
},
1317
{
1418
"id": "symbols.imports.exports",
@@ -36,6 +40,8 @@
3640
"goldenScenarios": [
3741
"references-product-card-perms",
3842
"bindings-createClient",
43+
"import-specifiers-consumer",
44+
"function-params-createClient",
3945
"find-symbol-references",
4046
"find-re-exported-bindings"
4147
]
@@ -79,6 +85,7 @@
7985
"goldenScenarios": [
8086
"type-members-client-config",
8187
"type-ancestors-dog",
88+
"type-ancestors-qualified-child",
8289
"type-descendants-animal"
8390
]
8491
},
@@ -119,6 +126,7 @@
119126
"goldenScenarios": [
120127
"find-barrel-files",
121128
"circular-imports",
129+
"module-cycles-cache-store",
122130
"dynamic-imports-prefetch",
123131
"re-export-chains-product-card",
124132
"barrel-chains"
@@ -140,7 +148,8 @@
140148
"try-catch-rethrow-heuristics",
141149
"decorators-sealed",
142150
"runtime-markers-env",
143-
"find-leftover-console"
151+
"find-leftover-console",
152+
"jsdoc-tags-createClient"
144153
]
145154
},
146155
{

‎fixtures/README.md‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,14 @@
22

33
**Single committed corpus** for regression-testing Codemap — no `CODEMAP_ROOT` pointing at external apps required for maintainers or CI.
44

5-
| Piece | Path | Role |
6-
| ----------------------- | --------------------------------------------------------------------------------------- | -------------------------------------------------------- |
7-
| **Corpus** | [`minimal/`](./minimal/) | Source tree indexed by goldens, benchmark, agent-eval |
8-
| **Golden scenarios** | [`golden/scenarios.json`](./golden/scenarios.json) | 124+ SQL/recipe scenarios |
9-
| **Golden snapshots** | [`golden/minimal/`](./golden/minimal/) | Expected query JSON (committed) |
10-
| **Capability map** | [`CAPABILITIES.json`](./CAPABILITIES.json) | Machine-readable: capability → fixture files → scenarios |
11-
| **Agent eval** | [`agent-eval/`](../scripts/agent-eval/) + probes in `scripts/agent-eval/scenarios.json` | MCP-on vs traditional arms |
12-
| **Benchmark scenarios** | [`benchmark/scenarios.example.json`](./benchmark/scenarios.example.json) | Speed comparisons (optional override) |
5+
| Piece | Path | Role |
6+
| ----------------------- | --------------------------------------------------------------------------------------- | --------------------------------------------------------- |
7+
| **Corpus** | [`minimal/`](./minimal/) | Source tree indexed by goldens, benchmark, agent-eval |
8+
| **Golden scenarios** | [`golden/scenarios.json`](./golden/scenarios.json) | 125 SQL/recipe scenarios |
9+
| **Golden snapshots** | [`golden/minimal/`](./golden/minimal/) | Expected query JSON (committed) |
10+
| **Capability map** | [`CAPABILITIES.json`](./CAPABILITIES.json) | Capability groups → fixture files → `goldenScenarios` ids |
11+
| **Agent eval** | [`agent-eval/`](../scripts/agent-eval/) + probes in `scripts/agent-eval/scenarios.json` | MCP-on vs traditional arms |
12+
| **Benchmark scenarios** | [`benchmark/scenarios.example.json`](./benchmark/scenarios.example.json) | Speed comparisons (optional override) |
1313

1414
## Commands (from repo root)
1515

0 commit comments

Comments
 (0)