Skip to content

Commit 16687a0

Browse files
feat(recipes): coverage-confirmed-dead with confidence columns (#176)
* feat(recipes): add coverage-confirmed-dead recipe (Plan 3 slice 3.1) Fork untested-and-dead with confidence/reason/caller_count columns — high when ingested coverage is 0%, medium when unmeasured; honors shared suppressions. * test(golden): coverage-confirmed-dead scenarios with preSetup clear-coverage Add no-ingest (all medium) and post-ingest goldens; extend query-golden with per-scenario preSetup, clear-coverage step, and everyRowFieldEquals match. * feat(recipes): ship coverage-confirmed-dead agent surfaces and classifier Document confidence columns in golden-queries, glossary, architecture, rule/skill/MCP; add cleanup intent mapping; changeset for patch release. * harden: retire coverage-deletion-confidence plan and lift contract Delete shipped plan doc, mark roadmap item done, slim agent-enrichment-wave to plan 4 only, cross-link untested-and-dead, refresh fixture README goldens.
1 parent af86d10 commit 16687a0

26 files changed

Lines changed: 707 additions & 201 deletions
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"@stainless-code/codemap": patch
3+
---
4+
5+
Add `coverage-confirmed-dead` recipe: static dead exports with `confidence` (`high` when ingested 0% coverage, `medium` when unmeasured), plus `reason` and `caller_count`. Golden scenarios and agent rule/skill/MCP documented.

docs/architecture.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -194,6 +194,8 @@ Three **mutually exclusive** CLI entry shapes; all converge on `applyDiffPayload
194194

195195
**Coverage columns (CRAP recipes):** `high-crap-score` adds **`coverage_source`** and **`effective_coverage_pct`** — measured vs graph-estimated undertest signal. Contract: [golden-queries.md § Coverage columns](./golden-queries.md#coverage-columns-crap--enrichment-recipes).
196196

197+
**Confidence columns (deletion recipes):** `coverage-confirmed-dead` adds **`confidence`** (`high` \| `medium`), **`reason`**, and **`caller_count`** on static dead exports — ingested zero vs unmeasured coverage. Contract: [golden-queries.md § Confidence columns](./golden-queries.md#confidence-columns-deletion-confidence-recipes).
198+
197199
**Recipes wiring:** **`src/application/recipes-loader.ts`** (pure transport-agnostic loader) + **`src/application/query-recipes.ts`** (cache + public API — `getQueryRecipeSql` / `getQueryRecipeActions` / `getQueryRecipeParams` / `listQueryRecipeIds` / `listQueryRecipeCatalog` / `getQueryRecipeCatalogEntry`, shared by CLI + MCP). Recipes live as file pairs: **`<id>.sql`** + optional **`<id>.md`**. The loader reads `templates/recipes/` (bundled, ships in npm package next to `templates/agents/`) and `<state-dir>/recipes/` (project-local — default `.codemap/recipes/`; honors `--state-dir` / `CODEMAP_STATE_DIR`; root-only resolution per the registry plan, no walk-up). Project recipes win on id collision; entries that override a bundled id carry **`shadows: true`** in the catalog so agents reading `codemap://recipes` at session start see when a recipe behaves differently from the documented bundled version. Per-row **`actions`** templates and recipe **`params`** declarations live in YAML frontmatter on each `<id>.md` — uniform shape across bundled + project. Param types are `string | number | boolean`; CLI passes values via repeatable `--params key=value[,key=value]`, MCP / HTTP pass nested `params: {key: value}` to `query_recipe`. Validation runs before SQL binding; missing / unknown / malformed params return the same `{error}` envelope as query failures. Hand-rolled YAML parser is scoped to block-list `actions:` and `params:` only (no `js-yaml` dep). Load-time validation rejects empty SQL and DML / DDL keywords (`INSERT` / `UPDATE` / `DELETE` / `DROP` / `CREATE` / `ALTER` / `ATTACH` / `DETACH` / `REPLACE` / `TRUNCATE` / `VACUUM` / `PRAGMA`) with recipe-aware error messages — defence in depth alongside the runtime `PRAGMA query_only=1` backstop in `query-engine.ts` (PR #35). `<state-dir>/index.db` is gitignored; `<state-dir>/recipes/` is NOT (verified via `git check-ignore`) — recipes are git-tracked source code authored for human review.
198200

199201
**Tool / resource handlers (transport-agnostic):** **`src/application/tool-handlers.ts`** + **`src/application/resource-handlers.ts`** — pure functions that take the args object an MCP tool / resource URI accepts and return a discriminated **`ToolResult`** (`{ok: true, format: 'json'|'sarif'|'annotations'|'mermaid'|'diff'|'diff-json'|'codeclimate'|'badge', payload}` — badge arm also carries `badgeStyle`; `{ok: false, error}`) or a **`ResourcePayload`** (`{mimeType, text}`). MCP and HTTP both wrap the same handlers — MCP translates to `{content: [{type: "text", text}]}`, HTTP translates to `(status, body)` with the right `Content-Type`. Engine layer untouched; transport changes don't ripple into the SQL.

docs/glossary.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,7 @@ Statement coverage ingested from Istanbul JSON, LCOV, or V8 runtime (`NODE_V8_CO
170170
Format auto-detected from extension (`.json` → istanbul, `.info` → lcov, directory → probe both, error if ambiguous); `--runtime` opts into V8 directory mode. Each statement projects onto the **innermost** enclosing symbol via JS-side `(line_end - line_start) ASC` tie-break — required because nested symbols (class methods inside classes, closures inside functions) would otherwise inflate `total_statements`. Statements that fall outside every symbol range (top-level expressions, side-effect imports) increment `skipped.statements_no_symbol` for observability. Three bundled recipes consume the table at first-class agent surface (no agent ever has to hand-compose the JOIN):
171171

172172
- `untested-and-dead` — exported functions with no callers AND zero coverage (the killer recipe; ships with a name-collision mitigation guide in the recipe `.md`).
173+
- `coverage-confirmed-dead` — same static dead predicate as `untested-and-dead` with explicit **`confidence`** (`high` when ingested `coverage_pct = 0`, `medium` when unmeasured) plus **`reason`** and **`caller_count`** — Moat A predicate columns for deletion triage after `ingest-coverage`.
173174
- `files-by-coverage` — files ranked ascending by statement coverage (replaces a deferred `file_coverage` rollup table; aggregates the symbol-level table via index-bounded `GROUP BY`).
174175
- `worst-covered-exports` — top-20 worst-covered exported functions.
175176

docs/golden-queries.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ We **do not** commit another product’s source tree, paths, business strings, o
6262

6363
## Scenario shape (implemented)
6464

65-
Scenarios live in **`fixtures/golden/scenarios.json`** (Tier A) or optional **`scenarios.external.json`** / **example** (Tier B). The file may be a **bare array** of scenarios (legacy) or an object `{ "setup": [...], "scenarios": [...] }`. Optional top-level **`setup`** runs once after index, before scenarios — today only **`{ "kind": "ingest-coverage", "path": "<relative-to-fixture>" }`** (see [run-setup.ts](../scripts/query-golden/run-setup.ts)); missing coverage files are skipped with a warning. Each scenario has **`id`**, **`sql` or `recipe`**, optional **`match`** (`exact`, `minRows`, `everyRowContains`), optional **`budgetMs`**. Goldens: **`fixtures/golden/minimal/*.json`** etc. Refresh: **`bun scripts/query-golden.ts --update`**.
65+
Scenarios live in **`fixtures/golden/scenarios.json`** (Tier A) or optional **`scenarios.external.json`** / **example** (Tier B). The file may be a **bare array** of scenarios (legacy) or an object `{ "setup": [...], "scenarios": [...] }`. Optional top-level **`setup`** runs once after index, before scenarios — today **`ingest-coverage`** and **`clear-coverage`** (see [run-setup.ts](../scripts/query-golden/run-setup.ts)); missing coverage files are skipped with a warning. Per-scenario **`preSetup`** runs after global setup (global setup restores after the scenario when `preSetup` mutates the index). Each scenario has **`id`**, **`sql` or `recipe`**, optional **`match`** (`exact`, `minRows`, `everyRowContains`, `everyRowFieldEquals`), optional **`budgetMs`**. Goldens: **`fixtures/golden/minimal/*.json`** etc. Refresh: **`bun scripts/query-golden.ts --update`**.
6666

6767
**Prompts** in JSON are **intent labels**, not pasted chat logs — pair with queries whose literals come from **fixture-owned** data (see [fixtures/qa/prompts.external.template.md](../fixtures/qa/prompts.external.template.md) for optional chat QA).
6868

@@ -74,6 +74,10 @@ Some bundled recipes add optional **`reason`** (TEXT) and **`evidence_json`** (T
7474

7575
`high-crap-score` adds **`coverage_source`** (`measured` \| `estimated`) and **`effective_coverage_pct`** on each row — measured when `coverage` has a matching symbol row after `ingest-coverage`; otherwise graph-estimated tiers from test reachability. Goldens assert `coverage_source` when the recipe ships coverage semantics (`high-crap-score`); measured override is covered by `scripts/high-crap-score-measured.test.mjs`.
7676

77+
### Confidence columns (deletion-confidence recipes)
78+
79+
`coverage-confirmed-dead` adds **`confidence`** (`high` \| `medium`) on each row — **`high`** when static dead and ingested `coverage_pct = 0`; **`medium`** when static dead but the symbol has no ingested coverage row. Also **`reason`**, **`caller_count`**. Goldens: `coverage-confirmed-dead` (post-ingest mix) and `coverage-confirmed-dead-no-ingest` (`preSetup: clear-coverage`, `everyRowFieldEquals` on `confidence: medium`).
80+
7781
---
7882

7983
## Status
Lines changed: 20 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -1,42 +1,29 @@
1-
# Agent enrichment wave — tracer workflow (plans 3–4)
1+
# Agent enrichment wave — tracer workflow (plan 4)
22

3-
> **Status:** in-flight · **Scope:** remaining P2 plans ranked by consumer/agent ROI
3+
> **Status:** in-flight · **Scope:** remaining P2 plan ranked by consumer/agent ROI
44
>
55
> **Goal:** Ship tracer bullets that cut agent round-trips, improve answer trust, and sharpen PR/CI deltas — all Moat-A (predicate columns, no verdict primitives).
66
>
7-
> **Shipped (plans retired):** Evidence chains ([#174](https://github.com/stainless-code/codemap/pull/174)) · Graph-estimated CRAP ([#175](https://github.com/stainless-code/codemap/pull/175)) — durable contract in `golden-queries.md` + `architecture.md`; plan files deleted per [docs-governance](../../.agents/skills/docs-governance/SKILL.md) § Closing a plan.
7+
> **Shipped (plans retired):** Evidence chains ([#174](https://github.com/stainless-code/codemap/pull/174)) · Graph-estimated CRAP ([#175](https://github.com/stainless-code/codemap/pull/175)) · Coverage deletion confidence (PR **#D**) — durable contracts in `golden-queries.md` + `architecture.md`; plan files deleted per [docs-governance](../../.agents/skills/docs-governance/SKILL.md) § Closing a plan.
88
>
9-
> **Remaining:** [coverage-deletion-confidence](./coverage-deletion-confidence.md)[audit-delta-attribution](./audit-delta-attribution.md)
9+
> **Remaining:** [audit-delta-attribution](./audit-delta-attribution.md)
1010
1111
---
1212

1313
## Shared conventions (locked)
1414

15-
| Convention | Applies to |
16-
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- |
17-
| **Moat A** — no `pass`/`fail` engine verdict; extra columns only | All |
18-
| **`reason` TEXT** — machine code + short clause where useful | #3 |
19-
| **`evidence_json` TEXT** — bounded JSON array (≤3 hops) | shipped #1 |
20-
| **`confidence` / `coverage_source` / `attribution`** — recipe-specific enums | #3, #4 |
21-
| **Golden update per slice**`fixtures/golden/minimal/*.json` + `scenarios.json` | All |
22-
| **`/harden-pr lite`** after each tracer commit; **`/harden-pr full`** before PR merge | All |
23-
| **Retire plan on merge** — delete `docs/plans/<topic>.md` + lift to reference docs/roadmap in the **same PR** (never leave shipped plans as leftovers) | All |
24-
| **No deferring complements** — agent surfaces (rule/skill/MCP), glossary, golden/script tests, and plan acceptance items ship **in the same PR** unless explicitly listed under plan **Out of scope** | All |
15+
| Convention | Applies to |
16+
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- |
17+
| **Moat A** — no `pass`/`fail` engine verdict; extra columns only | All |
18+
| **`reason` TEXT** — machine code + short clause where useful | shipped |
19+
| **`evidence_json` TEXT** — bounded JSON array (≤3 hops) | shipped |
20+
| **`confidence` / `coverage_source` / `attribution`** — recipe-specific enums | shipped #3, #4 |
21+
| **Golden update per slice**`fixtures/golden/minimal/*.json` + `scenarios.json` | All |
22+
| **`/harden-pr lite`** after each tracer commit; **`/harden-pr full`** before PR merge | All |
23+
| **Retire plan on merge** — delete `docs/plans/<topic>.md` + lift to reference docs/roadmap in the **same PR** (never leave shipped plans as leftovers) | All |
24+
| **No deferring complements** — agent surfaces (rule/skill/MCP), glossary, golden/script tests, and plan acceptance items ship in the **same PR** unless explicitly listed under plan **Out of scope** | All |
2525

26-
**Cross-plan synergy:** shipped evidence `reason` complements #4 `attribution` on audit `added` rows. CRAP `coverage_source` (#175) ships before #3 so deletion-confidence can narrow rows with coverage semantics.
27-
28-
---
29-
30-
## Plan 3 — Coverage deletion confidence (`coverage-deletion-confidence.md`)
31-
32-
| Slice | Deliverable | Verify |
33-
| -------------------------- | -------------------------------------------------------------- | ------------- |
34-
| **3.1 recipe fork** | `coverage-confirmed-dead.sql` + `.md` from `untested-and-dead` | query CLI |
35-
| **3.2 golden no-ingest** | `confidence: medium` policy (per D.4) | `test:golden` |
36-
| **3.3 golden with ingest** | fixture coverage → `confidence: high` | `test:golden` |
37-
| **3.4 classifier** | intent keywords if needed | optional |
38-
39-
**Grill before 3.1:** Q3 without ingest — `medium` rows vs empty + stderr (plan D.4 leans medium rows).
26+
**Cross-plan synergy:** shipped evidence `reason` complements #4 `attribution` on audit `added` rows. Shipped `confidence` (#D) narrows deletion triage after `ingest-coverage`.
4027

4128
---
4229

@@ -54,15 +41,14 @@
5441

5542
## PR cadence
5643

57-
| PR | Contents | Changeset | Retire plan on merge |
58-
| -------------------------- | --------------- | --------- | --------------------------------- |
59-
| **#D Deletion confidence** | Plan 3 complete | patch | `coverage-deletion-confidence.md` |
60-
| **#E Audit attribution** | Plan 4 complete | patch | `audit-delta-attribution.md` |
44+
| PR | Contents | Changeset | Retire plan on merge |
45+
| ------------------------ | --------------- | --------- | ---------------------------- |
46+
| **#E Audit attribution** | Plan 4 complete | patch | `audit-delta-attribution.md` |
6147

62-
Each PR: `harden-pr full` (includes plan retirement) → merge. Do not batch plans 3–4 into one PR.
48+
Each PR: `harden-pr full` (includes plan retirement) → merge.
6349

6450
---
6551

6652
## Current slice
6753

68-
**Active:** Plan 3 slice **3.1** on `feat/high-crap-score` or fresh branch from `main` after **#175** merges — `coverage-confirmed-dead` recipe fork.
54+
**Active:** Plan 4 slice **4.1** on fresh branch from `main` after **#D** merges — `findingKey()` helper + unit tests.

0 commit comments

Comments
 (0)