Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/coverage-confirmed-dead.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@stainless-code/codemap": patch
---

Add `coverage-confirmed-dead` recipe: static dead exports with `confidence` (`high` when ingested 0% coverage, `medium` when unmeasured), plus `reason` and `caller_count`. Golden scenarios and agent rule/skill/MCP documented.
2 changes: 2 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,8 @@ Three **mutually exclusive** CLI entry shapes; all converge on `applyDiffPayload

**Coverage columns (CRAP recipes):** `high-crap-score` adds **`coverage_source`** and **`effective_coverage_pct`** — measured vs graph-estimated undertest signal. Contract: [golden-queries.md § Coverage columns](./golden-queries.md#coverage-columns-crap--enrichment-recipes).

**Confidence columns (deletion recipes):** `coverage-confirmed-dead` adds **`confidence`** (`high` \| `medium`), **`reason`**, and **`caller_count`** on static dead exports — ingested zero vs unmeasured coverage. Contract: [golden-queries.md § Confidence columns](./golden-queries.md#confidence-columns-deletion-confidence-recipes).

**Recipes wiring:** **`src/application/recipes-loader.ts`** (pure transport-agnostic loader) + **`src/application/query-recipes.ts`** (cache + public API — `getQueryRecipeSql` / `getQueryRecipeActions` / `getQueryRecipeParams` / `listQueryRecipeIds` / `listQueryRecipeCatalog` / `getQueryRecipeCatalogEntry`, shared by CLI + MCP). Recipes live as file pairs: **`<id>.sql`** + optional **`<id>.md`**. The loader reads `templates/recipes/` (bundled, ships in npm package next to `templates/agents/`) and `<state-dir>/recipes/` (project-local — default `.codemap/recipes/`; honors `--state-dir` / `CODEMAP_STATE_DIR`; root-only resolution per the registry plan, no walk-up). Project recipes win on id collision; entries that override a bundled id carry **`shadows: true`** in the catalog so agents reading `codemap://recipes` at session start see when a recipe behaves differently from the documented bundled version. Per-row **`actions`** templates and recipe **`params`** declarations live in YAML frontmatter on each `<id>.md` — uniform shape across bundled + project. Param types are `string | number | boolean`; CLI passes values via repeatable `--params key=value[,key=value]`, MCP / HTTP pass nested `params: {key: value}` to `query_recipe`. Validation runs before SQL binding; missing / unknown / malformed params return the same `{error}` envelope as query failures. Hand-rolled YAML parser is scoped to block-list `actions:` and `params:` only (no `js-yaml` dep). Load-time validation rejects empty SQL and DML / DDL keywords (`INSERT` / `UPDATE` / `DELETE` / `DROP` / `CREATE` / `ALTER` / `ATTACH` / `DETACH` / `REPLACE` / `TRUNCATE` / `VACUUM` / `PRAGMA`) with recipe-aware error messages — defence in depth alongside the runtime `PRAGMA query_only=1` backstop in `query-engine.ts` (PR #35). `<state-dir>/index.db` is gitignored; `<state-dir>/recipes/` is NOT (verified via `git check-ignore`) — recipes are git-tracked source code authored for human review.

**Tool / resource handlers (transport-agnostic):** **`src/application/tool-handlers.ts`** + **`src/application/resource-handlers.ts`** — pure functions that take the args object an MCP tool / resource URI accepts and return a discriminated **`ToolResult`** (`{ok: true, format: 'json'|'sarif'|'annotations'|'mermaid'|'diff'|'diff-json'|'codeclimate'|'badge', payload}` — badge arm also carries `badgeStyle`; `{ok: false, error}`) or a **`ResourcePayload`** (`{mimeType, text}`). MCP and HTTP both wrap the same handlers — MCP translates to `{content: [{type: "text", text}]}`, HTTP translates to `(status, body)` with the right `Content-Type`. Engine layer untouched; transport changes don't ripple into the SQL.
Expand Down
1 change: 1 addition & 0 deletions docs/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,7 @@ Statement coverage ingested from Istanbul JSON, LCOV, or V8 runtime (`NODE_V8_CO
Format auto-detected from extension (`.json` → istanbul, `.info` → lcov, directory → probe both, error if ambiguous); `--runtime` opts into V8 directory mode. Each statement projects onto the **innermost** enclosing symbol via JS-side `(line_end - line_start) ASC` tie-break — required because nested symbols (class methods inside classes, closures inside functions) would otherwise inflate `total_statements`. Statements that fall outside every symbol range (top-level expressions, side-effect imports) increment `skipped.statements_no_symbol` for observability. Three bundled recipes consume the table at first-class agent surface (no agent ever has to hand-compose the JOIN):

- `untested-and-dead` — exported functions with no callers AND zero coverage (the killer recipe; ships with a name-collision mitigation guide in the recipe `.md`).
- `coverage-confirmed-dead` — same static dead predicate as `untested-and-dead` with explicit **`confidence`** (`high` when ingested `coverage_pct = 0`, `medium` when unmeasured) plus **`reason`** and **`caller_count`** — Moat A predicate columns for deletion triage after `ingest-coverage`.
- `files-by-coverage` — files ranked ascending by statement coverage (replaces a deferred `file_coverage` rollup table; aggregates the symbol-level table via index-bounded `GROUP BY`).
- `worst-covered-exports` — top-20 worst-covered exported functions.

Expand Down
6 changes: 5 additions & 1 deletion docs/golden-queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ We **do not** commit another product’s source tree, paths, business strings, o

## Scenario shape (implemented)

Scenarios live in **`fixtures/golden/scenarios.json`** (Tier A) or optional **`scenarios.external.json`** / **example** (Tier B). The file may be a **bare array** of scenarios (legacy) or an object `{ "setup": [...], "scenarios": [...] }`. Optional top-level **`setup`** runs once after index, before scenarios — today only **`{ "kind": "ingest-coverage", "path": "<relative-to-fixture>" }`** (see [run-setup.ts](../scripts/query-golden/run-setup.ts)); missing coverage files are skipped with a warning. Each scenario has **`id`**, **`sql` or `recipe`**, optional **`match`** (`exact`, `minRows`, `everyRowContains`), optional **`budgetMs`**. Goldens: **`fixtures/golden/minimal/*.json`** etc. Refresh: **`bun scripts/query-golden.ts --update`**.
Scenarios live in **`fixtures/golden/scenarios.json`** (Tier A) or optional **`scenarios.external.json`** / **example** (Tier B). The file may be a **bare array** of scenarios (legacy) or an object `{ "setup": [...], "scenarios": [...] }`. Optional top-level **`setup`** runs once after index, before scenarios — today **`ingest-coverage`** and **`clear-coverage`** (see [run-setup.ts](../scripts/query-golden/run-setup.ts)); missing coverage files are skipped with a warning. Per-scenario **`preSetup`** runs after global setup (global setup restores after the scenario when `preSetup` mutates the index). Each scenario has **`id`**, **`sql` or `recipe`**, optional **`match`** (`exact`, `minRows`, `everyRowContains`, `everyRowFieldEquals`), optional **`budgetMs`**. Goldens: **`fixtures/golden/minimal/*.json`** etc. Refresh: **`bun scripts/query-golden.ts --update`**.

**Prompts** in JSON are **intent labels**, not pasted chat logs — pair with queries whose literals come from **fixture-owned** data (see [fixtures/qa/prompts.external.template.md](../fixtures/qa/prompts.external.template.md) for optional chat QA).

Expand All @@ -74,6 +74,10 @@ Some bundled recipes add optional **`reason`** (TEXT) and **`evidence_json`** (T

`high-crap-score` adds **`coverage_source`** (`measured` \| `estimated`) and **`effective_coverage_pct`** on each row — measured when `coverage` has a matching symbol row after `ingest-coverage`; otherwise graph-estimated tiers from test reachability. Goldens assert `coverage_source` when the recipe ships coverage semantics (`high-crap-score`); measured override is covered by `scripts/high-crap-score-measured.test.mjs`.

### Confidence columns (deletion-confidence recipes)

`coverage-confirmed-dead` adds **`confidence`** (`high` \| `medium`) on each row — **`high`** when static dead and ingested `coverage_pct = 0`; **`medium`** when static dead but the symbol has no ingested coverage row. Also **`reason`**, **`caller_count`**. Goldens: `coverage-confirmed-dead` (post-ingest mix) and `coverage-confirmed-dead-no-ingest` (`preSetup: clear-coverage`, `everyRowFieldEquals` on `confidence: medium`).

---

## Status
Expand Down
54 changes: 20 additions & 34 deletions docs/plans/agent-enrichment-wave.md
Original file line number Diff line number Diff line change
@@ -1,42 +1,29 @@
# Agent enrichment wave — tracer workflow (plans 3–4)
# Agent enrichment wave — tracer workflow (plan 4)

> **Status:** in-flight · **Scope:** remaining P2 plans ranked by consumer/agent ROI
> **Status:** in-flight · **Scope:** remaining P2 plan ranked by consumer/agent ROI
>
> **Goal:** Ship tracer bullets that cut agent round-trips, improve answer trust, and sharpen PR/CI deltas — all Moat-A (predicate columns, no verdict primitives).
>
> **Shipped (plans retired):** Evidence chains ([#174](https://github.com/stainless-code/codemap/pull/174)) · Graph-estimated CRAP ([#175](https://github.com/stainless-code/codemap/pull/175)) — durable contract in `golden-queries.md` + `architecture.md`; plan files deleted per [docs-governance](../../.agents/skills/docs-governance/SKILL.md) § Closing a plan.
> **Shipped (plans retired):** Evidence chains ([#174](https://github.com/stainless-code/codemap/pull/174)) · Graph-estimated CRAP ([#175](https://github.com/stainless-code/codemap/pull/175)) · Coverage deletion confidence (PR **#D**) — durable contracts in `golden-queries.md` + `architecture.md`; plan files deleted per [docs-governance](../../.agents/skills/docs-governance/SKILL.md) § Closing a plan.
>
> **Remaining:** [coverage-deletion-confidence](./coverage-deletion-confidence.md) → [audit-delta-attribution](./audit-delta-attribution.md)
> **Remaining:** [audit-delta-attribution](./audit-delta-attribution.md)

---

## Shared conventions (locked)

| Convention | Applies to |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- |
| **Moat A** — no `pass`/`fail` engine verdict; extra columns only | All |
| **`reason` TEXT** — machine code + short clause where useful | #3 |
| **`evidence_json` TEXT** — bounded JSON array (≤3 hops) | shipped #1 |
| **`confidence` / `coverage_source` / `attribution`** — recipe-specific enums | #3, #4 |
| **Golden update per slice** — `fixtures/golden/minimal/*.json` + `scenarios.json` | All |
| **`/harden-pr lite`** after each tracer commit; **`/harden-pr full`** before PR merge | All |
| **Retire plan on merge** — delete `docs/plans/<topic>.md` + lift to reference docs/roadmap in the **same PR** (never leave shipped plans as leftovers) | All |
| **No deferring complements** — agent surfaces (rule/skill/MCP), glossary, golden/script tests, and plan acceptance items ship **in the same PR** unless explicitly listed under plan **Out of scope** | All |
| Convention | Applies to |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- |
| **Moat A** — no `pass`/`fail` engine verdict; extra columns only | All |
| **`reason` TEXT** — machine code + short clause where useful | shipped |
| **`evidence_json` TEXT** — bounded JSON array (≤3 hops) | shipped |
| **`confidence` / `coverage_source` / `attribution`** — recipe-specific enums | shipped #3, #4 |
| **Golden update per slice** — `fixtures/golden/minimal/*.json` + `scenarios.json` | All |
| **`/harden-pr lite`** after each tracer commit; **`/harden-pr full`** before PR merge | All |
| **Retire plan on merge** — delete `docs/plans/<topic>.md` + lift to reference docs/roadmap in the **same PR** (never leave shipped plans as leftovers) | All |
| **No deferring complements** — agent surfaces (rule/skill/MCP), glossary, golden/script tests, and plan acceptance items ship in the **same PR** unless explicitly listed under plan **Out of scope** | All |

**Cross-plan synergy:** shipped evidence `reason` complements #4 `attribution` on audit `added` rows. CRAP `coverage_source` (#175) ships before #3 so deletion-confidence can narrow rows with coverage semantics.

---

## Plan 3 — Coverage deletion confidence (`coverage-deletion-confidence.md`)

| Slice | Deliverable | Verify |
| -------------------------- | -------------------------------------------------------------- | ------------- |
| **3.1 recipe fork** | `coverage-confirmed-dead.sql` + `.md` from `untested-and-dead` | query CLI |
| **3.2 golden no-ingest** | `confidence: medium` policy (per D.4) | `test:golden` |
| **3.3 golden with ingest** | fixture coverage → `confidence: high` | `test:golden` |
| **3.4 classifier** | intent keywords if needed | optional |

**Grill before 3.1:** Q3 without ingest — `medium` rows vs empty + stderr (plan D.4 leans medium rows).
**Cross-plan synergy:** shipped evidence `reason` complements #4 `attribution` on audit `added` rows. Shipped `confidence` (#D) narrows deletion triage after `ingest-coverage`.

---

Expand All @@ -54,15 +41,14 @@

## PR cadence

| PR | Contents | Changeset | Retire plan on merge |
| -------------------------- | --------------- | --------- | --------------------------------- |
| **#D Deletion confidence** | Plan 3 complete | patch | `coverage-deletion-confidence.md` |
| **#E Audit attribution** | Plan 4 complete | patch | `audit-delta-attribution.md` |
| PR | Contents | Changeset | Retire plan on merge |
| ------------------------ | --------------- | --------- | ---------------------------- |
| **#E Audit attribution** | Plan 4 complete | patch | `audit-delta-attribution.md` |

Each PR: `harden-pr full` (includes plan retirement) → merge. Do not batch plans 3–4 into one PR.
Each PR: `harden-pr full` (includes plan retirement) → merge.

---

## Current slice

**Active:** Plan 3 slice **3.1** on `feat/high-crap-score` or fresh branch from `main` after **#175** merges — `coverage-confirmed-dead` recipe fork.
**Active:** Plan 4 slice **4.1** on fresh branch from `main` after **#D** merges — `findingKey()` helper + unit tests.
Loading
Loading