Commit 114303f
authored
feat: codemap audit v1 (B.5) — structural-drift command with per-delta baselines (#33)
* feat(audit): tracer 1 — CLI scaffold + auto-incremental-index prelude (B.5 v1)
First vertical slice per docs/plans/codemap-audit.md § 8 tracer-bullet
sequence step 1. End-to-end working: bare `codemap audit` exits 1 with
a clean message; --baseline missing emits {"error":"…"}; --baseline
<existing> emits the {base, head, deltas} envelope (deltas: {} until
tracer 2 fills it in).
File layout per the design pass (Q6):
- src/cli/cmd-audit.ts — argv parse, --no-index opt-out, runAuditCmd
bootstrap + auto-incremental-index prelude
(calls runCodemapIndex({mode: "incremental",
quiet: true}) unless --no-index), stub
renderer.
- src/application/audit-engine.ts — types (AuditEnvelope / AuditBase /
AuditHead / AuditDelta / AuditError),
runAudit({db, baselineName}) — loads
baseline via getQueryBaseline (B.6
helper), assembles envelope. Caller owns
the DB lifecycle and the index-prelude
decision (engine reads whatever's there).
Tests:
- src/cli/cmd-audit.test.ts — 9 parser tests (all four flags +
mutex / error paths).
- src/application/audit-engine.test.ts — 2 engine tests (missing baseline
error path; happy-path envelope shape).
- src/cli.test.ts — `audit --help` + `audit` (no flags)
integration.
Wiring: src/cli/main.ts dispatch + src/cli/bootstrap.ts validateIndexModeArgs
allow-list ("audit").
No deltas yet — tracer 2 ships `files`. No verdict, no exit codes 1/2,
no codemap.config.audit (per Q3). No --base <ref> (per Q1, defers to v1.x).
* feat(audit): tracer 2 — files delta + canonical-projection registry (B.5 v1)
Per docs/plans/codemap-audit.md § 8 step 2. End-to-end working: a
files-shaped baseline produces a real diff (added/removed paths);
column-mismatch baselines surface the clean re-save guidance per § 4.
Engine additions (src/application/audit-engine.ts):
- AuditDeltaSpec — {key, sql, requiredColumns, recipeIdHint}.
- V1_DELTAS — readonly registry (one entry: files; tracer 3+ adds the
rest). New deltas land here as plain data.
- runAudit now iterates V1_DELTAS, populating the deltas map. First
delta error short-circuits the audit and propagates the
AuditError envelope verbatim.
- computeDelta(db, baselineName, baselineRows, spec) — pure function:
validates baseline column-set membership, projects baseline rows
down to spec.requiredColumns (drops extras — schema-drift-resilient
per Q4), runs spec.sql against the live DB via the caller's
connection, set-diffs via diffRows.
- projectRow helper picks required columns into a fresh object.
Pre-req refactor: extract diffRows from cli/cmd-query.ts to src/
diff-rows.ts. Engine importing from cli/ would be a layering back-edge
(application → cli); same pattern as src/git-changed.ts where pure
helpers needed by both layers live at src/. cli/cmd-query.ts updated
to import from the new location; tests moved to src/diff-rows.test.ts;
parseQueryRest tests previously mis-nested inside the diffRows
describe block re-labeled (functional no-op — describe is just a
label — but stops being misleading).
Tests:
- src/application/audit-engine.test.ts grows from 2 to 9 tests:
end-to-end runAudit flow (existing baseline → empty deltas;
wrong-shape → error envelope), then 6 computeDelta cases for
files (no-diff, added, removed, schema-drift projection,
column-mismatch error, empty-baseline-treated-as-all-added).
- src/diff-rows.test.ts — 6 multiset cases ported verbatim from
cmd-query.test.ts.
Smoke: tested against this clone — files-snap baseline saved BEFORE
this commit's two new files diff-rows.ts/.test.ts surfaced exactly
those two as `added`. wrong-shape baseline saved from
`SELECT name FROM symbols` produces:
codemap audit: baseline "wrong-shape" is missing required columns
for delta "files": got [name], need [path]. Re-save with: codemap
query --save-baseline=wrong-shape a query that returns `path`
(e.g. `--recipe files-hashes` or `"SELECT path FROM files"`)
bun:sqlite null vs better-sqlite3 undefined coercion already handled
in the B.6 getQueryBaseline helper; audit reuses it cleanly.
* feat(audit): tracer 3 — per-delta baselines (A+B hybrid) + dependencies + deprecated
Design fork surfaced during tracer 2 implementation: a baseline saved
from `SELECT path FROM files` satisfies the files delta but not
dependencies (needs from_path/to_path) or deprecated (needs
name/kind/file_path). One baseline can't naturally satisfy every
delta. Grilled the user, who picked option A+B (per-delta CLI flags
as primary + --baseline <prefix> as auto-resolve sugar). Plan §3 / §6
/ §7 / §8 updated inline with the new shape.
Engine (src/application/audit-engine.ts):
- AuditDelta now carries its own `base` metadata — different deltas
can reference different baselines (mixed-baseline audits are
first-class; e.g. `--baseline base --dependencies-baseline override`).
- AuditEnvelope drops the top-level `base` (was a single source of
truth that doesn't fit the per-delta model). New shape: `{head, deltas}`.
- AuditBaselineMap = Partial<Record<deltaKey, baselineName>>. Audit
iterates the map keys, not V1_DELTAS — only requested deltas run;
others are absent from the envelope.
- runAudit(opts: {db, baselines}) — empty map errors with a clean
"missing snapshot source" message; explicit baseline-not-found
errors name the delta key for context.
- computeDelta now returns just `{added, removed} | AuditError`;
runAudit wraps with the per-delta `base` metadata. Cleaner
separation: computeDelta is pure-diff, runAudit is composer.
- V1_DELTAS adds two more entries: dependencies + deprecated. Each
has its own canonical SQL, required columns, and recipe-id hint
for the column-mismatch error message.
CLI (src/cli/cmd-audit.ts):
- parseAuditRest now handles two flag families:
- `--baseline <prefix>`: auto-resolve sugar; looks up
`<prefix>-<key>` in query_baselines for each known delta key.
Slots that don't exist are silently absent (no error per missing
slot — convention-driven users can save just what they need).
- `--<key>-baseline <name>`: explicit per-delta override. Names
must exist or audit exits 1 with a delta-tagged error message.
- PER_DELTA_FLAGS map is generated from V1_DELTAS — adding a delta
in the engine surfaces a `--<key>-baseline` flag automatically.
- New `resolveAuditBaselines({db, baselinePrefix, perDelta})` exported
for testing; per-delta flags override auto-resolved slots.
- consumeFlagValue helper consolidates the `--flag <v>` /
`--flag=<v>` parsing pattern (used by all four baseline flags).
- Error message when no slot resolves explains both flag families
+ the v1.x --base <ref> path.
Help text updated with the new flag matrix + 4 example invocations
(convention path, explicit per-delta, mixed override, --no-index).
Tests:
- src/cli/cmd-audit.test.ts grows from 9 to 16 tests: parser cases
for each flag shape (--baseline / --baseline= / --<key>-baseline /
--<key>-baseline= / mixed mode / no-value errors) + the new
resolveAuditBaselines describe (4 cases covering auto-resolve,
per-delta override, per-delta-only, and no-match-empty-map).
- src/application/audit-engine.test.ts restructured: 5 runAudit cases
(empty map error / explicit-baseline-not-found / single-delta-only /
mixed-baseline per-delta `base` metadata / column-mismatch error
propagation), 6 computeDelta cases for files (unchanged shape).
- src/cli.test.ts integration test asserts the new --help output
surface (--baseline <prefix> + all three --<delta>-baseline flags).
Smoke (against this clone):
codemap query --save-baseline=base-files "SELECT path FROM files"
codemap query --save-baseline=base-dependencies "SELECT from_path, to_path FROM dependencies"
codemap query --save-baseline=base-deprecated -r deprecated-symbols
codemap audit --json --summary --baseline base
→ {head, deltas: {files: {base, added: 0, removed: 0},
dependencies: {base, added: 2, removed: 0},
deprecated: {base, added: 0, removed: 0}}}
codemap audit --files-baseline base-files
→ only files delta runs
codemap audit --baseline nonexistent
→ exit 1, "no delta baselines provided" with both flag families
explained
* feat(audit): tracer 5 — terminal-mode renderer per plan §7.1
Per plan §8 step 4. Three output modes from one envelope:
No-drift (default terminal):
audit: 3 delta(s), no drift across files / dependencies / deprecated.
files ← base-files @ abc12345 (no drift)
dependencies ← base-dependencies @ abc12345 (no drift)
deprecated ← base-deprecated @ abc12345 (no drift)
With drift (default terminal):
audit: 1 delta(s), drift in 1 (+2 / -0)
files ← base-files @ abc12345 (+2 / -0)
files added (+2):
┌─────┬──────────────────────┐
│ │ path │
├─────┼──────────────────────┤
│ 0 │ src/cli/cmd-audit.ts │
│ 1 │ src/diff-rows.ts │
└─────┴──────────────────────┘
--summary (terminal): same header + per-delta lines, no console.table
blocks. JSON --summary unchanged from tracer 3.
Adapted from §7.1's single-baseline-header sketch to the per-delta
`base` shape settled in tracer 3 — each delta carries its own
provenance line (`← <name> @ <sha8>`) so mixed-baseline audits
(e.g. `--baseline base --dependencies-baseline override`) make their
divergence visible without burying it in JSON.
Per-delta header layout:
- Drifting deltas show counts (`(+N / -M)`); no-drift deltas show
`(no drift)` as a literal label so the line shape is uniform.
- Per-delta key padded to a column width derived from the longest
key — keeps the `←` markers vertically aligned.
- Sha rendered as a 8-char prefix (consistent with git's default
short-sha length) and elided when `git_ref` is null.
Without `--summary`, drifting deltas get added/removed
`console.table` blocks one after the other (no `(no results)`
placeholders for empty deltas — they're just absent from the
expanded output).
Smoke (this clone, three baselines saved with the convention):
- 3 deltas, no drift → uniform 4-line output (header + 3 lines).
- 1 delta, contrived drift via 3-row baseline → header counts
match per-delta line counts; `console.table` renders the 169 new
paths cleanly.
- --summary on both modes drops the row tables, keeps the per-delta
provenance.
No envelope shape change — this is pure presentation. JSON output
unchanged (still `{head, deltas}` with per-delta `base`).
* docs(audit): tracer 6 — lift plan into canonical homes; ship & retire docs/plans/codemap-audit.md
Final tracer per docs/plans/codemap-audit.md § 8 step 5/6 (plan
itself is now deleted per Rule 2: "delete plans on ship; lift the
canonical bits into architecture.md").
Lifted into canonical homes:
- docs/architecture.md § CLI usage — new "Audit wiring" paragraph
next to "Query / Validate / Context wiring": cmd-audit.ts (CLI) +
application/audit-engine.ts (engine), V1_DELTAS registry + canonical
SQL projections, computeDelta + per-delta `base` metadata,
resolveAuditBaselines auto-resolve sugar + per-delta override
composition, runAuditCmd's auto-incremental-index prelude with
--no-index opt-out, v1 vs v1.x scope (no verdict / no --base <ref>).
- docs/glossary.md — new "audit" entry under § A. Disambiguates from
`codemap query --baseline` (one query, one diff) and from
`fallow audit` (code-quality verdicts — explicit non-goal per
roadmap.md § Non-goals).
- README.md § CLI — 4 new example lines under the Daily commands
block (auto-resolve, --json --summary, explicit per-delta, --no-index).
Per Rule 10 (core surface change → both rule + skill in lockstep):
- .agents/rules/codemap.md + templates/agents/rules/codemap.md — new
"Per-delta audit" CLI table row + new bold "Audit (`codemap audit`)"
section explaining the two snapshot-source flag families.
- .agents/skills/codemap/SKILL.md + templates/agents/skills/codemap/
SKILL.md — new "Audit" subsection in the Output flags block,
covering --baseline / --<delta>-baseline / per-delta `base`
metadata / --summary / --no-index / v1 no-verdict + jq idiom /
schema-bump-resilient projection.
Roadmap entries (docs/roadmap.md):
- v1 line replaced with v1.x scope notes for the two deferred slices:
--base <ref> (worktree+reindex) and verdict + threshold config.
Each carries the explicit revisit trigger settled during the
grill-me design pass.
Plan deleted:
- docs/plans/codemap-audit.md — Plan lifecycle is "delete on ship"
per docs/README.md Rule 2 + § Document Lifecycle. The decisions
of record (snapshot strategy, delta registry, verdict scope, file
layout, terminal output, per-delta baselines design fork) are now
in architecture.md / glossary.md / roadmap.md / README.md /
rule + skill. Anything else (the grilling rationale, the rejected
alternatives, the open-question triggers) lives in the PR
description and git log.
Patch changeset (no schema bump; reuses query_baselines; pre-v1
default per .agents/lessons.md "changesets bump policy").
* fix(audit): address PR #33 CodeRabbit feedback (1 Major + 3 Minor)
All 4 verified correct against the actual code; all applied.
Major:
- audit-engine.ts: validate baseline rows_json is an array before
diffing. JSON.parse("null") / "{}" / "true" all parse successfully
but produce non-array values that crash computeDelta on .length /
.map. New explicit check returns a structured AuditError naming
the actual type so the user knows what to re-save. Tests cover
both the null and object cases.
Minor:
- README.md: add a fourth example line showing the prefix + per-delta
override composition (`--baseline base --files-baseline hotfix-files`).
The behavior is implemented + tested in resolveAuditBaselines but
the README only showed the standalone forms.
- cmd-audit.ts: harden consumeFlagValue's two-token path to reject
empty-string values (`--flag ""`) and whitespace-only values
(`--flag " "`). The `--flag=` path was already strict; the
positional path silently accepted these and the failure surfaced
later as a less-clear baseline-not-found error. Two new parser
tests cover both cases.
- SKILL.md (mirrored across .agents/ and templates/agents/ per Rule
10): add the "no slot resolves → exit 1" failure mode to the
Audit subsection. The "silently absent" wording read like
--baseline <prefix> could produce an empty envelope; clarified
that audit errors instead of doing nothing.1 parent 035ee23 commit 114303f
21 files changed
Lines changed: 1392 additions & 455 deletions
File tree
- .agents
- rules
- skills/codemap
- .changeset
- docs
- plans
- src
- application
- cli
- templates/agents
- rules
- skills/codemap
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | | - | |
16 | | - | |
17 | | - | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
25 | | - | |
26 | | - | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
27 | 28 | | |
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
31 | 32 | | |
| 33 | + | |
| 34 | + | |
32 | 35 | | |
33 | 36 | | |
34 | 37 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
49 | 56 | | |
50 | 57 | | |
51 | 58 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
94 | 94 | | |
95 | 95 | | |
96 | 96 | | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
97 | 106 | | |
98 | 107 | | |
99 | 108 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
121 | 121 | | |
122 | 122 | | |
123 | 123 | | |
| 124 | + | |
| 125 | + | |
124 | 126 | | |
125 | 127 | | |
126 | 128 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
34 | 38 | | |
35 | 39 | | |
36 | 40 | | |
| |||
0 commit comments