Commit f7c159e
authored
feat: npm run audit — read-only drift detector for state/dashboard divergence (#27)
* feat: npm run audit command for orphan/duplicate/drift detection
Adds `npm run audit -- <org>` — single-command audit for the state-vs-dashboard
drift conditions that have been accumulating cruft in customer-fork repos.
Detects (read-only):
- orphan local YAML files (no state entry — Scenario B leftovers)
- state ghosts (state UUID missing on dashboard)
- state UUID collisions (cascade-duplicate fingerprint)
- content-identical resources (same lastPulledHash)
- sibling base-slug clusters (cascade-risk warning)
- dashboard orphans (UUID not in state; suppressed by .vapi-ignore)
- assistants with inline model.tools (suspected duplicate-spawn surface)
Exit code: 0 if clean, 1 if any findings.
Designed for DI: state loader, local file lister, remote fetcher are all
injectable, making tests filesystem-free and network-free.
Promotes `listExistingResourceIds` in src/pull.ts from `function` to
`export function` (one-word edit) to avoid duplicating the directory walker.
Tests in tests/audit.test.ts will be added in a follow-up commit on this
branch.
* test: add coverage for npm run audit (7 checks + formatter + integration)
Covers all 7 audit checks via DI fixtures (no filesystem, no network):
- orphan-yaml (3 cases)
- state-ghost (3 cases inc. fetchRemote=false short-circuit)
- state-uuid-collision (2 cases)
- content-identical (3 cases inc. missing-hash safety)
- sibling-base-slug (3 cases inc. cross-ref overlap)
- dashboard-orphan (4 cases inc. .vapi-ignore suppression)
- inline-tools (4 cases inc. async-Promise branch)
Plus: 1 integration test combining multiple checks, 1 exit-code mapping
test, 3 formatter tests.
* refactor: extract exitCodeForFindings helper, pin test to it
Closes the gap surfaced by the test-writer phase: audit-cmd.ts had inlined
`findings.length === 0 ? 0 : 1` at every exit-code call site, so the
exit-code test in tests/audit.test.ts could only assert on a parallel
re-derivation rather than the real CLI behavior.
Extracts a tiny exported `exitCodeForFindings(findings)` helper and routes
both exit sites through it. Test imports and pins to the helper, so future
changes to the severity bar (e.g. a `--strict` flag in v2) will surface in
the existing assertion instead of silently drifting.
No behavior change. 155/155 tests pass.
* fix(audit): use Promise.allSettled for per-type fetches + README entry
Addresses two non-blocking code-review findings before opening the PR:
1. **Fail-fast → fail-graceful for dashboard fetches.** Switched the
parallel per-type API calls from `Promise.all` to `Promise.allSettled`.
A transient 500 / 429 / network blip on one resource type used to abort
the entire audit, leaving the operator with zero findings instead of
findings-for-the-types-that-succeeded.
Now: each failed fetch emits a `fetch-failed` finding (severity: warn,
message includes the underlying error). The per-type loop checks
`remoteByType.has(type)` before running state-ghost and dashboard-orphan
checks — preventing the would-be false-positive where an empty-array
fallback marks every state entry as a ghost.
New rule: `AuditRule = ... | "fetch-failed"`.
2. **README command table missing `audit`.** Added a row under the
`validate` entry so operators discover the command from the same
surface that lists `pull`/`push`/`cleanup`/`rollback`/etc.
New test pinning the fail-graceful path: one type's `remoteFetcher`
throws → exactly 1 `fetch-failed` finding for that type, 0 false-positive
state-ghost findings for any state entry of that type, and other types'
checks proceed normally.
Suite: 156/156 pass (+1 test).1 parent 0c7a7aa commit f7c159e
6 files changed
Lines changed: 1216 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
72 | 72 | | |
73 | 73 | | |
74 | 74 | | |
| 75 | + | |
75 | 76 | | |
76 | 77 | | |
77 | 78 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| 16 | + | |
16 | 17 | | |
17 | 18 | | |
18 | 19 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
0 commit comments