|
| 1 | +--- |
| 2 | +name: parity |
| 3 | +description: Audit WASM/native engine correctness parity across all resolution fixtures and fix any divergence at the root cause — both engines must produce identical graphs |
| 4 | +argument-hint: "[--langs js,python] [--hybrid] [--audit-only]" |
| 5 | +allowed-tools: Bash, Read, Write, Edit, Glob, Grep, Agent |
| 6 | +--- |
| 7 | + |
| 8 | +# /parity — Engine Correctness Parity Audit & Fix |
| 9 | + |
| 10 | +Codegraph has two engines that MUST produce identical results (see CLAUDE.md): |
| 11 | + |
| 12 | +- **wasm** — JS pipeline + JS extractors + JS edge resolution |
| 13 | +- **native** — full Rust orchestrator (`crates/codegraph-core/src/domain/graph/builder/pipeline.rs`) |
| 14 | +- **hybrid** — JS pipeline + napi `buildCallEdges` (the fallback when the |
| 15 | + orchestrator is skipped: forced full rebuilds, older addons) |
| 16 | + |
| 17 | +This skill runs `scripts/parity-compare.mjs`, which builds every |
| 18 | +resolution-benchmark fixture with each engine and compares the **full node and |
| 19 | +edge multisets** (kind, name, file, line, confidence, dynamic flag). Any |
| 20 | +difference is a bug in the less-accurate engine — never an acceptable gap, and |
| 21 | +never something to document as expected. The skill finds the root cause, fixes |
| 22 | +it, and re-verifies until the audit is clean. |
| 23 | + |
| 24 | +## Arguments |
| 25 | + |
| 26 | +- `$ARGUMENTS` may contain: |
| 27 | + - `--langs a,b,c` — restrict to specific fixture names (e.g. `javascript,pts-javascript`) |
| 28 | + - `--hybrid` — also audit the hybrid path (recommended; slower) |
| 29 | + - `--audit-only` — report divergences without fixing them |
| 30 | + - No arguments — full audit across all fixtures, then fix divergences |
| 31 | + |
| 32 | +## Phase 0 — Pre-flight |
| 33 | + |
| 34 | +All steps run from the repo root. |
| 35 | + |
| 36 | +1. Confirm `scripts/parity-compare.mjs` exists. If not, this repo doesn't have |
| 37 | + the parity tooling — stop and report. |
| 38 | +2. Build the TypeScript dist (the script imports `dist/index.js`, and extractor |
| 39 | + changes in `src/` are invisible until rebuilt): |
| 40 | + ```bash |
| 41 | + npm run build |
| 42 | + ``` |
| 43 | +3. Ensure the native addon reflects the local Rust source: |
| 44 | + ```bash |
| 45 | + cd crates/codegraph-core && npx napi build --platform --release && cd ../.. |
| 46 | + ``` |
| 47 | + On macOS, locally built binaries must be re-signed or Node kills the process |
| 48 | + (exit 137): |
| 49 | + ```bash |
| 50 | + codesign --sign - --force crates/codegraph-core/*.node |
| 51 | + ``` |
| 52 | +4. Verify the loader picks it up: |
| 53 | + ```bash |
| 54 | + node -e "import('./dist/infrastructure/native.js').then(m => console.log(m.isNativeAvailable()))" |
| 55 | + ``` |
| 56 | + If `false`, stop and report — auditing parity without the native engine is |
| 57 | + meaningless. Note: if the repo (or a parent) has |
| 58 | + `node_modules/@optave/codegraph-<platform>-<arch>/` installed, Node resolves |
| 59 | + that package **before** the crate-local build — copy the freshly built |
| 60 | + binary over `codegraph-core.node` in that package dir, or the audit will |
| 61 | + silently test the published binary instead of your changes. |
| 62 | + |
| 63 | +## Phase 1 — Audit |
| 64 | + |
| 65 | +Run the comparison (pass through `--langs` / `--hybrid` from `$ARGUMENTS`): |
| 66 | + |
| 67 | +```bash |
| 68 | +node scripts/parity-compare.mjs [--langs ...] [--hybrid] 2>/dev/null |
| 69 | +``` |
| 70 | + |
| 71 | +- Exit 0 → parity holds. Skip to Phase 4 and report a clean audit. |
| 72 | +- Exit 1 → divergences or fixture build failures. Collect every `[node]` / |
| 73 | + `[edge]` diff line and any `BUILD FAILED` fixtures. |
| 74 | +- Exit 2 → pre-flight failure; go back to Phase 0. |
| 75 | + |
| 76 | +For machine-readable output (useful when many fixtures diverge), re-run with |
| 77 | +`--json` and parse `fixtures[].comparisons[].nodeDiffs/edgeDiffs`. |
| 78 | + |
| 79 | +If `--audit-only` was passed: report the diffs (Phase 4 format) and stop. |
| 80 | + |
| 81 | +## Phase 2 — Root-cause and fix |
| 82 | + |
| 83 | +For each divergence, identify which engine is wrong — the one missing edges or |
| 84 | +producing lower-quality resolution is usually the buggy one, but verify by |
| 85 | +reading the fixture source and deciding what the *correct* graph is. |
| 86 | + |
| 87 | +**Localize the bug by which paths disagree:** |
| 88 | + |
| 89 | +| wasm | hybrid | native | Bug location | |
| 90 | +|------|--------|--------|--------------| |
| 91 | +| A | A | B | Rust pipeline prep (`pipeline.rs`) or Rust extractor (`crates/.../extractors/`) — the napi solver gets correct input from JS but the orchestrator's own input differs | |
| 92 | +| A | B | B | Rust `build_edges.rs` solver (shared by hybrid + native) | |
| 93 | +| A | B | A | JS↔napi boundary: `NativeFileEntry` plumbing in `build-edges.ts` or the wasm-worker protocol | |
| 94 | +| B | A | A | JS extractor or JS resolution (`src/extractors/`, `src/domain/graph/builder/stages/build-edges.ts`) | |
| 95 | + |
| 96 | +**Fix rules (from CLAUDE.md — non-negotiable):** |
| 97 | + |
| 98 | +- Fix the extraction/resolution layer that produces incorrect results. Never |
| 99 | + add comments, tests, or fixture exclusions that frame wrong output as |
| 100 | + expected. |
| 101 | +- Changes may land in either language or both — create the best version based |
| 102 | + on both implementations, don't restrict the fix to one side. |
| 103 | +- The module layout is mirrored between `src/` and `crates/codegraph-core/src/` |
| 104 | + — read the TS and Rust counterparts side by side (e.g. |
| 105 | + `src/domain/graph/builder/stages/build-edges.ts` ↔ |
| 106 | + `crates/.../domain/graph/builder/stages/build_edges.rs`). |
| 107 | +- Mirror *semantics exactly*: confidence constants, hop penalties, tie-breaking |
| 108 | + order, first-wins vs highest-wins rules. A 0.05 confidence difference is a |
| 109 | + parity failure. |
| 110 | +- Add a focused unit test next to the fix (Rust `#[cfg(test)]` or vitest) that |
| 111 | + pins the behavior. |
| 112 | + |
| 113 | +**Gotchas that mask fixes:** |
| 114 | + |
| 115 | +- `src/` changes need `npm run build` before the script (which imports dist) |
| 116 | + sees them. |
| 117 | +- Rust changes need the napi rebuild + macOS codesign from Phase 0. |
| 118 | +- New `ExtractorOutput` fields must be added to `SerializedExtractorOutput` in |
| 119 | + `src/domain/wasm-worker-{protocol,entry,pool}.ts` or they are silently |
| 120 | + dropped at the Worker-thread boundary. |
| 121 | +- New per-file fields crossing the napi boundary need: the `FileSymbols` / |
| 122 | + `FileEdgeInput` structs in `crates/.../types.rs` & `build_edges.rs`, the |
| 123 | + `NativeFileEntry` assembly in `build-edges.ts`, and the orchestrator's own |
| 124 | + assembly in `pipeline.rs` (`build_and_insert_call_edges`). Missing the last |
| 125 | + one produces hybrid-OK/native-broken splits. |
| 126 | +- Out-of-scope findings discovered along the way (pre-existing bugs, refactor |
| 127 | + opportunities) → `gh issue create` immediately, then continue. |
| 128 | + |
| 129 | +## Phase 3 — Verify |
| 130 | + |
| 131 | +Repeat until the audit is clean — never stop at "fewer diffs than before": |
| 132 | + |
| 133 | +1. Rebuild whichever side changed (`npm run build` / napi build + codesign). |
| 134 | +2. Re-run the Phase 1 audit command. Any remaining divergence → back to Phase 2. |
| 135 | +3. Once clean, run the full verification suite — all must pass: |
| 136 | + ```bash |
| 137 | + cargo test --manifest-path crates/codegraph-core/Cargo.toml |
| 138 | + npm test |
| 139 | + npx vitest run tests/benchmarks/resolution/resolution-benchmark.test.ts |
| 140 | + ``` |
| 141 | + (From a `.claude` worktree, vitest needs the worktree override config — |
| 142 | + check memory/project notes if no tests are found.) |
| 143 | +4. If any verification step cannot run, STOP and report it — never proceed |
| 144 | + with unverified changes. |
| 145 | + |
| 146 | +## Phase 4 — Report |
| 147 | + |
| 148 | +Print a summary: |
| 149 | + |
| 150 | +``` |
| 151 | +PARITY AUDIT — <date> |
| 152 | +Fixtures audited: N (wasm vs native[, hybrid]) |
| 153 | +Divergences found: M |
| 154 | +Fixed: <file:line summary per fix, with engine + root cause> |
| 155 | +Verification: cargo test ✓ | npm test ✓ | resolution benchmark ✓ |
| 156 | +Issues filed: #NNN (out-of-scope findings) |
| 157 | +``` |
| 158 | + |
| 159 | +- If divergences were found and fixed, list each root cause in one line — |
| 160 | + which engine was wrong, which layer, what semantic was mismatched. |
| 161 | +- If `--audit-only`: list divergences grouped by fixture with the |
| 162 | + wasm/hybrid/native localization table applied. |
| 163 | +- Suggest committing engine fixes separately from unrelated work (one PR = one |
| 164 | + concern). |
| 165 | + |
| 166 | +## Rules |
| 167 | + |
| 168 | +- **Zero divergence is the only passing state** — a single edge differing in |
| 169 | + confidence is a failure. |
| 170 | +- **Never exclude a fixture or file to make the audit pass.** |
| 171 | +- **Never run the audit against a stale dist or stale native binary** — Phase 0 |
| 172 | + is mandatory after any code change. |
| 173 | +- **The wasm/hybrid/native disagreement pattern localizes the bug** — use the |
| 174 | + table before reading code. |
| 175 | +- **Both engines evolve together**: a feature added to one engine without the |
| 176 | + other is a parity bug from day one. New resolution techniques must land in |
| 177 | + `src/` and `crates/codegraph-core/src/` in the same PR. |
0 commit comments