Commit e7a2255
committed
fix: port Leiden algorithm to native Rust for cross-engine community-detection parity
`codegraph communities`/`--drift` ran genuinely different algorithms
depending on which engine was active: the native path ran classic Louvain
(crates/codegraph-core/src/graph/algorithms/louvain.rs, undirected
modularity optimization), while the JS/WASM fallback ran Leiden
(src/graph/algorithms/leiden/*, hardened by #1734/#1755/#1770 earlier in
this batch). Louvain and Leiden are related but distinct algorithms with
different guarantees (Leiden avoids Louvain's disconnected-community
defect), so results diverged purely based on whether the native addon
loaded, independent of any change to the analyzed codebase.
Ported Leiden to Rust (crates/codegraph-core/src/graph/algorithms/leiden.rs,
~950 lines) as a faithful line-by-line translation of the TS reference:
graph adapter with undirected symmetrization (averaging reciprocal edge
weights, not summing them -- a second, independent divergence the old
Louvain implementation had), partition with incremental aggregate deltas,
modularity quality/diff functions, greedy local-move phase, true Leiden
refinement (singleton start, singleton guard, Boltzmann probabilistic
selection), post-refinement disconnected-community splitting, and
multi-level coarsening. Also ported a mulberry32 PRNG matching the TS
reference's exact bit-pattern arithmetic (u32 wrapping ops), since the
refinement phase's random draws must stay in lockstep with the JS engine
bit-for-bit.
Scope: covers exactly the option surface `louvainCommunities`/
`LouvainOptions` (src/graph/algorithms/louvain.ts) ever exercises --
undirected, modularity-only, "neighbors" candidate strategy, refine always
on, uniform edge weight/node size. The TS reference's directed mode, CPM
quality, alternate candidate strategies, allowNewCommunity, fixedNodes, and
preserveLabels are not reachable from this binding and are not ported; see
leiden.rs's module doc and follow-up issue #1936.
Determinism (per #1734's exact failure mode): every place the TS reference
relies on Map/Array *insertion order* for tie-breaking, this port uses an
insertion-order-preserving Vec + HashMap-as-lookup-only pattern rather than
a BTreeMap -- a BTreeMap would be deterministic but *sorted*, which does
not match the JS engine's insertion order and would silently reintroduce
cross-engine divergence.
Retired the old classic-Louvain louvain.rs entirely (confirmed via
repo-wide grep that its only consumer was the mod.rs re-export feeding this
one napi binding) and renamed the native binding louvainCommunities ->
leidenCommunities (types.ts's NativeAddon.leidenCommunities is marked
optional so older published native packages that predate this rename
correctly fall back to the JS engine instead of silently running the old,
now-removed algorithm). Updated the misleading "native: Louvain, JS:
Leiden" doc comments in louvain.ts, config.ts, and the communities CLI/MCP
descriptions that documented the mismatch as intentional.
Verification:
- Before fix: this repo's own dependency graph via native vs JS differed
in both community count and modularity at file level (322 vs 335
communities, Q=0.566 vs 0.536) and function level (5540 vs 6072
communities, Q=0.850 vs 0.655), confirming the bug was real.
- First native Leiden port attempt still diverged from TS beyond level 0
of the multi-level pipeline; root-caused to the coarse-graph builder
reusing raw first-seen-edge order instead of replicating the TS
reference's two-stage process (building a synthetic ascending-id
CodeGraph, then re-reading it through the undirected dedup generator,
which reorders each community's adjacency list). Fixed by making
build_coarse_graph replicate both stages exactly.
- After fix: native and JS produce byte-identical assignments and
modularity (including exact community-id labels, not just equivalent
groupings) across 40 seed/resolution/knob combinations on this repo's
own file- and function-level graphs, 72 combinations across 8 other
fixture projects (multiple languages, this repo's own Rust crate at
6072/13062 nodes/edges), and a suite of hand-built/synthetic graphs
specifically covering reciprocal edges, self-loops, and forced
multi-level coarsening.
- Determinism re-verified for the new implementation: 12 separate process
invocations plus 2 full clean rebuilds (3 independently compiled
binaries total) all produce byte-identical (SHA-256-matching) output on
this repo's own graph.
- npm test: 223 files, 3665 passed, 0 failed, 30 skipped, 2 todo.
- npm run lint: clean.
- cargo test: 513 passed, 0 failed. cargo clippy: leiden.rs clean.
- Resolution-benchmark suite: 206 passed (community detection isn't part
of it, but confirms nothing else regressed).
Fixes #1804
Impact: 2 functions changed, 12 affected1 parent d84f6b5 commit e7a2255
12 files changed
Lines changed: 1749 additions & 574 deletions
File tree
- crates/codegraph-core/src
- graph/algorithms
- src
- cli/commands
- graph/algorithms
- infrastructure
- mcp
- tests/graph/algorithms
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
128 | 128 | | |
129 | 129 | | |
130 | 130 | | |
131 | | - | |
| 131 | + | |
132 | 132 | | |
133 | 133 | | |
134 | 134 | | |
| |||
148 | 148 | | |
149 | 149 | | |
150 | 150 | | |
151 | | - | |
| 151 | + | |
152 | 152 | | |
153 | 153 | | |
154 | 154 | | |
| |||
Large diffs are not rendered by default.
0 commit comments