Skip to content

Commit 0e41ceb

Browse files
carlos-almgithub-actions[bot]
carlos-alm
and
github-actions[bot]
authored
feat: Halstead/LOC/MI metrics + benchmark dev/release versioning (#161)
* feat: benchmark dev/release versioning and npm source resolution Add shared bench-config.js with --version and --npm CLI flags so benchmark scripts can install and measure the actual published npm package instead of always using local source labeled with package.json version. - New scripts/lib/bench-config.js: parseArgs(), resolveBenchmarkSource(), srcImport() with retry/backoff for npm propagation - All 4 benchmark runners use CLI-driven version (default "dev") and srcImport() for dynamic imports; call cleanup() on exit - All 4 report updaters: dev entries are rolling (one at a time), releases remove dev entry, findPrevRelease() skips dev for trends - Workflow: workflow_dispatch accepts version input, detects npm vs local mode, adds npm propagation wait, removes artifact download steps - Re-benchmarked 2.4.0 from actual npm package (123 files, 801 nodes) Impact: 10 functions changed, 10 affected * refactor: simplify redundant unwrap_or pattern in complexity.rs Impact: 1 functions changed, 0 affected --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
1 parent 44fe899 commit 0e41ceb

13 files changed

Lines changed: 459 additions & 304 deletions

‎.github/workflows/benchmark.yml‎

Lines changed: 182 additions & 140 deletions
Large diffs are not rendered by default.

‎README.md‎

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -423,14 +423,12 @@ Self-measured on every release via CI ([build benchmarks](generated/BUILD-BENCHM
423423

424424
| Metric | Latest |
425425
|---|---|
426-
| Build speed (native) | **1.9 ms/file** |
427-
| Build speed (WASM) | **9.3 ms/file** |
428-
| Query time | **3ms** |
429-
| No-op rebuild (native) | **4ms** |
430-
| 1-file rebuild (native) | **89ms** |
431-
| Query: fn-deps | **2.1ms** |
432-
| Query: path | **1.2ms** |
433-
| ~50,000 files (est.) | **~95.0s build** |
426+
| Build speed | **5.1 ms/file** |
427+
| Query time | **2ms** |
428+
| No-op rebuild | **5ms** |
429+
| 1-file rebuild | **192ms** |
430+
| Query: fn-deps | **0.5ms** |
431+
| ~50,000 files (est.) | **~255.0s build** |
434432

435433
Metrics are normalized per file for cross-version comparability. Times above are for a full initial build — incremental rebuilds only re-parse changed files.
436434

‎crates/codegraph-core/src/complexity.rs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -999,7 +999,7 @@ pub fn compute_all_metrics(
999999
cognitive,
10001000
cyclomatic,
10011001
max_nesting,
1002-
halstead: Some(halstead.unwrap_or(HalsteadMetrics {
1002+
halstead: halstead.or(Some(HalsteadMetrics {
10031003
n1: 0, n2: 0, big_n1: 0, big_n2: 0,
10041004
vocabulary: 0, length: 0,
10051005
volume: 0.0, difficulty: 0.0, effort: 0.0, bugs: 0.0,

‎generated/BUILD-BENCHMARKS.md‎

Lines changed: 32 additions & 95 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,7 @@ Metrics are normalized per file for cross-version comparability.
55

66
| Version | Engine | Date | Files | Build (ms/file) | Query (ms) | Nodes/file | Edges/file | DB (bytes/file) |
77
|---------|--------|------|------:|----------------:|-----------:|-----------:|-----------:|----------------:|
8-
| 2.4.0 | native | 2026-02-27 | 122 | 1.9 ~ | 2.5 ↑67% | 6.4 ↑10% | 10.9 ↑20% | 5238 ↑36% |
9-
| 2.4.0 | wasm | 2026-02-27 | 122 | 9.3 ↑41% | 3.4 ↑62% | 6.4 ↑10% | 10.9 ↑20% | 5506 ↑43% |
8+
| 2.4.0 | wasm | 2026-02-28 | 123 | 5.1 ↓23% | 2.2 ↑5% | 6.5 ↑12% | 10.7 ↑18% | 4695 ↑22% |
109
| 2.3.0 | native | 2026-02-24 | 99 | 1.9 ~ | 1.5 ↑7% | 5.8 ↑7% | 9.1 ~ | 3848 ~ |
1110
| 2.3.0 | wasm | 2026-02-24 | 99 | 6.6 ~ | 2.1 ↑11% | 5.8 ~ | 9.1 ↑3% | 3848 ~ |
1211
| 2.1.0 | native | 2026-02-23 | 92 | 1.9 ↓24% | 1.4 ↑17% | 5.4 ↑6% | 9.1 ↓47% | 3829 ↓14% |
@@ -16,64 +15,39 @@ Metrics are normalized per file for cross-version comparability.
1615

1716
### Raw totals (latest)
1817

19-
#### Native (Rust)
20-
21-
| Metric | Value |
22-
|--------|-------|
23-
| Build time | 229ms |
24-
| Query time | 3ms |
25-
| Nodes | 778 |
26-
| Edges | 1,333 |
27-
| DB size | 624 KB |
28-
| Files | 122 |
29-
3018
#### WASM
3119

3220
| Metric | Value |
3321
|--------|-------|
34-
| Build time | 1.1s |
35-
| Query time | 3ms |
36-
| Nodes | 778 |
37-
| Edges | 1,333 |
38-
| DB size | 656 KB |
39-
| Files | 122 |
40-
41-
### Build Phase Breakdown (latest)
42-
43-
| Phase | Native | WASM |
44-
|-------|-------:|-----:|
45-
| Parse | 116 ms | 639.2 ms |
46-
| Insert nodes | 14.1 ms | 15.9 ms |
47-
| Resolve imports | 10.2 ms | 13.4 ms |
48-
| Build edges | 61.9 ms | 59.1 ms |
49-
| Structure | 4.1 ms | 7.3 ms |
50-
| Roles | 4.7 ms | 5.2 ms |
51-
| Complexity | 4.6 ms | 367.8 ms |
22+
| Build time | 630ms |
23+
| Query time | 2ms |
24+
| Nodes | 801 |
25+
| Edges | 1,320 |
26+
| DB size | 564 KB |
27+
| Files | 123 |
5228

5329
### Estimated performance at 50,000 files
5430

5531
Extrapolated linearly from per-file metrics above.
5632

5733
| Metric | Native (Rust) | WASM |
5834
|--------|---:|---:|
59-
| Build time | 95.0s | 465.0s |
60-
| DB size | 249.8 MB | 262.5 MB |
61-
| Nodes | 320,000 | 320,000 |
62-
| Edges | 545,000 | 545,000 |
35+
| Build time | n/a | 255.0s |
36+
| DB size | n/a | 223.9 MB |
37+
| Nodes | n/a | 325,000 |
38+
| Edges | n/a | 535,000 |
6339

6440
### Incremental Rebuilds
6541

6642
| Version | Engine | No-op (ms) | 1-file (ms) |
6743
|---------|--------|----------:|-----------:|
68-
| 2.4.0 | native | 4 | 89 |
69-
| 2.4.0 | wasm | 4 | 362 |
44+
| 2.4.0 | wasm | 5 | 192 |
7045

7146
### Query Latency
7247

7348
| Version | Engine | fn-deps (ms) | fn-impact (ms) | path (ms) | roles (ms) |
7449
|---------|--------|------------:|--------------:|----------:|----------:|
75-
| 2.4.0 | native | 2.1 | 1.6 | 1.2 | 1.1 |
76-
| 2.4.0 | wasm | 2.1 | 1.6 | 1.2 | 1.1 |
50+
| 2.4.0 | wasm | 0.5 | 0.5 | null | 0.9 |
7751

7852
<!-- NOTES_START -->
7953
### Notes
@@ -99,68 +73,31 @@ extractor is needed to recover the regression.
9973
[
10074
{
10175
"version": "2.4.0",
102-
"date": "2026-02-27",
103-
"files": 122,
76+
"date": "2026-02-28",
77+
"files": 123,
10478
"wasm": {
105-
"buildTimeMs": 1138,
106-
"queryTimeMs": 3.4,
107-
"nodes": 778,
108-
"edges": 1333,
109-
"dbSizeBytes": 671744,
79+
"buildTimeMs": 630,
80+
"queryTimeMs": 2.2,
81+
"nodes": 801,
82+
"edges": 1320,
83+
"dbSizeBytes": 577536,
11084
"perFile": {
111-
"buildTimeMs": 9.3,
112-
"nodes": 6.4,
113-
"edges": 10.9,
114-
"dbSizeBytes": 5506
85+
"buildTimeMs": 5.1,
86+
"nodes": 6.5,
87+
"edges": 10.7,
88+
"dbSizeBytes": 4695
11589
},
116-
"noopRebuildMs": 4,
117-
"oneFileRebuildMs": 362,
90+
"noopRebuildMs": 5,
91+
"oneFileRebuildMs": 192,
11892
"queries": {
119-
"fnDepsMs": 2.1,
120-
"fnImpactMs": 1.6,
121-
"pathMs": 1.2,
122-
"rolesMs": 1.1
93+
"fnDepsMs": 0.5,
94+
"fnImpactMs": 0.5,
95+
"pathMs": null,
96+
"rolesMs": 0.9
12397
},
124-
"phases": {
125-
"parseMs": 639.2,
126-
"insertMs": 15.9,
127-
"resolveMs": 13.4,
128-
"edgesMs": 59.1,
129-
"structureMs": 7.3,
130-
"rolesMs": 5.2,
131-
"complexityMs": 367.8
132-
}
98+
"phases": null
13399
},
134-
"native": {
135-
"buildTimeMs": 229,
136-
"queryTimeMs": 2.5,
137-
"nodes": 778,
138-
"edges": 1333,
139-
"dbSizeBytes": 638976,
140-
"perFile": {
141-
"buildTimeMs": 1.9,
142-
"nodes": 6.4,
143-
"edges": 10.9,
144-
"dbSizeBytes": 5238
145-
},
146-
"noopRebuildMs": 4,
147-
"oneFileRebuildMs": 89,
148-
"queries": {
149-
"fnDepsMs": 2.1,
150-
"fnImpactMs": 1.6,
151-
"pathMs": 1.2,
152-
"rolesMs": 1.1
153-
},
154-
"phases": {
155-
"parseMs": 116,
156-
"insertMs": 14.1,
157-
"resolveMs": 10.2,
158-
"edgesMs": 61.9,
159-
"structureMs": 4.1,
160-
"rolesMs": 4.7,
161-
"complexityMs": 4.6
162-
}
163-
}
100+
"native": null
164101
},
165102
{
166103
"version": "2.3.0",

‎scripts/benchmark.js‎

Lines changed: 13 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -12,24 +12,24 @@
1212
import fs from 'node:fs';
1313
import path from 'node:path';
1414
import { performance } from 'node:perf_hooks';
15-
import { fileURLToPath, pathToFileURL } from 'node:url';
15+
import { fileURLToPath } from 'node:url';
1616
import Database from 'better-sqlite3';
17+
import { resolveBenchmarkSource, srcImport } from './lib/bench-config.js';
1718

1819
const __dirname = path.dirname(fileURLToPath(import.meta.url));
1920
const root = path.resolve(__dirname, '..');
2021

21-
// Read version from package.json
22-
const pkg = JSON.parse(fs.readFileSync(path.join(root, 'package.json'), 'utf8'));
22+
const { version, srcDir, cleanup } = await resolveBenchmarkSource();
2323

2424
const dbPath = path.join(root, '.codegraph', 'graph.db');
2525

2626
// Import programmatic API (use file:// URLs for Windows compatibility)
27-
const { buildGraph } = await import(pathToFileURL(path.join(root, 'src', 'builder.js')).href);
27+
const { buildGraph } = await import(srcImport(srcDir, 'builder.js'));
2828
const { fnDepsData, fnImpactData, pathData, rolesData, statsData } = await import(
29-
pathToFileURL(path.join(root, 'src', 'queries.js')).href
29+
srcImport(srcDir, 'queries.js')
3030
);
3131
const { isNativeAvailable } = await import(
32-
pathToFileURL(path.join(root, 'src', 'native.js')).href
32+
srcImport(srcDir, 'native.js')
3333
);
3434

3535
const INCREMENTAL_RUNS = 3;
@@ -133,10 +133,10 @@ async function benchmarkEngine(engine) {
133133
}
134134

135135
const queries = {
136-
fnDepsMs: benchQuery(fnDepsData, targets.hub, dbPath, { depth: 3, noTests: true }),
137-
fnImpactMs: benchQuery(fnImpactData, targets.hub, dbPath, { depth: 3, noTests: true }),
138-
pathMs: benchQuery(pathData, targets.hub, targets.leaf, dbPath, { noTests: true }),
139-
rolesMs: benchQuery(rolesData, dbPath, { noTests: true }),
136+
fnDepsMs: fnDepsData ? benchQuery(fnDepsData, targets.hub, dbPath, { depth: 3, noTests: true }) : null,
137+
fnImpactMs: fnImpactData ? benchQuery(fnImpactData, targets.hub, dbPath, { depth: 3, noTests: true }) : null,
138+
pathMs: pathData ? benchQuery(pathData, targets.hub, targets.leaf, dbPath, { noTests: true }) : null,
139+
rolesMs: rolesData ? benchQuery(rolesData, dbPath, { noTests: true }) : null,
140140
};
141141

142142
return {
@@ -173,7 +173,7 @@ if (isNativeAvailable()) {
173173
console.log = origLog;
174174

175175
const result = {
176-
version: pkg.version,
176+
version,
177177
date: new Date().toISOString().slice(0, 10),
178178
files: wasm.files,
179179
wasm: {
@@ -205,3 +205,5 @@ const result = {
205205
};
206206

207207
console.log(JSON.stringify(result, null, 2));
208+
209+
cleanup();

‎scripts/embedding-benchmark.js‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,15 +17,16 @@ import path from 'node:path';
1717
import { performance } from 'node:perf_hooks';
1818
import { fileURLToPath } from 'node:url';
1919
import Database from 'better-sqlite3';
20+
import { resolveBenchmarkSource, srcImport } from './lib/bench-config.js';
2021

2122
const __dirname = path.dirname(fileURLToPath(import.meta.url));
2223
const root = path.resolve(__dirname, '..');
2324

24-
const pkg = JSON.parse(fs.readFileSync(path.join(root, 'package.json'), 'utf8'));
25+
const { version, srcDir, cleanup } = await resolveBenchmarkSource();
2526
const dbPath = path.join(root, '.codegraph', 'graph.db');
2627

2728
const { buildEmbeddings, MODELS, searchData, disposeModel } = await import(
28-
new URL('../src/embedder.js', import.meta.url).href
29+
srcImport(srcDir, 'embedder.js')
2930
);
3031

3132
// Redirect console.log to stderr so only JSON goes to stdout
@@ -136,11 +137,13 @@ for (const key of modelKeys) {
136137
console.log = origLog;
137138

138139
const output = {
139-
version: pkg.version,
140+
version,
140141
date: new Date().toISOString().slice(0, 10),
141142
strategy: 'structured',
142143
symbols: symbols.length,
143144
models: results,
144145
};
145146

146147
console.log(JSON.stringify(output, null, 2));
148+
149+
cleanup();

‎scripts/incremental-benchmark.js‎

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -13,20 +13,22 @@
1313
import fs from 'node:fs';
1414
import path from 'node:path';
1515
import { performance } from 'node:perf_hooks';
16-
import { fileURLToPath, pathToFileURL } from 'node:url';
16+
import { fileURLToPath } from 'node:url';
17+
import { resolveBenchmarkSource, srcImport } from './lib/bench-config.js';
18+
1719
const __dirname = path.dirname(fileURLToPath(import.meta.url));
1820
const root = path.resolve(__dirname, '..');
1921

20-
const pkg = JSON.parse(fs.readFileSync(path.join(root, 'package.json'), 'utf8'));
22+
const { version, srcDir, cleanup } = await resolveBenchmarkSource();
2123
const dbPath = path.join(root, '.codegraph', 'graph.db');
2224

23-
const { buildGraph } = await import(pathToFileURL(path.join(root, 'src', 'builder.js')).href);
24-
const { statsData } = await import(pathToFileURL(path.join(root, 'src', 'queries.js')).href);
25+
const { buildGraph } = await import(srcImport(srcDir, 'builder.js'));
26+
const { statsData } = await import(srcImport(srcDir, 'queries.js'));
2527
const { resolveImportPath, resolveImportsBatch, resolveImportPathJS } = await import(
26-
pathToFileURL(path.join(root, 'src', 'resolve.js')).href
28+
srcImport(srcDir, 'resolve.js')
2729
);
2830
const { isNativeAvailable } = await import(
29-
pathToFileURL(path.join(root, 'src', 'native.js')).href
31+
srcImport(srcDir, 'native.js')
3032
);
3133

3234
// Redirect console.log to stderr so only JSON goes to stdout
@@ -181,7 +183,7 @@ console.error(` native=${resolve.nativeBatchMs}ms js=${resolve.jsFallbackMs}ms`
181183
console.log = origLog;
182184

183185
const result = {
184-
version: pkg.version,
186+
version,
185187
date: new Date().toISOString().slice(0, 10),
186188
files,
187189
wasm: {
@@ -200,3 +202,5 @@ const result = {
200202
};
201203

202204
console.log(JSON.stringify(result, null, 2));
205+
206+
cleanup();

0 commit comments

Comments
 (0)