Skip to content

Commit 53fc34f

Browse files
fix: benchmark script regex + workflow branch naming (#124)
* docs: update build performance benchmarks * fix: benchmark script regex preserves h3 subsections in README The performance section regex in update-benchmark-report.js was matching from ## Performance all the way to the next ## heading, swallowing any ### subsections (like Lightweight Footprint) in between. Changed the lookahead to also stop at h3 headings and restored the deleted section. * fix: use docs/ prefix for benchmark workflow branch names The benchmark workflow created branches with benchmark/ prefix which doesn't match the required branch naming convention (feat/, fix/, docs/, etc.). Changed all four jobs to use docs/benchmark-* instead. --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
1 parent 6a700b2 commit 53fc34f

4 files changed

Lines changed: 58 additions & 25 deletions

File tree

.github/workflows/benchmark.yml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ jobs:
6262
git config user.name "github-actions[bot]"
6363
git config user.email "github-actions[bot]@users.noreply.github.com"
6464
65-
BRANCH="benchmark/build-$(date +%Y%m%d-%H%M%S)"
65+
BRANCH="docs/benchmark-build-$(date +%Y%m%d-%H%M%S)"
6666
git checkout -b "$BRANCH"
6767
git add generated/BUILD-BENCHMARKS.md README.md
6868
git commit -m "docs: update build performance benchmarks"
@@ -139,7 +139,7 @@ jobs:
139139
git config user.name "github-actions[bot]"
140140
git config user.email "github-actions[bot]@users.noreply.github.com"
141141
142-
BRANCH="benchmark/embedding-$(date +%Y%m%d-%H%M%S)"
142+
BRANCH="docs/benchmark-embedding-$(date +%Y%m%d-%H%M%S)"
143143
git checkout -b "$BRANCH"
144144
git add generated/EMBEDDING-BENCHMARKS.md
145145
git commit -m "docs: update embedding benchmarks"
@@ -204,7 +204,7 @@ jobs:
204204
git config user.name "github-actions[bot]"
205205
git config user.email "github-actions[bot]@users.noreply.github.com"
206206
207-
BRANCH="benchmark/query-$(date +%Y%m%d-%H%M%S)"
207+
BRANCH="docs/benchmark-query-$(date +%Y%m%d-%H%M%S)"
208208
git checkout -b "$BRANCH"
209209
git add generated/QUERY-BENCHMARKS.md
210210
git commit -m "docs: update query benchmarks"
@@ -269,7 +269,7 @@ jobs:
269269
git config user.name "github-actions[bot]"
270270
git config user.email "github-actions[bot]@users.noreply.github.com"
271271
272-
BRANCH="benchmark/incremental-$(date +%Y%m%d-%H%M%S)"
272+
BRANCH="docs/benchmark-incremental-$(date +%Y%m%d-%H%M%S)"
273273
git checkout -b "$BRANCH"
274274
git add generated/INCREMENTAL-BENCHMARKS.md
275275
git commit -m "docs: update incremental benchmarks"

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -403,10 +403,10 @@ Self-measured on every release via CI ([build benchmarks](generated/BUILD-BENCHM
403403

404404
| Metric | Latest |
405405
|---|---|
406-
| Build speed (native) | **1.9 ms/file** |
407-
| Build speed (WASM) | **6.6 ms/file** |
406+
| Build speed (native) | **2.2 ms/file** |
407+
| Build speed (WASM) | **6.4 ms/file** |
408408
| Query time | **2ms** |
409-
| ~50,000 files (est.) | **~95.0s build** |
409+
| ~50,000 files (est.) | **~110.0s build** |
410410

411411
Metrics are normalized per file for cross-version comparability. Times above are for a full initial build — incremental rebuilds only re-parse changed files.
412412

generated/BUILD-BENCHMARKS.md

Lines changed: 47 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,8 @@ Metrics are normalized per file for cross-version comparability.
55

66
| Version | Engine | Date | Files | Build (ms/file) | Query (ms) | Nodes/file | Edges/file | DB (bytes/file) |
77
|---------|--------|------|------:|----------------:|-----------:|-----------:|-----------:|----------------:|
8+
| 2.4.0 | native | 2026-02-26 | 109 | 2.2 ↑16% | 1.6 ↑7% | 5.9 ~ | 9.6 ↑5% | 4397 ↑14% |
9+
| 2.4.0 | wasm | 2026-02-26 | 109 | 6.4 ↓3% | 2.1 ~ | 5.9 ~ | 9.6 ↑5% | 4397 ↑14% |
810
| 2.3.0 | native | 2026-02-24 | 99 | 1.9 ~ | 1.5 ↑7% | 5.8 ↑7% | 9.1 ~ | 3848 ~ |
911
| 2.3.0 | wasm | 2026-02-24 | 99 | 6.6 ~ | 2.1 ↑11% | 5.8 ~ | 9.1 ↑3% | 3848 ~ |
1012
| 2.1.0 | native | 2026-02-23 | 92 | 1.9 ↓24% | 1.4 ↑17% | 5.4 ↑6% | 9.1 ↓47% | 3829 ↓14% |
@@ -18,37 +20,68 @@ Metrics are normalized per file for cross-version comparability.
1820

1921
| Metric | Value |
2022
|--------|-------|
21-
| Build time | 183ms |
23+
| Build time | 235ms |
2224
| Query time | 2ms |
23-
| Nodes | 575 |
24-
| Edges | 897 |
25-
| DB size | 372 KB |
26-
| Files | 99 |
25+
| Nodes | 640 |
26+
| Edges | 1,049 |
27+
| DB size | 468 KB |
28+
| Files | 109 |
2729

2830
#### WASM
2931

3032
| Metric | Value |
3133
|--------|-------|
32-
| Build time | 649ms |
34+
| Build time | 700ms |
3335
| Query time | 2ms |
34-
| Nodes | 575 |
35-
| Edges | 897 |
36-
| DB size | 372 KB |
37-
| Files | 99 |
36+
| Nodes | 640 |
37+
| Edges | 1,049 |
38+
| DB size | 468 KB |
39+
| Files | 109 |
3840

3941
### Estimated performance at 50,000 files
4042

4143
Extrapolated linearly from per-file metrics above.
4244

4345
| Metric | Native (Rust) | WASM |
4446
|--------|---:|---:|
45-
| Build time | 95.0s | 330.0s |
46-
| DB size | 183.5 MB | 183.5 MB |
47-
| Nodes | 290,000 | 290,000 |
48-
| Edges | 455,000 | 455,000 |
47+
| Build time | 110.0s | 320.0s |
48+
| DB size | 209.7 MB | 209.7 MB |
49+
| Nodes | 295,000 | 295,000 |
50+
| Edges | 480,000 | 480,000 |
4951

5052
<!-- BENCHMARK_DATA
5153
[
54+
{
55+
"version": "2.4.0",
56+
"date": "2026-02-26",
57+
"files": 109,
58+
"wasm": {
59+
"buildTimeMs": 700,
60+
"queryTimeMs": 2.1,
61+
"nodes": 640,
62+
"edges": 1049,
63+
"dbSizeBytes": 479232,
64+
"perFile": {
65+
"buildTimeMs": 6.4,
66+
"nodes": 5.9,
67+
"edges": 9.6,
68+
"dbSizeBytes": 4397
69+
}
70+
},
71+
"native": {
72+
"buildTimeMs": 235,
73+
"queryTimeMs": 1.6,
74+
"nodes": 640,
75+
"edges": 1049,
76+
"dbSizeBytes": 479232,
77+
"perFile": {
78+
"buildTimeMs": 2.2,
79+
"nodes": 5.9,
80+
"edges": 9.6,
81+
"dbSizeBytes": 4397
82+
}
83+
}
84+
},
5285
{
5386
"version": "2.3.0",
5487
"date": "2026-02-24",

scripts/update-benchmark-report.js

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -214,10 +214,10 @@ ${rows}
214214
Metrics are normalized per file for cross-version comparability. Times above are for a full initial build — incremental rebuilds only re-parse changed files.
215215
`;
216216

217-
// Match the performance section from header to next h2 (## ) header or end.
218-
// The lookahead must reject h3+ (###) so subsections like "### Lightweight
219-
// Footprint" are preserved and not swallowed by the replacement.
220-
const perfRegex = /## 📊 Performance\r?\n[\s\S]*?(?=\r?\n## (?!#)|$)/;
217+
// Match the performance section from header to next h2/h3 header or end.
218+
// The lookahead stops at ## (h2) or ### (h3) so subsections like
219+
// "### Lightweight Footprint" are preserved and not swallowed.
220+
const perfRegex = /## 📊 Performance\r?\n[\s\S]*?(?=\r?\n#{2,3} |$)/;
221221
if (perfRegex.test(readme)) {
222222
readme = readme.replace(perfRegex, perfSection);
223223
} else {

0 commit comments

Comments
 (0)