chore(release): add 0.17.1 benchmark snapshot

[benvinegar]

Summary

• Add the committed 0.17.1 release benchmark snapshot required by the tag-triggered publish workflow.
• Confirm the release benchmark gate passes against 0.17.0 with no accepted regressions.
• Record improved large-review results, including an 18% lower median hunk-navigation latency and a 14% faster windowed scroll benchmark.

Verification

• bun run bench:release
• bun run bench:release:compare -- --version 0.17.1 --out dist/release/benchmark-comparison.json
• bunx oxfmt --check benchmarks/release/bench-0.17.1.json

This PR description was generated by Pi using OpenAI GPT-5.2

[greptile-apps]

Greptile Summary

Adds the committed bench-0.17.1.json release benchmark snapshot required for the tag-triggered publish workflow. All 1430-line JSON file's derived statistics (median, p75, p95, min, max) check out correctly against the raw samples arrays for every entry inspected, and gitSha points to the correct base commit (35934407).

• The snapshot's gitSha matches the base commit of the 0.17.x branch, which is the expected artifact for a patch-release publish gate.
• Two interaction-latency benchmarks exhibit notably high within-run variance: first_frame_ms has one sample at 3.98 ms against four others at 18–22 ms, and hunk_nav_press_p95_ms splits into two distinct clusters (~87–91 ms vs ~129–131 ms). These don't affect correctness of the recorded statistics but may reduce the reliability of this snapshot as a regression baseline.

Confidence Score: 4/5

Safe to merge; the snapshot is structurally correct and the publish gate will use the accurately computed medians.

The JSON is well-formed and all derived statistics are correct. Two interaction-latency benchmarks show unusually high within-run variance — one with a 5× outlier sample and one with a bimodal spread — which slightly weakens this snapshot's value as a future regression baseline, but neither affects correctness of the recorded data.

The variance in interaction-latency/first_frame_ms and interaction-latency/hunk_nav_press_p95_ms samples is worth a second look before treating this snapshot as a long-lived regression anchor.

Important Files Changed

Filename	Overview
benchmarks/release/bench-0.17.1.json	New benchmark snapshot for 0.17.1; all derived statistics are correctly computed, but two benchmarks show notably high within-run variance that could affect future regression comparisons.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Tag push: v0.17.1] --> B[Publish workflow triggered]
    B --> C{Release benchmark\ngate check}
    C --> D[Load bench-0.17.1.json\ngitSha: 35934407]
    C --> E[Load bench-0.17.0.json\nfor comparison]
    D --> F{Compare medians\nagainst thresholds}
    E --> F
    F -->|All within maxRegressionRatio\nand minAbsoluteRegression| G[Gate PASSES]
    F -->|Any metric exceeds threshold| H[Gate FAILS — block publish]
    G --> I[npm publish hunk@0.17.1]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[Tag push: v0.17.1] --> B[Publish workflow triggered]
    B --> C{Release benchmark\ngate check}
    C --> D[Load bench-0.17.1.json\ngitSha: 35934407]
    C --> E[Load bench-0.17.0.json\nfor comparison]
    D --> F{Compare medians\nagainst thresholds}
    E --> F
    F -->|All within maxRegressionRatio\nand minAbsoluteRegression| G[Gate PASSES]
    F -->|Any metric exceeds threshold| H[Gate FAILS — block publish]
    G --> I[npm publish hunk@0.17.1]

Loading

Prompt To Fix All With AI

Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
benchmarks/release/bench-0.17.1.json:552-569
**High within-run variance in `first_frame_ms`**

One sample (3.98 ms) is ~5× lower than the remaining four (18–22 ms), producing a spread of 18 ms across just 5 runs. The median is computed correctly as 21.03 ms and the regression gate only compares medians, so this doesn't cause an incorrect gate result today. However, with `minAbsoluteRegression: 5`, a future version would need to push the median above ~26 ms before the ratio gate even activates — and if any run catches the same cold-path luck the outlier did, a rerun could produce a very different median. Worth checking whether the benchmark harness warms the runtime consistently before sampling.

### Issue 2 of 2
benchmarks/release/bench-0.17.1.json:582-601
**Bimodal sample distribution in `hunk_nav_press_p95_ms`**

The five samples split cleanly into two clusters: 86.98–90.81 ms (runs 3–5) and 129.25–131.54 ms (runs 1–2), a ~1.45× spread. The median (90.81 ms) sits in the lower cluster, so the effective regression threshold becomes ~104 ms. A future version that consistently lands at ~115 ms would correctly trip the gate — but one that happens to produce more "high" runs in a single bench pass could produce a median above the threshold even without a true regression, since the upper cluster is already at 129–131 ms. This snapshot as a baseline is less stable than typical.

_{Reviews (1): Last reviewed commit: "chore(release): add 0.17.1 benchmark sna..." | Re-trigger Greptile}

[greptile-apps]

High within-run variance in first_frame_ms

One sample (3.98 ms) is ~5× lower than the remaining four (18–22 ms), producing a spread of 18 ms across just 5 runs. The median is computed correctly as 21.03 ms and the regression gate only compares medians, so this doesn't cause an incorrect gate result today. However, with minAbsoluteRegression: 5, a future version would need to push the median above ~26 ms before the ratio gate even activates — and if any run catches the same cold-path luck the outlier did, a rerun could produce a very different median. Worth checking whether the benchmark harness warms the runtime consistently before sampling.

Prompt To Fix With AI

This is a comment left during a code review.
Path: benchmarks/release/bench-0.17.1.json
Line: 552-569

Comment:
**High within-run variance in `first_frame_ms`**

One sample (3.98 ms) is ~5× lower than the remaining four (18–22 ms), producing a spread of 18 ms across just 5 runs. The median is computed correctly as 21.03 ms and the regression gate only compares medians, so this doesn't cause an incorrect gate result today. However, with `minAbsoluteRegression: 5`, a future version would need to push the median above ~26 ms before the ratio gate even activates — and if any run catches the same cold-path luck the outlier did, a rerun could produce a very different median. Worth checking whether the benchmark harness warms the runtime consistently before sampling.

How can I resolve this? If you propose a fix, please make it concise.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[greptile-apps]

Bimodal sample distribution in hunk_nav_press_p95_ms

The five samples split cleanly into two clusters: 86.98–90.81 ms (runs 3–5) and 129.25–131.54 ms (runs 1–2), a ~1.45× spread. The median (90.81 ms) sits in the lower cluster, so the effective regression threshold becomes ~104 ms. A future version that consistently lands at ~115 ms would correctly trip the gate — but one that happens to produce more "high" runs in a single bench pass could produce a median above the threshold even without a true regression, since the upper cluster is already at 129–131 ms. This snapshot as a baseline is less stable than typical.

Prompt To Fix With AI

This is a comment left during a code review.
Path: benchmarks/release/bench-0.17.1.json
Line: 582-601

Comment:
**Bimodal sample distribution in `hunk_nav_press_p95_ms`**

The five samples split cleanly into two clusters: 86.98–90.81 ms (runs 3–5) and 129.25–131.54 ms (runs 1–2), a ~1.45× spread. The median (90.81 ms) sits in the lower cluster, so the effective regression threshold becomes ~104 ms. A future version that consistently lands at ~115 ms would correctly trip the gate — but one that happens to produce more "high" runs in a single bench pass could produce a median above the threshold even without a true regression, since the upper cluster is already at 129–131 ms. This snapshot as a baseline is less stable than typical.

How can I resolve this? If you propose a fix, please make it concise.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!