Skip to content

Commit 43d00a3

Browse files
committed
feat(ci): annotate compare table cells with ratio (1.5x faster/slower)
Δ% is easier to interpret at scale when paired with a ratio — e.g. `-33.3% (1.50x faster)` for a big prove-time drop; `+15.2% (1.15x slower)` for a real execute-time regression. Only shown when the factor is >= 1.05x (roughly ±5% in either direction) so sub-noise deltas don't get `(1.02x slower)` clutter next to `+1.1%`. Cell format cascades cleanly: `{Δ%} [(1.NN× {slower|faster})] [emoji]`.
1 parent 6e0b5c1 commit 43d00a3

1 file changed

Lines changed: 15 additions & 0 deletions

File tree

.github/scripts/bench.py

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -252,6 +252,15 @@ def _delta(main, pr):
252252
return (pr - main) / main * 100.0
253253

254254

255+
def _ratio(main, pr):
256+
"""(factor, direction) with `factor` always ≥ 1.0. Metrics are lower-
257+
is-better, so `pr > main` reads as slower / larger; `pr < main` as
258+
faster / smaller. Returns None if either side is missing or non-positive."""
259+
if main is None or pr is None or main <= 0 or pr <= 0:
260+
return None
261+
return (pr / main, "slower") if pr >= main else (main / pr, "faster")
262+
263+
255264
def _load(path):
256265
try:
257266
with open(path) as f:
@@ -321,6 +330,12 @@ def _oom(d, n):
321330
dp = _delta(mv, pv)
322331
cell = "n/a" if dp is None else f"{dp:+.1f}%"
323332
if dp is not None:
333+
# Ratio only when the change is big enough that "1.18× slower"
334+
# carries new signal beyond the percentage — sub-5% deltas would
335+
# just add `(1.03× slower)` noise to the cell.
336+
r = _ratio(mv, pv)
337+
if r is not None and r[0] >= 1.05:
338+
cell += f" ({r[0]:.2f}× {r[1]})"
324339
if dp > a.threshold:
325340
cell += " ⚠️"; regressed.add(n)
326341
elif dp < -a.threshold:

0 commit comments

Comments
 (0)