Skip to content

Commit 08d14a3

Browse files
Floor perf-test timing denominator to deflake CI (#1073)
test_generate_does_not_scale_linearly flaked twice this week (5.3x on windows-3.10, 5.9x on macos-3.10) with both absolute times under a quarter second, where scheduler noise dominates the ratio. Floor the denominator at 100ms; a real pathological regression still trips the assertion. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 73d85de commit 08d14a3

2 files changed

Lines changed: 8 additions & 1 deletion

File tree

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Floor the generate-phase performance test's timing denominator at 100ms so scheduler noise on fast CI runners no longer flakes the 5x scaling assertion.

tests/test_performance.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -183,7 +183,13 @@ def test_generate_does_not_scale_linearly(self):
183183
times[n] = time.time() - t0
184184
dataset.cleanup()
185185

186-
ratio = times[500] / max(times[100], 0.01)
186+
# Floor the denominator at 100ms: when the 100-record base runs in a
187+
# few tens of milliseconds, scheduler noise dominates the ratio and
188+
# the test flakes on CI (observed 5.3x/5.9x on runs where both
189+
# absolute times were well under a quarter second). A genuinely
190+
# pathological (e.g. quadratic row-loop) regression still trips the
191+
# assertion because times[500] grows into whole seconds.
192+
ratio = times[500] / max(times[100], 0.1)
187193
assert ratio < 5.0, (
188194
f"Generate phase scaled {ratio:.1f}x for 5x more records "
189195
f"(100: {times[100]:.2f}s, 500: {times[500]:.2f}s). "

0 commit comments

Comments
 (0)