Skip to content

Commit 5b688a9

Browse files
connortsui20claude
andauthored
Fix remaining flaky CodSpeed microbenchmarks (#8861)
## Rationale for this change - Closes: #8807 Three benchmarks stayed flaky after #8742, flipping between the same two values on PRs that can't affect them. Same root causes and fixes as #8742: | Benchmark | Seen flaky on | Why | Fix | | --- | --- | --- | --- | | `true_count_vortex_buffer[128]` | ±11.17% on 9 unrelated PRs (#8805, #8811, #8812, #8820, #8843, #8803, …) | a 128-bit popcount measures harness overhead and code layout, not the count | drop the 128 size | | runend `compress[(100000, 4)]` | ±11.9% on #8805, #8750, #8856 | allocates in the timed region; glibc malloc differs across runner images | mimalloc as global allocator | | `cast_decimal` `copy_*[65536]` | identical flags on #8838 and #8724 | same glibc-malloc cause (512 KB alloc per iteration) | mimalloc as global allocator | Left alone: `compact_sliced[(4096, 90)]` (single sighting) and the CUDA walltime benches (hosted-runner walltime noise, a runner config issue). The allocator swap shifts every benchmark in the two touched binaries once — see the comment below. Needs a one-time CodSpeed acknowledgment, like #8742. ## What changes are included in this PR? One commit per benchmark; bench files only. Ran `cargo check` + `clippy` on the three bench targets, smoke-ran the binaries, `cargo +nightly fmt`. --------- Signed-off-by: Claude <noreply@anthropic.com> Co-authored-by: Claude <noreply@anthropic.com>
1 parent add6944 commit 5b688a9

5 files changed

Lines changed: 23 additions & 2 deletions

File tree

Cargo.lock

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

encodings/runend/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ workspace = true
3030
[dev-dependencies]
3131
divan = { workspace = true }
3232
itertools = { workspace = true }
33+
mimalloc = { workspace = true }
3334
rand = { workspace = true }
3435
rstest = { workspace = true }
3536
vortex-array = { workspace = true, features = ["_test-harness"] }

encodings/runend/benches/run_end_compress.rs

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ use std::sync::LazyLock;
77

88
use divan::Bencher;
99
use itertools::repeat_n;
10+
use mimalloc::MiMalloc;
1011
use vortex_array::IntoArray;
1112
use vortex_array::RecursiveCanonical;
1213
use vortex_array::VortexSessionExecute;
@@ -19,6 +20,12 @@ use vortex_runend::RunEnd;
1920
use vortex_runend::compress::runend_encode;
2021
use vortex_session::VortexSession;
2122

23+
// `runend_encode` allocates its output buffers inside the timed region, so route allocation
24+
// through vendored mimalloc to keep glibc malloc (which varies across runner images) out of
25+
// the measured trace.
26+
#[global_allocator]
27+
static GLOBAL: MiMalloc = MiMalloc;
28+
2229
fn main() {
2330
divan::main();
2431
}

vortex-array/benches/cast_decimal.rs

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
use std::sync::LazyLock;
1515

1616
use divan::Bencher;
17+
use mimalloc::MiMalloc;
1718
use rand::prelude::*;
1819
use vortex_array::ArrayRef;
1920
use vortex_array::Canonical;
@@ -28,6 +29,12 @@ use vortex_array::validity::Validity;
2829
use vortex_buffer::BufferMut;
2930
use vortex_session::VortexSession;
3031

32+
// The copy casts allocate their output buffer inside the timed region, so route allocation
33+
// through vendored mimalloc to keep glibc malloc (which varies across runner images) out of
34+
// the measured trace.
35+
#[global_allocator]
36+
static GLOBAL: MiMalloc = MiMalloc;
37+
3138
fn main() {
3239
divan::main();
3340
}

vortex-buffer/benches/vortex_bitbuffer.rs

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -176,7 +176,12 @@ fn slice_arrow_buffer(bencher: Bencher, length: usize) {
176176
});
177177
}
178178

179-
#[divan::bench(args = INPUT_SIZE)]
179+
/// A 128-bit `true_count` is a popcount over two `u64` words, so the measurement is fixed
180+
/// dispatch and harness overhead plus binary code layout rather than the count itself. Only
181+
/// sizes where the popcount loop dominates are worth measuring.
182+
const TRUE_COUNT_INPUT_SIZE: &[usize] = &[1024, 2048, 16_384, 65_536];
183+
184+
#[divan::bench(args = TRUE_COUNT_INPUT_SIZE)]
180185
fn true_count_vortex_buffer(bencher: Bencher, length: usize) {
181186
let buffer = BitBuffer::from_iter((0..length).map(true_count_pattern));
182187

@@ -185,7 +190,7 @@ fn true_count_vortex_buffer(bencher: Bencher, length: usize) {
185190
.bench_refs(|buffer| buffer.true_count())
186191
}
187192

188-
#[divan::bench(args = INPUT_SIZE)]
193+
#[divan::bench(args = TRUE_COUNT_INPUT_SIZE)]
189194
fn true_count_arrow_buffer(bencher: Bencher, length: usize) {
190195
let buffer = Arrow(BooleanBuffer::from_iter(
191196
(0..length).map(true_count_pattern),

0 commit comments

Comments
 (0)