Skip to content

Commit 1ee87f7

Browse files
author
B Vadlamani
committed
address docu review comments
1 parent e483e59 commit 1ee87f7

2 files changed

Lines changed: 36 additions & 8 deletions

File tree

benchmarks/src/hj.rs

Lines changed: 22 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -302,8 +302,17 @@ const HASH_QUERIES: &[HashJoinQuery] = &[
302302
probe_size: "60M",
303303
},
304304
// RightSemi Join benchmarks with Int32 keys
305-
// Matching ratio: 1:1 (build keys are unique, each probe matches at most 1 build row)
306-
// Semi joins can short-circuit after first match, so hit rate affects performance.
305+
//
306+
// Fanout (average build rows matched per probe row, as measured by running
307+
// the equivalent INNER JOIN under `EXPLAIN ANALYZE` and reading the
308+
// `HashJoinExec` metrics): 1 for Q16-Q18. Build keys here are primary
309+
// keys (`n_nationkey`, `s_suppkey`), so each probe row matches at most
310+
// one build row. `prob_hit` controls what fraction of probe rows find
311+
// that one match.
312+
//
313+
// Fanout still matters because semi joins short-circuit after the first
314+
// match. Coverage of fanout > 1 (build-side duplicates) is left for a
315+
// follow-up.
307316
//
308317
// Q16: RightSemi, Small build (25 rows), 100% Hit rate
309318
// Build Side: nation (25 rows) | Probe Side: customer (1.5M rows)
@@ -345,8 +354,17 @@ const HASH_QUERIES: &[HashJoinQuery] = &[
345354
probe_size: "60M_RightSemi",
346355
},
347356
// RightAnti Join benchmarks with Int32 keys
348-
// Matching ratio: 1:1 (build keys are unique, each probe matches at most 1 build row)
349-
// Anti joins can short-circuit after first match, so hit rate affects performance.
357+
//
358+
// Fanout (average build rows matched per probe row, as measured by running
359+
// the equivalent INNER JOIN under `EXPLAIN ANALYZE` and reading the
360+
// `HashJoinExec` metrics): 1 for Q19-Q21. Build keys here are primary
361+
// keys (`n_nationkey`, `s_suppkey`), so each probe row matches at most
362+
// one build row. `prob_hit` controls what fraction of probe rows find
363+
// that one match (and are therefore filtered *out* by anti).
364+
//
365+
// Fanout still matters because anti joins short-circuit after the first
366+
// match. Coverage of fanout > 1 (build-side duplicates) is left for a
367+
// follow-up.
350368
//
351369
// Q19: RightAnti, Small build (25 rows), 100% Hit rate (no output)
352370
// Build Side: nation (25 rows) | Probe Side: customer (1.5M rows)

datafusion/physical-plan/benches/hash_join_semi_anti.rs

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,10 +19,20 @@
1919
//!
2020
//! ## Key Benchmark Axes
2121
//!
22-
//! - **Density**: The ratio of distinct key values to the key range span.
23-
//! For example, keys [0, 2, 4, 6, 8] have density = 5/9 ≈ 55% (5 distinct values
24-
//! in a range of 9). Density affects hash table memory layout and cache behavior.
25-
//! Lower density means keys are more spread out, leading to more cache misses.
22+
//! - **Density**: How tightly distinct keys pack into their numeric range.
23+
//! `density = num_distinct_keys / (max_key - min_key + 1)`.
24+
//! Examples for 5 distinct keys:
25+
//! - `[0, 1, 2, 3, 4]` → 5/5 = 100% (fully packed)
26+
//! - `[0, 2, 4, 6, 8]` → 5/9 ≈ 55% (every 2nd slot)
27+
//! - `[0, 10, 20, 30, 40]` → 5/41 ≈ 12% (every 10th slot)
28+
//!
29+
//! Why it matters for this workload: future potential semi/anti-join
30+
//! fast paths could exploit densely packed build keys to outperform the
31+
//! general hash-table path, which is largely insensitive to density.
32+
//! Varying density across benchmarks helps surface those potential gains
33+
//! under different key distributions. Density describes only the
34+
//! build-side key layout; the per-probe match count is tracked
35+
//! separately as fanout.
2636
//!
2737
//! - **Hit Rate**: The percentage of probe rows that find a match in the build side.
2838
//! This controls how often the join produces output rows.

0 commit comments

Comments
 (0)