Skip to content

Commit 6b27d2d

Browse files
Rich-T-kidblagininkumarUjjawal
authored
Rich t kid/introduce dict benchmarks (apache#21860)
## Which issue does this PR close? This PR provides the benchmarks mentioned in apache#7647 & apache#9017 <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Works towards closing apache#7647. ## Rationale for this change Currently the benchmark suite doesn't have any dictionary-encoded tables with aggregations performed on them. This makes it difficult to prove performance improvements, for example, a separate PR I'm working on (apache#21765) is hard to validate because the existing benchmarks don't exercise this path. This PR attempts to close that gap. <!-- Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed. Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes. --> ## What changes are included in this PR? Adds a new dict benchmark to dfbench that measures group-by performance on dictionary-encoded columns across varying cardinality (5/10/25%), null rates (0/15%), and value types (Utf8 and List<Utf8>), covering both single and multi-column group-by scenarios. <!-- There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR. --> ## Are these changes tested? -- <!-- We typically require tests for all PRs in order to: 1. Prevent the code from being accidentally broken by subsequent changes 2. Serve as another way to document the expected behavior of the code If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> ## Are there any user-facing changes? no <!-- If there are user-facing changes then we may require documentation to be updated before approving the PR. --> <!-- If there are any breaking changes to public APIs, please add the `api change` label. --> --------- Co-authored-by: Dmitrii Blaginin <dmitrii@blaginin.me> Co-authored-by: Kumar Ujjawal <ujjawalpathak6@gmail.com>
1 parent 52f23ae commit 6b27d2d

4 files changed

Lines changed: 458 additions & 2 deletions

File tree

benchmarks/bench.sh

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,7 @@ cancellation: How long cancelling a query takes
147147
nlj: Benchmark for simple nested loop joins, testing various join scenarios
148148
hj: Benchmark for simple hash joins, testing various join scenarios
149149
smj: Benchmark for simple sort merge joins, testing various join scenarios
150+
dict: Benchmark for dictionary-encoded group-by scenarios
150151
compile_profile: Compile and execute TPC-H across selected Cargo profiles, reporting timing and binary size
151152
152153
@@ -351,6 +352,10 @@ main() {
351352
# smj uses range() function, no data generation needed
352353
echo "SMJ benchmark does not require data generation"
353354
;;
355+
dict)
356+
# dict generates in-memory data, no data generation needed
357+
echo "DICT benchmark does not require data generation"
358+
;;
354359
compile_profile)
355360
data_tpch "1" "parquet"
356361
;;
@@ -430,6 +435,7 @@ main() {
430435
run_hj
431436
run_tpcds
432437
run_smj
438+
run_dict
433439
;;
434440
tpch)
435441
run_tpch "1" "parquet"
@@ -564,6 +570,9 @@ main() {
564570
smj)
565571
run_smj
566572
;;
573+
dict)
574+
run_dict
575+
;;
567576
compile_profile)
568577
run_compile_profile "${PROFILE_ARGS[@]}"
569578
;;
@@ -1460,6 +1469,14 @@ run_smj() {
14601469
debug_run $CARGO_COMMAND --bin dfbench -- smj --iterations 5 -o "${RESULTS_FILE}" ${QUERY_ARG} ${LATENCY_ARG}
14611470
}
14621471

1472+
# Runs the dict benchmark
1473+
run_dict() {
1474+
RESULTS_FILE="${RESULTS_DIR}/dict.json"
1475+
echo "RESULTS_FILE: ${RESULTS_FILE}"
1476+
echo "Running dict benchmark..."
1477+
debug_run $CARGO_COMMAND --bin dfbench -- dict --iterations 5 -o "${RESULTS_FILE}" ${QUERY_ARG} ${LATENCY_ARG}
1478+
}
1479+
14631480

14641481
compare_benchmarks() {
14651482
BASE_RESULTS_DIR="${SCRIPT_DIR}/results"

benchmarks/src/bin/dfbench.rs

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717

1818
//! DataFusion benchmark runner
1919
use datafusion::error::Result;
20+
use datafusion_benchmarks::sort_pushdown;
2021

2122
use clap::{Parser, Subcommand};
2223

@@ -31,8 +32,7 @@ static ALLOC: snmalloc_rs::SnMalloc = snmalloc_rs::SnMalloc;
3132
static ALLOC: mimalloc::MiMalloc = mimalloc::MiMalloc;
3233

3334
use datafusion_benchmarks::{
34-
cancellation, clickbench, h2o, hj, imdb, nlj, smj, sort_pushdown, sort_tpch, tpcds,
35-
tpch,
35+
cancellation, clickbench, dict, h2o, hj, imdb, nlj, smj, sort_tpch, tpcds, tpch,
3636
};
3737

3838
#[derive(Debug, Parser)]
@@ -46,6 +46,7 @@ struct Cli {
4646
enum Options {
4747
Cancellation(cancellation::RunOpt),
4848
Clickbench(clickbench::RunOpt),
49+
Dict(dict::RunOpt),
4950
H2o(h2o::RunOpt),
5051
HJ(hj::RunOpt),
5152
Imdb(imdb::RunOpt),
@@ -66,6 +67,7 @@ pub async fn main() -> Result<()> {
6667
match cli.command {
6768
Options::Cancellation(opt) => opt.run().await,
6869
Options::Clickbench(opt) => opt.run().await,
70+
Options::Dict(opt) => opt.run().await,
6971
Options::H2o(opt) => opt.run().await,
7072
Options::HJ(opt) => opt.run().await,
7173
Options::Imdb(opt) => Box::pin(opt.run()).await,

0 commit comments

Comments
 (0)