Add wide-schema benchmark suite for measuring per-file metadata overhead

Adds a new sql_benchmarks suite that isolates the wide-schema scan
overhead in selective parquet queries: the regime where most of the
work is loading footers / column-chunk metadata rather than reading
row data, and that cost scales with the number of column chunks in
the dataset rather than with the number of columns the query touches.

The wide_schema suite has two subgroups (selected via BENCH_SUBGROUP):

  - wide:   1024 cols x 256 files x 50k rows (~225 MB) — the workload
  - narrow:    8 cols x 256 files x 50k rows (~110 MB) — internal
              baseline, only meaningful as a comparison point

Both share row count, file count, and per-file row-group structure;
only schema width differs. Most queries (Q01/Q03/Q10/Q13) run against
both subgroups; Q02/Q08/Q11/Q12 reference suffix-renamed columns that
only exist in the wide dataset and hardcode subgroup=wide.

A new gen_wide_data binary synthesizes both datasets in ~60 s with no
external data source. The 8-column base schema (id, value, count, ts,
category, flag, status, text) carries deterministic data; copies 2..N
from the suffix-renamed widening are zero-filled (zero rather than
null since reader-side null-array shortcuts mute the slowdown by
~35 %).

Headline query Q13 isolates the overhead clearly: filter on two
low-cardinality string columns plus a non-stat-prunable modulo
predicate for ~0.005 % selectivity, project two columns, no LIMIT or
ORDER BY. Cold-start datafusion-cli shows ~15x slowdown wide vs
narrow; EXPLAIN ANALYZE shows metadata_load_time scaling 141x while
bloom_filter_eval_time and statistics_eval_time stay flat.

bench.sh adds:
  - data wide_schema:  synthesizes both wide and narrow datasets
  - run wide_schema:   runs the wide subgroup, then the narrow
                       baseline subgroup, for query-by-query comparison

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: adriangb

benchmarks/README.md

@@ -620,6 +620,65 @@ This benchmarks is derived from the [TPC-H][1] version
 [2]: https://github.com/databricks/tpch-dbgen.git,
 [2.17.1]: https://www.tpc.org/tpc_documents_current_versions/pdf/tpc-h_v2.17.1.pdf
 
+## Wide-schema benchmark
+
+`wide_schema` measures the per-file metadata overhead of a wide schema
+in selective parquet scans — the regime where most of the work is
+loading footers / column-chunk metadata rather than reading row data,
+and that cost scales linearly with the number of column chunks in the
+dataset rather than with the number of columns the query references.
+
+The suite has two subgroups, selected via `BENCH_SUBGROUP`:
+
+- **`wide`** — runs against a 1024-col synthetic dataset. This is the
+  actual workload.
+- **`narrow`** — runs the same SQL against an 8-col version of the same
+  dataset (same row count, file count, per-file row-group shape).
+  This subgroup exists **only as a baseline for the wide subgroup** —
+  reading its numbers in isolation tells you very little. The
+  per-query wide-vs-narrow ratio is what isolates the schema-width
+  cost.
+
+A few queries (Q02, Q08, Q11, Q12) reference suffix-renamed columns
+that only exist in the wide dataset, so they hardcode
+`subgroup wide` and are skipped on the narrow pass.
+
+The data preparation step (`gen_wide_data`) synthesizes a generic
+8-column base schema (`id`, `value`, `count`, `ts`, `category`,
+`flag`, `status`, `text`) with deterministic data, then replicates it
+128× via suffix renaming (`_2`, `_3`, …) for 1024 columns total —
+written across 256 files at 50 k rows per file with one row group per
+file and ZSTD(1) compression. Copies 2..128 are zero-filled arrays so
+the schema is wide (every column still has its own footer / page
+index / column-chunk metadata) but the on-disk size stays around
+225 MB. The narrow dataset is written the same way without the suffix
+copies. The only variable between wide and narrow is schema width.
+
+```shell
+./benchmarks/bench.sh data wide_schema    # synthesizes wide (1024 cols × 256 files) + narrow (8 cols × 256 files), ~60 s, ~335 MB
+./benchmarks/bench.sh run  wide_schema    # runs both 'wide' and 'narrow' subgroups; compare the per-query times for the slowdown ratio
+```
+
+The queries are deliberately small-projection (touch ≤ 4 columns) so
+the wide-schema overhead is the dominant signal. Coverage:
+
+- `Q01` — filter + project + `ORDER BY` + `LIMIT` (TopK shortcut)
+- `Q02` — same shape as Q01 but filtering on suffix-renamed columns
+  far from the start of the schema (wide-only)
+- `Q03` — project 1 column with a tight filter and `LIMIT 1`
+- `Q08` — `GROUP BY` a suffix-renamed column (wide-only)
+- `Q10` — tight filter + small projection, no sort
+- `Q11` — `ORDER BY` a suffix-renamed column with `LIMIT` (wide-only)
+- `Q12` — same as Q03 with an `expect_plan` regression guard for
+  projection pushdown (wide-only)
+- `Q13` — two low-cardinality string filters + a non-stat-prunable
+  modulo predicate for tight selectivity, project two columns, no
+  `LIMIT` or `ORDER BY`
+
+For cold-start measurements that include planner setup (the regime
+where this overhead is most visible), invoke `datafusion-cli`
+directly against `data/wide_schema/{wide,narrow}/`.
+
 ## TPCDS
 
 Run the tpcds benchmark.

benchmarks/bench.sh

@@ -100,6 +100,8 @@ sort_tpch:              Benchmark of sorting speed for end-to-end sort queries o
 sort_tpch10:            Benchmark of sorting speed for end-to-end sort queries on TPC-H dataset (SF=10)
 topk_tpch:              Benchmark of top-k (sorting with limit) queries on TPC-H dataset (SF=1)
 external_aggr:          External aggregation benchmark on TPC-H dataset (SF=1)
+wide_schema:            Small-projection queries on a wide synthetic dataset (1024 cols × 256 files) — measures per-file metadata overhead
+                          (runs both 'wide' and 'narrow' subgroups: narrow is an internal baseline; the wide-vs-narrow ratio is the signal)
 
 # ClickBench Benchmarks
 clickbench_1:           ClickBench queries against a single parquet file
@@ -239,6 +241,9 @@ main() {
                 tpch_csv10)
                     data_tpch "10" "csv"
                     ;;
+                wide_schema)
+                    data_wide_schema
+                    ;;
                 tpcds)
                     data_tpcds
                     ;;
@@ -444,6 +449,9 @@ main() {
                 tpch_mem10)
                     run_tpch_mem "10"
                     ;;
+                wide_schema)
+                    run_wide_schema
+                    ;;
                 tpcds)
                     run_tpcds
                     ;;
@@ -698,6 +706,68 @@ run_tpch() {
       bash -c "$SQL_CARGO_COMMAND"
 }
 
+# Synthesizes two parquet datasets used to measure per-file metadata
+# overhead of a wide schema:
+#
+#   - data/wide_schema/wide/    1024-col events × 256 files (~225 MB)
+#   - data/wide_schema/narrow/    8-col events × 256 files (~110 MB)
+#
+# Both share row count, file count, and per-file row-group shape; only
+# schema width differs. No external data source required — gen_wide_data
+# synthesizes everything from scratch in ~60 s.
+data_wide_schema() {
+    NUM_FILES=256
+    ROWS_PER_FILE=50000
+    WIDTH_FACTOR=128
+
+    DST_DIR="${DATA_DIR}/wide_schema"
+    WIDE_DIR="${DST_DIR}/wide"
+    NARROW_DIR="${DST_DIR}/narrow"
+
+    if [ -d "${WIDE_DIR}" ] && [ "$(ls -A "${WIDE_DIR}" 2>/dev/null | wc -l)" -ge ${NUM_FILES} ]; then
+        echo " wide parquet exists (${WIDE_DIR})."
+    else
+        mkdir -p "${WIDE_DIR}"
+        echo " synthesizing wide -> ${WIDE_DIR} (factor ${WIDTH_FACTOR}, ${NUM_FILES} files × ${ROWS_PER_FILE} rows) ..."
+        debug_run $CARGO_COMMAND --bin gen_wide_data -- \
+            --dst-dir "${WIDE_DIR}" \
+            --width-factor ${WIDTH_FACTOR} \
+            --num-files ${NUM_FILES} \
+            --rows-per-file ${ROWS_PER_FILE}
+    fi
+
+    if [ -d "${NARROW_DIR}" ] && [ "$(ls -A "${NARROW_DIR}" 2>/dev/null | wc -l)" -ge ${NUM_FILES} ]; then
+        echo " narrow parquet exists (${NARROW_DIR})."
+    else
+        mkdir -p "${NARROW_DIR}"
+        echo " synthesizing narrow -> ${NARROW_DIR} (8 base cols, ${NUM_FILES} files × ${ROWS_PER_FILE} rows) ..."
+        debug_run $CARGO_COMMAND --bin gen_wide_data -- \
+            --dst-dir "${NARROW_DIR}" \
+            --width-factor 1 \
+            --num-files ${NUM_FILES} \
+            --rows-per-file ${ROWS_PER_FILE}
+    fi
+}
+
+# Runs the wide_schema benchmark. Each query has a `subgroup`
+# directive that picks up BENCH_SUBGROUP, so we invoke the framework
+# twice — once with subgroup=wide (the actual workload) and once with
+# subgroup=narrow (the baseline). The wide-only queries (Q02/Q08/Q11/Q12)
+# hardcode `subgroup wide`, so they're skipped on the narrow pass.
+run_wide_schema() {
+    echo "Running wide_schema benchmark (wide subgroup)..."
+    debug_run env BENCH_NAME=wide_schema BENCH_SUBGROUP=wide \
+      SIMULATE_LATENCY="${SIMULATE_LATENCY}" \
+      ${QUERY:+BENCH_QUERY="${QUERY}"}  \
+      bash -c "$SQL_CARGO_COMMAND"
+
+    echo "Running wide_schema benchmark (narrow baseline subgroup)..."
+    debug_run env BENCH_NAME=wide_schema BENCH_SUBGROUP=narrow \
+      SIMULATE_LATENCY="${SIMULATE_LATENCY}" \
+      ${QUERY:+BENCH_QUERY="${QUERY}"}  \
+      bash -c "$SQL_CARGO_COMMAND"
+}
+
 # Runs the tpch in memory (needs tpch parquet data)
 run_tpch_mem() {
     SCALE_FACTOR=$1

benchmarks/sql_benchmarks/README.md

@@ -41,6 +41,7 @@ in the community:
 | `taxi`                | NYC taxi dataset benchmark                                         |
 | `tpcds`               | TPC‑DS queries                                                     |
 | `tpch`                | TPC‑H queries                                                      |
+| `wide_schema`         | Small-projection queries on a wide (1024-col, 256-file) synthetic dataset; runs `wide` + `narrow` subgroups for comparison |
 
 # Running Benchmarks
 

benchmarks/sql_benchmarks/wide_schema/benchmarks/q01.benchmark

@@ -0,0 +1,25 @@
+-- Filter on three low-cardinality columns, project four columns,
+-- ORDER BY + LIMIT (TopK shortcut). Runs on both wide and narrow
+-- datasets via BENCH_SUBGROUP.
+
+name Q01
+group wide_schema
+subgroup ${BENCH_SUBGROUP:-wide}
+
+load sql_benchmarks/wide_schema/init/load.sql
+
+assert I
+SELECT COUNT(*) > 0 from events;
+----
+true
+
+run
+SELECT id, ts, value, text
+FROM events
+WHERE category = 'c0'
+  AND flag     = 'f0'
+  AND status   = 's0'
+ORDER BY ts DESC
+LIMIT 100;
+
+cleanup sql_benchmarks/wide_schema/init/cleanup.sql

benchmarks/sql_benchmarks/wide_schema/benchmarks/q02.benchmark

@@ -0,0 +1,26 @@
+-- Same shape as Q01 but filtering on duplicated columns far from the
+-- start of the schema (suffix copies _10). Tests whether planning /
+-- pruning cost depends on column position. Wide-only — the narrow
+-- dataset doesn't have suffix-renamed copies.
+
+name Q02
+group wide_schema
+subgroup wide
+
+load sql_benchmarks/wide_schema/init/load.sql
+
+assert I
+SELECT COUNT(*) > 0 from events;
+----
+true
+
+run
+SELECT id, ts, value, text
+FROM events
+WHERE category_10 = 'c0'
+  AND flag_10     = 'f0'
+  AND status_10   = 's0'
+ORDER BY ts DESC
+LIMIT 100;
+
+cleanup sql_benchmarks/wide_schema/init/cleanup.sql

benchmarks/sql_benchmarks/wide_schema/benchmarks/q03.benchmark

@@ -0,0 +1,22 @@
+-- Project 1 column with a very tight filter. Stresses minimum-
+-- projection pushdown over a wide schema. Runs on both wide and
+-- narrow datasets via BENCH_SUBGROUP.
+
+name Q03
+group wide_schema
+subgroup ${BENCH_SUBGROUP:-wide}
+
+load sql_benchmarks/wide_schema/init/load.sql
+
+assert I
+SELECT COUNT(*) > 0 from events;
+----
+true
+
+run
+SELECT value
+FROM events
+WHERE id = 12345
+LIMIT 1;
+
+cleanup sql_benchmarks/wide_schema/init/cleanup.sql

benchmarks/sql_benchmarks/wide_schema/benchmarks/q08.benchmark

@@ -0,0 +1,23 @@
+-- GROUP BY a duplicated column, single aggregate. Checks aggregator
+-- behaviour when the grouping column comes from late in a wide
+-- schema. Wide-only — the narrow dataset doesn't have suffix copies.
+
+name Q08
+group wide_schema
+subgroup wide
+
+load sql_benchmarks/wide_schema/init/load.sql
+
+assert I
+SELECT COUNT(*) > 0 from events;
+----
+true
+
+run
+SELECT category_5, COUNT(*) AS c
+FROM events
+GROUP BY category_5
+ORDER BY c DESC
+LIMIT 10;
+
+cleanup sql_benchmarks/wide_schema/init/cleanup.sql

benchmarks/sql_benchmarks/wide_schema/benchmarks/q10.benchmark

@@ -0,0 +1,23 @@
+-- Tight filter + small projection without ORDER BY / LIMIT — measures
+-- pure filter+project throughput against a wide schema (no TopK
+-- shortcut). The filter is tight so the result set stays small.
+-- Runs on both wide and narrow datasets via BENCH_SUBGROUP.
+
+name Q10
+group wide_schema
+subgroup ${BENCH_SUBGROUP:-wide}
+
+load sql_benchmarks/wide_schema/init/load.sql
+
+assert I
+SELECT COUNT(*) > 0 from events;
+----
+true
+
+run
+SELECT id, ts, value, text
+FROM events
+WHERE id = 12345
+  AND category = 'c0';
+
+cleanup sql_benchmarks/wide_schema/init/cleanup.sql

benchmarks/sql_benchmarks/wide_schema/benchmarks/q11.benchmark

@@ -0,0 +1,22 @@
+-- TopK on a duplicated sort column with LIMIT — tests sort/TopK
+-- physical operator over a wide schema. Wide-only — sorts on a
+-- suffix copy that doesn't exist in the narrow dataset.
+
+name Q11
+group wide_schema
+subgroup wide
+
+load sql_benchmarks/wide_schema/init/load.sql
+
+assert I
+SELECT COUNT(*) > 0 from events;
+----
+true
+
+run
+SELECT id, ts
+FROM events
+ORDER BY ts_8 DESC
+LIMIT 100;
+
+cleanup sql_benchmarks/wide_schema/init/cleanup.sql

benchmarks/sql_benchmarks/wide_schema/benchmarks/q12.benchmark

@@ -0,0 +1,25 @@
+-- Same as Q03 but with `expect_plan` — guards against regressions
+-- where projection pushdown stops trimming the parquet read down to
+-- a single column. Wide-only since the projection-pushdown signal
+-- only matters when there are many columns to prune.
+
+name Q12
+group wide_schema
+subgroup wide
+
+load sql_benchmarks/wide_schema/init/load.sql
+
+assert I
+SELECT COUNT(*) > 0 from events;
+----
+true
+
+expect_plan projection=[value]
+
+run
+SELECT value
+FROM events
+WHERE id = 12345
+LIMIT 1;
+
+cleanup sql_benchmarks/wide_schema/init/cleanup.sql