Skip to content

Commit abfa5f2

Browse files
adriangbclaude
andauthored
refactor(parquet-datasource): split bloom_filter out of row_group_filter.rs (apache#22348)
## Which issue does this PR close? Relates to the discussion in apache#22024 about the Parquet datasource crate becoming hard to navigate. Split out of apache#22156, which bundled several code-motion moves into one PR — this is one of three smaller, independently-reviewable PRs that replace it. ## Rationale for this change `row_group_filter.rs` had grown to ~1,900 LOC. It mixes "data we loaded from the file" with "the access-plan filter that consumes it." This PR is **code motion only**: no behavior change and no public API change. ## What changes are included in this PR? Extracts `BloomFilterStatistics` — the loaded Split Block Bloom Filter (SBBF) data plus its `PruningStatistics` adapter — from `row_group_filter.rs` into a new `bloom_filter.rs`, and moves the bloom-filter tests alongside it. This separates `BloomFilterStatistics` (data loaded from the file) from `RowGroupAccessPlanFilter` (the access-plan filter that consumes it), leaving `row_group_filter.rs` focused on the latter. Each commit builds green on its own: 1. **Split `bloom_filter` out of `row_group_filter.rs`** — move `BloomFilterStatistics` and its `PruningStatistics` adapter into a new `bloom_filter.rs`. 2. **Build `BloomFilterStatistics` via its constructors in tests** — the test built it with a struct literal; use the existing `with_capacity`/`insert` constructors (the pattern `opener.rs` already uses) so the `column_sbbf` field stays private. 3. **Extract `ExpectedPruning` into a shared `test_util` module** — the one test helper shared between the row-group and bloom-filter tests, so the bloom-filter tests can move out. Adds a `#[cfg(test)] pub(crate) fn access_plan()` accessor on `RowGroupAccessPlanFilter` so the helper can assert from a sibling module without widening field visibility. 4. **Move the bloom-filter tests into `bloom_filter.rs`** — relocate `test_row_group_bloom_*`, the `BloomFilterTest` builder, and its helper next to the code they exercise. `BloomFilterStatistics` is crate-internal; `row_group_filter` re-exports it (`pub(crate) use`) so the existing `crate::row_group_filter::BloomFilterStatistics` path keeps resolving for in-crate callers. Aside from `row_group_filter.rs` and `bloom_filter.rs`, this adds a new `src/test_util.rs` and a one-line module declaration in `mod.rs`. ## Are these changes tested? Yes, covered by existing tests. `cargo test -p datafusion-datasource-parquet --all-features` (122 passing) and `cargo clippy -p datafusion-datasource-parquet --all-targets --all-features -- -D warnings` both pass. ## Are there any user-facing changes? No. `BloomFilterStatistics` is crate-internal; this only reorganizes files inside the crate. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 557a4f0 commit abfa5f2

4 files changed

Lines changed: 650 additions & 561 deletions

File tree

0 commit comments

Comments
 (0)