Skip to content

Commit 1a413c7

Browse files
dfa1claude
andcommitted
deprecate(compute): mark Mask and Compute.filter for removal
A materialized selection mask is a slower primitive than a fused single-pass scan — it builds a positional bitmap the reduce must then re-scan (two-pass filter->Mask->reduce measured ~1.5x a fused one-pass; Compute.filter's bitmap build ~6x a vectorizable count). Mask and Compute.filter are now @deprecated(forRemoval) in favour of fused kernels like Compute.filteredSum (and a forthcoming fused multi-column filteredReduce). Nothing is removed: the Calcite boundary fold still uses Mask until it migrates, guarded by narrow @SuppressWarnings("removal") on the transitional internal call sites. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 57d2225 commit 1a413c7

16 files changed

Lines changed: 60 additions & 0 deletions

File tree

CHANGELOG.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Added
11+
12+
- `Compute.filteredSum(filterColumn, predicate, aggColumn)` fuses a filter and a sum into a single scan — a row folds into the total only when the predicate selects it (a null filter row is excluded) and the aggregate value is non-null — with no intermediate selection bitmap. It matches a hand-written fused loop and is ~1.5× faster than the two-pass `filter` + `sum`. ([57d2225b](https://github.com/dfa1/vortex-java/commit/57d2225b))
13+
14+
### Deprecated
15+
16+
- The selection `Mask` and `Compute.filter(Array, Predicate, Arena)` are deprecated for removal: a materialized selection mask is a slower primitive than a fused single-pass scan (it builds a positional bitmap the reduce must re-scan). Prefer the fused `Compute.filteredSum` (and the forthcoming fused multi-column `filteredReduce`), which fold filter and aggregate in one pass with no intermediate bitmap. ([319cfd97](https://github.com/dfa1/vortex-java/commit/319cfd97))
17+
1018
## [0.11.0] — 2026-06-28
1119

1220
SQL over Vortex grows a **compute layer**: a Calcite `WHERE`-filtered `SUM`/`COUNT`/`MIN`/`MAX` is now answered from the zone-map statistics — folding the chunks the predicate fully selects without decoding them and decoding only the one or two chunks its range cuts through (ADR 0013 §6 / ADR 0018 boundary tier). On a 100-chunk file a `SELECT SUM(x) WHERE id BETWEEN …` answers ~12× faster than a full scan when the range is wide. Plus the security hardening of the untrusted-input parse paths (ADR 0003) and a `vortex.zstd` binding bump.

calcite/src/main/java/io/github/dfa1/vortex/calcite/VortexTable.java

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -324,6 +324,7 @@ public Optional<FilteredFold> filteredFold(RowFilter filter, String aggColumn) {
324324
/// [VortexException] disables the sum (mirroring a zone that records no usable sum) rather than
325325
/// failing the whole fold — a `MIN`/`MAX` over the same column still folds. The aggregate
326326
/// column's null count among the selected rows is the selected count minus the non-null count.
327+
@SuppressWarnings("removal") // transitional — uses the deprecated Mask until the fused multi-column filteredReduce lands
327328
private static void foldBoundaryZone(VortexReader reader, int zone, List<String> columns,
328329
RowFilter filter, String aggColumn, Fold fold) {
329330
try (Chunk chunk = reader.decodeChunk(zone, columns);
@@ -353,6 +354,7 @@ private static void foldBoundaryZone(VortexReader reader, int zone, List<String>
353354
/// column's decoded array, and an n-ary `AND` intersects the per-leaf masks (a row is selected
354355
/// only when every leaf accepts it). The kernels apply SQL three-valued logic, so a null row
355356
/// never satisfies a value predicate — matching the fold's null semantics.
357+
@SuppressWarnings("removal") // transitional — uses the deprecated Mask until the fused multi-column filteredReduce lands
356358
private static Mask buildMask(Chunk chunk, RowFilter filter, Arena arena) {
357359
long n = chunk.rowCount();
358360
List<Mask> masks = new ArrayList<>();
@@ -378,6 +380,7 @@ private static Mask buildMask(Chunk chunk, RowFilter filter, Arena arena) {
378380
/// A [RowFilter.Column] already holds the exact compute [Predicate], so the kernel runs it
379381
/// directly against the bound column — no translation seam, since `>=`, `<=` and `<>` are now
380382
/// first-class predicate variants.
383+
@SuppressWarnings("removal") // transitional — uses the deprecated Mask and Compute.filter until the fused multi-column filteredReduce lands
381384
private static void collectLeafMasks(Chunk chunk, RowFilter filter, Arena arena, List<Mask> out) {
382385
switch (filter) {
383386
case RowFilter.And(var parts) -> {

performance/src/main/java/io/github/dfa1/vortex/performance/ComputeKernelBenchmark.java

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,7 @@
100100
"--sun-misc-unsafe-memory-access=allow",
101101
"-Xmx4g"
102102
})
103+
@SuppressWarnings("removal") // transitional — benchmarks the deprecated Mask / Compute.filter two-pass path against the fused single-pass kernel
103104
public class ComputeKernelBenchmark {
104105

105106
/// Total rows scanned per op — ≈ 800 MB per column, far larger than L3, so memory-bound.

reader/src/main/java/io/github/dfa1/vortex/reader/compute/Compute.java

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,14 @@ private Compute() {
3434
/// @param arena the arena for the output bitmap; its [Arena#allocate(long)] zero-fills, which
3535
/// seeds the unselected bits to 0
3636
/// @return a mask of `array.length()` positions selected where `predicate` holds
37+
/// @deprecated Materializing a selection [Mask] is a slower primitive than a fused single-pass
38+
/// scan: it builds a positional bitmap over the whole filter column, which a later
39+
/// reduce must re-scan. Prefer the fused [#filteredSum(Array, Predicate, Array)] (and
40+
/// the forthcoming fused multi-column `filteredReduce`), which fold filter and
41+
/// aggregate in one pass with no intermediate bitmap. Scheduled for removal once the
42+
/// Calcite boundary fold migrates off it.
43+
@Deprecated(since = "0.12.0", forRemoval = true)
44+
@SuppressWarnings("removal") // suppresses only this method's own use of the to-be-removed Mask return type — callers still see filter() as deprecated
3745
public static Mask filter(Array array, Predicate predicate, Arena arena) {
3846
Objects.requireNonNull(array, "array");
3947
Objects.requireNonNull(predicate, "predicate");
@@ -43,10 +51,14 @@ public static Mask filter(Array array, Predicate predicate, Arena arena) {
4351

4452
/// Sums the selected non-null values of `array`.
4553
///
54+
/// Pairs with the (deprecated) [Mask]; transitional pending the fused reductions such as
55+
/// [#filteredSum(Array, Predicate, Array)].
56+
///
4657
/// @param array the array to reduce
4758
/// @param mask the selection mask, must have the same length as `array`
4859
/// @return the sum as a [Long] for integer columns or a [Double] for floating columns; the
4960
/// additive identity (`0`) when no selected position is non-null
61+
@SuppressWarnings("removal") // transitional — consumes the deprecated Mask until the fused multi-column filteredReduce lands
5062
public static Number sum(Array array, Mask mask) {
5163
return Reductions.SUM.apply(array, requireMask(array, mask));
5264
}
@@ -77,27 +89,39 @@ public static Number filteredSum(Array filterColumn, Predicate predicate, Array
7789

7890
/// Counts the selected non-null values of `array`.
7991
///
92+
/// Pairs with the (deprecated) [Mask]; transitional pending the fused reductions such as
93+
/// [#filteredSum(Array, Predicate, Array)].
94+
///
8095
/// @param array the array to reduce
8196
/// @param mask the selection mask, must have the same length as `array`
8297
/// @return the count of selected non-null values, `0` when none
98+
@SuppressWarnings("removal") // transitional — consumes the deprecated Mask until the fused multi-column filteredReduce lands
8399
public static long count(Array array, Mask mask) {
84100
return Reductions.COUNT.apply(array, requireMask(array, mask));
85101
}
86102

87103
/// Finds the smallest selected non-null value of `array` under its dtype-aware order.
88104
///
105+
/// Pairs with the (deprecated) [Mask]; transitional pending the fused reductions such as
106+
/// [#filteredSum(Array, Predicate, Array)].
107+
///
89108
/// @param array the array to reduce
90109
/// @param mask the selection mask, must have the same length as `array`
91110
/// @return the minimum value, or `null` when no selected position is non-null
111+
@SuppressWarnings("removal") // transitional — consumes the deprecated Mask until the fused multi-column filteredReduce lands
92112
public static Object min(Array array, Mask mask) {
93113
return Reductions.MIN.apply(array, requireMask(array, mask));
94114
}
95115

96116
/// Finds the largest selected non-null value of `array` under its dtype-aware order.
97117
///
118+
/// Pairs with the (deprecated) [Mask]; transitional pending the fused reductions such as
119+
/// [#filteredSum(Array, Predicate, Array)].
120+
///
98121
/// @param array the array to reduce
99122
/// @param mask the selection mask, must have the same length as `array`
100123
/// @return the maximum value, or `null` when no selected position is non-null
124+
@SuppressWarnings("removal") // transitional — consumes the deprecated Mask until the fused multi-column filteredReduce lands
101125
public static Object max(Array array, Mask mask) {
102126
return Reductions.MAX.apply(array, requireMask(array, mask));
103127
}
@@ -107,6 +131,7 @@ public static Object max(Array array, Mask mask) {
107131
/// @param array the array to reduce
108132
/// @param mask the selection mask
109133
/// @return the validated mask
134+
@SuppressWarnings("removal") // transitional — consumes the deprecated Mask until the fused multi-column filteredReduce lands
110135
private static Mask requireMask(Array array, Mask mask) {
111136
Objects.requireNonNull(array, "array");
112137
Objects.requireNonNull(mask, "mask");

reader/src/main/java/io/github/dfa1/vortex/reader/compute/FilterKernel.java

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,5 +33,6 @@ interface FilterKernel {
3333
/// @param arena the arena for the output bitmap; its [Arena#allocate(long)] zero-fills, which
3434
/// seeds the unselected bits to 0
3535
/// @return a mask of `array.length()` positions, selected where `current` and `predicate` agree
36+
@SuppressWarnings("removal") // transitional — uses the deprecated Mask until the fused multi-column filteredReduce lands
3637
Mask apply(Array array, Mask current, Predicate predicate, Arena arena);
3738
}

reader/src/main/java/io/github/dfa1/vortex/reader/compute/Mask.java

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,14 @@
2121
/// `compare` / `between` / `is_null` kernels.
2222
///
2323
/// All variants are read-only views: a mask never copies or allocates a bitmap of its own.
24+
///
25+
/// @deprecated The materialized selection mask is a slower primitive than a fused single-pass scan:
26+
/// producing it forces a positional bitmap and then a second pass over the aggregate
27+
/// column. Prefer the fused `Compute.filteredSum` (and the forthcoming fused
28+
/// multi-column `filteredReduce`), which fuse filter and aggregate in one pass with no
29+
/// intermediate bitmap. Scheduled for removal once the Calcite boundary fold migrates
30+
/// off it.
31+
@Deprecated(since = "0.12.0", forRemoval = true)
2432
public sealed interface Mask permits Mask.AllTrue, Mask.AllFalse, Mask.RangeMask, Mask.BitmapMask {
2533

2634
/// Returns the number of positions this mask covers.

reader/src/main/java/io/github/dfa1/vortex/reader/compute/PrimitiveFilter.java

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@ private PrimitiveFilter() {
4949
/// @param predicate the predicate to evaluate
5050
/// @param arena the arena for the result bitmap; its zero-fill seeds the unselected bits to 0
5151
/// @return the selection mask, or `null` if the input is not specializable
52+
@SuppressWarnings("removal") // transitional — uses the deprecated Mask until the fused multi-column filteredReduce lands
5253
static Mask tryFilter(Array array, Mask current, Predicate predicate, Arena arena) {
5354
Array data;
5455
BoolArray validity;
@@ -427,6 +428,7 @@ private static MemorySegment nullLeaf(BoolArray validity, long n, Arena arena, b
427428
/// @param bits the predicate-result bitmap, mutated in place
428429
/// @param current the incoming selection mask
429430
/// @param n the row count
431+
@SuppressWarnings("removal") // transitional — uses the deprecated Mask until the fused multi-column filteredReduce lands
430432
private static void applyCurrent(MemorySegment bits, Mask current, long n) {
431433
if (current instanceof Mask.AllTrue) {
432434
return;

reader/src/main/java/io/github/dfa1/vortex/reader/compute/ReduceKernel.java

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,5 +21,6 @@ interface ReduceKernel<R> {
2121
/// @param array the input array, possibly a lazy variant; never materialized whole
2222
/// @param current the selection mask, must have the same length as `array`
2323
/// @return the reduction result
24+
@SuppressWarnings("removal") // transitional — uses the deprecated Mask until the fused multi-column filteredReduce lands
2425
R apply(Array array, Mask current);
2526
}

reader/src/main/java/io/github/dfa1/vortex/reader/compute/Reductions.java

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,8 @@
2525
/// columns fold into a [Long] (wrapping at 2^63, like SQL `SUM(BIGINT)`), floating columns into a
2626
/// [Double], nulls are skipped, and the Rust empty / all-null results are mirrored exactly — `SUM` is
2727
/// the additive identity (`0L` or `0.0`, never null), `COUNT` is `0`, and `MIN` / `MAX` are `null`.
28+
// transitional — every reduction consumes the deprecated Mask until the fused multi-column filteredReduce lands
29+
@SuppressWarnings("removal")
2830
final class Reductions {
2931

3032
/// Sums the selected non-null values: a [Long] for integer columns, a [Double] for floats.

reader/src/main/java/io/github/dfa1/vortex/reader/compute/StreamingFilterKernel.java

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@
2222
final class StreamingFilterKernel implements FilterKernel {
2323

2424
@Override
25+
@SuppressWarnings("removal") // transitional — uses the deprecated Mask until the fused multi-column filteredReduce lands
2526
public Mask apply(Array array, Mask current, Predicate predicate, Arena arena) {
2627
Objects.requireNonNull(array, "array");
2728
Objects.requireNonNull(current, "current");
@@ -54,6 +55,7 @@ public Mask apply(Array array, Mask current, Predicate predicate, Arena arena) {
5455
/// @param predicate the predicate to evaluate
5556
/// @param arena the arena for the result bitmap
5657
/// @return the selection mask the generic per-element path produces
58+
@SuppressWarnings("removal") // transitional — uses the deprecated Mask until the fused multi-column filteredReduce lands
5759
Mask applyGeneric(Array array, Mask current, Predicate predicate, Arena arena) {
5860
long n = array.length();
5961
if (n == 0 || current instanceof Mask.AllFalse) {
@@ -62,6 +64,7 @@ Mask applyGeneric(Array array, Mask current, Predicate predicate, Arena arena) {
6264
return applyGeneric(array, current, predicate, arena, n);
6365
}
6466

67+
@SuppressWarnings("removal") // transitional — uses the deprecated Mask until the fused multi-column filteredReduce lands
6568
private Mask applyGeneric(Array array, Mask current, Predicate predicate, Arena arena, long n) {
6669
long bytes = (n + 7) >>> 3;
6770
// Arena.allocate zero-fills (FFM spec), so the unselected bits are already 0 — the loop

0 commit comments

Comments
 (0)