feat[fastlanes]: allow delta to support signed bases#7923
Merged
Conversation
Lifts the `is_unsigned_int` gate on `DeltaArray` so `i8` / `i16` / `i32` / `i64` columns can be delta-encoded. The upstream FastLanes kernels (`Delta::delta`, `Transpose::transpose`) are bounded on `T: FastLanes: Unsigned`, so signed inputs are processed by reinterpret-casting the underlying buffer to the same-width unsigned counterpart, running the existing kernel, and reinterpret-casting back. `wrapping_sub`/`wrapping_add` are bit-identical for signed and unsigned operands under two's-complement, so the round-trip is exact. Note that the encoded delta bytes for inputs that cross zero have the high bits set (e.g. delta `-1i8` = `0xFF`); naively bit-packing those would force the bit width to `T`. A follow-up should compose `Delta` with `FoR` so the deltas are stored as `value - min(delta)` before bit-packing. See encodings/fastlanes/src/delta/FUSED_DECODE.md for a design note on a fused triple-kernel (unpack + add-reference + undelta) that addresses the decode bandwidth. Also guards `Delta::cast` against signed sources: value-preserving casts of signed deltas (e.g. `-1i8` -> `4294967295u32`) break the wrapping-add invariant during decompression, so signed sources fall back to the decompress-and-reencode path. Signed-off-by: Claude <noreply@anthropic.com>
Measures the encoded byte budget under three bit-packing strategies for four representative signed `i32` shapes (monotone, sensor-like wobble around zero, large-negative offset, near-monotone with backtracks): | Workload | range | Wnaive | Wffor | Wzz | ratio | |-----------------------------------|----------------|-------:|------:|----:|--------:| | monotone i32 (0..N) | [0, 1] | 1 | 1 | 2 | 15.97x | | sensor i32 in [-100, 100] | [-196, 199] | 32 | 9 | 9 | 3.20x | | offset i32 base=-1e9 | [0, 1] | 1 | 1 | 2 | 15.97x | | near-monotone i32 (5% backtrack) | [-2, 1] | 32 | 2 | 3 | 10.65x | The "naive" column is the OR-mask of the raw delta bit-patterns: a single negative delta sets every high bit and forces `W = T`, which is why the two workloads with negative deltas (`sensor`, `near-monotone`) blow up to 32 bits. FFoR brings them to 9 and 2 bits. ZigZag matches FFoR only on the symmetric `sensor` workload and loses on every asymmetric column. Asserts that FFoR never exceeds naive, drops below `T` whenever a negative delta is present, and beats ZigZag on the asymmetric workloads. Run with `--nocapture` to see the table. Signed-off-by: Claude <noreply@anthropic.com>
Extends the synthetic workload report with two extra columns: bases byte size and the FFoR bit-width those bases would pack to. For 8K-element i32 inputs the bases buffer is ~50% of the FFoR total on monotone-like columns, and the bases sequence inherits the smoothness of the input, so recursively packing the bases with FoR gives a further ~1.4x on top of FFoR(deltas): workload FFoR (B) ratio bases (B) Wb +bcomp ratio monotone i32 (0..N) 2052 15.97x 1024 13 1448 22.63x sensor i32 in [-100, 100] 10244 3.20x 1024 8 9480 3.46x offset i32 base=-1e9 2052 15.97x 1024 13 1448 22.63x near-monotone i32 (5% backtrack) 3076 10.65x 1024 13 2472 13.26x This is already structurally enabled: the bases child is an `ArrayRef`, and the btrblocks compressor at vortex-btrblocks/src/schemes/integer.rs:917 already routes bases through `compress_child` so the cascading compressor picks whatever encoding fits (typically FoR + BitPacked). Signed-off-by: Claude <noreply@anthropic.com>
REUSE compliance — markdown files outside the patterns in REUSE.toml need inline SPDX comments. Signed-off-by: Claude <noreply@anthropic.com>
Pre-merge polish across the three things a reviewer would notice: * DeltaArray docstring: add a signed `i32` example next to the unsigned one so users see signed support is first-class. Verified by doctest. * Conformance: extend `test_delta_consistency` and `test_delta_binary_numeric` with i32 / i64 / i8 cases (crossing zero, all-negative, single-negative). These run the array-trait conformance harness, so any operation that's silently broken for signed inputs surfaces here. * cast.rs: expand the comment justifying why signed sources fall back to decompress-and-re-encode (the wrapping-add invariant breaks under value-preserving widening; the same hazard applies to cross-signedness). * synthetic_workload_compression table: rename duplicate "ratio" columns to `FFoR x` / `+bcomp x` so the report is unambiguous. 256 -> 263 tests, all pass. Clippy clean. Fmt clean. Signed-off-by: Claude <noreply@anthropic.com>
Merging this PR will not alter performance
|
joseph-isaacs
changed the title
robert3005
approved these changes
May 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Allow signed bases in delta.