Skip to content

Commit 70053ea

Browse files
dfa1claude
andcommitted
docs: apply terse CHANGELOG convention retroactively
Rewrite the 5 remaining paragraph-style Unreleased entries (nullable global dict, Bool constant/sparse/runend/rle) as one-sentence titles with commit links, matching the new changelog skill. Splits the combined Sparse+Sequence and Constant+RunEnd bullets into one entry per logical change (same source commit, distinct fixes). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1 parent a920ff4 commit 70053ea

1 file changed

Lines changed: 7 additions & 5 deletions

File tree

CHANGELOG.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,11 +9,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
99

1010
### Fixed
1111

12-
- Nullable low-cardinality columns now use the shared global dictionary across chunks instead of re-emitting a per-chunk dictionary. The writer previously excluded any nullable column from global-dict candidacy, so a nullable low-cardinality string/numeric column (extremely common in real data — most Raincloud categorical columns have nulls) re-wrote its dictionary in every chunk. Nullable columns now run the same cardinality/ratio gate against their non-null values and emit one shared `vortex.dict` layout with per-chunk masked codes; the reader combines codes-side and pool-side validity per row. On a 200k-row, 10-chunk nullable low-cardinality Utf8 column this shrinks the file 54% (232 KB → 107 KB) and closes the gap to the Rust reference from ~4× to ~1.5×. (internal)
13-
- `MaskedEncodingEncoder` now encodes an all-valid or all-invalid validity bitmap as `vortex.constant` instead of a raw per-row bitmap. A nullable column chunk with zero nulls previously still paid a full `n/8`-byte bitmap for no information; an all-valid or all-invalid chunk now costs a few bytes regardless of row count. (internal)
14-
- `MaskedEncodingEncoder` also tries `vortex.sparse` for a mixed-validity bitmap (fill = the majority value, patches = the minority positions) and keeps it over the raw bitmap when smaller — the patch-index array is compressed through a dedicated `vortex.sequence`/`fastlanes.delta`/FoR/bitpack cascade, so a clustered or regular null pattern (e.g. one null every 10 rows) collapses to a handful of bytes instead of a full bitmap. On the 200k-row, 10-chunk nullable low-cardinality Utf8 benchmark this drops the file a further 20% (107 KB → 86 KB) and the gap to the Rust reference from ~1.45× to ~1.17×. Also: `SequenceEncodingEncoder.encodeCascade` no longer lets `encode`'s "not an arithmetic sequence" exception escape when a stratified sample looked arithmetic but the full data wasn't — it now reports the cascade step as not applicable, like every other encoder's contract requires. (internal)
15-
- `ConstantEncodingEncoder` now accepts `DType.Bool` (previously `DType.Primitive` only), so any all-true or all-false dense Bool column — not just a masked/nullable one — can win `vortex.constant` through the normal per-column cascade, not only through `MaskedEncodingEncoder`'s dedicated check. `MaskedEncodingEncoder`'s validity encoding now delegates to it instead of duplicating the scalar-building logic. `RunEndEncodingEncoder` also gains a `vortex.runend` path for Bool: `MaskedEncodingEncoder` tries it alongside `vortex.sparse` and keeps whichever is smallest — `vortex.runend` wins on long clustered runs of valid/invalid rows (regardless of which value dominates), a shape `vortex.sparse`'s per-flip patch cost handles poorly. (internal)
16-
- `RleEncodingEncoder`/`RleEncodingDecoder` (`fastlanes.rle`) round out Bool coverage: the same FastLanes 1024-row chunked layout the numeric path uses, with a `vortex.bool`-encoded values pool (at most 2 distinct values per chunk) instead of a ptype-width one. `MaskedEncodingEncoder` tries it alongside `vortex.sparse`/`vortex.runend` and keeps the smallest — `DType.Bool` now has every general-purpose encoding family (`vortex.constant`, `vortex.sparse`, `vortex.runend`, `fastlanes.rle`) that `DType.Primitive` has. (internal)
12+
- Nullable low-cardinality columns now share one global dictionary across chunks instead of re-emitting a per-chunk dictionary. ([5fe8b544](https://github.com/dfa1/vortex-java/commit/5fe8b544))
13+
- `MaskedEncodingEncoder` encodes an all-valid or all-invalid validity bitmap as `vortex.constant` instead of a raw per-row bitmap. ([ecd47ead](https://github.com/dfa1/vortex-java/commit/ecd47ead))
14+
- `MaskedEncodingEncoder` tries `vortex.sparse` for a mixed-validity bitmap and keeps it over the raw bitmap when smaller. ([506d036f](https://github.com/dfa1/vortex-java/commit/506d036f))
15+
- `SequenceEncodingEncoder.encodeCascade` no longer lets a full-data encode exception escape when a sampled prefix looked arithmetic but the full data wasn't. ([506d036f](https://github.com/dfa1/vortex-java/commit/506d036f))
16+
- `ConstantEncodingEncoder` now accepts `DType.Bool` (previously `Primitive` only). ([5379af66](https://github.com/dfa1/vortex-java/commit/5379af66))
17+
- `RunEndEncodingEncoder` gains a `vortex.runend` path for Bool. ([5379af66](https://github.com/dfa1/vortex-java/commit/5379af66))
18+
- `fastlanes.rle` now supports `DType.Bool`, completing Bool coverage across all general-purpose encodings (`vortex.constant`, `vortex.sparse`, `vortex.runend`, `fastlanes.rle`). ([fe6f132b](https://github.com/dfa1/vortex-java/commit/fe6f132b))
1719
- `Utf8`/`Binary` columns now use cost-based encoding selection instead of first-match dispatch (Dict no longer always beats FSST regardless of size). ([1bbc3549](https://github.com/dfa1/vortex-java/commit/1bbc3549))
1820
- `FsstEncodingDecoder` no longer throws on valid files with an all-default (empty) `vortex.fsst` metadata segment. ([1bbc3549](https://github.com/dfa1/vortex-java/commit/1bbc3549))
1921
- `FsstEncodingEncoder` writes symbol tables in the order the Rust reference requires; previously an out-of-order table could fail to open in the JNI/Rust reader. ([95e0dfb4](https://github.com/dfa1/vortex-java/commit/95e0dfb4))

0 commit comments

Comments
 (0)