Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `Compute.filteredSum` over a dictionary-encoded filter column is ~20× faster (best runs ~30×): the predicate is resolved against the dictionary's value pool once and the raw `u8` codes are scanned directly from their backing segments, instead of decoding every row through the per-element accessor — a fused `SUM(measure) WHERE category = …` over 100M rows drops from ~760 ms to ~38 ms. ([85e251cc](https://github.com/dfa1/vortex-java/commit/85e251cc))
- `Compute.filteredAggregate` takes the same dictionary code-scan lane (`COUNT(*)` included) — ~22× faster on the same workload (~980 ms → ~46 ms over 100M rows), which the Calcite `WHERE`-filtered aggregate push-down inherits on its boundary chunks. ([145791c7](https://github.com/dfa1/vortex-java/commit/145791c7), [6e6d7dd0](https://github.com/dfa1/vortex-java/commit/6e6d7dd0))
- A multi-column `AND` filter no longer forfeits the dictionary lane: the dict-encoded leaf drives the code scan and the remaining predicates are evaluated only on its matches — `SUM(…) WHERE category = 7 AND price > 500` over 100M rows drops from ~2.3 s to ~200 ms (~11×). ([12e13501](https://github.com/dfa1/vortex-java/commit/12e13501))
- `reader.decode.ArrayNode` is now a single record carrying the raw encoding-id string from the wire; the `KnownArrayNode` / `UnknownArrayNode` split is gone. Custom `EncodingDecoder` implementations keep working unchanged — decode dispatch is keyed by the id string, and the `allowUnknown` passthrough behaves as before. ([3b25a97d](https://github.com/dfa1/vortex-java/commit/3b25a97d))
- `core.model.EncodingId` is now a sealed interface: the spec constants live in the nested `WellKnown` enum (re-exported, so `EncodingId.VORTEX_FOO` call sites compile unchanged) and `Custom` wraps any other wire string, which for the first time lets third-party `EncodingDecoder`/`EncodingEncoder` implementations declare ids outside the spec set. `parse` is total over non-blank ids — an unknown id yields a typed `Custom` instead of an empty `Optional`. ([ea88a91b](https://github.com/dfa1/vortex-java/commit/ea88a91b))
- `reader.decode.ArrayNode` is a single record carrying the typed `EncodingId`; the `KnownArrayNode`/`UnknownArrayNode` split and the `ArrayNode.of` factory are gone. Decode dispatch, the `allowUnknown` passthrough, and error messages are unchanged. A crafted file with a blank encoding id now fails as `VortexException` instead of escaping as `IllegalArgumentException`. ([21810d7e](https://github.com/dfa1/vortex-java/commit/21810d7e))

## [0.11.0] — 2026-06-28

Expand Down
12 changes: 6 additions & 6 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,12 @@ core — everything lives under `io.github.dfa1.vortex.core.*`:
core.error VortexException
core.compute FastLanes, PrimitiveArrays
core.fbs / core.proto — generated wire codecs + their runtimes
reader — VortexReader, VortexHttpReader, VortexHandle, ReadRegistry, ExtensionDecoder,
Chunk, ArrayStats, ScanOptions, RowFilter; file internals (Footer, Layout, Trailer,
reader — VortexReader, VortexHttpReader, VortexHandle, ReadRegistry, Chunk, ArrayStats,
ScanOptions, RowFilter; file internals (Footer, Layout, Trailer,
PostscriptParser, …)
reader.array — Array + all subtypes (decode outputs)
reader.decode — EncodingDecoder, DecodeContext, ArrayNode + *EncodingDecoder impls
reader.extension — Date/Time/Timestamp/Uuid ExtensionDecoder
reader.extension — ExtensionDecoder + Date/Time/Timestamp/Uuid impls
writer — VortexWriter, WriteRegistry, WriteOptions, ExtensionEncoder
writer.encode — EncodingEncoder, EncodeContext, NullableData + *EncodingEncoder impls,
extension encoders
Expand Down Expand Up @@ -135,9 +135,9 @@ Add `EncodingId` enum constant `VORTEX_FOO("vortex.foo")`, then per side:
### Adding an extension type

Add `ExtensionId` constant, then per side:
- **Decode:** `FooExtensionDecoder implements ExtensionDecoder` in `reader.extension`; register via
`ReadRegistry.builder().register(new FooExtensionDecoder())` — **no service file**
(`registerServiceLoaded()` does not discover `ExtensionDecoder`).
- **Decode:** singleton `FooExtensionDecoder implements ExtensionDecoder` in `reader.extension` +
a `case VORTEX_FOO` in `Chunk.as()` — not registry-managed, **no service file**
(`registerServiceLoaded()` only discovers `EncodingDecoder`).
- **Encode:** `FooExtensionEncoder implements ExtensionEncoder` in `writer` + FQN in
`writer/.../META-INF/services/io.github.dfa1.vortex.writer.ExtensionEncoder`

Expand Down
Loading
Loading