Skip to content

Commit 2960655

Browse files
dfa1claude
andcommitted
docs: Chunk typed-columns surface in reference + changelog
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent f8ad15d commit 2960655

2 files changed

Lines changed: 16 additions & 7 deletions

File tree

CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1717

1818
### Changed
1919

20+
- `Chunk.columns()` returns an order-preserving `SequencedMap<ColumnName, Chunk.Column>` — one map instead of two parallel string-keyed maps, with each column's `Array` and `DType` traveling together in the `Column` carrier. `column(String)` stays as boundary sugar (plus a `column(ColumnName)` overload); iteration order is the schema/projection order, now guaranteed even for 1–2 column chunks. Typed names originate in `ScanIterator` from the file's already-certified schema. ([f8ad15d1](https://github.com/dfa1/vortex-java/commit/f8ad15d1))
21+
2022
- `Compute.filteredSum` over a dictionary-encoded filter column is ~20× faster (best runs ~30×): the predicate is resolved against the dictionary's value pool once and the raw `u8` codes are scanned directly from their backing segments, instead of decoding every row through the per-element accessor — a fused `SUM(measure) WHERE category = …` over 100M rows drops from ~760 ms to ~38 ms. ([85e251cc](https://github.com/dfa1/vortex-java/commit/85e251cc))
2123
- `Compute.filteredAggregate` takes the same dictionary code-scan lane (`COUNT(*)` included) — ~22× faster on the same workload (~980 ms → ~46 ms over 100M rows), which the Calcite `WHERE`-filtered aggregate push-down inherits on its boundary chunks. ([145791c7](https://github.com/dfa1/vortex-java/commit/145791c7), [6e6d7dd0](https://github.com/dfa1/vortex-java/commit/6e6d7dd0))
2224
- A multi-column `AND` filter no longer forfeits the dictionary lane: the dict-encoded leaf drives the code scan and the remaining predicates are evaluated only on its matches — `SUM(…) WHERE category = 7 AND price > 500` over 100M rows drops from ~2.3 s to ~200 ms (~11×). ([12e13501](https://github.com/dfa1/vortex-java/commit/12e13501))

docs/reference.md

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -183,13 +183,20 @@ columnar buffers; closing the chunk releases the arena. After `close()`, touchin
183183
any `Array` previously returned by `column(...)` or `columns()` raises FFM's scope
184184
check (`IllegalStateException`).
185185

186-
| Method | Notes |
187-
|-----------------------------------------|----------------------------------------------------------|
188-
| `rowCount()` | Rows in this chunk |
189-
| `columns()` | All columns in this chunk |
190-
| `<T extends Array> column(String name)` | Typed column lookup; throws `VortexException` if unknown |
191-
| `isClosed()` | Whether `close()` has run |
192-
| `close()` | Releases the chunk's arena. Idempotent. |
186+
Columns are stored as one order-preserving map keyed by the validated [`ColumnName`]; each
187+
entry is a `Chunk.Column(Array array, DType dtype)` carrier, so a column's data and type can
188+
never desync. `column(String)` is boundary sugar: the name is wrapped in a `ColumnName` (a
189+
policy-invalid name fails fast — it could never match a certified column).
190+
191+
| Method | Notes |
192+
|---------------------------------------------|----------------------------------------------------------|
193+
| `rowCount()` | Rows in this chunk |
194+
| `columns()` | `SequencedMap<ColumnName, Chunk.Column>`, schema order, unmodifiable |
195+
| `<T extends Array> column(String name)` | Typed column lookup; throws `VortexException` if absent |
196+
| `<T extends Array> column(ColumnName name)` | Same, for callers that validated early |
197+
| `as(String name, Class<T> domainType)` | Extension column → typed `List<T>` |
198+
| `isClosed()` | Whether `close()` has run |
199+
| `close()` | Releases the chunk's arena. Idempotent. |
193200

194201
---
195202

0 commit comments

Comments
 (0)