Skip to content

Commit 8012c13

Browse files
dfa1claude
andcommitted
docs: move ADRs from docs/adr to top-level adr/
Relative links rewritten: docs pages point at ../adr/, ADR upward references drop one level, ADR links into docs/ gain the prefix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 2ca759d commit 8012c13

25 files changed

Lines changed: 43 additions & 43 deletions

CHANGELOG.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -170,7 +170,7 @@ A hardening release: no new file-format capability, but a large step up in verif
170170

171171
- Zero-warning rule: `-Xlint:all -Werror` across all modules. The `classfile` lint (which only flags missing annotation class files inside third-party Arrow bytecode) is scoped off in the two Arrow-using modules only. ([dab467e5](https://github.com/dfa1/vortex-java/commit/dab467e5), [43f6f840](https://github.com/dfa1/vortex-java/commit/43f6f840))
172172
- Mutation testing (PIT): opt-in `pitest` profiles in core, reader, and writer, scoped to the bounds/parse classes (`IoBounds`, `PTypeIO`, `WriteRegistry`, `ChunkImpl`, …), with common config hoisted into the parent POM. ([46904b24](https://github.com/dfa1/vortex-java/commit/46904b24), [ed8c98a1](https://github.com/dfa1/vortex-java/commit/ed8c98a1), [1200c76b](https://github.com/dfa1/vortex-java/commit/1200c76b), [840cc46a](https://github.com/dfa1/vortex-java/commit/840cc46a))
173-
- SonarCloud: generated `fbs/` and `proto/` sources excluded from analysis (machine output, not hand-maintained); the deliberate per-width SIMD-loop duplication is documented in [ADR 0005](docs/adr/0005-vector-api-adoption.md) rather than refactored away. Code smells dropped 857→394; coverage ~81%, all ratings A, zero bugs/vulnerabilities. ([6c591293](https://github.com/dfa1/vortex-java/commit/6c591293))
173+
- SonarCloud: generated `fbs/` and `proto/` sources excluded from analysis (machine output, not hand-maintained); the deliberate per-width SIMD-loop duplication is documented in [ADR 0005](adr/0005-vector-api-adoption.md) rather than refactored away. Code smells dropped 857→394; coverage ~81%, all ratings A, zero bugs/vulnerabilities. ([6c591293](https://github.com/dfa1/vortex-java/commit/6c591293))
174174

175175
### Tests
176176

@@ -185,7 +185,7 @@ Read and write Vortex Variant (semi-structured, JSON-shaped) columns from Java.
185185

186186
### Added
187187

188-
- Writer: `vortex.variant` encoder. Encodes a variant column as the canonical `vortex.variant` container over `core_storage` — an all-equal column becomes a single `vortex.constant`, a row-varying column a `vortex.chunked` of per-run constants — with an optional row-aligned typed `shredded` child recorded in `VariantMetadata.shredded_dtype`. Input is `VariantData(List<Scalar>)` with `.constant(n, v)` / `.shredded(...)` factories. Java↔Rust (JNI) round-trip verified for constant, row-varying, and shredded columns. Scalar values only — arbitrary nested objects need `vortex.parquet.variant` (deferred, [ADR 0014](docs/adr/0014-variant-encoding-strategy.md)). ([35da529d](https://github.com/dfa1/vortex-java/commit/35da529d), [e4e44980](https://github.com/dfa1/vortex-java/commit/e4e44980), [4566dca0](https://github.com/dfa1/vortex-java/commit/4566dca0))
188+
- Writer: `vortex.variant` encoder. Encodes a variant column as the canonical `vortex.variant` container over `core_storage` — an all-equal column becomes a single `vortex.constant`, a row-varying column a `vortex.chunked` of per-run constants — with an optional row-aligned typed `shredded` child recorded in `VariantMetadata.shredded_dtype`. Input is `VariantData(List<Scalar>)` with `.constant(n, v)` / `.shredded(...)` factories. Java↔Rust (JNI) round-trip verified for constant, row-varying, and shredded columns. Scalar values only — arbitrary nested objects need `vortex.parquet.variant` (deferred, [ADR 0014](adr/0014-variant-encoding-strategy.md)). ([35da529d](https://github.com/dfa1/vortex-java/commit/35da529d), [e4e44980](https://github.com/dfa1/vortex-java/commit/e4e44980), [4566dca0](https://github.com/dfa1/vortex-java/commit/4566dca0))
189189
- Reader: variant columns now decode Java-side. `ConstantEncodingDecoder` and `ChunkedEncodingDecoder` handle `DType.Variant` (materializing the inner-typed array); `VariantEncodingDecoder` wraps the result as `VariantArray`, exposing `coreStorage()` and `shredded()`. ([76e4c741](https://github.com/dfa1/vortex-java/commit/76e4c741), [4566dca0](https://github.com/dfa1/vortex-java/commit/4566dca0))
190190

191191
### Security
@@ -200,7 +200,7 @@ Read and write Vortex Variant (semi-structured, JSON-shaped) columns from Java.
200200

201201
### Changed
202202

203-
- Decode shape: transform encodings now decode **lazy-only**. The eager `Materialized*Array` fallbacks were removed from `vortex.zigzag` (all PTypes + broadcast, [cd59fefa](https://github.com/dfa1/vortex-java/commit/cd59fefa)), `fastlanes.for` (all integer PTypes, [d7953e1f](https://github.com/dfa1/vortex-java/commit/d7953e1f)), `vortex.alp` (broadcast-without-patches, [deab8067](https://github.com/dfa1/vortex-java/commit/deab8067)), `vortex.constant` (Decimal → `LazyConstantDecimalArray`, [a6a9611e](https://github.com/dfa1/vortex-java/commit/a6a9611e)), `vortex.runend` (Bool → `LazyRunEndBoolArray`, [0bbcb81f](https://github.com/dfa1/vortex-java/commit/0bbcb81f)), `vortex.sparse` (Bool → `LazySparseBoolArray`, [db2e955b](https://github.com/dfa1/vortex-java/commit/db2e955b)), and `fastlanes.rle` (validity → `OffsetBoolArray`, empty → `LazyConstantXxxArray`, [5e83a5c3](https://github.com/dfa1/vortex-java/commit/5e83a5c3)). Decompression encodings (`bitpacked`, `pco`, `zstd`, `fsst`, `delta`, `patched`), the primitive base, the `vortex.dict` encoding-level path, and the `vortex.alp` patches path stay Materialized by design. See [ADR 0015](docs/adr/0015-drop-materialized-fallbacks.md).
203+
- Decode shape: transform encodings now decode **lazy-only**. The eager `Materialized*Array` fallbacks were removed from `vortex.zigzag` (all PTypes + broadcast, [cd59fefa](https://github.com/dfa1/vortex-java/commit/cd59fefa)), `fastlanes.for` (all integer PTypes, [d7953e1f](https://github.com/dfa1/vortex-java/commit/d7953e1f)), `vortex.alp` (broadcast-without-patches, [deab8067](https://github.com/dfa1/vortex-java/commit/deab8067)), `vortex.constant` (Decimal → `LazyConstantDecimalArray`, [a6a9611e](https://github.com/dfa1/vortex-java/commit/a6a9611e)), `vortex.runend` (Bool → `LazyRunEndBoolArray`, [0bbcb81f](https://github.com/dfa1/vortex-java/commit/0bbcb81f)), `vortex.sparse` (Bool → `LazySparseBoolArray`, [db2e955b](https://github.com/dfa1/vortex-java/commit/db2e955b)), and `fastlanes.rle` (validity → `OffsetBoolArray`, empty → `LazyConstantXxxArray`, [5e83a5c3](https://github.com/dfa1/vortex-java/commit/5e83a5c3)). Decompression encodings (`bitpacked`, `pco`, `zstd`, `fsst`, `delta`, `patched`), the primitive base, the `vortex.dict` encoding-level path, and the `vortex.alp` patches path stay Materialized by design. See [ADR 0015](adr/0015-drop-materialized-fallbacks.md).
204204
- **Breaking — sealed `Array` permits changed.** `DecimalArray` is now a `non-sealed` family interface (decimal arrays moved from `implements Array` to `implements DecimalArray`), so decimal joins the per-dtype family layer. Downstream exhaustive `switch` over `Array` must add a `case DecimalArray`. ([a6a9611e](https://github.com/dfa1/vortex-java/commit/a6a9611e))
205205
- **Breaking — `Array` API.** `Array.truncate(rows)` renamed to `Array.limited(rows)` and made an abstract operation implemented by every array (composites slice their children); raw-segment access moved off the `ArraySegments` utility onto `Array.materialize(SegmentAllocator)` and `Array.segmentIfPresent()`. ([87ab65e2](https://github.com/dfa1/vortex-java/commit/87ab65e2), [4d9ac1f8](https://github.com/dfa1/vortex-java/commit/4d9ac1f8), [332b067e](https://github.com/dfa1/vortex-java/commit/332b067e), [32a35e03](https://github.com/dfa1/vortex-java/commit/32a35e03))
206206
- CSV import reports progress every 10K rows instead of per-chunk. ([07a056e7](https://github.com/dfa1/vortex-java/commit/07a056e7))
@@ -211,7 +211,7 @@ Read and write Vortex Variant (semi-structured, JSON-shaped) columns from Java.
211211

212212
### Documentation
213213

214-
- [ADR 0016](docs/adr/0016-vortex-arrow-bridge.md): captures `vortex-arrow` bridge interop options (separate module / Arrow C-Data / none); deferred until a concrete downstream need. ([a6126f29](https://github.com/dfa1/vortex-java/commit/a6126f29))
214+
- [ADR 0016](adr/0016-vortex-arrow-bridge.md): captures `vortex-arrow` bridge interop options (separate module / Arrow C-Data / none); deferred until a concrete downstream need. ([a6126f29](https://github.com/dfa1/vortex-java/commit/a6126f29))
215215

216216
### Tests
217217

TODO.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,10 @@
88

99
## Performance
1010

11-
- [ ] **Benchmark publishing** — drop CI workflow, add `bench-publish` script; see [ADR-0006](docs/adr/0006-benchmark-publishing.md).
11+
- [ ] **Benchmark publishing** — drop CI workflow, add `bench-publish` script; see [ADR-0006](adr/0006-benchmark-publishing.md).
1212
- [ ] Performance tests must be peer-reviewed
1313
- [ ] Run performance tests on other machines (I have access only to Apple M5)
14-
- [ ] **Vector API adoption** — deferred; see [ADR-0005](docs/adr/0005-vector-api-adoption.md) for adoption criteria and candidate loops.
14+
- [ ] **Vector API adoption** — deferred; see [ADR-0005](adr/0005-vector-api-adoption.md) for adoption criteria and candidate loops.
1515

1616
## Security
1717

@@ -44,7 +44,7 @@ Per-encoding gotchas:
4444

4545
### Resource caps
4646

47-
- [ ] **Implement `ResourceLimits` + `ReadOptions`** — see [ADR-0004](docs/adr/0004-resource-caps-read-options.md) for design, defaults, and enforcement points. Also covers Pco page/bin caps.
47+
- [ ] **Implement `ResourceLimits` + `ReadOptions`** — see [ADR-0004](adr/0004-resource-caps-read-options.md) for design, defaults, and enforcement points. Also covers Pco page/bin caps.
4848

4949
### Fuzz infrastructure
5050

@@ -70,23 +70,23 @@ Per-encoding gotchas:
7070

7171
## Tooling
7272

73-
- [ ] Optional `vortex-arrow` bridge module for Arrow ecosystem interop — see [ADR-0016](docs/adr/0016-vortex-arrow-bridge.md)
73+
- [ ] Optional `vortex-arrow` bridge module for Arrow ecosystem interop — see [ADR-0016](adr/0016-vortex-arrow-bridge.md)
7474

7575
## API
7676

7777
- [ ] **Error messages — structural sanitization of `VortexException`**
7878
Phase E (bounds typing via `IoBounds`) shipped; remaining is Phases A–D (the `Sanitize`
79-
helper + `VortexError` catalog). See [ADR-0003](docs/adr/0003-vortex-exception-sanitization.md)
79+
helper + `VortexError` catalog). See [ADR-0003](adr/0003-vortex-exception-sanitization.md)
8080
for design and phasing.
8181
- [ ] Use domain primitives (`UInt32`, `UInt64`, etc.) as value classes via Project Valhalla instead of raw `long`/`int`
82-
- See [ADR-0008](docs/adr/0008-domain-primitives-unsigned-integers.md) and https://dfa1.github.io/articles/rethink-domain-primitives-with-valhalla
82+
- See [ADR-0008](adr/0008-domain-primitives-unsigned-integers.md) and https://dfa1.github.io/articles/rethink-domain-primitives-with-valhalla
8383
- Candidates: `PType` integer kinds, buffer offsets, row indices, byte lengths
8484
- Goal: type-safety at zero cost (value class = no heap alloc, no boxing)
8585

8686
## Compute
8787

8888
- [ ] **Compute primitives — encoded-domain specialization & façade** — the remaining ADR-0013
89-
follow-ups now the fused kernels have shipped. See [ADR-0013](docs/adr/0013-compute-primitives.md).
89+
follow-ups now the fused kernels have shipped. See [ADR-0013](adr/0013-compute-primitives.md).
9090
Done: §4 `Predicate`; §5 `RowFilter` unified over `Predicate`; §6 zone-map aggregate push-down in
9191
both tiers — the whole-zone `ZoneReducer` fold wired into `VortexAggregatePushDownRule` (rewrites a
9292
whole-table `MIN`/`MAX`/`COUNT`/`SUM`/`AVG` to a single-row `Values`, auto-registered over a bare
@@ -100,7 +100,7 @@ Per-encoding gotchas:
100100
residual leaves tested per match). Multi-fork numbers: `fusedFilteredSumDict` 762 → 38 ms/op
101101
≈ 20×; `fusedFilteredAggregateDict` 983 → 46 ms/op ≈ 22×; `fusedFilteredAggregateMulti`
102102
(2-leaf `AND` × 2 aggregates) 2269 → 201 ms/op ≈ 11×.
103-
Next: the columnar transducer façade — [ADR-0019](docs/adr/0019-columnar-transducer-facade.md)
103+
Next: the columnar transducer façade — [ADR-0019](adr/0019-columnar-transducer-facade.md)
104104
drafted (Proposed): declarative column-bound stages compiled to one fused pass; the remaining
105105
measured lever is the multi-aggregate single scan (≈ 2×) plus composition ergonomics for the
106106
Calcite boundary tier; review, then implement.

docs/adr/0001-split-read-and-write-runtimes.md renamed to adr/0001-split-read-and-write-runtimes.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -353,6 +353,6 @@ of CI / integration-test fallout, plus reviewer time. Not a weekend.
353353

354354
- [PR #27`sec(parser): BoundedSegment + audit trail for untrusted asSlice`](https://github.com/dfa1/vortex-java/pull/27)
355355
- [Phase 1–4 commits — BoundedSegment introduction and migration](https://github.com/dfa1/vortex-java/pull/27/commits)
356-
- [SECURITY.md — the contract this work hardens](../../SECURITY.md)
357-
- [CLAUDE.md — current "three touch-points" rule for adding an encoding](../../CLAUDE.md)
358-
- [TODO.md — parser hardening backlog](../../TODO.md)
356+
- [SECURITY.md — the contract this work hardens](../SECURITY.md)
357+
- [CLAUDE.md — current "three touch-points" rule for adding an encoding](../CLAUDE.md)
358+
- [TODO.md — parser hardening backlog](../TODO.md)

docs/adr/0002-pluggable-dtype-layout-compute.md renamed to adr/0002-pluggable-dtype-layout-compute.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
- **Status:** Deferred — awaiting a real downstream consumer use case
44
- **Date:** 2026-06-11
55
- **Deciders:** project maintainer
6-
- **Related:** [ADR 0001 — Split read and write runtimes](0001-split-read-and-write-runtimes.md), [TODO.md §Audit runtime pluggability vs Rust impl](../../TODO.md)
6+
- **Related:** [ADR 0001 — Split read and write runtimes](0001-split-read-and-write-runtimes.md), [TODO.md §Audit runtime pluggability vs Rust impl](../TODO.md)
77

88
## Context
99

@@ -197,10 +197,10 @@ Until those four boxes can be ticked, the deferral stands.
197197

198198
## References
199199

200-
- [TODO.md §"Audit runtime pluggability vs Rust impl"](../../TODO.md)
200+
- [TODO.md §"Audit runtime pluggability vs Rust impl"](../TODO.md)
201201
(line ~215, dated 2026-06-04)
202202
- [ADR 0001 — Split read and write runtimes out of `core`](0001-split-read-and-write-runtimes.md)
203-
- [docs/compatibility.md — Known wire-format gaps](../compatibility.md)
203+
- [docs/compatibility.md — Known wire-format gaps](../docs/compatibility.md)
204204
(notes `DType::Union` as one new variant Java does not yet decode)
205205
- Rust upstream:
206206
[`VortexSession`](https://docs.rs/vortex/latest/vortex/session/struct.VortexSession.html)

docs/adr/0003-vortex-exception-sanitization.md renamed to adr/0003-vortex-exception-sanitization.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
- **Deciders:** project maintainer
66
- **Related:** [ADR 0001 — Split read and write runtimes](0001-split-read-and-write-runtimes.md),
77
[ADR 0004 — Resource caps and `ReadOptions`](0004-resource-caps-read-options.md),
8-
[SECURITY.md](../../SECURITY.md)
8+
[SECURITY.md](../SECURITY.md)
99

1010
## Context
1111

@@ -415,7 +415,7 @@ programmatically inspect exception types, this decision can be revisited.
415415

416416
## References
417417

418-
- [SECURITY.md — injection threat model](../../SECURITY.md)
418+
- [SECURITY.md — injection threat model](../SECURITY.md)
419419
- [PR #27`BoundedSegment` + audit trail for untrusted `asSlice`](https://github.com/dfa1/vortex-java/pull/27)
420420
- [ADR 0001 — Split read and write runtimes](0001-split-read-and-write-runtimes.md)
421-
- [TODO.md §"Error messages — structural sanitization"](../../TODO.md)
421+
- [TODO.md §"Error messages — structural sanitization"](../TODO.md)

docs/adr/0004-resource-caps-read-options.md renamed to adr/0004-resource-caps-read-options.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
- **Status:** Accepted — implementation pending
44
- **Date:** 2026-06-13
55
- **Deciders:** project maintainer
6-
- **Related:** [SECURITY.md](../../SECURITY.md), [TODO.md §Resource caps](../../TODO.md)
6+
- **Related:** [SECURITY.md](../SECURITY.md), [TODO.md §Resource caps](../TODO.md)
77

88
## Context
99

@@ -167,6 +167,6 @@ a user-facing query limit, not a security cap.
167167

168168
## References
169169

170-
- [SECURITY.md — zip-bomb and depth-bomb mitigations](../../SECURITY.md)
171-
- [TODO.md §Resource caps](../../TODO.md)
170+
- [SECURITY.md — zip-bomb and depth-bomb mitigations](../SECURITY.md)
171+
- [TODO.md §Resource caps](../TODO.md)
172172
- [ADR 0003 — VortexException sanitization](0003-vortex-exception-sanitization.md)
Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33
- **Status:** Deferred — adoption gated on API finalization + benchmark evidence
44
- **Date:** 2026-06-13
55
- **Deciders:** project maintainer
6-
- **Related:** [CLAUDE.md §Memory model — Hot-loop rule](../../CLAUDE.md),
7-
[TODO.md §Vector API](../../TODO.md)
6+
- **Related:** [CLAUDE.md §Memory model — Hot-loop rule](../CLAUDE.md),
7+
[TODO.md §Vector API](../TODO.md)
88

99
## Context
1010

@@ -150,9 +150,9 @@ loop structure.
150150

151151
## References
152152

153-
- [CLAUDE.md §Hot-loop rule](../../CLAUDE.md) — the primary vectorization
153+
- [CLAUDE.md §Hot-loop rule](../CLAUDE.md) — the primary vectorization
154154
strategy in use today
155-
- [TODO.md §Vector API items](../../TODO.md)
155+
- [TODO.md §Vector API items](../TODO.md)
156156
- JEP 469: Vector API (8th Incubator) —
157157
https://openjdk.org/jeps/469
158158
- JMH benchmark harness: `./bench` command, `JavaVsJniReadBenchmark`
Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
- **Status:** Accepted — CI workflow deleted; `bench-publish` script pending
44
- **Date:** 2026-06-13
55
- **Deciders:** project maintainer
6-
- **Related:** [TODO.md §Performance](../../TODO.md),
6+
- **Related:** [TODO.md §Performance](../TODO.md),
77
`.github/workflows/benchmark.yml`
88

99
## Context
@@ -156,5 +156,5 @@ longer updated.
156156
multi-source comparison
157157
- [benchmark-action/github-action-benchmark](https://github.com/benchmark-action/github-action-benchmark)
158158
— the tool being retired
159-
- [TODO.md §Publish benchmarks](../../TODO.md)
160-
- [TODO.md §Publish reproducible perf artifacts](../../TODO.md)
159+
- [TODO.md §Publish benchmarks](../TODO.md)
160+
- [TODO.md §Publish reproducible perf artifacts](../TODO.md)
File renamed without changes.

docs/adr/0008-domain-primitives-unsigned-integers.md renamed to adr/0008-domain-primitives-unsigned-integers.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
- **Status:** Proposed
44
- **Date:** 2026-06-13
55
- **Deciders:** project maintainer
6-
- **Related:** [TODO.md — domain primitives / Valhalla item](../../TODO.md),
6+
- **Related:** [TODO.md — domain primitives / Valhalla item](../TODO.md),
77
[ADR 0005 — Vector API adoption](0005-vector-api-adoption.md)
88

99
## Context
@@ -20,7 +20,7 @@ as their widened signed counterparts (`int`, `int`, `long`, `long`) with
2020
no enforcement of the unsigned range and no indication at the call site that
2121
the value must be treated as unsigned.
2222

23-
The [TODO.md](../../TODO.md) item reads:
23+
The [TODO.md](../TODO.md) item reads:
2424
> Use domain primitives (`UInt32`, `UInt64`, etc.) as value classes via
2525
> Project Valhalla instead of raw `long`/`int`.
2626
@@ -155,7 +155,7 @@ additional tooling.
155155

156156
## References
157157

158-
- [TODO.md — domain primitives / Valhalla item](../../TODO.md)
158+
- [TODO.md — domain primitives / Valhalla item](../TODO.md)
159159
- [Rethink Domain Primitives with Valhalla](https://dfa1.github.io/articles/rethink-domain-primitives-with-valhalla)
160160
- JEP 401: Value Classes and Objects — https://openjdk.org/jeps/401
161161
- [ADR 0005 — Vector API adoption](0005-vector-api-adoption.md) — analogous

0 commit comments

Comments
 (0)