Skip to content

Commit 0637ed3

Browse files
dfa1claude
andcommitted
docs: note Java writer file size vs Rust trade-off
Java files up to ~2x larger due to incomplete encoding set; still smaller than CSV. References FileSizeComparisonIntegrationTest. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent d95f6cb commit 0637ed3

1 file changed

Lines changed: 4 additions & 0 deletions

File tree

README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,10 @@ format translation at the boundary.
1616
- Local (mmap) or Remote (HTTPS, single read of last 65K) (stable)
1717
- Writer: in progress
1818
- Benchmark vs Rust+JNI: Java beats JNI 1.5×–11.5× across read/write workloads (see Benchmarks)
19+
- **File size trade-off:** Java-written files are larger than Rust-written files (up to ~
20+
with `cascading(3)`). The Rust writer applies more compression passes; the Java writer
21+
covers ALP, bitpacking, and FSST but not the full Rust encoding set yet. Files are still
22+
significantly smaller than CSV. Cross-implementation verified by `FileSizeComparisonIntegrationTest`.
1923
- Full encoding coverage: in progress
2024
- Vectorized decode paths (Panama Vector API): planned
2125
- Iceberg/Spark/Flink integration: not available yet

0 commit comments

Comments
 (0)