|
| 1 | +# Compatibility |
| 2 | + |
| 3 | +Tested against the [Rust reference implementation](https://github.com/spiraldb/vortex) v0.72.0. |
| 4 | + |
| 5 | +## Encodings |
| 6 | + |
| 7 | +| Encoding ID | Class | Decode | Encode | Notes | |
| 8 | +|------------------------------|------------------------------|--------|--------|-------| |
| 9 | +| `vortex.primitive` | `PrimitiveEncoding` | ✅ | ✅ | All `PType` (I8–I64, U8–U64, F32, F64) | |
| 10 | +| `vortex.bool` | `BoolEncoding` | ✅ | ✅ | Bool (bit-packed) | |
| 11 | +| `vortex.null` | `NullEncoding` | ✅ | ✅ | Null | |
| 12 | +| `vortex.bytebool` | `ByteBoolEncoding` | ✅ | ✅ | Bool (byte-per-element) | |
| 13 | +| `vortex.zigzag` | `ZigZagEncoding` | ✅ | ✅ | Signed integer PTypes | |
| 14 | +| `vortex.constant` | `ConstantEncoding` | ✅ | ✅ | Primitive, Utf8, Binary, Bool, Null, Decimal, Extension | |
| 15 | +| `vortex.ext` | `ExtEncoding` | ✅ | ✅ | Extension | |
| 16 | +| `vortex.runend` | `RunEndEncoding` | ✅ | ✅ | Primitive, Utf8/Binary, Bool | |
| 17 | +| `vortex.varbin` | `VarBinEncoding` | ✅ | ✅ | Utf8, Binary | |
| 18 | +| `vortex.varbinview` | `VarBinViewEncoding` | ✅ | ✅ | Utf8, Binary | |
| 19 | +| `vortex.alp` | `AlpEncoding` | ✅ | ✅ | F64, F32 | |
| 20 | +| `vortex.alprd` | `AlpRdEncoding` | ✅ | ✅ | F64, F32 | |
| 21 | +| `vortex.dict` | `DictEncoding` | ✅ | ✅ | Primitive, Utf8/Binary | |
| 22 | +| `vortex.sparse` | `SparseEncoding` | ✅ | ✅ | Primitive | |
| 23 | +| `vortex.sequence` | `SequenceEncoding` | ✅ | ✅ | Primitive | |
| 24 | +| `vortex.struct` | `StructEncoding` | ✅ | ✅ | Struct | |
| 25 | +| `vortex.chunked` | `ChunkedEncoding` | ✅ | ✅ | Primitive + Struct concat | |
| 26 | +| `vortex.fsst` | `FsstEncoding` | ✅ | ✅ | Utf8, Binary | |
| 27 | +| `vortex.list` | `ListEncoding` | ✅ | ✅ | | |
| 28 | +| `vortex.listview` | `ListViewEncoding` | ✅ | ✅ | | |
| 29 | +| `vortex.fixed_size_list` | `FixedSizeListEncoding` | ✅ | ✅ | | |
| 30 | +| `vortex.zstd` | `ZstdEncoding` | ✅ | ✅ | Primitive, Utf8, Binary | |
| 31 | +| `vortex.masked` | `MaskedEncoding` | ✅ | ❌ | Encode not yet implemented | |
| 32 | +| `vortex.decimal` | `DecimalEncoding` | ✅ | ✅ | | |
| 33 | +| `vortex.decimal_byte_parts` | `DecimalBytePartsEncoding` | ✅ | ✅ | | |
| 34 | +| `vortex.datetimeparts` | `DateTimePartsEncoding` | ✅ | ✅ | | |
| 35 | +| `vortex.pco` | `PcoEncoding` | ✅ | ❌ | Decode: all modes; encode not yet implemented | |
| 36 | +| `fastlanes.bitpacked` | `BitpackedEncoding` | ✅ | ✅ | Unsigned integer PTypes | |
| 37 | +| `fastlanes.delta` | `DeltaEncoding` | ✅ | ✅ | Integer PTypes | |
| 38 | +| `fastlanes.for` | `FrameOfReferenceEncoding` | ✅ | ✅ | Integer PTypes | |
| 39 | +| `fastlanes.rle` | `RleEncoding` | ✅ | ✅ | Chunk-based RLE | |
| 40 | +| `vortex.patched` | — | ❌ | ❌ | No decoder yet; reads as `UnknownArray` when `allowUnknown()` enabled | |
| 41 | +| `vortex.variant` | — | ❌ | ❌ | No decoder yet; reads as `UnknownArray` when `allowUnknown()` enabled | |
| 42 | + |
| 43 | +### Unknown encodings |
| 44 | + |
| 45 | +Files containing unrecognised encoding IDs throw `VortexException` by default. Opt in to |
| 46 | +passthrough mode to read such files without failing: |
| 47 | + |
| 48 | +```java |
| 49 | +EncodingRegistry registry = EncodingRegistry.loadAll().allowUnknown(); |
| 50 | +try (VortexReader vf = VortexReader.open(path, registry)) { |
| 51 | + // columns with unknown encodings are returned as UnknownArray |
| 52 | +} |
| 53 | +``` |
| 54 | + |
| 55 | +## S3 Fixture Status (v0.72.0) |
| 56 | + |
| 57 | +Cross-language round-trips tested against Rust-written fixture files hosted at |
| 58 | +`s3://vortex-compat-fixtures/v0.72.0/arrays/`. |
| 59 | + |
| 60 | +| Fixture | Status | |
| 61 | +|-------------------------------------|--------| |
| 62 | +| `primitives.vortex` | ✅ | |
| 63 | +| `alp.vortex` | ✅ | |
| 64 | +| `bitpacked.vortex` | ✅ | |
| 65 | +| `booleans.vortex` | ✅ | |
| 66 | +| `constant.vortex` | ✅ | |
| 67 | +| `for.vortex` | ✅ | |
| 68 | +| `fsst.vortex` | ✅ | |
| 69 | +| `runend.vortex` | ✅ | |
| 70 | +| `sequence.vortex` | ✅ | |
| 71 | +| `varbin.vortex` | ✅ | |
| 72 | +| `struct_nested.vortex` | ✅ | |
| 73 | +| `null.vortex` | ✅ | |
| 74 | +| `bytebool.vortex` | ✅ | |
| 75 | +| `zigzag.vortex` | ✅ | |
| 76 | +| `datetime.vortex` | ✅ | |
| 77 | +| `dict.vortex` | ✅ | |
| 78 | +| `sparse.vortex` | ✅ | |
| 79 | +| `varbinview.vortex` | ✅ | |
| 80 | +| `chunked.vortex` | ✅ | |
| 81 | +| `rle.vortex` | ✅ | |
| 82 | +| `alprd.vortex` | ✅ | |
| 83 | +| `decimal.vortex` | ✅ | |
| 84 | +| `decimal_byte_parts.vortex` | ✅ | |
| 85 | +| `datetimeparts.vortex` | ✅ | |
| 86 | +| `list.vortex` | ✅ | |
| 87 | +| `listview.vortex` | ✅ | |
| 88 | +| `fixed_size_list.vortex` | ✅ | |
| 89 | +| `zstd.vortex` | ✅ | |
| 90 | +| `tpch_lineitem.compact.vortex` | ✅ | |
| 91 | +| `tpch_lineitem.regular.vortex` | ✅ | |
| 92 | +| `tpch_orders.compact.vortex` | ✅ | |
| 93 | +| `tpch_orders.regular.vortex` | ✅ | |
| 94 | +| `pco.vortex` | ✅ | |
| 95 | +| `clickbench_hits_5k.compact.vortex` | ✅ | |
| 96 | +| `clickbench_hits_5k.regular.vortex` | ✅ | |
| 97 | +| `masked.vortex` | ❓ | No fixture in v0.72.0 | |
| 98 | +| `patched.vortex` | ❓ | No fixture in v0.72.0 | |
| 99 | +| `variant.vortex` | ❓ | No fixture in v0.72.0 | |
0 commit comments