|
49 | 49 |
|
50 | 50 | ## Encodings |
51 | 51 |
|
| 52 | +### Implemented |
| 53 | + |
| 54 | +| Encoding ID | Class | Decode | Encode | Encode effort | Dtypes supported | |
| 55 | +|------------------------|---------------------|--------|--------|---------------|------------------| |
| 56 | +| `vortex.primitive` | `PrimitiveEncoding` | ✅ | ✅ | — | all `PType` (I8–I64, U8–U64, F32, F64) | |
| 57 | +| `vortex.bool` | `BoolEncoding` | ✅ | ✅ | — | Bool (bit-packed) | |
| 58 | +| `vortex.dict` | `DictEncoding` | ✅ | ✅ | — | Primitive (VarBin via dict.vortex blocked by VarBinView) | |
| 59 | +| `fastlanes.delta` | `DeltaEncoding` | ✅ | ✅ | — | integer PTypes | |
| 60 | +| `fastlanes.bitpacked` | `BitpackedEncoding` | ✅ | ✅ | — | unsigned integer PTypes | |
| 61 | +| `vortex.null` | `NullEncoding` | ✅ | ✅ | — | Null | |
| 62 | +| `vortex.bytebool` | `ByteBoolEncoding` | ✅ | trivial | `boolean[]` → 1 byte/elem | Bool | |
| 63 | +| `vortex.zigzag` | `ZigZagEncoding` | ✅ | trivial | `(v<<1)^(v>>63)`, delegate | signed integer PTypes | |
| 64 | +| `fastlanes.for` | `FrameOfReferenceEncoding` | ✅ | low | find min, emit deltas child | integer PTypes | |
| 65 | +| `vortex.runend` | `RunEndEncoding` | ✅ | low | scan runs → ends + values arrays | Primitive, Utf8/Binary, Bool | |
| 66 | +| `vortex.constant` | `ConstantEncoding` | ✅ | low | validate uniform, emit `ScalarValue` proto | Primitive, Utf8, Binary, Bool, Null, Decimal, Extension | |
| 67 | +| `vortex.sparse` | `SparseEncoding` | ✅ | medium | collect non-fill indices + values; needs fill-value detection | Primitive | |
| 68 | +| `vortex.varbin` | `VarBinEncoding` | ✅ | medium | offsets buf + bytes buf + `VarBinMetadata` proto | Utf8, Binary | |
| 69 | +| `vortex.sequence` | `SequenceEncoding` | ✅ | ❌ stub | detect arithmetic progression (base + i×step) | Primitive | |
| 70 | +| `vortex.struct` | `StructEncoding` | ✅ | medium | encode each field, emit children | Struct | |
| 71 | +| `vortex.ext` | `ExtEncoding` | ✅ | ✅ | — | Extension | |
| 72 | +| `vortex.alp` | `AlpEncoding` | ✅ | hard | ALP float quantization + patch residuals | F64, F32 | |
| 73 | +| `vortex.fsst` | `FsstEncoding` | ✅ | hard | FSST symbol-table building | Utf8, Binary | |
| 74 | +| `vortex.varbinview` | `VarBinViewEncoding` | ✅ | hard | 16-byte view layout + inline vs heap split | Utf8, Binary | |
| 75 | +| `vortex.pco` | `PcoEncoding` | ❌ stub | ❌ | very hard — ANS + bin tokenization not ported | Primitive | |
| 76 | + |
| 77 | +### Missing |
| 78 | + |
| 79 | +| Encoding ID | Effort | Unblocks | |
| 80 | +|----------------------------|---------|-------------------------------------------------| |
| 81 | +| `vortex.chunked` | medium | `chunked.vortex` (segment-level chunked array) | |
| 82 | +| `fastlanes.rle` | medium | `rle.vortex` | |
| 83 | +| `vortex.alprd` | medium | `alprd.vortex` | |
| 84 | +| `vortex.decimal` | medium | `decimal.vortex` | |
| 85 | +| `vortex.decimal_byte_parts`| medium | `decimal_byte_parts.vortex` | |
| 86 | +| `vortex.datetimeparts` | medium | `datetimeparts.vortex` | |
| 87 | +| `vortex.list` | hard | `list.vortex` (needs list array model) | |
| 88 | +| `vortex.listview` | hard | `listview.vortex` | |
| 89 | +| `vortex.fixed_size_list` | hard | `fixed_size_list.vortex` | |
| 90 | +| `vortex.zstd` | hard | `zstd.vortex` (needs zstd native lib) | |
| 91 | +| `vortex.pco` (full) | very hard | `pco.vortex`, `tpch_*.vortex`, `clickbench_*.vortex` | |
| 92 | + |
| 93 | +### S3 Fixture Status (`v0.72.0/arrays/`) |
| 94 | + |
| 95 | +| Fixture | Status | Blocker | |
| 96 | +|----------------------------------|--------|-------------------------------| |
| 97 | +| `primitives.vortex` | ✅ | | |
| 98 | +| `alp.vortex` | ✅ | | |
| 99 | +| `bitpacked.vortex` | ✅ | | |
| 100 | +| `booleans.vortex` | ✅ | | |
| 101 | +| `constant.vortex` | ✅ | | |
| 102 | +| `for.vortex` | ✅ | | |
| 103 | +| `fsst.vortex` | ✅ | | |
| 104 | +| `runend.vortex` | ✅ | | |
| 105 | +| `sequence.vortex` | ✅ | | |
| 106 | +| `varbin.vortex` | ✅ | | |
| 107 | +| `struct_nested.vortex` | ✅ | | |
| 108 | +| `null.vortex` | ✅ | | |
| 109 | +| `bytebool.vortex` | ✅ | | |
| 110 | +| `zigzag.vortex` | ✅ | | |
| 111 | +| `datetime.vortex` | ✅ | | |
| 112 | +| `dict.vortex` | ✅ | | |
| 113 | +| `sparse.vortex` | ✅ | | |
| 114 | +| `varbinview.vortex` | ✅ | | |
| 115 | +| `chunked.vortex` | ❌ | `vortex.chunked` at segment level | |
| 116 | +| `rle.vortex` | ❌ | `fastlanes.rle` missing | |
| 117 | +| `alprd.vortex` | ❌ | `vortex.alprd` missing | |
| 118 | +| `decimal.vortex` | ❌ | `vortex.decimal` missing | |
| 119 | +| `decimal_byte_parts.vortex` | ❌ | `vortex.decimal_byte_parts` missing | |
| 120 | +| `datetimeparts.vortex` | ❌ | `vortex.datetimeparts` missing | |
| 121 | +| `list.vortex` | ❌ | `vortex.list` + list array model | |
| 122 | +| `listview.vortex` | ❌ | `vortex.listview` missing | |
| 123 | +| `fixed_size_list.vortex` | ❌ | `vortex.fixed_size_list` missing | |
| 124 | +| `zstd.vortex` | ❌ | needs zstd native library | |
| 125 | +| `tpch_lineitem.compact.vortex` | ❌ | `vortex.pco` | |
| 126 | +| `tpch_lineitem.regular.vortex` | ❌ | `vortex.pco` | |
| 127 | +| `tpch_orders.compact.vortex` | ❌ | `vortex.pco` | |
| 128 | +| `tpch_orders.regular.vortex` | ❌ | `vortex.pco` | |
| 129 | +| `pco.vortex` | ❌ | `vortex.pco` | |
| 130 | +| `clickbench_hits_5k.compact.vortex` | ❌ | `vortex.pco` | |
| 131 | +| `clickbench_hits_5k.regular.vortex` | ❌ | `vortex.pco` | |
| 132 | + |
| 133 | +**Score: 18/35** (including `for.vortex` scanned separately from `scan_fixture_decodesAllRows`) |
| 134 | + |
0 commit comments