|
30 | 30 | - `pcodec` — float compression |
31 | 31 | - `fsst` — string compression (in-flight, see `FsstCodec.java`) |
32 | 32 |
|
33 | | -- [ ] **#7a `fastlanes.bitpacked` — patches (step 3)** |
34 | | - - Steps 1, 2, 4, 5 landed: protobuf metadata (tags 1+2), unified `unpack(buf, bitWidth, offset, T, rowCount)` using |
35 | | - FastLanes algorithm, encoder writes protobuf, round-trip test green. |
36 | | - - Step 3 still open: decode child slots (indices + values), overwrite output at patch indices. |
| 33 | +- [ ] **#7a Fix `fastlanes.bitpacked` — spec-compliant rewrite** |
| 34 | + - Root cause: current code guesses format by metadata byte size (9 = Java, 2 = JNI). Wrong. |
| 35 | + The spec always uses protobuf metadata regardless of writer origin. |
| 36 | + - **Spec** (from `encodings/fastlanes/src/bitpacking/vtable/mod.rs`): |
| 37 | + - Metadata: protobuf `BitPackedMetadata` — `bit_width u32` (tag 1), `offset u32` (tag 2, 0≤offset<1024), |
| 38 | + `patches PatchesMetadata` (tag 3, optional) |
| 39 | + - Buffer size: `ceil((len + offset) / 1024) * 128 * bit_width` bytes |
| 40 | + - FastLanes block layout: `LANES = 1024 / T` (T = element bit-width); `FL_ORDER = [0,4,2,6,1,5,3,7]`; logical |
| 41 | + index for `(row, lane)` = `FL_ORDER[row/8]*16 + (row%8)*128 + lane` — same formula for all types |
| 42 | + - Step 1: delete `decodeJni()` and `decodeJava()`; parse metadata as protobuf (tags 1+2) |
| 43 | + - Step 2: implement single unified `unpack(buf, bitWidth, offset, T, rowCount) → long[]` using the FastLanes |
| 44 | + algorithm above |
| 45 | + - Step 3: handle patches — decode child slots (indices + values), overwrite output at patch indices (**missing**; steps 1,2,4,5 done) |
| 46 | + - Step 4: align `encode()` to write protobuf metadata (tags 1+2) instead of the 9-byte custom format |
| 47 | + - Step 5: update `BitpackedCodecTest` for round-trip with spec-compliant metadata |
37 | 48 | - Reference: `spiraldb/vortex` `encodings/fastlanes/src/bitpacking/`, `spiraldb/fastlanes-rs` `src/bitpacking.rs` + |
38 | 49 | `src/macros.rs` |
39 | 50 |
|
|
0 commit comments