Skip to content

Commit 3bb9dc7

Browse files
docs(vortex-onpair): simplify decode buffer padding comments
The previous comments described try_decode_into's safe, buffer-derived bound and fast-path internals, which are implementation-specific and would not hold if the decode path switches to unsafe decode_into. Keep only the durable point: pad the output for the decoder's per-token over-copy and verify the exact size afterward. Signed-off-by: Francesco Gargiulo <francesco@spiraldb.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent f838da3 commit 3bb9dc7

2 files changed

Lines changed: 4 additions & 8 deletions

File tree

encodings/experimental/onpair/src/canonical.rs

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -95,10 +95,8 @@ pub(crate) fn onpair_decode_views(
9595
CompactDictionaryView::validate(array.dict_bytes().as_slice(), dict_offsets.as_slice())
9696
.map_err(|e| vortex_err!(InvalidArgument: "Invalid OnPair dictionary: {e}"))?;
9797

98-
// `try_decode_into` derives its write bound from the buffer itself, so it
99-
// is sound even when the file-borne `uncompressed_lengths` understate the
100-
// real decoded size; the extra DECODE_PADDING merely keeps it on the
101-
// all-over-copy fast path.
98+
// Pad the output with DECODE_PADDING to absorb the decoder's fixed
99+
// per-token over-copy; the exact decoded size is checked below.
102100
let mut out_bytes = ByteBufferMut::with_capacity(total_size + onpair::DECODE_PADDING);
103101
let written = onpair::try_decode_into(codes.as_slice(), dict, out_bytes.spare_capacity_mut())
104102
.map_err(|_| {

encodings/experimental/onpair/src/ops.rs

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -46,10 +46,8 @@ impl OperationsVTable<OnPair> for OnPair {
4646
.as_primitive()
4747
.as_::<usize>()
4848
.ok_or_else(|| vortex_err!("OnPair uncompressed_lengths[{index}] is null"))?;
49-
// `try_decode_into` derives its write bound from the buffer itself, so
50-
// it is sound even when the file-borne `uncompressed_lengths` child
51-
// understates the row's real decoded size; the extra DECODE_PADDING
52-
// merely keeps it on the all-over-copy fast path.
49+
// Pad the row buffer with DECODE_PADDING to absorb the decoder's fixed
50+
// per-token over-copy; the exact decoded size is checked below.
5351
let mut buf: Vec<u8> = Vec::with_capacity(len + onpair::DECODE_PADDING);
5452
let written = onpair::try_decode_into(codes.as_slice(), dict, buf.spare_capacity_mut())
5553
.map_err(|_| {

0 commit comments

Comments
 (0)