Skip to content

Fix/issue 94#96

Merged
noamteyssier merged 4 commits into
mainfrom
fix/issue-94
Jul 1, 2026
Merged

Fix/issue 94#96
noamteyssier merged 4 commits into
mainfrom
fix/issue-94

Conversation

@noamteyssier

Copy link
Copy Markdown
Collaborator

No description provided.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request optimizes Elias-Fano decompression in src/cbq/core/block.rs by replacing resize with reserve on ef_bytes before decoding, and adds a new unit test in src/cbq/write.rs to verify round-tripping of sequences containing 'N's. The review feedback correctly points out that self.ef_bytes should be explicitly cleared before calling reserve and copy_decode to prevent appending to stale data in case the vector is reused.

Comment thread src/cbq/core/block.rs
// decompress npos
if !self.z_npos.is_empty() {
self.ef_bytes.resize(self.len_nef, 0);
self.ef_bytes.reserve(self.len_nef);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To ensure defensive programming and idempotency, self.ef_bytes should be cleared before calling copy_decode. Since copy_decode takes &mut self.ef_bytes as impl std::io::Write, it appends the decompressed bytes to the vector. If self.ef_bytes is not cleared first, any pre-existing data in the vector (for example, if decompress_columns is called multiple times or if the buffer wasn't fully cleared) will remain at the beginning of the vector, causing EliasFano::deserialize_from to read stale/invalid data or fail.

Suggested change
self.ef_bytes.reserve(self.len_nef);
self.ef_bytes.clear();
self.ef_bytes.reserve(self.len_nef);

@noamteyssier noamteyssier merged commit fc5889d into main Jul 1, 2026
28 checks passed
@noamteyssier noamteyssier deleted the fix/issue-94 branch July 1, 2026 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant