Fix/issue 94#96
Conversation
There was a problem hiding this comment.
Code Review
This pull request optimizes Elias-Fano decompression in src/cbq/core/block.rs by replacing resize with reserve on ef_bytes before decoding, and adds a new unit test in src/cbq/write.rs to verify round-tripping of sequences containing 'N's. The review feedback correctly points out that self.ef_bytes should be explicitly cleared before calling reserve and copy_decode to prevent appending to stale data in case the vector is reused.
| // decompress npos | ||
| if !self.z_npos.is_empty() { | ||
| self.ef_bytes.resize(self.len_nef, 0); | ||
| self.ef_bytes.reserve(self.len_nef); |
There was a problem hiding this comment.
To ensure defensive programming and idempotency, self.ef_bytes should be cleared before calling copy_decode. Since copy_decode takes &mut self.ef_bytes as impl std::io::Write, it appends the decompressed bytes to the vector. If self.ef_bytes is not cleared first, any pre-existing data in the vector (for example, if decompress_columns is called multiple times or if the buffer wasn't fully cleared) will remain at the beginning of the vector, causing EliasFano::deserialize_from to read stale/invalid data or fail.
| self.ef_bytes.reserve(self.len_nef); | |
| self.ef_bytes.clear(); | |
| self.ef_bytes.reserve(self.len_nef); |
No description provided.