fix(cbq): correct silent N to A corruption on CBQ decode (#94)

[BenjaminDEMAILLE]

Fixes #94.

Problem

A .cbq written with N bases does not round-trip: on decode every N comes back as A. In debug builds this panics inside sucds (elias_fano/iter.rs:23, debug_assert_ne!(num_ones, 0)); in the shipped release config it is silent, returning a real A where the read had a no-call N. For methylation/variant consumers this is a silent correctness violation.

Root cause

CBQ stores N as a 2-bit placeholder A plus a sucds EliasFano index of the N-positions, backfilled on decode. In the serial Reader path, ColumnarBlock::decompress_columns() did:

self.ef_bytes.resize(self.len_nef, 0);              // pre-fills len_nef ZERO bytes
copy_decode(self.z_npos.as_slice(), &mut self.ef_bytes)?;  // io::Write for Vec<u8> APPENDS

copy_decode's destination is a &mut Vec<u8>, whose io::Write impl appends. So ef_bytes became [0u8; len_nef] ++ [real EF bytes], and EliasFano::deserialize_from read the leading zeros, producing an empty index (num_ones() == 0). backfill_npos() then panicked in debug or silently no-oped in release, leaving the placeholder As.

This is not a sucds bug: its serialize/deserialize are symmetric. The other columns are fine because numeric columns decode into cast_slice_mut(&mut vec) (an in-place &mut [u8]), and headers/qual append into an already-cleared buffer with no resize. Only ef_bytes did both resize(_, 0) and append.

The mmap / process_parallel path (decompress_from_bytes, using dctx.decompress) writes at offset 0 via zstd-safe's WriteBuf, so it was already correct.

Fix

Clear ef_bytes before copy_decode so it holds exactly the decompressed bytes, mirroring the headers/qual columns. Also reset len_nef in ColumnarBlock::clear() for consistent block reuse.

No on-disk format change: z_npos and len_nef are written correctly and already read back correctly by the mmap path, so FILE_VERSION is unchanged and existing files (N-free or not) are unaffected.

Tests

Existing CBQ tests were ACGT-only, which is why this shipped. Added N-nucleotide round-trip coverage (leading / interior / trailing / all-N, single and multi-block, with and without quality+headers) over both the serial Reader path and the parallel MmapReader/process_parallel path. Verified these new tests fail (serial) on the pre-fix code and pass after; full suite green in debug and release; the exact issue repro prints NACGTACGT, NNNNNNNN, ACGTNACGT unchanged in both profiles.

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request fixes a bug (issue #94) where N-nucleotides were silently dropped during decoding. Instead of resizing self.ef_bytes with zeros (which caused subsequent decodes to append and fail to reconstruct the Elias-Fano index), the code now clears self.ef_bytes before decoding. Additionally, self.len_nef is now properly reset when clearing the block, and comprehensive serial and parallel round-trip tests have been added. The reviewer suggested a performance improvement to pre-allocate capacity for self.ef_bytes using reserve after clearing it, avoiding unnecessary reallocations during decompression.

[noamteyssier]

done with #96