Skip to content

fix: RNTuple footer leak on extend, broken WritableNTuple.compression, directory __delitem__ KeyError#1650

Open
henryiii wants to merge 2 commits into
mainfrom
fix-rntuple-writing-writable
Open

fix: RNTuple footer leak on extend, broken WritableNTuple.compression, directory __delitem__ KeyError#1650
henryiii wants to merge 2 commits into
mainfrom
fix-rntuple-writing-writable

Conversation

@henryiii

@henryiii henryiii commented Jun 10, 2026

Copy link
Copy Markdown
Member

🤖 AI text below 🤖

What was broken

1. RNTuple extend() leaked footer-sized regions into FreeSegments on every call

NTuple.extend in _cascadentuple.py called freesegments.release(start, start + old_footer_key.allocation), but Key.allocation is just Key.num_bytes (the ~70-byte key record header). The full allocated region is num_bytes + compressed_bytes. Every call to extend() left the footer payload stranded in FreeSegments, causing permanent file bloat proportional to the number of extends. This parallels the TTree path in _cascadetree.py:487 which correctly uses num_bytes + compressed_bytes.

Reproducer: Create an RNTuple and call extend() 10 times — the file is larger than it needs to be and FreeSegments entries accumulate. After the fix the freed region is fully reclaimed.

2. WritableNTuple.compression getter/setter raised AttributeError

The property was copy-pasted verbatim from WritableTree but NTuple has no _branch_data; it always raised AttributeError: 'NTuple' object has no attribute '_branch_data'. The getter now returns the actual file-level compression from self._cascading._freesegments.fileheader.compression. The setter raises NotImplementedError with a clear message since per-RNTuple compression is not currently supported by the cascading implementation.

Reproducer:

with uproot.recreate("f.root") as f:
    f['t'] = {'x': np.array([1.0], 'f4')}
    print(f['t'].compression)  # was AttributeError, now returns ZLIB(1)

3. WritableDirectory.__delitem__ raised AttributeError for nonexistent keys

_del called key.seek_location without checking whether get_key() returned None. Deleting a nonexistent key raised AttributeError: 'NoneType' object has no attribute 'seek_location' instead of a KeyError subclass as the MutableMapping contract requires. Now raises uproot.exceptions.KeyInFileError (a KeyError subclass), exactly like _get.

Reproducer:

with uproot.recreate("f.root") as f:
    f['t'] = {'x': np.array([1], 'i4')}
    del f['nonexistent']  # was AttributeError, now KeyInFileError

4. Dead/broken code removed

  • Ntuple_PageLink class: unreferenced anywhere; its serialize packed two values into a single-field struct → guaranteed struct.error.
  • NTuple.branch_types and NTuple.field_name properties: referenced self._branch_types / self._field_name which are never set on NTuple → guaranteed AttributeError.
  • NTuple_Footer.__repr__: referenced nonexistent self.metadata_block_envelope_links → guaranteed AttributeError.
  • NTuple.write_updates: a one-line stub (sink.flush()) that is never called from anywhere.
  • Tree.branch_types in _cascadetree.py: referenced self._branch_types which is never set (branch types are stored in _branch_data).

What changed

  • src/uproot/writing/_cascadentuple.py: fix footer release extent; remove 5 dead/broken code items
  • src/uproot/writing/_cascadetree.py: remove Tree.branch_types dead property
  • src/uproot/writing/writable.py: fix WritableNTuple.compression; fix WritableDirectory._del null check

Skipped

Nothing was skipped. All 4 categories of findings were verified and fixed.

Test results

29 tests passed, 6 skipped (ROOT not installed) across the relevant test files.

Part of #1646.

henryiii added 2 commits June 10, 2026 14:56
…, directory __delitem__ KeyError

- _cascadentuple.py extend(): release full key+payload region (num_bytes+compressed_bytes) instead of just the key header (allocation/num_bytes), preventing footer-sized FreeSegments leak on every extend call
- WritableNTuple.compression getter/setter: replace copied-from-WritableTree code that referenced nonexistent _branch_data; getter now returns actual file-level compression; setter raises NotImplementedError with a clear message
- WritableDirectory._del: check get_key() result for None and raise KeyInFileError, fixing AttributeError on nonexistent key deletion (MutableMapping contract)
- Remove dead/broken NTuple code: Ntuple_PageLink class (unused, broken serialize), NTuple.branch_types and NTuple.field_name properties (referencing unset attributes), NTuple_Footer.__repr__ (referenced nonexistent metadata_block_envelope_links), NTuple.write_updates (stub never called)
- Remove Tree.branch_types property in _cascadetree.py (referencing unset _branch_types)

Assisted-by: ClaudeCode:claude-sonnet-4-6
Add tests/test_1650_rntuple_writing_bugfixes.py covering:
- RNTuple extend() FreeSegments leak: verify file stays small and is readable after 10 extends
- WritableNTuple.compression getter: returns file-level compression without AttributeError
- WritableNTuple.compression setter: raises NotImplementedError as expected
- WritableDirectory.__delitem__ on nonexistent key: raises KeyError (was AttributeError)
- WritableDirectory.__delitem__ on nonexistent key: specifically raises KeyInFileError

Assisted-by: ClaudeCode:claude-sonnet-4-6
@codecov

codecov Bot commented Jun 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 73.86%. Comparing base (1c06db0) to head (da0aef6).
⚠️ Report is 16 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/uproot/writing/_cascadentuple.py 0.00% 1 Missing ⚠️

❌ Your patch check has failed because the patch coverage (80.00%) is below the target coverage (98.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
Files with missing lines Coverage Δ
src/uproot/writing/_cascadetree.py 83.78% <ø> (+0.07%) ⬆️
src/uproot/writing/writable.py 80.10% <100.00%> (+2.11%) ⬆️
src/uproot/writing/_cascadentuple.py 88.96% <0.00%> (+0.99%) ⬆️

... and 1 file with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant