fix(flatfiles): reject zero-column and over-width header drift in block decode#1101
Merged
Conversation
…ck decode decode_block silently emitted zero rows for a header declaring zero columns over a non-empty DATA block, and silently clipped rows carrying more fields than the header column count. Both now return a typed decode error, matching the FPSS delta path width guard and the existing mid-row truncation guard so a drifted header fails loud. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR tightens flat-file FIT block decoding so drifted/hostile headers fail fast instead of silently producing empty output or clipped rows, aligning behavior with the FPSS delta decoding path and the existing mid-row truncation guard.
Changes:
- Rejects non-empty DATA blocks when
n_columns == 0with a typed codec decode error. - Rejects rows whose decoded field count exceeds the header column count (
n > n_columns) to prevent silent truncation. - Adds regression tests for both drifted-header shapes and documents the behavior change in both changelogs.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| thetadatadx-rs/src/flatfiles/decode.rs | Adds explicit guards for zero-column headers over non-empty blocks and over-width rows, plus regression tests. |
| docs-site/docs/changelog.md | Documents the new “fail loud” behavior for drifted headers in the docs changelog. |
| CHANGELOG.md | Documents the new “fail loud” behavior for drifted headers in the root changelog. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
decode_blockinthetadatadx-rs/src/flatfiles/decode.rspassed two drifted/hostile-header shapes silently, unlike the mid-row truncation guard that already fails loud on the same surface:fmt_count == 0over a non-empty DATA block decoded to zero rows with no error. A drifted zero-column header vanished silently.fpss/delta.rs) rejects the same width drift.Solution
Both now return
Error::decode_codec, matching the FPSS delta path's width guard and the existing truncation guard:n_columns == 0over a non-empty block returns a zero-column decode error (an empty block still returnsOkwith no rows).n > n_columnsreturns an over-width decode error. Delta rows legitimately carryn < n_columns(trailing carry-forward), so only wider rows are rejected.Regression tests cover both shapes.
Impact
A drifted flat-files header fails loud instead of producing silently wrong output. No behavior change on well-formed blocks; no version bump.
🤖 Generated with Claude Code