fix(decoder): byte-exact sub-range (col_range) IDWT at component boundaries#447
Merged
Conversation
…daries The reuse-path sub-range horizontal IDWT (idwt_1d_row_inplace_range) clamped its lifting-pass range to [0, width-1]. When a window edge coincided with a component boundary (col_lo == u0 or col_hi == u1) this dropped the PSE-region writes the full-width kernel makes, leaving the first/last ~2-3 output columns wrong; the per-level error then compounded down the streaming-IDWT cascade, so a column-range decode was not byte-exact with full-width decode for windows touching a component edge (the scalar build's cr_ht_06_reuse failure; worst on subsampled chroma, whose window clamps to the component edge at every level). Interior window edges, which keep a finite real-neighbour margin, were exact. At a component boundary, run the lifting pass to its natural extent (INT32_MIN / INT32_MAX make the kernel's clip resolve to the natural start/end, which writes into the row's PSE prefix/suffix slack exactly as the full-width kernel does); keep the finite margin only for interior edges. The fix is in the shared scalar/interleave fallback, which every ISA build uses for coarse DWT levels. 449/449 conformance on AVX2 and scalar (scalar previously failed cr_ht_06_reuse); full-image decode is byte-identical (the full-width fast path is untouched). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WaH1Ay9hPfnqfHsJ7wxh8M
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes a correctness mismatch in the decoder’s windowed (column-range / JPIP zoom) horizontal IDWT on the reuse path when the requested window edge lands at/near a component boundary, ensuring byte-exact output versus full-width decode and preventing error compounding in the streaming IDWT cascade.
Changes:
- Adjust sub-range horizontal IDWT range selection so that when a window edge is within filter support of a component boundary, the lifting pass runs to its natural extent (matching full-width PSE-region writes).
- Preserve the finite-neighbor “widen” margin behavior for interior window edges.
- Add
<cstdint>include to supportINT32_MIN/INT32_MAXsentinels used by the range-clip logic.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The reuse-path sub-range horizontal IDWT (the column-range / JPIP-zoom decode) was not byte-exact with full-width decode when a window edge fell on a component boundary — which surfaced as the scalar build's
cr_ht_06_reuseconformance failure.idwt_1d_row_inplace_rangeclamped its lifting-pass range to[0, width-1]. At a component boundary (col_lo == u0orcol_hi == u1) that dropped the PSE-region writes the full-width kernel makes, leaving the first/last ~2-3 output columns wrong; the per-level error then compounded down the streaming-IDWT cascade. Subsampled chroma was worst hit (its window clamps to the component edge at every level). Interior window edges, which keep a finite real-neighbour margin, were already exact.Fix: at a component boundary, run the lifting pass to its natural extent (writing into the row's PSE prefix/suffix slack exactly as the full-width kernel does) instead of clamping; keep the finite margin only for interior edges. The change is in the shared scalar/interleave fallback that every ISA build uses for coarse DWT levels (
N < 16), so it covers all ISAs.Verification
cr_ht_06_reuse).Deferred follow-ups
Two related improvements were prototyped and intentionally left out of this PR. They exposed a separate, pre-existing latent fault in the windowed-decode path that reproduces only on certain CI toolchains (GCC 13 / MSVC / AppleClang, non-deterministically) and not under local AddressSanitizer (pooled allocator disabled, so all buffers redzoned — decoder is memory-clean) or valgrind, so they need dedicated investigation:
set_col_rangeto actually apply on the non-reuse line-based path (it is currently a silent no-op there, because it runs beforeinit_line_decodebuilds the tree).col_range_compare's reuse probe.🤖 Generated with Claude Code
https://claude.ai/code/session_01WaH1Ay9hPfnqfHsJ7wxh8M