Skip to content

fix(decoder): byte-exact sub-range (col_range) IDWT at component boundaries#447

Merged
osamu620 merged 1 commit into
mainfrom
fix/col-range-idwt-edge
Jun 24, 2026
Merged

fix(decoder): byte-exact sub-range (col_range) IDWT at component boundaries#447
osamu620 merged 1 commit into
mainfrom
fix/col-range-idwt-edge

Conversation

@osamu620

Copy link
Copy Markdown
Owner

Summary

The reuse-path sub-range horizontal IDWT (the column-range / JPIP-zoom decode) was not byte-exact with full-width decode when a window edge fell on a component boundary — which surfaced as the scalar build's cr_ht_06_reuse conformance failure.

idwt_1d_row_inplace_range clamped its lifting-pass range to [0, width-1]. At a component boundary (col_lo == u0 or col_hi == u1) that dropped the PSE-region writes the full-width kernel makes, leaving the first/last ~2-3 output columns wrong; the per-level error then compounded down the streaming-IDWT cascade. Subsampled chroma was worst hit (its window clamps to the component edge at every level). Interior window edges, which keep a finite real-neighbour margin, were already exact.

Fix: at a component boundary, run the lifting pass to its natural extent (writing into the row's PSE prefix/suffix slack exactly as the full-width kernel does) instead of clamping; keep the finite margin only for interior edges. The change is in the shared scalar/interleave fallback that every ISA build uses for coarse DWT levels (N < 16), so it covers all ISAs.

Verification

  • 449/449 conformance on AVX2 and scalar (scalar previously failed cr_ht_06_reuse).
  • Full-image decode is byte-identical (the full-width fast path is untouched; the change only affects an actual sub-range window).

Deferred follow-ups

Two related improvements were prototyped and intentionally left out of this PR. They exposed a separate, pre-existing latent fault in the windowed-decode path that reproduces only on certain CI toolchains (GCC 13 / MSVC / AppleClang, non-deterministically) and not under local AddressSanitizer (pooled allocator disabled, so all buffers redzoned — decoder is memory-clean) or valgrind, so they need dedicated investigation:

  • Wiring set_col_range to actually apply on the non-reuse line-based path (it is currently a silent no-op there, because it runs before init_line_decode builds the tree).
  • Broader edge / narrow-window coverage in col_range_compare's reuse probe.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WaH1Ay9hPfnqfHsJ7wxh8M

…daries

The reuse-path sub-range horizontal IDWT (idwt_1d_row_inplace_range) clamped
its lifting-pass range to [0, width-1]. When a window edge coincided with a
component boundary (col_lo == u0 or col_hi == u1) this dropped the PSE-region
writes the full-width kernel makes, leaving the first/last ~2-3 output columns
wrong; the per-level error then compounded down the streaming-IDWT cascade, so
a column-range decode was not byte-exact with full-width decode for windows
touching a component edge (the scalar build's cr_ht_06_reuse failure; worst on
subsampled chroma, whose window clamps to the component edge at every level).
Interior window edges, which keep a finite real-neighbour margin, were exact.

At a component boundary, run the lifting pass to its natural extent (INT32_MIN /
INT32_MAX make the kernel's clip resolve to the natural start/end, which writes
into the row's PSE prefix/suffix slack exactly as the full-width kernel does);
keep the finite margin only for interior edges. The fix is in the shared
scalar/interleave fallback, which every ISA build uses for coarse DWT levels.

449/449 conformance on AVX2 and scalar (scalar previously failed
cr_ht_06_reuse); full-image decode is byte-identical (the full-width fast path
is untouched).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WaH1Ay9hPfnqfHsJ7wxh8M

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a correctness mismatch in the decoder’s windowed (column-range / JPIP zoom) horizontal IDWT on the reuse path when the requested window edge lands at/near a component boundary, ensuring byte-exact output versus full-width decode and preventing error compounding in the streaming IDWT cascade.

Changes:

  • Adjust sub-range horizontal IDWT range selection so that when a window edge is within filter support of a component boundary, the lifting pass runs to its natural extent (matching full-width PSE-region writes).
  • Preserve the finite-neighbor “widen” margin behavior for interior window edges.
  • Add <cstdint> include to support INT32_MIN/INT32_MAX sentinels used by the range-clip logic.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@osamu620 osamu620 merged commit 679e873 into main Jun 24, 2026
10 checks passed
@osamu620 osamu620 deleted the fix/col-range-idwt-edge branch June 24, 2026 06:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants