Skip to content

Fix arm_transpose_conv_s8 row misalignment when pad_h % stride_h != 0#233

Open
94xhn wants to merge 2 commits into
ARM-software:mainfrom
94xhn:fix/transpose-conv-s8-pad-stride-row-scheduling
Open

Fix arm_transpose_conv_s8 row misalignment when pad_h % stride_h != 0#233
94xhn wants to merge 2 commits into
ARM-software:mainfrom
94xhn:fix/transpose-conv-s8-pad-stride-row-scheduling

Conversation

@94xhn

@94xhn 94xhn commented Jul 12, 2026

Copy link
Copy Markdown

Summary

arm_transpose_conv_s8() streams input rows through a rolling circular output-row buffer, flushing finished rows to the output as it goes. The number of rows to flush after processing input row j was decided by a binary gate:

if (skip_rows_top == 0) { flush stride_y rows }

skip_rows_top is MAX(0, pad_y - j * stride_y), i.e. it only becomes zero once input row j has advanced past the top padding. Before that point the gate flushes nothing at all — even on the specific row where the padding region ends part-way through — so it never flushes a partial count.

That's correct only when pad_y % stride_y == 0, because in that case the row where padding ends coincides exactly with a full stride_y-row boundary, so "flush 0 until skip_rows_top==0, then flush stride_y every row" happens to be exactly right.

When pad_y % stride_y != 0, the transition row should flush a partial count (between 1 and stride_y - 1), not 0. Skipping it permanently shifts every later flush by one buffer position, so subsequent input rows accumulate into the wrong rolling-buffer row, and the deficit is never made up: one full output row is left completely unwritten (still holding whatever the caller's buffer contained beforehand), while other rows mix contributions meant for their neighbour.

Fixes #230.

Fix

Lines ~141-153 of Source/ConvolutionFunctions/arm_transpose_conv_s8.c: replaced the binary gate with the exact number of rows that just became final, using the same MAX(0, ...) padding-clip idiom already used elsewhere in this function for skip_rows_top/skip_rows_bottom:

rows_to_flush = MAX(0, (j + 1) * stride_y - pad_y) - MAX(0, j * stride_y - pad_y)

This telescopes to stride_y in steady state and to 0 while still fully inside the padding region — exactly matching the old behaviour whenever pad_y % stride_y == 0 (provable algebraically, and confirmed empirically below) — while correctly producing a partial count on the transition row otherwise.

The identical binary-gate pattern also appeared in the NULL-bias reset branch inside the same flush block; it has the same fix applied.

Also bumped the file's $Date/$Revision header (V.2.0.0 -> V.2.0.1) per the repo's versioning convention (Version & Date section of the Contribution Guideline).

Horizontal (pad_x) misalignment is not affected by this bug and needed no change: a full input row's horizontal accumulation completes within a single arm_nn_transpose_conv_row_s8_s32 call, so there is no cross-call scheduling state on that axis. transpose_conv_3 already exercises pad_x % stride_x != 0 and passes before and after this change.

Verification

All verification was done by compiling the real, unmodified library source (not a reimplementation of the logic under test) standalone on host with gcc, and running it directly — no simulator, no modification to the Unity test harness.

  1. Reproduced the reported bug against unmodified upstream arm_transpose_conv_s8.c + arm_nn_transpose_conv_row_s8_s32.c using the configuration from the issue (input 4x4x8, filter 4x4x4x8 (out_ch), stride 2x2, pad 1x1, pad % stride = 1): 152/256 output elements wrong (59.4%), matching the issue report exactly.

  2. Poison-fill test: filled the output buffer with a 0xAA sentinel before calling the unmodified function. Output row 7 (the last of 8 rows) came back entirely as sentinel bytes — proof the function returns having left part of the documented output tensor completely unwritten, not merely numerically wrong.

  3. Independent reference implementation: wrote a from-scratch reference for transposed convolution using the direct "gather/scatter" mathematical definition (a full accumulate-into-a-plain-array approach, not derived from this file's rolling-buffer scheduling), reusing only the library's own unmodified arm_nn_requantize() for rounding. This reference was itself validated against ARM's own official TensorFlow-generated golden data for the existing transpose_conv_1 unit test (nonzero input/output offsets, per-channel multiplier/shift, batch size 2): 0/1944 elements differ, establishing it as a trustworthy correctness oracle independent of the third-party issue's own attached golden file.

  4. With the fix applied, library output on the reported configuration matches the validated reference exactly: 0/256 elements differ (previously 152/256).

  5. No regressions — re-ran all 4 existing official unit tests whose in_ch routes through this basic (non-MVE-optimized) path (in_ch <= REVERSE_TCOL_EFFICIENT_THRESHOLD == 16), against their official TensorFlow-generated golden data, comparing the compiled binaries' output directly (bypassing the Unity harness):

    • transpose_conv_1: 0/1944 differ
    • transpose_conv_2: 0/2016 differ
    • transpose_conv_3: 0/1050 differ (this one already exercises pad_x % stride_x != 0, confirming the horizontal axis is untouched by this fix)
    • reverse_transpose_conv_4: 0/40 differ

    All four already passed before this change and still pass after.

  6. Synthetic fuzz sweep: 6 additional independently-chosen (pad, stride, filter, size, seed) combinations against the validated reference. Every case with pad_y % stride_y == 0 gave 0 mismatches on both old and new code (identical, as the algebra predicts they must be). Every case with pad_y % stride_y != 0 gave massive failure on the old code (97-100% of elements wrong) and 0 mismatches on the new code. One case isolated pad_x % stride_x != 0 with pad_y % stride_y == 0: both old and new gave 0 mismatches, confirming horizontal misalignment was already handled correctly and is untouched by this fix.

  7. Why this escaped CI: all 5 existing official test cases that exercise this function (transpose_conv_1..4, reverse_transpose_conv_4) happen to use padding/stride pairs where pad_y % stride_y == 0 (2%2, 0%1, 0%5, 0%1, 0%1) — exactly the condition under which the old and new code are provably identical. That is why a bug of this shape was never caught by the existing suite; it isn't a gap in test execution, it's a gap in the parameter space the existing fixtures happen to cover.

Two separate, pre-existing issues unrelated to this bug were noticed during fuzzing (the function crashing when the filter is smaller than the stride, and when padding exceeds the filter size); both reproduce identically on unmodified upstream and on this patched branch, so they are out of scope for this PR and are not addressed here.

Disclosure

This investigation, the reproduction/verification harnesses, and the fix implementation were done with Claude AI assistance (root-cause derivation, host-compilable test harnesses against the real library source, and the independent reference implementation used as the correctness oracle). I reviewed the diagnosis, the fix, and all verification results myself before submitting this PR, and I'm happy to answer questions or make changes based on maintainer feedback.

94xhn and others added 2 commits July 12, 2026 22:30
arm_transpose_conv_s8() streams input rows through a rolling circular
output-row buffer, flushing finished rows to the output as it goes. The
number of rows to flush after processing input row j was decided by a
binary gate:

    if (skip_rows_top == 0) { flush stride_y rows }

skip_rows_top is MAX(0, pad_y - j * stride_y), i.e. it is only zero once
input row j has advanced past the top padding. Before that point the
gate flushes nothing at all, even on the specific row where the padding
region ends part-way through -- so it never flushes a *partial* count.

That is fine when pad_y is a multiple of stride_y, because the input row
where padding ends is also the row where a full stride_y rows first
become available, so "flush 0 until skip_rows_top==0, then flush
stride_y every row" is exactly correct.

When pad_y % stride_y != 0, the transition row should flush a partial
count (between 1 and stride_y-1 rows), not 0. Skipping it permanently
shifts every later flush by one buffer position, so subsequent input
rows accumulate into the wrong rolling-buffer row, and the deficit is
never made up: one full output row is left completely unwritten (still
holding whatever the caller's buffer contained beforehand), while every
other row mixes contributions meant for its neighbour.

Root cause, verified empirically (not just by inspection):
- Added temporary instrumentation to Source/ConvolutionFunctions/arm_transpose_conv_s8.c
  and traced the exact flush schedule for input=4x4x8, filter=4x4x4x8,
  stride=2x2, pad=1x1 (output 8x8x4): total rows flushed by the main
  loop + the "leftover rows" tail came to 7, not the required 8.
- Poisoned the output buffer (0xAA fill) before calling the function
  with unmodified upstream code: output row 7 (the last of 8) came back
  entirely as poison bytes -- proof the function returns with part of
  the documented output tensor never written, not just wrong values.

Fix: replace the binary gate with the exact number of rows that just
became final, using the same MAX(0, ...) padding-clip idiom already
used elsewhere in this function for skip_rows_top/skip_rows_bottom:

    rows_to_flush = MAX(0, (j+1)*stride_y - pad_y) - MAX(0, j*stride_y - pad_y)

This telescopes to stride_y in steady state and to 0 while still fully
inside the padding region, exactly matching the old behaviour whenever
pad_y % stride_y == 0 (provable algebraically, and confirmed by the
unit tests below), while correctly producing a partial count on the
transition row otherwise.

Verification
------------
1. Reproduced the reported bug against unmodified upstream
   arm_transpose_conv_s8.c + arm_nn_transpose_conv_row_s8_s32.c, compiled
   standalone on host (gcc) and run directly (no simulator): 152/256
   output elements wrong, matching the issue's reported 59.4% exactly.

2. Wrote an independent reference implementation of transposed
   convolution (direct "gather" form from the mathematical definition,
   not derived from this file's rolling-buffer scheduling), reusing only
   the library's own unmodified arm_nn_requantize() for rounding. This
   reference was itself validated against ARM's own official
   TensorFlow-generated golden data for the existing transpose_conv_1
   unit test (nonzero input/output offsets, per-channel multiplier/shift,
   batch size 2): 0/1944 elements differ.

3. With the fix applied, the library's output on the reported
   configuration matches that independently-validated reference exactly:
   0/256 elements differ (previously 152/256).

4. No regressions: re-ran all four existing unit tests that exercise
   this function (transpose_conv_1/2/3 and reverse_transpose_conv_4 --
   the ones with in_ch <= REVERSE_TCOL_EFFICIENT_THRESHOLD so the
   wrapper dispatches here) against their official TensorFlow-generated
   golden data. All still match exactly:
     transpose_conv_1: 0/1944 differ
     transpose_conv_2: 0/2016 differ
     transpose_conv_3: 0/1050 differ
     reverse_transpose_conv_4: 0/40 differ
   (transpose_conv_3 already exercises pad_x % stride_x != 0 on the
   horizontal axis with no issue -- horizontal accumulation completes
   within a single row-processing call, so it never needed this kind of
   incremental flush schedule; only the vertical/row direction streams
   across multiple input rows and was affected.)

5. Additionally swept 6 synthetic (pad, stride, filter, offset)
   combinations against the validated reference -- multi-row padding
   ramps, combined horizontal+vertical misalignment, stride=1 -- all
   0 mismatches with the fix, all previously failing on unmodified
   upstream for the pad % stride != 0 cases among them.

   Two unrelated, pre-existing configurations (filter smaller than
   stride; padding larger than the filter) were found to crash
   identically on both unmodified upstream and the patched version, so
   they are a separate issue and intentionally out of scope here.

Fixes ARM-software#230.

Disclosure: This fix was investigated and implemented with Claude AI
assistance (reproduction harness, root-cause tracing, and the reference
implementation used to validate correctness). All findings and the
patch were reviewed by the human submitter before this commit.

Signed-off-by: 94xhn <87560781+94xhn@users.noreply.github.com>
The pinned formatting check expects the rows_to_flush assignment on one line. Match the repository style so CI can evaluate the functional fix.

Constraint: Upstream CI pins clang-format-18
Confidence: high
Scope-risk: narrow
Reversibility: clean
Tested: Repository .clang-format dry-run with clang-format 14; git diff --check
Not-tested: clang-format-18 unavailable locally; upstream CI will verify the pinned version
Related: ARM-software#233
@ArmRyan

ArmRyan commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Hey @94xhn, Thanks for the contribution! I will need to take this through our internal CI and try to understand it better before I approve. Thanks for the detailed information :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

arm_transpose_conv_s8: wrong output when pad_h % stride_h != 0

2 participants