[PyTorch FE] Restore dynamic batch in window-reverse view shapes#36555
Closed
evkotov wants to merge 5 commits into
Closed
[PyTorch FE] Restore dynamic batch in window-reverse view shapes#36555evkotov wants to merge 5 commits into
evkotov wants to merge 5 commits into
Conversation
A model traced with a fixed batch bakes the batch into a view shape such as `x.view(B, H, W, -1)` as a literal leading constant, while the channel becomes the single `-1` infer slot. After the converted model is reshaped to a different batch, the frozen leading constant cannot propagate the new batch and the `-1` channel absorbs it instead, scrambling the data (seen as a large accuracy drop on SwinIR-style models at batch > 1). Add ReshapeBatchDimResolver, a normalize() MatcherPass that detects this shape Concat (positive-int leading constant, a single trailing `-1`, at least one dynamic interior dim, dynamic data batch) and rebuilds it per reshape so the leading dim becomes `-1` (inferred from the real element count) and the channel is pinned to the data tensor's last dimension via Gather(ShapeOf(data), -1). The rewrite is a no-op at the traced batch and tracks the real batch afterwards. Add a layer test covering rebatch accuracy, the structural rewrite, and a negative case proving the pass does not fire on ordinary reshapes.
…k-back Tracing a window-reverse style model at a fixed batch bakes the batch as a literal Constant in the leading slot of a view-shape Concat, with the channel as the trailing -1. After model.reshape to a new batch the frozen leading constant cannot carry it, so the -1 channel absorbs the extra batch and the windows are mis-partitioned, producing wrong values. ReshapeBatchDimResolver previously pinned the restored channel to a runtime Gather of the data's last dimension, which is only correct for window-reverse and corrupted ordinary reshapes whose -1 is a product of several dimensions (spatial flatten, attention head-merge). Gating on a static data last dim fixed those false positives but left the second of the two chained window-reverse views unrewritten, because OpenVINO does not propagate the first view's static channel through the intervening permute. Recover the channel structurally instead of relying on shape propagation: walk back from the reshape's data, taking a statically known last dimension directly, recursing through a last-axis-preserving Transpose, or reusing the channel of a reshape already selected for rewrite. Collect matches in topological order, guard each against the statically inferred output channel, then rewrite. The channel is pinned to a static Constant (batch-independent) and the leading dimension becomes -1. Add a window-reverse layer test that projects to a fixed channel width (as the real model does) so the static-channel path is exercised, and keep the negative tests for the ordinary-reshape idioms.
…hapes ReshapeBatchDimResolver could fire on an ordinary reshape that shares the window-reverse signature -- a leading positive-int constant, a dynamic interior dimension and a single trailing -1 -- and corrupt the result even at the traced batch. A view such as Linear(D, D) followed by reshape(1, T//2, -1) has a static data last dimension (so the channel walk-back resolves it) and a dynamic output last dimension (so the existing static-output-channel guard is vacuous), yet its -1 spans more than data's last dimension, so pinning the channel to that last dim and freeing the batch produces the wrong shape. Add a trailing-block value-preservation guard on the direct resolution path (channel recovered from the data's own static last dimension): the rewrite is applied only when it merely re-partitions data's leading dimension and keeps data's entire trailing block, i.e. the new shape's last rank_data-1 elements statically equal data's trailing dimensions. This is exactly the window-reverse semantics and rejects merge/split reshapes. The walk-back path (the second view, whose data last dimension is dynamic) is unaffected. Also skip the full-model re-inference when no reshape carries the structural signature: a cheap structural pre-scan (which needs no shape inference) gates the validation, so the common case of models without this pattern pays nothing. Add regression tests covering the traced-batch false positive, idempotency, the dynamic-channel scope boundary, and walk-back fragility.
…ts tests Reuse shared constant helpers (ov::op::util::has_constant_value / get_constant_value) in ReshapeBatchDimResolver instead of hand-rolled constant inspection, and add a static_last_dim() helper to replace three copy-pasted partial-shape last-dimension extractions. Behavior is unchanged. In the layer test, extract the duplicated scalar-constant walk and the compile/compare-to-torch boilerplate into module-level helpers, and merge the three structurally identical must-not-fire cases into one parametrized class. Note: merging those test classes renames some pytest node IDs (now under TestResolverDoesNotFire::test_not_fired_and_b1_ok).
… graphs Rewrite the ReshapeBatchDimResolver header doc to match the convention used by common transformations (before/after Unicode box-drawn graphs). Add a graph of the shape-Concat rewrite (leading Constant(B) -> Constant(-1), trailing Constant(-1) -> Constant(channel), interior dims kept) and a channel-recovery walk-back graph for the two chained window-reverse views. Documentation only.
mvafin
requested changes
Jun 25, 2026
Contributor
There was a problem hiding this comment.
This is not a good fix. The problem is in model, the model is not-reshapable. @evkotov please check the actual pytorch model code, does it really supports any other batch then 1? We also have SmartReshape transformations, maybe it can be covered there?
Contributor
Author
|
created #36574 with alternative approach with fix in SmartReshape |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
SwinIR-style
window_reversecomputes the batch as plain Python arithmetic:Under
torch.jit.tracetheint(...)collapses to a literal. In the IR theview shape becomes a Concat where only the spatial dims stay live (
aten::size),the leading batch is a baked
Constant, and the channel is the trailing-1:At the traced batch=1 this is correct:
After
model.reshape(batch=2)the input becomes(2, 3, 152, 152). The spatialdims do not depend on batch, so the live
H//ws, W//wsstay 19, 19, and theleading
Const(1)stays 1 (it is a literal with no input for reshape to rewrite):The doubled element count divides by the same known product, so the only free
axis (
-1) must absorb the extra factor of 2: the batch leaks into the channel,180 -> 360, and the window grid is scrambled.
The corruption is silent because the final view rebuilds a correct shape -- its
batch is live and its channel is the literal
embed_dim:Compilation, inference, and output shapes all pass; only the values are wrong
(max_abs_diff ~3.47 on the SwinIR_Classical models at batch=2).
Fix
Add
ReshapeBatchDimResolver, a PyTorch-FEModelPassrun innormalize().For a reshape carrying the window-reverse signature (leading positive-int
constant, exactly one trailing
-1, at least one dynamic interior dim) it freesthe leading dim to
-1and pins the channel to its batch-independent value as aConstant, so the batch is inferred from the real element count and no longerleaks into the channel.
The channel is recovered by a deterministic walk-back rather than OV shape
inference: window_reverse uses two chained views and the second takes its data
through a permute whose output stays dynamic. The pass walks back from the data
to a static last dim, through a last-axis-preserving
Transpose, or to a viewalready selected for rewrite.
Two guards keep it from touching ordinary reshapes that share the structure:
the data's entire trailing block, which rejects merge/split reshapes such as
view(1, T//2, -1)whose-1spans more than the data's last dim.Tickets: