Delegate even-kernel 'same'-padding convs via a quantized static pad (#20553)#20553
Delegate even-kernel 'same'-padding convs via a quantized static pad (#20553)#20553JakeStevens wants to merge 1 commit into
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20553
Note: Links to docs will display an error until the docs builds have been completed. ❌ 2 New Failures, 2 Unrelated FailuresAs of commit 0d1649d with merge base b1ef9a5 ( NEW FAILURES - The following jobs have failed:
FLAKY - The following job failed but was likely due to flakiness present on trunk:
BROKEN TRUNK - The following job failed but was present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@JakeStevens has exported this pull request. If you are a Meta employee, you can view the originating Diff in D109871964. |
…ytorch#20553) Summary: When using padding="same" for a conv with even kernels, `torch.export` creates an explicit pad node, leading to a dq -> pad -> conv chain, which is not recognized, so conv ends up undelegated. The ReLU ends up in its own partition with a quantize. XNNPACK does not support this, resulting in `xnn_status_unsupported_parameter`. This PR fixes this, allowing for full delegation to XNNPACK in this case, by pulling the zero-valued spatial pad into the conv's partition (`ConvConfig._get_act_deps`) so both delegate together: for fp32 the pad lowers directly as an `XNNStaticConstantPad`. For quantized graphs the pad is created by `to_edge` decomposition after the quantizer runs, so it arrives as `dq -> pad -> conv` with no quantize on its output and would serialize as fp32 (the conv would then reject its unquantized activation). `InsertPadQDQPass` inserts an implicit `quantize -> dequantize` after the pad, reusing the feeding dequant's params (a zero pad preserves quantization), so it lowers as a quantized static pad and the conv sees a proper dequantized activation. Note: This results in full delegation, an improvement over the existing runtime error (or portable fallback as naive fix), but the pad is executed as a separate op and results in overhead. Differential Revision: D109871964
a04093b to
3c5db00
Compare
…ytorch#20553) Summary: When using padding="same" for a conv with even kernels, `torch.export` creates an explicit pad node, leading to a dq -> pad -> conv chain, which is not recognized, so conv ends up undelegated. The ReLU ends up in its own partition with a quantize. XNNPACK does not support this, resulting in `xnn_status_unsupported_parameter`. This PR fixes this, allowing for full delegation to XNNPACK in this case, by pulling the zero-valued spatial pad into the conv's partition (`ConvConfig._get_act_deps`) so both delegate together: for fp32 the pad lowers directly as an `XNNStaticConstantPad`. For quantized graphs the pad is created by `to_edge` decomposition after the quantizer runs, so it arrives as `dq -> pad -> conv` with no quantize on its output and would serialize as fp32 (the conv would then reject its unquantized activation). `InsertPadQDQPass` inserts an implicit `quantize -> dequantize` after the pad, reusing the feeding dequant's params (a zero pad preserves quantization), so it lowers as a quantized static pad and the conv sees a proper dequantized activation. Note: This results in full delegation, an improvement over the existing runtime error (or portable fallback as naive fix), but the pad is executed as a separate op and results in overhead. Differential Revision: D109871964
3c5db00 to
f94078b
Compare
…ytorch#20553) Summary: When using padding="same" for a conv with even kernels, `torch.export` creates an explicit pad node, leading to a dq -> pad -> conv chain, which is not recognized, so conv ends up undelegated. The ReLU ends up in its own partition with a quantize. XNNPACK does not support this, resulting in `xnn_status_unsupported_parameter`. This PR fixes this, allowing for full delegation to XNNPACK in this case, by pulling the zero-valued spatial pad into the conv's partition (`ConvConfig._get_act_deps`) so both delegate together: for fp32 the pad lowers directly as an `XNNStaticConstantPad`. For quantized graphs the pad is created by `to_edge` decomposition after the quantizer runs, so it arrives as `dq -> pad -> conv` with no quantize on its output and would serialize as fp32 (the conv would then reject its unquantized activation). `InsertPadQDQPass` inserts an implicit `quantize -> dequantize` after the pad, reusing the feeding dequant's params (a zero pad preserves quantization), so it lowers as a quantized static pad and the conv sees a proper dequantized activation. Note: This results in full delegation, an improvement over the existing runtime error (or portable fallback as naive fix), but the pad is executed as a separate op and results in overhead. Differential Revision: D109871964
f94078b to
e46184b
Compare
…ytorch#20553) Summary: When using padding="same" for a conv with even kernels, `torch.export` creates an explicit pad node, leading to a dq -> pad -> conv chain, which is not recognized, so conv ends up undelegated. The ReLU ends up in its own partition with a quantize. XNNPACK does not support this, resulting in `xnn_status_unsupported_parameter`. This PR fixes this, allowing for full delegation to XNNPACK in this case, by pulling the zero-valued spatial pad into the conv's partition (`ConvConfig._get_act_deps`) so both delegate together: for fp32 the pad lowers directly as an `XNNStaticConstantPad`. For quantized graphs the pad is created by `to_edge` decomposition after the quantizer runs, so it arrives as `dq -> pad -> conv` with no quantize on its output and would serialize as fp32 (the conv would then reject its unquantized activation). `InsertPadQDQPass` inserts an implicit `quantize -> dequantize` after the pad, reusing the feeding dequant's params (a zero pad preserves quantization), so it lowers as a quantized static pad and the conv sees a proper dequantized activation. Note: This results in full delegation, an improvement over the existing runtime error (or portable fallback as naive fix), but the pad is executed as a separate op and results in overhead. Differential Revision: D109871964
e46184b to
0d1649d
Compare
|
|
||
| An even-kernel 'same'-padding conv decomposes (after quantization) into | ||
| dequant -> constant_pad_nd -> convolution. Because the pad is introduced by | ||
| to_edge decomposition -- after the quantizer has run -- it is never annotated, |
There was a problem hiding this comment.
can we fuse it inside the preprocess for xnnpack or we have to run pad + conv in the XNNPACK graph? I am asking because if we can fuse it we might as well write that pass instead.
There was a problem hiding this comment.
We can fuse, I have some work in that direction, was going to publish as a quick follow up since there is an external contributor with the 1d work and the change is slightly larger so preferred to do it once these both land.
Summary:
When using padding="same" for a conv with even kernels,
torch.exportcreates an explicit pad node, leading to a dq -> pad -> conv chain, which is not recognized, so conv ends up undelegated. The ReLU ends up in its own partition with a quantize. XNNPACK does not support this, resulting inxnn_status_unsupported_parameter.This PR fixes this, allowing for full delegation to XNNPACK in this case, by pulling the zero-valued spatial pad into the conv's partition (
ConvConfig._get_act_deps) so both delegate together: for fp32 the pad lowers directly as anXNNStaticConstantPad. For quantized graphs the pad is created byto_edgedecomposition after the quantizer runs, so it arrives asdq -> pad -> convwith no quantize on its output and would serialize as fp32 (the conv would then reject its unquantized activation).InsertPadQDQPassinserts an implicitquantize -> dequantizeafter the pad, reusing the feeding dequant's params (a zero pad preserves quantization), so it lowers as a quantized static pad and the conv sees a proper dequantized activation.Note: This results in full delegation, an improvement over the existing runtime error (or portable fallback as naive fix), but the pad is executed as a separate op and results in overhead.
Differential Revision: D109871964