[CPU][ARM] Int8 conv swish fq fusion by Passavee-Losripat · Pull Request #34931 · openvinotoolkit/openvino

Passavee-Losripat · 2026-03-25T16:53:24Z

Details:

Re-enable ConvolutionTransformation on ARM by removing CPU_DISABLE_PASS_ARM in transformation_pipeline.cpp
Extend ConvMulAddFQBlock pattern matcher to optionally match an activation node between Add and FakeQuantize, enabling recognition of Conv -> Mul -> Add -> Activation -> FQ in addition to the existing Conv -> Mul -> Add -> FQ. Currently supports Swish and Relu via wrap_type<Swish, Relu>
- designed to handle for other activation in addition to Swish support in the future without structural changes
Update ConvertConvolutionBias and FallbackUnsupportedLPConvToFP16 to retrieve and handle the optional activation anchor without breaking existing patterns
Improve ACLConvolutionExecutor to accept Activation + FakeQuantize as simultaneous post-ops, replacing the hard single-post-op limit with an iteration loop
Extend canFuse() in conv.cpp to allow FakeQuantize fusion after a single Eltwise activation is already fused, enabling the full post-op chain to reach the ACL executor
Introduce isI8DstF32 / isU8DstF32 flags and use_direct_i8_s8_f32 / use_direct_u8_u8_f32 paths in ACLConvolutionExecutor so Conv can output F32 directly, letting the activation run in floating point before FQ requantizes
Add subgraph tests covering static/dynamic shapes, per-channel and per-tensor weight scales, i8 and u8 quantized paths, with and without Swish.

Known Issues

Per-tensor quantization (i8 and u8) fails numerically and it's under investigation. - Per-channel quantization passes.

Tickets:

This PR is related to GSoC2026 Project 5 Optimize Quantized Model Inference Performance on ARM Devices with OpenVINO

AI Assistance:

AI assistance used: yes
Claude was used for writing external analyzing script and explaining codebase. All code in this PR was manually written and validated through YOLO26 detection accuracy on Apple M4 Max (ARM64). Analysis artifacts are not included in this PR.

…swish_fq_chain

Copilot

Pull request overview

This PR updates the Intel CPU plugin’s ARM/ACL low-precision convolution flow to recognize and fuse a Conv -> (Mul/Add bias/scale) -> Activation -> FakeQuantize pattern (notably Swish), re-enables relevant ARM convolution transformations, and extends the ACL convolution executor to handle an activation + FakeQuantize post-op chain and direct-to-F32 output paths.

Changes:

Extend ARM pattern matching / transformation plumbing to optionally include an activation node between Add and FakeQuantize and propagate an optional “activation” anchor through affected passes.
Update ACL convolution execution constraints and configuration to support up to two post-ops (Activation + FakeQuantize) and add an i8→f32 direct output configuration.
Add/extend ARM subgraph tests and test target wiring for ARM-specific custom subgraph suites.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/plugins/intel_cpu/tests/functional/custom/subgraph_tests/src/arm/conv_fq.cpp	Adds optional Swish into the Conv→FQ test graph and instantiates a Swish test suite.
src/plugins/intel_cpu/tests/functional/cmake/target_per_test.cmake	Adds an ARM subgraph test target directory for per-test targets.
src/plugins/intel_cpu/src/transformations/utils.hpp	Declares a new ARM matcher helper for Conv→Swish→FQ chains.
src/plugins/intel_cpu/src/transformations/utils.cpp	Implements Conv→Swish→FQ matcher used to skip FakeQuantize decomposition on ARM.
src/plugins/intel_cpu/src/transformations/transformation_pipeline.cpp	Extends the ARM callback to avoid FakeQuantize decomposition for Swish chains.
src/plugins/intel_cpu/src/transformations/snippets/aarch64/pass/snippets_mark_skipped.cpp	Marks Conv→Swish→FQ chains as skipped for snippets on ARM.
src/plugins/intel_cpu/src/transformations/cpu_opset/arm/pass/fallback_unsupported_lp_conv_to_fp16.cpp	Propagates optional activation anchor handling during fallback-to-FP16 transformation.
src/plugins/intel_cpu/src/transformations/cpu_opset/arm/pass/convert_conv_bias.cpp	Updates bias-conversion transformation to handle optional activation presence.
src/plugins/intel_cpu/src/transformations/cpu_opset/arm/pass/conv_mul_add_fq_block.cpp	Extends the block pattern to optionally include an activation (Swish/Relu) between Add and FQ.
src/plugins/intel_cpu/src/nodes/executors/convolution_implementations.cpp	Adds an ACL lowp type mapping for i8→f32 conv.
src/plugins/intel_cpu/src/nodes/executors/acl/acl_conv.hpp	Adds flags for i8/u8 direct-to-f32 paths.
src/plugins/intel_cpu/src/nodes/executors/acl/acl_conv.cpp	Iterates post-ops (up to 2), adjusts supports() checks, and wires direct-to-f32 ACL configuration parameters.
src/plugins/intel_cpu/src/nodes/conv.cpp	Extends fusion eligibility rules to allow post-op chaining with Activation + FakeQuantize (but contains a Swish-specific fusion blocker).
src/plugins/intel_cpu/src/dnnl_extension_utils.cpp	Adds Swish to the set of unary post-ops supported under ACL builds.

+            if (any_of(convInPrc, ov::element::u8, ov::element::i8) &&
+                fusedWith.back()->getAlgorithm() == Algorithm::EltwiseSwish) {
+                return false;
+            }


+    // After ConvertConvolutionBias swaps Mul and Add:
+    // Conv -> Add(bias) -> Mul(scales) -> Swish -> FQ
+    auto conv  = wrap_type<ov::op::v1::Convolution>();
+    auto add   = wrap_type<ov::op::v1::Add>({conv, any_input()});
+    auto mul   = wrap_type<ov::op::v1::Multiply>({add, any_input()});
+    auto swish = wrap_type<ov::op::v4::Swish>({mul});
+    auto fq    = wrap_type<ov::op::v0::FakeQuantize>(
+                     {swish, any_input(), any_input(), any_input(), any_input()});
+


+                         ::testing::Combine(::testing::ValuesIn(inputShapes),
+                                            ::testing::Values(element::f32),
+                                            ::testing::ValuesIn(quantizationParams),
+                                            ::testing::Values(true), //withBias
+                                            ::testing::Values(true), //withSwish
                                            ::testing::Values(ov::test::utils::DEVICE_CPU)),


+    const bool isQuantizedU8       = srcDesc->getPrecision() == ov::element::u8 &&
+                                     any_of(weiDesc->getPrecision(), ov::element::u8, ov::element::i8) &&
+                                     dstDesc->getPrecision() == ov::element::u8 && hasQuantizationPostOp;
+    const bool isQuantizedI8       = srcDesc->getPrecision() == ov::element::i8 &&
+                                     weiDesc->getPrecision() == ov::element::i8 &&
+                                     dstDesc->getPrecision() == ov::element::i8 && hasQuantizationPostOp;
+    const bool isQuantizedI8DstF32 = srcDesc->getPrecision() == ov::element::i8 &&
+                                     weiDesc->getPrecision() == ov::element::i8 &&
+                                     dstDesc->getPrecision() == ov::element::f32;
+
+    VERIFY(isQuantizedU8 || isQuantizedI8 || isQuantizedI8DstF32, UNSUPPORTED_BY_EXECUTOR);


… regression

Passavee-Losripat force-pushed the int8-conv-swish-fq-fusion branch from 25f294b to c3d0c62 Compare March 25, 2026 16:54

github-actions Bot added the category: CPU OpenVINO CPU plugin label Mar 28, 2026

sys-openvino-ci added the ExternalPR External contributor label Mar 28, 2026

v-Golubev self-assigned this May 26, 2026

Passavee-Losripat force-pushed the int8-conv-swish-fq-fusion branch from 3ea426d to d8eb502 Compare May 28, 2026 15:00

github-actions Bot added the category: build OpenVINO cmake script / infra label Jun 11, 2026

Passavee-Losripat force-pushed the int8-conv-swish-fq-fusion branch from 653e0c4 to fa7be2b Compare June 29, 2026 17:41

Passavee-Losripat added 16 commits June 30, 2026 01:04

support combination of activation + fakequantize

2acc91c

support optional activation type

b41748e

allow fakequantized after fusion

652fbf4

support multi-Eltwise chains

f92c8ac

add EltwiseSwish as supported ACL unary post-op

c951980

mark Conv+Swish+FQ as skipped by snippets

388064d

add subgraph test for conv+swish+fq

d778b54

remove fq existence check

ac8825b

fix match_acl_int8_conv_swish_fq_chain to check Conv input type

fbbd326

add conv+swish+fq pattern in FakeQuantizeDecomposition callback

b08b7d0

support Conv+Swish+FQ pattern in ConvertConvolutionBias

30890fd

extend canFuse and snippets marking for Conv+Add+Mul+Swish+FQ chain

5ba0698

Extend Subgraph test to include optional Swish

d5e7f78

check activation type instead of weights type in match_acl_int8_conv_…

ddfe18a

…swish_fq_chain

enable ACL with i8+i8->32 path for Swish

be8c059

enable custom i8/i8/f32 acl path

d2b04d6

Passavee-Losripat force-pushed the int8-conv-swish-fq-fusion branch from fa7be2b to d2b04d6 Compare June 29, 2026 18:04

Passavee-Losripat marked this pull request as ready for review June 29, 2026 18:05

Passavee-Losripat requested review from a team as code owners June 29, 2026 18:05

alvoron requested a review from Copilot June 29, 2026 18:10

Copilot started reviewing on behalf of alvoron June 29, 2026 18:11 View session

Copilot AI reviewed Jun 29, 2026

View reviewed changes

v-Golubev reviewed Jul 1, 2026

View reviewed changes

Passavee-Losripat and others added 7 commits July 2, 2026 01:49

extend u8+i8->f32 ACL path and Swish subgraph support; found accuracy…

037676e

… regression

check conv input type in ConvertConvolutionBias Swish branch

dbd4e16

use optional Swish instead of separate matchers

c7b1353

use make_activation for Swish subgraph test

d2f9c35

remove Swish chain from ConvertConvolutionBias and fp16 fallback

06ab33b

Merge branch 'master' into int8-conv-swish-fq-fusion

d0cd950

add i8 weights on direct u8->f32 path + per-channel dequantize scales

01a58fa

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[CPU][ARM] Int8 conv swish fq fusion#34931

[CPU][ARM] Int8 conv swish fq fusion#34931
Passavee-Losripat wants to merge 23 commits into
openvinotoolkit:masterfrom
Passavee-Losripat:int8-conv-swish-fq-fusion

Passavee-Losripat commented Mar 25, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Passavee-Losripat commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details:

Known Issues

Tickets:

AI Assistance:

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Passavee-Losripat commented Mar 25, 2026 •

edited

Loading