Skip to content

[GPU] Block the fc horizontal fusion for weights that are not constants#36572

Merged
e-ddykim merged 1 commit into
openvinotoolkit:masterfrom
intbf:block_fc_horiz_fusion_for_inputs
Jun 26, 2026
Merged

[GPU] Block the fc horizontal fusion for weights that are not constants#36572
e-ddykim merged 1 commit into
openvinotoolkit:masterfrom
intbf:block_fc_horiz_fusion_for_inputs

Conversation

@intbf

@intbf intbf commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Details:

Found when using weight sharing with llms. In that case weights are converted into inputs/parameters, and the fusion might not work as expected. With inputs/parameters the constant folding won't happen, so the fusion will be invoked on every inference. Plus in some cases the concat operator might not be enabled for all weight types (like int4)

Tickets:

CVS-189072

AI Assistance:

  • AI assistance used: yes
  • do the code changes + unit tests

@intbf
intbf requested review from a team as code owners June 25, 2026 14:31
@github-actions github-actions Bot added the category: GPU OpenVINO GPU plugin label Jun 25, 2026
@maxnick maxnick added this to the 2026.3 milestone Jun 25, 2026
@maxnick
maxnick requested a review from Copilot June 25, 2026 15:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Intel GPU plugin’s FullyConnectedHorizontalFusion transformation to avoid fusing horizontal FC patterns when FC weights are not compile-time constants, addressing weight-sharing scenarios where weights become runtime Parameters and would otherwise force an unfused Concat into GPU program build (and/or cause per-inference overhead).

Changes:

  • Add a guard in the horizontal-fusion pattern predicate to require FC weight inputs to be constant (or constant-derived) before allowing fusion.
  • Add unit tests covering weights-as-Parameters and Convert(Parameter) weight forms to ensure fusion is blocked in these cases.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/plugins/intel_gpu/src/plugin/transformations/fc_horizontal_fusion.cpp Blocks horizontal FC fusion when weight input is not constant, preventing unfused runtime Concat and avoiding per-inference concatenation overhead.
src/plugins/intel_gpu/tests/unit/transformations/horizontal_fc_fusion_test.cpp Adds regression tests ensuring no fusion occurs when weights are provided as Parameter (including Convert(Parameter)).

@intbf
intbf force-pushed the block_fc_horiz_fusion_for_inputs branch from 1c5f36e to 2a9cccf Compare June 25, 2026 17:58

@e-ddykim e-ddykim left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@e-ddykim
e-ddykim added this pull request to the merge queue Jun 26, 2026
Merged via the queue into openvinotoolkit:master with commit 0cd964a Jun 26, 2026
191 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: GPU OpenVINO GPU plugin

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants