Skip to content

[GPU] Fix FullyConnected accuracy regression for large f16 matmuls on non-immad devices#36883

Open
alvoron wants to merge 5 commits into
openvinotoolkit:masterfrom
alvoron:alvoron_gpu_f16_matmul_accuracy_fix
Open

[GPU] Fix FullyConnected accuracy regression for large f16 matmuls on non-immad devices#36883
alvoron wants to merge 5 commits into
openvinotoolkit:masterfrom
alvoron:alvoron_gpu_f16_matmul_accuracy_fix

Conversation

@alvoron

@alvoron alvoron commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Details:

  • [GPU] Selectively keep FullyConnected weights non-transposed for large matmuls #36437 introduced a weights_transposed (transpose_b) flag and, for large non-compressed f16 matmuls (K >= 8192 && (M <= 512 || N <= 4096)), keeps the weights in the non-transposed [K, N] layout so they can be executed with the faster onednn abc weight layout.
  • However, the non-transposed weight layout is only consumed by the onednn FullyConnected implementation. The OCL FullyConnected kernels always assume transposed [N, K] weights and ignore the flag - they only reference weights_transposed for output-shape calculation, not for weight indexing.
  • On devices without immad support, the OCL FC path is used. There, feeding non-transposed weights while the kernel indexes them as transposed makes the kernel read the wrong weights.

Fix:

  • The fix gates the non-transposed-weights decision on supports_immad, so the layout is only chosen when onednn will actually consume it. On non-immad devices the pass now keeps the previous transposed behavior.

Tickets:

@alvoron
alvoron requested review from a team as code owners July 14, 2026 15:33
@github-actions github-actions Bot added the category: GPU OpenVINO GPU plugin label Jul 14, 2026
@maxnick

maxnick commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

@clee30 , could you please review?

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes an Intel GPU plugin accuracy regression where large f16 MatMul→FullyConnected conversions could keep weights in non-transposed [K, N] layout even when the execution path uses OCL FullyConnected kernels (non-immad devices) that assume transposed weights, causing incorrect weight indexing.

Changes:

  • Gate the “keep weights non-transposed for large f16 matmuls” heuristic on supports_immad, ensuring the non-transposed layout is only selected when the oneDNN FullyConnected implementation will consume it.
  • Add a unit test covering the non-immad case to ensure the transformation keeps the transposed behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
src/plugins/intel_gpu/src/plugin/transformations/convert_matmul_to_fc.cpp Gates the large-f16 non-transposed weights decision on supports_immad to avoid OCL FC misinterpreting weight layout.
src/plugins/intel_gpu/tests/unit/transformations/convert_matmul_to_fc_test.cpp Adds a regression test ensuring non-immad devices keep the transposed weights path.

Comment thread src/plugins/intel_gpu/tests/unit/transformations/convert_matmul_to_fc_test.cpp Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Comment thread src/plugins/intel_gpu/src/plugin/transformations/convert_matmul_to_fc.cpp Outdated
@alvoron

alvoron commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

@maxnick could you please review?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: GPU OpenVINO GPU plugin

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants