Skip to content

[GPU] Fix reorder_data_fast_b1 performance regression for unaligned blocked formats#36410

Merged
p-durandin merged 2 commits into
openvinotoolkit:masterfrom
maxnick:fix_padding_reorder
Jun 25, 2026
Merged

[GPU] Fix reorder_data_fast_b1 performance regression for unaligned blocked formats#36410
p-durandin merged 2 commits into
openvinotoolkit:masterfrom
maxnick:fix_padding_reorder

Conversation

@maxnick

@maxnick maxnick commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Details:

Commit 8932d86 ("[GPU] Fix unit tests for Xe2+ platforms") introduced zero-fill for padding. Por this purpose GWS was expanded by ceil(features/fsv) ratio to cover padding positions. For small features count (first convolutions have 3 input features) this means that the most of the SIMD lanes are used to insert zeroes, wasting SIMD lanes and causing branch divergence.

In this PR padding zero-fill is moved from the kernel to clEnqueueFillBuffer. The reorder kernel is restored to its original GWS (logical size only). The fill is only enqueued when the output format has a feature-blocking dimension that is not aligned.

Tickets:

AI Assistance:

  • AI assistance used: yes

@maxnick
maxnick requested review from a team as code owners June 15, 2026 10:24
@github-actions github-actions Bot added the category: GPU OpenVINO GPU plugin label Jun 15, 2026
@maxnick maxnick added this to the 2026.3 milestone Jun 15, 2026
Comment thread src/plugins/intel_gpu/src/graph/reorder.cpp Outdated
Comment thread src/plugins/intel_gpu/src/graph/reorder.cpp Outdated
@maxnick
maxnick requested a review from Lyamin-Roman June 16, 2026 15:49
@maxnick
maxnick requested a review from sungeunk June 22, 2026 08:18
@maxnick

maxnick commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

@sungeunk , could you please review?

@sungeunk

Copy link
Copy Markdown
Contributor

I have a few concerns before approval:

  1. High: Padding handling appears to regress from kernel-level fine-grained writes to full output buffer clear.
    Previously, padding lanes were handled inside the fast reorder kernel path (FILL_FEATURE_PADDING). In this PR, that path is removed and replaced with graph-level fill on output memory. This may trigger full-buffer clear plus synchronization overhead, not only for the original fast_b1 case but also for broader blocked reorder scenarios.
    Risk: this can reintroduce the same performance regression the PR is trying to fix, especially on static paths.

  2. Medium: No end-to-end test coverage for the padding-zero contract.
    The behavioral contract moved from kernel logic to graph-level memory reset logic, but I do not see an E2E test that validates padding zeroing and prevents NaN propagation/regression across real execution flows.
    Risk: future changes in memory reuse, scheduling, or kernel selection could silently break correctness/performance.

Could you please add model-level validation details to the PR?

  1. Which models were tested (model names, key shapes/configs, static vs dynamic).
  2. Test scope (accuracy/perf, device and driver, relevant precision/layout combinations).
  3. Results summary (pass/fail, regressions, notable deltas).
  4. Explicit confirmation of static-model coverage and observed impact after the need_reset_output_memory() changes.

Given the scope of this change, I am particularly concerned about impact on static models due to the need_reset_output_memory() behavior changes.

@maxnick

maxnick commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

I have a few concerns before approval:

  1. High: Padding handling appears to regress from kernel-level fine-grained writes to full output buffer clear.
    Previously, padding lanes were handled inside the fast reorder kernel path (FILL_FEATURE_PADDING). In this PR, that path is removed and replaced with graph-level fill on output memory. This may trigger full-buffer clear plus synchronization overhead, not only for the original fast_b1 case but also for broader blocked reorder scenarios.
    Risk: this can reintroduce the same performance regression the PR is trying to fix, especially on static paths.
  2. Medium: No end-to-end test coverage for the padding-zero contract.
    The behavioral contract moved from kernel logic to graph-level memory reset logic, but I do not see an E2E test that validates padding zeroing and prevents NaN propagation/regression across real execution flows.
    Risk: future changes in memory reuse, scheduling, or kernel selection could silently break correctness/performance.

Could you please add model-level validation details to the PR?

  1. Which models were tested (model names, key shapes/configs, static vs dynamic).
  2. Test scope (accuracy/perf, device and driver, relevant precision/layout combinations).
  3. Results summary (pass/fail, regressions, notable deltas).
  4. Explicit confirmation of static-model coverage and observed impact after the need_reset_output_memory() changes.

Given the scope of this change, I am particularly concerned about impact on static models due to the need_reset_output_memory() behavior changes.

Thank you very much for raising these questions.
Regarding the performance validation, I triggered internal performance validation for static models.
As for your second concern. Yes, you're right, there are no dedicated tests precisely checking if the padding are filled with zeroes. The main reason of these modification is the activity of fixing unit tests for Xe2+ platforms. Specifically:

TEST(reorder_gpu_optimization, compare_with_ref__bfyx_to_blocked_format_different_datatype) {
compare_bfyx2blocked_with_ref("reorder_data_bfyx_to_blocked_format", data_types::f32, data_types::f16, format::bfyx, format::b_fs_yx_fsv16, 3, 32 + 4, 16 + 7, 2, 0, 0, false);
compare_bfyx2blocked_with_ref("reorder_data_bfyx_to_blocked_format", data_types::i8, data_types::f32, format::bfyx, format::b_fs_yx_fsv16, 3, 32 + 4, 16 + 7, 2, 0, 0, false);
compare_bfyx2blocked_with_ref("reorder_data_bfyx_to_blocked_format", data_types::i64, data_types::f32, format::bfyx, format::b_fs_yx_fsv16, 3, 32 + 4, 16 + 7, 2, 0, 0, false);
}
actually compare entire blobs including paddings, and the fact that the padding values weren't initialized with specific values led to a test fail. So zero filling was added.
Probably this don't even needed in inference scenarios as paddings values aren't used anyway and don't impact the result. So may be it does make sense to roll back these changes to pre 8932d86 and adjust the tests? What do you think?

@maxnick
maxnick force-pushed the fix_padding_reorder branch from 370ebfc to 74fd82d Compare June 24, 2026 16:56
@maxnick

maxnick commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

@sungeunk , it turned out that enqueueing zero fill impacts throughput performance of CV models significantly.
Given the fact, that in the real applications it doesn't really matter which values fill paddings the state preceding 8932d86 has been restored. Zero fill of paddings is removed (however it still persists for weights repacking).

@sungeunk sungeunk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@p-durandin
p-durandin added this pull request to the merge queue Jun 25, 2026
Merged via the queue into openvinotoolkit:master with commit 1100e92 Jun 25, 2026
205 of 207 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants