[GPU] add oneDNN SDPA primitive as a debug feature#36342
Open
wilson-seok wants to merge 7 commits into
Open
Conversation
wilson-seok
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a preview oneDNN GPU implementation for scaled_dot_product_attention in the Intel GPU plugin, with opt-in selection via a new internal property and a GPU unit test that compares oneDNN output against the existing OCL reference implementation.
Changes:
- Register a new oneDNN-based SDPA implementation manager ahead of existing OCL SDPA implementations (with fallback preserved via validation/property gating).
- Gate oneDNN SDPA selection behind internal property
ov::intel_gpu::enable_onednn_sdpa_primitive/OV_GPU_ENABLE_ONEDNN_SDPA_PRIMITIVE, and enable the primitive in the layout optimizer when configured. - Add a focused GPU unit test validating selection of oneDNN SDPA and numerical agreement vs OCL reference.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/plugins/intel_gpu/tests/unit/test_cases/sdpa_gpu_test.cpp | Adds parameterized unit test comparing oneDNN SDPA vs OCL ref and asserts oneDNN impl selection. |
| src/plugins/intel_gpu/src/graph/registry/sdpa_impls.cpp | Registers the oneDNN SDPA implementation manager before OCL SDPA implementations. |
| src/plugins/intel_gpu/src/graph/program.cpp | Enables oneDNN SDPA via layout_optimizer when the new config property is set. |
| src/plugins/intel_gpu/src/graph/layout_optimizer.cpp | Adds SDPA inst header include needed for enabling oneDNN for SDPA in layout optimizer flow. |
| src/plugins/intel_gpu/src/graph/impls/onednn/sdpa_onednn.hpp | Introduces oneDNN SDPA implementation manager with capability/shape/type validation. |
| src/plugins/intel_gpu/src/graph/impls/onednn/sdpa_onednn.cpp | Implements oneDNN SDPA primitive creation, argument binding, and (de)serialization support. |
| src/plugins/intel_gpu/include/intel_gpu/runtime/options.inl | Adds internal option declaration for enabling the oneDNN SDPA primitive. |
| src/plugins/intel_gpu/include/intel_gpu/runtime/internal_properties.hpp | Adds internal property key GPU_ENABLE_ONEDNN_SDPA_PRIMITIVE. |
wilson-seok
force-pushed
the
onednn_sdpa_implementation
branch
from
June 15, 2026 01:12
98bd57e to
1bcfe47
Compare
wilson-seok
changed the title
Enable oneDNN SDPA to work correctly with real LLM inference where K/V buffers have upper padding from KV-cache pre-allocation and shapes are dynamic. Key changes: - Use explicit strides from layout::get_pitches() when creating oneDNN memory descriptors for padded layouts, instead of assuming contiguous format tags. This fixes accuracy issues where oneDNN read wrong data for heads beyond the first due to stride mismatch. - Add dynamic shape support: create() returns unconfigured impl for dynamic shapes; update() calls configure() at runtime with actual shapes/strides. - Register dynamic_shape variant in sdpa_impls.cpp so the impl is selected for IndirectSDPA nodes with dynamic dimensions. - Add support_shapes() override with beam_table check for IndirectSDPA and shared validate_common() for both graph-build and runtime checks. - Add runtime attention mask support with proper descriptor creation and argument binding. - Extend unit tests with dynamic shape and runtime mask/scale variants. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
isanghao
changed the title
isanghao
reviewed
Jun 18, 2026
…ync, instrumentation Tooling + workarounds from the llama-3.1 Xe3p PagedAttention INT4-KV micro-kernel nan/inf accuracy investigation: - Add OV_GPU_DISABLE_SDPA_MICRO knob (internal_properties.hpp, options.inl) and wire it into both SDPA backends: PagedAttentionOpt::supports_micro_sdpa (paged_attention_opt.cpp) and SDPAOpt::supports_micro_sdpa (sdpa_opt.cpp). When set, forces SDPA onto the OCL sdpa_opt path instead of the oneDNN-derived micro kernel -- proven accuracy workaround for the Xe3p corruption, no rebuild. - Sync choose_config_xe3p in sdpa_gen_micro.cpp to faithfully mirror oneDNN's data-driven xe3p config table (configs.cpp): head_size ceiling match + exact second_token/quantized match + xe3p->xe2 fallback, fma rows folded in. - Add deduped [choose_config] + [micro_selected] diagnostic dumps in sdpa_gen_micro.cpp to capture the resolved tuning hint and the actual selected microkernel strategy per model/phase. - Keep the xe3p head_size<=64 micro-SDPA WA (load-bearing for gpt-oss accuracy). - CMakeLists.txt: gate DX11/DX12 remote-tensor interop tests on <atlbase.h> so environments without the VS ATL workload don't break the build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rename the opt-in property to use_onednn_sdpa and expose it as the debug-only OV_GPU_USE_ONEDNN_SDPA option. Guard oneDNN SDPA selection and related tests when debug caps are disabled.
Address Copilot review comments on PR openvinotoolkit#36342: - Route DISABLE_SDPA_MICRO knob messages through GPU verbose infra - Wrap choose_config/micro_selected logging in GPU_DEBUG_IF guard - Add comment explaining why PA skips xe3p tuning table - Remove unused <iostream>/<mutex> includes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
|
Please consider PR-36705. It will conflict this this PR: #36705 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
scaled_dot_product_attentionfor debugging feature.onednn::sdpaahead of OCL SDPA implementations while preserving OCL fallback.ov::intel_gpu::enable_onednn_sdpa_primitive/OV_GPU_ENABLE_ONEDNN_SDPA_PRIMITIVE.Validation
cmake --build . --target ov_gpu_unit_tests -j23 -- -scmake --build . --target openvino_intel_gpu_plugin -j23 -- -s./bin/intel64/Release/ov_gpu_unit_tests --gtest_filter='*onednn_sdpa*' --device_suffix=1