Skip to content

[GPU] Support IncreasePositionIdsPrecisionForGemma4 for PA backend#36915

Open
ahnyoung-paul wants to merge 1 commit into
openvinotoolkit:masterfrom
ahnyoung-paul:fix_increase_position_id_for_gemma4_pa
Open

[GPU] Support IncreasePositionIdsPrecisionForGemma4 for PA backend#36915
ahnyoung-paul wants to merge 1 commit into
openvinotoolkit:masterfrom
ahnyoung-paul:fix_increase_position_id_for_gemma4_pa

Conversation

@ahnyoung-paul

Copy link
Copy Markdown
Contributor

Details

Problem: The IncreasePositionIdsPrecisionForGemma4 pass introduced in PR #36896 does not match the rotary embedding graph pattern when using the PA (Paged Attention) backend. Under PA, upstream transformations fold Transpose into Reshape and Unsqueeze into Reshape, producing a different subgraph shape (MatMul → Reshape → Concat → Sin/Cos → Reshape → RoPE) that the original pattern (MatMul → Convert → Transpose → Concat → Sin/Cos → Unsqueeze → RoPE) cannot match. As a result, the rotary embedding MatMul remains in FP16 and the infinite repetition loop persists on PA backend.

Fix: Updated the IncreasePositionIdsPrecisionForGemma4 MatcherPass pattern to accept the PA backend's graph structure:

  • Added MatMul → Reshape as an alternative concat input path (alongside the existing MatMul → Convert → Transpose)
  • Added Sin/Cos → Reshape as an alternative RoPE input path (alongside the existing Sin/Cos → Unsqueeze)
  • Made the Convert after MatMul optional in the callback (present in SDPA path, absent in PA path)

Tickets

AI Assistance

  • AI assistance used: yes
  • AI was used for per-iteration logit comparison analysis, tensor dump comparison scripting, exec graph subgraph extraction, and matcher log interpretation. All code changes were manually reviewed and validated with end-to-end accuracy tests.

@ahnyoung-paul ahnyoung-paul requested review from a team as code owners July 16, 2026 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant