Ggml moe gathermatmul gpu

[cavusmustafa]

Details:

• Add a GPU-plugin MatcherPass (ov::intel_gpu::ConvertGgmlMoeGatherToGatherMatmul) that captures the ggml-openvino frontend's public-op MoE expert-matmul carrier — MatMul(activations, Reshape([Convert(]Gather(CompressedWeightsBlock[n_expert, m*k], ids)[)]), transpose_b=true) — and rewrites it to the internal ov::op::internal::GatherMatmul.
• The matched block is rank-2 [n_expert, m*k], but GatherMatmul needs rank-3 [n_expert, N, K]; the pass rebuilds the block's low-bit weight / scale / (optional) zero-point Constants at the grouped rank-4 [n_expert, N, K/group, group] shape over the same data (a bit-identical reinterpret), reshapes to rank-3, and feeds it to GatherMatmul. The downstream stock ConvertGatherMatmulToGatherMatmulCompressed then folds it into a single GatherMatmulCompressed (gather + dequantize-selected-experts + matmul), keeping the low-bit weights compressed.
• Registered inside the existing supports_immad && use_onednn MoE-compressed pipeline, after ConvertTiledMoeBlockToGatherMatmuls and before ConvertGatherMatmulToGatherMatmulCompressed. Matching a genuine CompressedWeightsBlock (u4/i4/u8/i8) both guarantees the downstream fold can fire and structurally excludes any non-compressed expert matmul.
• The frontend needs no changes and keeps emitting public ops only: without this pass its rank-2 carrier folds to a plain GatherCompressed; with this pass it is upgraded to GatherMatmulCompressed. No env flag — the fusion is automatic when the plugin is present.
• Change is purely additive: 2 new pass files + 1 registration line in transformations_pipeline.cpp (0 deletions to existing code).
• Adds unit tests (convert_ggml_moe_gather_to_gathermatmul_test.cpp, 6 cases): fires on u4/u8 (with zero point) and symmetric i4 (no zero point); end-to-end folds to GatherMatmulCompressed with the u4 weight Constant surviving (weights stay compressed); negative cases confirm it does not fire on a non-compressed f16 expert Gather nor when transpose_b is false.
• Measured on Arc B390 (gemma-4-26B-A4B Q4_K_M, llama-bench, warmup on): prefill ~1.8x (7.7 -> 13.6 t/s), decode ~1.2x (3.96 -> 4.81 t/s) vs. the non-fused rank-2 path.

Tickets:

• CVS-191064

AI Assistance:

• AI assistance used: yes
• The pass implementation, the unit test, and this description were developed with AI (Claude) assistance. Validation performed: built the GPU plugin and the ov_gpu_unit_tests target; the 6 new unit tests all pass; ran the GPU transformation test subset (*Transform*:*Convert*:*Fuse*:*Compress*:*MoE*:*GatherMatmul*:*FullyConnected*:*MatMul* -> 260 passed, 1 pre-existing skip, 0 failed) to confirm no regression; verified end-to-end on the real 26B model (dumped IR shows all 60 expert matmuls rewritten and folded to GatherMatmulCompressed, a before/after constant census confirms no expert weights are materialized), a live 26B GPU run completes (EXIT 0), and the throughput numbers above were measured directly.

cc: @wine99