[Perf][FlyDSL] Tune DeepSeek V4 fused MoE for C1/C2/C32/C64 decode

[Fangzhou-Ai]

Summary

• Tune the gfx950 DeepSeek-V4 FP8-activation/FP4-weight fused-MoE decode routes for internal shape 7168/512/E385/top-k7 at C1, C2, C32, and C64.
• Select a BM32 kw4 stage-1 kernel for C1/C2 while retaining their existing stage-2 selections.
• Select measured BM16 stage-1/stage-2 routes for C32/C64.
• Add the three ordinary FlyDSL enablement changes required by BM16: stage-1 registry enumeration, 16-row sorting, and 32-row physical scale-resource bounds in both stages.
• Leave C4, C8, C16, and C128+ unchanged.

Stack and scope

This is the top layer of the DeepSeek-V4 stack and targets #4269's staging branch directly:

main aa6ee6102 -> #4269 FHMoE 2a4cb1339 -> #4314 tuning 57e848f53

The PR contains two tuning commits across four files. The C1/C2 follow-up is CSV-only and changes exactly two rows. The C32/C64 commit contains the tuned CSV, the three ordinary FlyDSL BM16 enablement changes, and the BM16 scale-boundary regression in op_tests/test_moe_2stage.py.

#4269 is the base because it moves the ordinary mixed-MoE builders into mixed_moe_gemm_2stage_common.py, which this PR modifies. The tuning itself remains separate from FHMoE dispatch and adds no FHMoE API fields.

Why the source changes are required

The CSV cannot enable BM16 alone:

• without the stage-1 tile_m=16 registry entry, runtime lookup rejects the configured kernel and AOT skips it;
• BM16 sorting emits one expert block per 16 rows, so retaining a 32-row stride skips or misaddresses alternating blocks;
• shuffled activation scales physically pack two 16-row groups into 32-row tiles, so stage-1 and stage-2 resource descriptors must advertise the padded 32-row bound.

An independent probe with only the registry entry injected produced NaNs and 99.9% differing elements. Applying all three enablement changes completed without NaNs.

Tuned routes

The table compares the base and tuned rows in dsv4_fp8fp4_tuned_fmoe.csv. Common flydsl_moe{1,2}_afp8_wfp4_bf16_ prefixes are omitted.

C	Stage 1, base -> tuned	Stage 2, base -> tuned	Total us, base -> tuned	Gain	Tuned err1 / err2
1	t32x128x256_w4_gui_fp8 -> t32x64x256_w4_gui_kw4_fp8	unchanged t32x256x256_atomic	27.0394 -> 23.4916	13.12%	0.0% / 0.0%
2	t32x128x256_w2_gui_fp8 -> t32x64x256_w4_gui_kw4_fp8	unchanged t32x256x256_atomic_xcd4	32.6325 -> 27.1002	16.95%	0.0% / 0.0%
32	t32x128x256_w3_gui_xcd4_fp8 -> t16x128x256_w3_gui_fp8	t32x256x256_atomic_bnt2_xcd4_persist -> t16x256x256_atomic	206.7745 -> 82.0800	60.30%	0.0% / 0.0%
64	t32x128x256_w3_gui_xcd4_fp8 -> t16x128x256_w3_gui_fp8	t32x256x256_atomic_bnt2_xcd4 -> t16x256x256_atomic	351.4854 -> 129.3620	63.20%	0.0% / 0.0%

The CSV err1/err2 values are the tuner's stored stage pass values, not a claim of bitwise end-to-end equality.

Rebase and validation

The two tuning commits were restacked patch-identically onto corrected #4269 head 2a4cb1339; git range-diff reports both commits unchanged. GitHub now reports this PR mergeable with a clean merge state.

The stack was installed editable with PREBUILD_KERNELS=0 and tested using the repository's unchanged scripts on 8×gfx950 with TP=8, expert parallelism disabled, and performance concurrency 4. The server selected native-FP8 shared-expert fusion, internal E=385/top-k=7, and the tuned C1/C2 rows on every rank.

GSM8K, all 1,319 samples, 20-shot, concurrency 64, max_tokens=2048, temperature 0:

Filter	Result
strict match	1269/1319 (96.209%)
flexible extract	1268/1319 (96.133%)

All requests returned HTTP 200 without retries or server errors. Compared with the immediately preceding same-script baseline, both metrics changed by -4/1319 answers (-0.303 percentage points).

InferenceX 8K-input/1K-output, concurrency 4, 40/40 requests:

Metric	Result
output throughput	206.858 tok/s
total throughput	1859.526 tok/s
mean TTFT	477.185 ms
mean TPOT	18.360 ms

Compared with the immediately preceding TP8/C4 run, output throughput improved 0.41%, TTFT improved 2.75%, and TPOT improved 0.38%.

AITER_ROOT=/home/amd/aiter-pr4269 TP=8 PORT=8888 GPU_MEM_UTIL=0.8 \
  bash dsv4_rocm_bench/start_server.sh
PORT=8888 CONC=64 bash dsv4_rocm_bench/run_gsm8k.sh
PORT=8888 CONCS=4 bash dsv4_rocm_bench/run_inferencex_matrix.sh

The final base-only change after the runtime run forwards pass_swiglu_limit in #4269's AOT adapter and does not alter these runtime kernels or routes.

Static validation:

uvx ruff==0.16.0 check \
  aiter/ops/flydsl/kernels/mixed_moe_gemm_2stage_common.py \
  aiter/ops/flydsl/moe_kernels.py op_tests/test_moe_2stage.py
/home/amd/vllm/.venv/bin/python -m py_compile \
  aiter/ops/flydsl/kernels/mixed_moe_gemm_2stage_common.py \
  aiter/ops/flydsl/moe_kernels.py
git diff --check 2a4cb1339...57e848f53

Result: passed.

Duplicate-work check

Open-PR searches found no equivalent DeepSeek-V4 C1/C2/C32/C64 FP8/FP4 tuning change. #3835 tunes the separate dsv4_fp4_tuned_fmoe.csv A4W4 configuration and does not overlap this patch.

AI assistance

OpenAI Codex was used to audit, implement, test, benchmark, restack, and document this change. The human submitter reviewed the changes and is responsible for understanding and defending them end-to-end.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4314 --add-label <label>