Add Mistral-3-8B + Qwen3-8B-FP8 + native triton attention backend for gfx1201 (RDNA4 / RX 9070 XT)#749

Open

carlushuang wants to merge 47 commits into

mainfrom

carhuang/support_gfx1201_mistral3

Commits on May 9, 2026

Commits on May 10, 2026

attentions: torch-native impl + RMSNorm fallback for gfx1201
carlushuang
committed
wip: NEXT_SESSION update — RMSNorm fallback in, FP8 GEMM is next blocker
carlushuang
committed
model_ops: gfx1201 fallbacks for FP8 GEMM, SiLU+Mul, sampler
carlushuang
committed
wip: NEXT_SESSION v3 — prefill works end-to-end; decode is the only piece left
carlushuang
committed
attn: full prefill + decode + KV cache; FP8 dequant uint8 reinterpret
carlushuang
committed
recipes: Ministral-3-8B on gfx1201 with torch-native attention; remove NEXT_SESSION.md (work complete)
carlushuang
committed
linear: route gfx1201 FP8 GEMM through aiter triton gemm_a8w8
carlushuang
committed
recipes: update Ministral-3-8B with triton FP8 GEMM perf + gsm8k 76.5% (n=200)
carlushuang
committed
attn: wire aiter triton paged_attention_decode + pivot KV layout
carlushuang
committed
recipes: document triton paged_attention_decode (~20% e2e win)
carlushuang
committed
attn: wire aiter triton context_attention_fwd into prefill
carlushuang
committed
recipes: triton context_attention_fwd prefill (gsm8k 78.5% n=200)
carlushuang
committed
attn: triton kv-cache write kernel; linear: BF16 unquantized gfx1201 fallback
carlushuang
committed
recipes: triton kv-write kernel + BF16 linear fallback
carlushuang
committed
attn/norm/act: triton kernels + cudagraph foundation
carlushuang
committed
recipes: triton RMSNorm + SiLU+Mul
carlushuang
committed
attn: enable cudagraph at decode bs<=2 (24% TPOT, 3.3x TTFT)
carlushuang
committed
model_ops: delete torch reference fallbacks (triton-only on gfx1201)
carlushuang
committed
attn/cudagraph: warmup on capture stream, twice (SGLang pattern)
carlushuang
committed
linear: fuse dynamic FP8 quant + cache per-channel weight scale
carlushuang
committed
recipes: document fused FP8 quant win + TP=2 host blocker
carlushuang
committed

Commits on May 11, 2026

Commits on May 14, 2026

Commits on May 18, 2026

Merge branch 'main' into carhuang/support_gfx1201_mistral3
chuanbowang2026
authored

Commits on May 26, 2026

Merge branch 'main' into carhuang/support_gfx1201_mistral3
chuanbowang2026
authored