update recipe

Signed-off-by: zejunchen-zejun <zejun.chen@amd.com>

Author: zejunchen-zejun

recipes/GLM-5.md

@@ -12,27 +12,95 @@ docker pull rocm/atom-dev:latest
 All the operations in the next will be executed inside the container.
 
 ## Launching server
-ATOM supports running the model with different parallelism, e.g., tensor parallel, expert parallel, data parallel.
+ATOM supports running the model with different parallelism, e.g., tensor parallel, expert parallel, data parallel. The examples below use the current ATOM server entrypoint and keep the recommended runtime environment close to the command.
 
-### Serving on 8xMI355 GPUs (TP8 + FP8 KV Cache)
+### GLM-5.2 FP8 Server
 
 ```bash
 #!/bin/bash
 
-python -m atom.entrypoints.openai_server --model zai-org/GLM-5-FP8 -tp 8 --kv_cache_dtype fp8 --port 5678 --server-port 7777
+model_path=zai-org/GLM-5-FP8
+export AITER_QUICK_REDUCE_QUANTIZATION=INT4
+export AITER_USE_FLYDSL_MOE_SORTING=1
+TP=8
+
+rm -rf /root/.cache/atom/*
+
+python -m atom.entrypoints.openai_server \
+  --model "$model_path" \
+  --server-port 7777 \
+  --kv_cache_dtype fp8 \
+  --no-enable_prefix_caching \
+  -tp $TP 2>&1 | tee server.log &
 ```
 
-### Offline Inference with DP Attention + Expert Parallel
+### GLM-5.2 FP8 Server with online quant to MXFP8 MOE
 
 ```bash
 #!/bin/bash
 
-python -m atom.examples.simple_inference --model zai-org/GLM-5-FP8 -tp 8 --enable-dp-attention --enable-expert-parallel
+model_path=/shared/data/amd_int/models/GLM-5.2-FP8
+export AITER_QUICK_REDUCE_QUANTIZATION=INT4
+export AITER_USE_FLYDSL_MOE_SORTING=1
+TP=4
+
+rm -rf /root/.cache/atom/*
+
+python -m atom.entrypoints.openai_server \
+  --model "$model_path" \
+  --server-port 8000 \
+  --kv_cache_dtype fp8 \
+  --no-enable_prefix_caching \
+  --online_quant_config '{"layer_quant_config":{"model.layers.*.mlp.experts":"mxfp8"}}' \
+  -tp $TP 2>&1 | tee server.log &
+```
+
+### GLM-5.2 MXFP4 Server
+
+```bash
+#!/bin/bash
+
+model_path=amd/GLM-5.2-MXFP4
+export AITER_QUICK_REDUCE_QUANTIZATION=INT4
+export AITER_USE_FLYDSL_MOE_SORTING=1
+TP=4
+
+rm -rf /root/.cache/atom/*
+
+python -m atom.entrypoints.openai_server \
+  --model "$model_path" \
+  --server-port 8000 \
+  --kv_cache_dtype fp8 \
+  --no-enable_prefix_caching \
+  --online_quant_config '{"global_quant_config": "ptpc_fp8", "exclude_layer": ["lm_head", "model.embed_tokens", "*.mlp.gate", "*expert*"]}' \
+  -tp $TP 2>&1 | tee server.log &
+```
+
+### GLM-5.2 MXFP4 MTP Server
+
+```bash
+#!/bin/bash
+
+model_path=amd/GLM-5.2-MXFP4
+export AITER_QUICK_REDUCE_QUANTIZATION=INT4
+export AITER_USE_FLYDSL_MOE_SORTING=1
+TP=4
+
+rm -rf /root/.cache/atom/*
+
+python -m atom.entrypoints.openai_server \
+  --model "$model_path" \
+  --server-port 8004 \
+  --kv_cache_dtype fp8 \
+  --no-enable_prefix_caching \
+  --online_quant_config '{"global_quant_config": "ptpc_fp8", "exclude_layer": ["lm_head", "model.embed_tokens", "*.mlp.gate", "*expert*"]}' \
+  --num-speculative-tokens 3 \
+  --method mtp \
+  -tp $TP 2>&1 | tee server_mtp.log &
 ```
 
 Tips on server configuration:
 - We suggest using fp8 kv cache for better memory efficiency in the serving mode.
-- DP attention + EP MoE mode does not support fp8 kv cache when gqa=8, so `--kv_cache_dtype fp8` should not be used with `--enable-dp-attention --enable-expert-parallel`.
 - GLM-5 reuses the DeepSeek v3 implementation in ATOM (MLA attention, MoE routing), so all DeepSeek v3 optimizations apply automatically.
 - No `--trust-remote-code` is needed since ATOM has built-in support for `GlmMoeDsaForCausalLM`.
 
@@ -107,17 +175,9 @@ Here is the reference value when deploying on 8 ranks:
 
 [GLM-5.2](https://huggingface.co/zai-org/GLM-5.2-FP8) builds on the same `glm_moe_dsa` architecture as GLM-5 and adds **IndexShare**: the DSA indexer is computed only on `"full"` attention layers and reused by the following `"shared"` layers (the per-layer schedule is declared in `indexer_types`). Shared layers carry no indexer weights of their own. ATOM detects this schedule and enables the indexer cache automatically — no extra flags required.
 
-### Serving on 8xMI355 GPUs (TP8)
-
-```bash
-#!/bin/bash
-
-python -m atom.entrypoints.openai_server --model zai-org/GLM-5.2-FP8 -tp 8 --kv_cache_dtype bf16 --gpu-memory-utilization 0.8 --server-port 7777
-```
-
 Tips on server configuration:
-- Use `--kv_cache_dtype bf16` for the DSA sparse-attention path on CDNA4 (gfx950).
-- `--gpu-memory-utilization 0.8` leaves headroom for the per-layer DSA index cache; higher values may OOM during KV-cache allocation.
+- Use the FP8, MXFP4, or MXFP4 MTP server recipes above for GLM-5.2.
+- Use `--kv_cache_dtype fp8` with the optimized GLM-5.2 server recipes unless you are intentionally comparing against the older bf16 KV-cache baseline.
 - No `--trust-remote-code` is needed — ATOM has built-in support for `GlmMoeDsaForCausalLM`.
 
 ### Performance baseline
@@ -132,83 +192,3 @@ Reference numbers on 8×MI355X (TP8, FP8 weights, bf16 KV cache), using the benc
 | 8192 | 1024 | 1   | 73   | 669   | 409 | 13.2 |
 | 8192 | 1024 | 16  | 645  | 5818  | 418 | 23.3 |
 | 8192 | 1024 | 64  | 1210 | 10853 | 483 | 51.3 |
-
-## GLM-5.2 FP8 and MXFP4 Server Recipes
-
-Use the following docker image for these recipes:
-
-```bash
-docker pull docker.io/rocm/atom-dev:nightly_202606301541
-```
-
-Inside the container, install ATOM from the `zejun/opt_GLM5.2_0701` branch. AITER can remain unchanged.
-
-```bash
-cd PATH_TO_ATOM
-git checkout zejun/opt_GLM5.2_0701
-pip install -e .
-```
-
-### GLM-5.2 FP8 Server
-
-```bash
-#!/bin/bash
-
-model_path=/shared/data/amd_int/models/GLM-5.2-FP8
-export AITER_QUICK_REDUCE_QUANTIZATION=INT4
-export AITER_USE_FLYDSL_MOE_SORTING=1
-TP=4
-
-rm -rf /root/.cache/atom/*
-
-python -m atom.entrypoints.openai_server \
-  --model "$model_path" \
-  --server-port 8000 \
-  --kv_cache_dtype fp8 \
-  --no-enable_prefix_caching \
-  --online_quant_config '{"layer_quant_config":{"model.layers.*.mlp.experts":"mxfp8"}}' \
-  -tp $TP 2>&1 | tee server.log &
-```
-
-### GLM-5.2 MXFP4 Server
-
-```bash
-#!/bin/bash
-
-model_path=/shared/data/amd_int/models/GLM-5.2-MXFP4
-export AITER_QUICK_REDUCE_QUANTIZATION=INT4
-export AITER_USE_FLYDSL_MOE_SORTING=1
-TP=4
-
-rm -rf /root/.cache/atom/*
-
-python -m atom.entrypoints.openai_server \
-  --model "$model_path" \
-  --server-port 8000 \
-  --kv_cache_dtype fp8 \
-  --no-enable_prefix_caching \
-  --online_quant_config '{"global_quant_config": "ptpc_fp8", "exclude_layer": ["lm_head", "model.embed_tokens", "*.mlp.gate", "*expert*"]}' \
-  -tp $TP 2>&1 | tee server.log &
-```
-
-### GLM-5.2 MXFP4 MTP Server
-```bash
-#!/bin/bash
-
-model_path=/shared/data/amd_int/models/GLM-5.2-MXFP4
-export AITER_QUICK_REDUCE_QUANTIZATION=INT4
-export AITER_USE_FLYDSL_MOE_SORTING=1
-TP=4
-
-rm -rf /root/.cache/atom/*
-
-python -m atom.entrypoints.openai_server \
-  --model "$model_path" \
-  --server-port 8004 \
-  --kv_cache_dtype fp8 \
-  --no-enable_prefix_caching \
-  --online_quant_config '{"global_quant_config": "ptpc_fp8", "exclude_layer": ["lm_head", "model.embed_tokens", "*.mlp.gate", "*expert*"]}' \
-  --num-speculative-tokens 3 \
-  --method mtp \
-  -tp $TP 2>&1 | tee server_mtp.log &
-```