|
| 1 | +--- |
| 2 | +title: "GLM-4.5 Meets SGLang: Reasoning, Coding, and Agentic Abilities" |
| 3 | +author: "GLM Team" |
| 4 | +date: "July 31, 2025" |
| 5 | +previewImg: /images/blog/glm_4_5/GLM-4-5-preview.png |
| 6 | +--- |
| 7 | + |
| 8 | +Today, we are excited to introduce our latest flagship models [GLM-4.5](https://huggingface.co/zai-org/GLM-4.5) and [GLM-4.5-Air](https://huggingface.co/zai-org/GLM-4.5-Air), along with their FP8 variants. All models are now available with day-one support on SGLang. |
| 9 | +GLM-4.5 and GLM-4.5-Air are both powerful models designed to unify reasoning, coding, and agentic capabilities, with **355B** total parameters (**32B** active) and **106B** total parameters (**12B** active) respectively. |
| 10 | + |
| 11 | + |
| 12 | +## Deploying GLM-4.5 with SGLang |
| 13 | + |
| 14 | +We recommend deploying the **GLM-4.5 series** of models using SGLang for optimal performance. Through close collaboration with the SGLang community, all GLM-4.5 models are fully supported on SGLang starting from day one. |
| 15 | + |
| 16 | +### Basic Usage |
| 17 | + |
| 18 | + |
| 19 | +**Install SGLang** |
| 20 | +``` |
| 21 | +pip install --upgrade pip |
| 22 | +pip install "sglang[all]>=0.4.9.post6" |
| 23 | +``` |
| 24 | + |
| 25 | +**355B Model** |
| 26 | +``` |
| 27 | +python3 -m sglang.launch_server --model zai-org/GLM-4.5 --tp 8 |
| 28 | +``` |
| 29 | +**106B Model** |
| 30 | +``` |
| 31 | +python3 -m sglang.launch_server --model zai-org/GLM-4.5-Air --tp 8 |
| 32 | +``` |
| 33 | +**355B FP8 Quantized Model** |
| 34 | +``` |
| 35 | +python3 -m sglang.launch_server --model zai-org/GLM-4.5-FP8 --tp 8 |
| 36 | +``` |
| 37 | +**106B FP8 Quantized Model** |
| 38 | +``` |
| 39 | +python3 -m sglang.launch_server --model zai-org/GLM-4.5-Air-FP8 --tp 4 |
| 40 | +``` |
| 41 | + |
| 42 | +### Tool Call |
| 43 | +Append the following parameter to the command: |
| 44 | +``` |
| 45 | +--tool-call-parser glm45 |
| 46 | +``` |
| 47 | +### Reasoning Parser |
| 48 | +Append the following parameter to the command: |
| 49 | +``` |
| 50 | +--reasoning-parser glm45 |
| 51 | +``` |
| 52 | + |
| 53 | +### Speculative Decoding with MTP |
| 54 | +Append the following parameters to the command: |
| 55 | +``` |
| 56 | +--speculative-algorithm EAGLE \ |
| 57 | +--speculative-num-steps [number of steps] \ |
| 58 | +--speculative-eagle-topk [top k] \ |
| 59 | +--speculative-num-draft-tokens [number of draft tokens] |
| 60 | +``` |
| 61 | + |
| 62 | +## GLM 4.5 Model Architecture and Highlights |
| 63 | +**GLM-4.5** adopts a MoE architecture with loss-free balance routing and sigmoid gates, enhancing compute efficiency. Compared to models like DeepSeek-V3 and Kimi K2, we prioritize depth over width—fewer experts and smaller hidden dimensions, but more layers—resulting in better reasoning performance. |
| 64 | + |
| 65 | +**Key architectural designs and highlights:** |
| 66 | +* Grouped-Query Attention with partial RoPE |
| 67 | +* 96 attention heads for 5120 hidden size (2.5× more than typical), improving reasoning on MMLU/BBH despite similar training loss |
| 68 | +* QK-Norm for stabilized attention logits |
| 69 | +* Muon optimizer, enabling faster convergence and larger batch sizes |
| 70 | +* MTP (Multi-Token Prediction) head for speculative decoding |
| 71 | +* RL training is powered by the open-source framework [slime](https://github.com/THUDM/slime), which was earlier open-sourced by [THUDM](https://github.com/thudm). |
| 72 | + |
| 73 | + |
| 74 | +### Performance |
| 75 | +We compare **GLM-4.5** with various models from OpenAI, Anthropic, Google DeepMind, xAI, Alibaba, Moonshot, and DeepSeek on 12 benchmarks covering agentic (3), reasoning (7), and Coding (2). Overall, **GLM-4.5** is ranked at the 3rd place and **GLM-4.5 Air** is ranked at the 6th. |
| 76 | + |
| 77 | + |
| 78 | + |
| 79 | + |
| 80 | +**Agentic Abilities** |
| 81 | +GLM-4.5 supports 128k context and native function calling. On both $\tau$-bench and BFCL-v3, it matches Claude 4 Sonnet, and on the BrowseComp web browsing benchmark, it surpasses Claude 4 Opus (26.4% vs. 18.8%) and approaches GPT o4-mini-high (28.3%). Its high tool-calling success rate (90.6%) highlights its reliability in agent-based workflows. |
| 82 | + |
| 83 | +**Reasoning** |
| 84 | +GLM-4.5 excels in mathematical and logical reasoning. It scores competitively on MMLU Pro (84.6), AIME-24 (91.0), and MATH500 (98.2), and demonstrates strong generalization across benchmarks like GPQA, LCB, and AA-Index. |
| 85 | + |
| 86 | +**Coding** |
| 87 | +GLM-4.5 shows comprehensive full-stack development ability and ranks among the top models on SWE-bench Verified (64.2) and Terminal-Bench (37.5). In head-to-head evaluations, it achieves a 53.9% win rate over Kimi K2 and 80.8% over Qwen3-Coder. Its high agentic reliability, multi-round coding task performance, and visual interface quality demonstrate its strength as an autonomous coding assistant. |
| 88 | + |
| 89 | +## Conclusion |
| 90 | +The **GLM-4.5 series** represents a new wave of large language models, excelling in long-context reasoning, agentic workflows, and coding tasks. Its hybrid MoE architecture—enhanced by techniques like grouped-query attention, MTP, and RL training—offers both efficiency and strong capability. |
| 91 | + |
| 92 | +**SGLang** provides a production-ready, high-performance inference stack, enabling seamless deployment through advanced memory management and request batching. |
| 93 | + |
| 94 | +Together, **GLM-4.5** and **SGLang** form a robust foundation for next-generation AI—powering intelligent, scalable solutions across code, documents, and agents. |
| 95 | + |
| 96 | +## Acknowledgement |
| 97 | +We would like to express our heartfelt gratitude to the following teams and collaborators in this [PR](https://github.com/sgl-project/sglang/pull/8224): |
| 98 | + |
| 99 | +- **[GLM Team](https://github.com/THUDM/GLM)**: Yuxuan Zhang, Chenhui Zhang, Xin Lv, Zilin Zhu and colleagues. |
| 100 | +- **[SGLang Team and community](https://docs.sglang.ai/index.html)**: Biao He, Lifu Huang, Binyao Jiang, Minglei Zhu, Cheng Wan, Chang Su and many others. |
0 commit comments