Skip to content

Commit a4f05dc

Browse files
authored
Update 2025-07-25-spec-forge.md (#177)
* Update 2025-07-25-spec-forge.md * Update 2025-07-25-spec-forge.md
1 parent 6858184 commit a4f05dc

1 file changed

Lines changed: 2 additions & 2 deletions

File tree

blog/2025-07-25-spec-forge.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ This high performance is largely driven by Eagle's novel Training-Time Test (TTT
3333

3434
### Two Training Modes: Online and Offline
3535

36-
SpecForge simplifies hidden state collection by offering two versatile modes for training: **Online** and **Offline**. This two-mode design ensures flexibility across workflows, regardless of your model access rights or hardware limitations.
36+
SpecForge simplifies hidden state collection by offering two versatile modes for training: **Online** and **Offline**. This two-mode design ensures flexibility across workflows, regardless of your model sizes or hardware limitations.
3737

3838
![offline_vs_online.svg](/images/blog/spec_forge/offline_online.jpg)
3939

@@ -55,7 +55,7 @@ To support large-scale models, SpecForge leverages PyTorch’s FSDP and integrat
5555

5656
Using SpecForge, we trained the Llama 4 Scout and Maverick models on a 320K-sample dataset from ShareGPT and UltraChat. The models' strong performance on benchmarks like MT-Bench demonstrates their effectiveness and readiness for Eagle3 inference. Our Llama 4 Maverick draft model achieves a 2.18× speedup on MT-Bench, while the Scout variant delivers a 2.0× acceleration—demonstrating SpecForge’s performance gains across model variants. Detailed results are summarized below.
5757

58-
We evaluated various draft token lengths for Scout and Maverick.
58+
We evaluated various draft token lengths for Scout and Maverick.
5959

6060
In all the tests shown in the figure below, the x-axis represents steps, corresponding to `speculative-num-steps` in SGLang. Meanwhile, we fixed SGLang's `speculative-eagle-topk` to 8 and `speculative-num-draft-tokens` to 10 to ensure that `tree attention` can be enabled. To find the optimal speculative decoding parameters, we can use the **[bench_speculative](https://github.com/sgl-project/sglang/blob/main/scripts/playground/bench_speculative.py)** script in the SGLang repository. It runs throughput benchmarks across different configurations and helps us tune for the best performance on the hardware.
6161

0 commit comments

Comments
 (0)