Skip to content

Commit 69a3498

Browse files
committed
docs: address eagle3 review comments
1 parent 5e579af commit 69a3498

1 file changed

Lines changed: 15 additions & 2 deletions

File tree

docs/speculative.md

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,8 @@ A draft model is the most used approach in speculative decoding.
1717

1818
EAGLE-3 uses a small draft model that reads the target model's hidden states to predict the next tokens, so it
1919
reaches higher acceptance than a standalone draft model of the same size. The draft is a one-layer transformer
20-
trained for a specific target model; it shares the target's tokenizer and (optionally) a reduced draft vocabulary
21-
mapped back with a `d2t` table.
20+
trained for a specific target model; it shares the target model's tokenizer and, optionally, uses a reduced draft
21+
vocabulary with its own `lm_head`, which is mapped back using a `d2t` table.
2222

2323
Convert the EAGLE-3 checkpoint with `--target-model-dir` so it inherits the target's tokenizer and the layer
2424
indices to read. Both the SpecForge `LlamaForCausalLMEagle3` and the vLLM/AngelSlim `Eagle3LlamaForCausalLM`
@@ -32,6 +32,18 @@ python convert_hf_to_gguf.py AngelSlim/Qwen3-4B_eagle3 \
3232
llama-server -m Qwen3-4B.gguf -md Qwen3-4B-eagle3.gguf --spec-type draft-eagle3
3333
```
3434

35+
Supported EAGLE-3 draft models include:
36+
37+
- [yuhuili/EAGLE3-LLaMA3.1-Instruct-8B](https://huggingface.co/yuhuili/EAGLE3-LLaMA3.1-Instruct-8B)
38+
- [yuhuili/EAGLE3-LLaMA3.3-Instruct-70B](https://huggingface.co/yuhuili/EAGLE3-LLaMA3.3-Instruct-70B)
39+
- [RedHatAI/gemma-4-31B-it-speculator.eagle3](https://huggingface.co/RedHatAI/gemma-4-31B-it-speculator.eagle3)
40+
- [RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3](https://huggingface.co/RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3)
41+
- [Tengyunw/qwen3_8b_eagle3](https://huggingface.co/Tengyunw/qwen3_8b_eagle3)
42+
- [Tengyunw/qwen3_30b_moe_eagle3](https://huggingface.co/Tengyunw/qwen3_30b_moe_eagle3)
43+
- [AngelSlim/Qwen3-8B_eagle3](https://huggingface.co/AngelSlim/Qwen3-8B_eagle3)
44+
- [AngelSlim/Qwen3-14B_eagle3](https://huggingface.co/AngelSlim/Qwen3-14B_eagle3)
45+
- [AngelSlim/Qwen3-32B_eagle3](https://huggingface.co/AngelSlim/Qwen3-32B_eagle3)
46+
3547
### n-gram Cache (`ngram-cache`)
3648

3749
An n-gram is a sequence of n tokens. The n-gram cache implementation maintains statistics about short n-gram sequences.
@@ -266,6 +278,7 @@ Specifies a comma-separated list of speculative decoding types to use.
266278
|------|-------------|
267279
| `none` | No speculative decoding (default) |
268280
| `draft-simple` | Use a simple draft model for speculation |
281+
| `draft-eagle3` | Use an EAGLE-3 draft model that reads the target's hidden states |
269282
| `draft-mtp` | Use Multi Token Prediction (MTP) heads from the main model |
270283
| `ngram-cache` | Use n-gram cache lookup |
271284
| `ngram-simple` | Use simple n-gram pattern matching |

0 commit comments

Comments
 (0)