@@ -17,8 +17,8 @@ A draft model is the most used approach in speculative decoding.
1717
1818EAGLE-3 uses a small draft model that reads the target model's hidden states to predict the next tokens, so it
1919reaches higher acceptance than a standalone draft model of the same size. The draft is a one-layer transformer
20- trained for a specific target model; it shares the target's tokenizer and ( optionally) a reduced draft vocabulary
21- mapped back with a ` d2t ` table.
20+ trained for a specific target model; it shares the target model 's tokenizer and, optionally, uses a reduced draft
21+ vocabulary with its own ` lm_head ` , which is mapped back using a ` d2t ` table.
2222
2323Convert the EAGLE-3 checkpoint with ` --target-model-dir ` so it inherits the target's tokenizer and the layer
2424indices to read. Both the SpecForge ` LlamaForCausalLMEagle3 ` and the vLLM/AngelSlim ` Eagle3LlamaForCausalLM `
@@ -32,6 +32,18 @@ python convert_hf_to_gguf.py AngelSlim/Qwen3-4B_eagle3 \
3232llama-server -m Qwen3-4B.gguf -md Qwen3-4B-eagle3.gguf --spec-type draft-eagle3
3333```
3434
35+ Supported EAGLE-3 draft models include:
36+
37+ - [ yuhuili/EAGLE3-LLaMA3.1-Instruct-8B] ( https://huggingface.co/yuhuili/EAGLE3-LLaMA3.1-Instruct-8B )
38+ - [ yuhuili/EAGLE3-LLaMA3.3-Instruct-70B] ( https://huggingface.co/yuhuili/EAGLE3-LLaMA3.3-Instruct-70B )
39+ - [ RedHatAI/gemma-4-31B-it-speculator.eagle3] ( https://huggingface.co/RedHatAI/gemma-4-31B-it-speculator.eagle3 )
40+ - [ RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3] ( https://huggingface.co/RedHatAI/gemma-4-26B-A4B-it-speculator.eagle3 )
41+ - [ Tengyunw/qwen3_8b_eagle3] ( https://huggingface.co/Tengyunw/qwen3_8b_eagle3 )
42+ - [ Tengyunw/qwen3_30b_moe_eagle3] ( https://huggingface.co/Tengyunw/qwen3_30b_moe_eagle3 )
43+ - [ AngelSlim/Qwen3-8B_eagle3] ( https://huggingface.co/AngelSlim/Qwen3-8B_eagle3 )
44+ - [ AngelSlim/Qwen3-14B_eagle3] ( https://huggingface.co/AngelSlim/Qwen3-14B_eagle3 )
45+ - [ AngelSlim/Qwen3-32B_eagle3] ( https://huggingface.co/AngelSlim/Qwen3-32B_eagle3 )
46+
3547### n-gram Cache (` ngram-cache ` )
3648
3749An n-gram is a sequence of n tokens. The n-gram cache implementation maintains statistics about short n-gram sequences.
@@ -266,6 +278,7 @@ Specifies a comma-separated list of speculative decoding types to use.
266278| ------| -------------|
267279| ` none ` | No speculative decoding (default) |
268280| ` draft-simple ` | Use a simple draft model for speculation |
281+ | ` draft-eagle3 ` | Use an EAGLE-3 draft model that reads the target's hidden states |
269282| ` draft-mtp ` | Use Multi Token Prediction (MTP) heads from the main model |
270283| ` ngram-cache ` | Use n-gram cache lookup |
271284| ` ngram-simple ` | Use simple n-gram pattern matching |
0 commit comments