Skip to content

Commit a1b7c7a

Browse files
committed
Add Jupyter notebook guide for training Eagle on the CosmosReason2 model
Signed-off-by: Slawek Kierat <skierat@nvidia.com>
1 parent 7c4c9fd commit a1b7c7a

4 files changed

Lines changed: 90107 additions & 0 deletions

File tree

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"draft_vocab_size": 32000,
3+
"initializer_range": 0.02,
4+
"rms_norm_eps": 1e-06,
5+
"_attn_implementation": "flex_attention",
6+
"rope_scaling": {
7+
"beta_fast": 32.0,
8+
"beta_slow": 1.0,
9+
"factor": 32.0,
10+
"original_max_position_embeddings": 4096,
11+
"rope_type": "yarn",
12+
"truncate": false
13+
},
14+
"rope_theta": 150000
15+
}

0 commit comments

Comments
 (0)