Skip to content

Commit 83509a9

Browse files
committed
remove --eagle3 from optimum command and switch model
1 parent 68a025b commit 83509a9

2 files changed

Lines changed: 11 additions & 7 deletions

File tree

demos/common/export_models/export_model.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -463,7 +463,7 @@ def export_text_generation_model(model_repository_path, source_model, model_name
463463
additional_options = ""
464464
if args["draft_eagle3_mode"]:
465465
print("Using eagle3 option for the draft model export")
466-
additional_options += " --eagle3 --task text-generation-with-past"
466+
additional_options += " --task text-generation-with-past"
467467
optimum_command = "optimum-cli export openvino --model {} --weight-format {} --trust-remote-code {} {}".format(draft_source_model, precision, additional_options, draft_llm_model_path)
468468
if os.system(optimum_command):
469469
raise ValueError("Failed to export llm model", source_model)

demos/continuous_batching/speculative_decoding/README.md

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ mkdir models
4040
Run `export_model.py` script to download and quantize the model:
4141

4242
```console
43-
python export_model.py text_generation --source_model Qwen/Qwen3-8B --draft_source_model Tengyunw/qwen3_8b_eagle3 --draft_eagle3_mode --weight-format int4 --config_file_path models/config.json --model_repository_path models
43+
python export_model.py text_generation --source_model Qwen/Qwen3-8B --draft_source_model AngelSlim/Qwen3-8B_eagle3 --draft_eagle3_mode --weight-format int4 --config_file_path models/config.json --model_repository_path models
4444
```
4545

4646
Draft model inherits all scheduler properties from the main model.
@@ -52,6 +52,12 @@ models
5252
└── Qwen
5353
└── Qwen3-8B
5454
├── added_tokens.json
55+
├── AngelSlim-Qwen3-8B_eagle3
56+
│   ├── config.json
57+
│   ├── generation_config.json
58+
│   ├── openvino_config.json
59+
│   ├── openvino_model.bin
60+
│   └── openvino_model.xml
5561
├── chat_template.jinja
5662
├── config.json
5763
├── generation_config.json
@@ -65,14 +71,10 @@ models
6571
├── openvino_tokenizer.bin
6672
├── openvino_tokenizer.xml
6773
├── special_tokens_map.json
68-
├── Tengyunw-qwen3_8b_eagle3
69-
│   ├── config.json
70-
│   ├── generation_config.json
71-
│   ├── openvino_model.bin
72-
│   └── openvino_model.xml
7374
├── tokenizer_config.json
7475
├── tokenizer.json
7576
└── vocab.json
77+
7678
```
7779

7880
## Server Deployment
@@ -313,6 +315,8 @@ for chunk in stream:
313315
```
314316

315317
Output:
318+
319+
```
316320
if len(numbers) <= 1:
317321
return numbers
318322
else:

0 commit comments

Comments
 (0)