Code Used :
from optimum.intel import OVModelForCausalLM
from transformers import AutoTokenizer, pipeline
model_id = "moonshotai/Kimi-K2.5"
model = OVModelForCausalLM.from_pretrained(model_id , trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(model_id)
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
results = pipe("Hey, how are you doing today?", max_new_tokens=100)
Output
No OpenVINO files were found for moonshotai/Kimi-K2.5, setting export=True to convert the model to the OpenVINO IR. Don't forget to save the resulting model with .save_pretrained()
configuration_kimi_k25.py:
5.43k/? [00:00<00:00, 117kB/s]
configuration_deepseek.py:
10.6k/? [00:00<00:00, 629kB/s]
A new version of the following files was downloaded from https://huggingface.co/moonshotai/Kimi-K2.5:
- configuration_deepseek.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
A new version of the following files was downloaded from https://huggingface.co/moonshotai/Kimi-K2.5:
- configuration_kimi_k25.py
- configuration_deepseek.py
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
ValueError Traceback (most recent call last)
/tmp/ipykernel_3111/4003728944.py in <cell line: 0>()
3
4 model_id = "moonshotai/Kimi-K2.5"
----> 5 model = OVModelForCausalLM.from_pretrained(model_id , trust_remote_code=True)
6 tokenizer = AutoTokenizer.from_pretrained(model_id)
7 pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
3 frames
/usr/local/lib/python3.12/dist-packages/optimum/exporters/openvino/main.py in main_export(model_name_or_path, output, task, device, framework, cache_dir, trust_remote_code, pad_token_id, subfolder, revision, force_download, local_files_only, token, model_kwargs, custom_export_configs, fn_get_submodels, ov_config, stateful, convert_tokenizer, library_name, model_loading_kwargs, variant, **kwargs_shapes)
338 if model_type not in TasksManager._SUPPORTED_MODEL_TYPE:
339 if custom_export_configs is None:
--> 340 raise ValueError(
341 f"Trying to export a {model_type} model, that is a custom or unsupported architecture, but no "
342 "custom export configuration was passed as custom_export_configs. Please refer to "
ValueError: Trying to export a kimi_k25 model, that is a custom or unsupported architecture, but no custom export configuration was passed as custom_export_configs. Please refer to https://huggingface.co/docs/optimum/main/en/exporters/onnx/usage_guides/export_a_model#custom-export-of-transformers-models for an example on how to export custom models. Please open an issue at https://github.com/huggingface/optimum-intel/issues if you would like the model type kimi_k25 to be supported natively in the OpenVINO export.
Code Used :
Output
No OpenVINO files were found for moonshotai/Kimi-K2.5, setting
export=Trueto convert the model to the OpenVINO IR. Don't forget to save the resulting model with.save_pretrained()configuration_kimi_k25.py:
5.43k/? [00:00<00:00, 117kB/s]
configuration_deepseek.py:
10.6k/? [00:00<00:00, 629kB/s]
A new version of the following files was downloaded from https://huggingface.co/moonshotai/Kimi-K2.5:
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
A new version of the following files was downloaded from https://huggingface.co/moonshotai/Kimi-K2.5:
. Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
ValueError Traceback (most recent call last)
/tmp/ipykernel_3111/4003728944.py in <cell line: 0>()
3
4 model_id = "moonshotai/Kimi-K2.5"
----> 5 model = OVModelForCausalLM.from_pretrained(model_id , trust_remote_code=True)
6 tokenizer = AutoTokenizer.from_pretrained(model_id)
7 pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
3 frames
/usr/local/lib/python3.12/dist-packages/optimum/exporters/openvino/main.py in main_export(model_name_or_path, output, task, device, framework, cache_dir, trust_remote_code, pad_token_id, subfolder, revision, force_download, local_files_only, token, model_kwargs, custom_export_configs, fn_get_submodels, ov_config, stateful, convert_tokenizer, library_name, model_loading_kwargs, variant, **kwargs_shapes)
338 if model_type not in TasksManager._SUPPORTED_MODEL_TYPE:
339 if custom_export_configs is None:
--> 340 raise ValueError(
341 f"Trying to export a {model_type} model, that is a custom or unsupported architecture, but no "
342 "custom export configuration was passed as
custom_export_configs. Please refer to "ValueError: Trying to export a kimi_k25 model, that is a custom or unsupported architecture, but no custom export configuration was passed as
custom_export_configs. Please refer to https://huggingface.co/docs/optimum/main/en/exporters/onnx/usage_guides/export_a_model#custom-export-of-transformers-models for an example on how to export custom models. Please open an issue at https://github.com/huggingface/optimum-intel/issues if you would like the model type kimi_k25 to be supported natively in the OpenVINO export.