Skip to content

Commit 9c24e2c

Browse files
authored
Fix Deepseek transformers model loading (#740)
## What does this PR do? **Type of change:** ? Bug fix **Overview:** ? For Deepseek, let's force the user to apply trust_remote_code and use AutoModelForCausalLM for loading the model. ## Testing python hf_ptq.py --pyt_ckpt_path <Kimi-K2-Thinking_path> --qformat nvfp4 --export_path <quantized_ckpt> --kv_cache_qformat none --calib_size 64 --trust_remote_code --dataset cnn_dailymail ## Before your PR is "*Ready for review*" <!-- If you haven't finished some of the above items you can still open `Draft` PR. --> - **Make sure you read and follow [Contributor guidelines](https://github.com/NVIDIA/Model-Optimizer/blob/main/CONTRIBUTING.md)** and your commits are signed. - **Is this change backward compatible?**: Yes/No <!--- If No, explain why. --> - **Did you write any new necessary tests?**: Yes/No - **Did you add or update any necessary documentation?**: Yes/No - **Did you update [Changelog](https://github.com/NVIDIA/Model-Optimizer/blob/main/CHANGELOG.rst)?**: Yes/No <!--- Only for new features, API changes, critical bug fixes or bw breaking changes. --> ## Additional Information <!-- E.g. related issue. --> Signed-off-by: Chenjie Luo <chenjiel@nvidia.com>
1 parent 68d604d commit 9c24e2c

File tree

1 file changed

+6
-5
lines changed

1 file changed

+6
-5
lines changed

examples/llm_ptq/example_utils.py

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -349,11 +349,12 @@ def get_model(
349349
else:
350350
architecture = hf_config.architectures[0]
351351

352-
if not hasattr(transformers, architecture):
353-
warnings.warn(
354-
f"Architecture {architecture} not found in transformers: {transformers.__version__}. "
355-
"Falling back to AutoModelForCausalLM."
356-
)
352+
if not hasattr(transformers, architecture) or "Deepseek" in architecture:
353+
if not hasattr(transformers, architecture):
354+
warnings.warn(
355+
f"Architecture {architecture} not found in transformers: {transformers.__version__}. "
356+
"Falling back to AutoModelForCausalLM."
357+
)
357358
assert trust_remote_code, (
358359
"Please set trust_remote_code to True if you want to use this architecture"
359360
)

0 commit comments

Comments
 (0)