vLLM: Unable to load modeling_iquestcoder.IQuestCoderModel from [insert any IQuestLab model here] on HF Hub.

[PathosEthosLogos]

Command: sudo docker run -d --runtime nvidia --gpus all -p 8000:8000 --ipc=host --platform "linux/arm64" vllm/vllm-openai:nightly --model IQuestLab/IQuest-Coder-V1-40B-Base --tensor-parallel-size 8 --dtype auto --max-model-len 1024 --max-num-batched-tokens=512 --served-model-name vllm-io --trust-remote-code

Tried with instruct, looping, and base models. All same error:

WARNING 01-06 10:34:06 [argparse_utils.py:195] With `vllm serve`, you should provide the model as a positional argument or in a config file instead of via the `--model` option. The `--model` option will be removed in v0.13.
(APIServer pid=1) INFO 01-06 10:34:06 [api_server.py:1277] vLLM API server version 0.14.0rc1.dev221+g97a01308e
(APIServer pid=1) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=1) INFO 01-06 10:34:06 [utils.py:253] non-default args: {'model_tag': 'IQuestLab/IQuest-Coder-V1-40B-Base', 'model': 'IQuestLab/IQuest-Coder-V1-40B-Base', 'trust_remote_code': True, 'max_model_len': 1024, 'enforce_eager': True, 'served_model_name': ['vllm-io'], 'tensor_parallel_size': 8, 'max_num_batched_tokens': 512}
(APIServer pid=1) The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=1) A new version of the following files was downloaded from https://huggingface.co/IQuestLab/IQuest-Coder-V1-40B-Base:
(APIServer pid=1) - configuration_iquestcoder.py
(APIServer pid=1) . Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
(APIServer pid=1) A new version of the following files was downloaded from https://huggingface.co/IQuestLab/IQuest-Coder-V1-40B-Base:
(APIServer pid=1) - modeling_iquestcoder.py
(APIServer pid=1) . Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51] Unable to load modeling_iquestcoder.IQuestCoderModel from IQuestLab/IQuest-Coder-V1-40B-Base on HF Hub.
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51] Traceback (most recent call last):
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/dynamic_module.py", line 33, in try_get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]     return get_class_from_dynamic_module(
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 616, in get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]     return get_class_in_module(class_name, final_module, force_reload=force_download)
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 311, in get_class_in_module
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]     module_spec.loader.exec_module(module)
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "<frozen importlib._bootstrap_external>", line 999, in exec_module
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "/root/.cache/huggingface/modules/transformers_modules/IQuestLab/IQuest_hyphen_Coder_hyphen_V1_hyphen_40B_hyphen_Base/4e7c31e9eb7976c47b1768eef7296a43a95a97cb/modeling_iquestcoder.py", line 50, in <module>
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]     from transformers.utils import (
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51] ImportError: cannot import name 'LossKwargs' from 'transformers.utils' (/usr/local/lib/python3.12/dist-packages/transformers/utils/__init__.py)
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51] Unable to load modeling_iquestcoder.IQuestCoderForCausalLM from IQuestLab/IQuest-Coder-V1-40B-Base on HF Hub.
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51] Traceback (most recent call last):
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/dynamic_module.py", line 33, in try_get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]     return get_class_from_dynamic_module(
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 616, in get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]     return get_class_in_module(class_name, final_module, force_reload=force_download)
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 311, in get_class_in_module
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]     module_spec.loader.exec_module(module)
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "<frozen importlib._bootstrap_external>", line 999, in exec_module
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "/root/.cache/huggingface/modules/transformers_modules/IQuestLab/IQuest_hyphen_Coder_hyphen_V1_hyphen_40B_hyphen_Base/4e7c31e9eb7976c47b1768eef7296a43a95a97cb/modeling_iquestcoder.py", line 50, in <module>
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]     from transformers.utils import (
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51] ImportError: cannot import name 'LossKwargs' from 'transformers.utils' (/usr/local/lib/python3.12/dist-packages/transformers/utils/__init__.py)
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51] Unable to load modeling_iquestcoder.IQuestCoderForSequenceClassification from IQuestLab/IQuest-Coder-V1-40B-Base on HF Hub.
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51] Traceback (most recent call last):
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/dynamic_module.py", line 33, in try_get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]     return get_class_from_dynamic_module(
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 616, in get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]     return get_class_in_module(class_name, final_module, force_reload=force_download)
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 311, in get_class_in_module
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]     module_spec.loader.exec_module(module)
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "<frozen importlib._bootstrap_external>", line 999, in exec_module
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "/root/.cache/huggingface/modules/transformers_modules/IQuestLab/IQuest_hyphen_Coder_hyphen_V1_hyphen_40B_hyphen_Base/4e7c31e9eb7976c47b1768eef7296a43a95a97cb/modeling_iquestcoder.py", line 50, in <module>
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]     from transformers.utils import (
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51] ImportError: cannot import name 'LossKwargs' from 'transformers.utils' (/usr/local/lib/python3.12/dist-packages/transformers/utils/__init__.py)
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51] Unable to load modeling_iquestcoder.IQuestCoderForTokenClassification from IQuestLab/IQuest-Coder-V1-40B-Base on HF Hub.
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51] Traceback (most recent call last):
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/dynamic_module.py", line 33, in try_get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]     return get_class_from_dynamic_module(
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 616, in get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]     return get_class_in_module(class_name, final_module, force_reload=force_download)
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 311, in get_class_in_module
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]     module_spec.loader.exec_module(module)
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "<frozen importlib._bootstrap_external>", line 999, in exec_module
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]   File "/root/.cache/huggingface/modules/transformers_modules/IQuestLab/IQuest_hyphen_Coder_hyphen_V1_hyphen_40B_hyphen_Base/4e7c31e9eb7976c47b1768eef7296a43a95a97cb/modeling_iquestcoder.py", line 50, in <module>
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51]     from transformers.utils import (
(APIServer pid=1) WARNING 01-06 10:34:08 [dynamic_module.py:51] ImportError: cannot import name 'LossKwargs' from 'transformers.utils' (/usr/local/lib/python3.12/dist-packages/transformers/utils/__init__.py)
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51] Unable to load modeling_iquestcoder.IQuestCoderForQuestionAnswering from IQuestLab/IQuest-Coder-V1-40B-Base on HF Hub.
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51] Traceback (most recent call last):
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/dynamic_module.py", line 33, in try_get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     return get_class_from_dynamic_module(
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 616, in get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     return get_class_in_module(class_name, final_module, force_reload=force_download)
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 311, in get_class_in_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     module_spec.loader.exec_module(module)
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "<frozen importlib._bootstrap_external>", line 999, in exec_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/root/.cache/huggingface/modules/transformers_modules/IQuestLab/IQuest_hyphen_Coder_hyphen_V1_hyphen_40B_hyphen_Base/4e7c31e9eb7976c47b1768eef7296a43a95a97cb/modeling_iquestcoder.py", line 50, in <module>
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     from transformers.utils import (
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51] ImportError: cannot import name 'LossKwargs' from 'transformers.utils' (/usr/local/lib/python3.12/dist-packages/transformers/utils/__init__.py)
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51] Unable to load modeling_iquestcoder.IQuestCoderModel from IQuestLab/IQuest-Coder-V1-40B-Base on HF Hub.
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51] Traceback (most recent call last):
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/dynamic_module.py", line 33, in try_get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     return get_class_from_dynamic_module(
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 616, in get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     return get_class_in_module(class_name, final_module, force_reload=force_download)
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 311, in get_class_in_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     module_spec.loader.exec_module(module)
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "<frozen importlib._bootstrap_external>", line 999, in exec_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/root/.cache/huggingface/modules/transformers_modules/IQuestLab/IQuest_hyphen_Coder_hyphen_V1_hyphen_40B_hyphen_Base/4e7c31e9eb7976c47b1768eef7296a43a95a97cb/modeling_iquestcoder.py", line 50, in <module>
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     from transformers.utils import (
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51] ImportError: cannot import name 'LossKwargs' from 'transformers.utils' (/usr/local/lib/python3.12/dist-packages/transformers/utils/__init__.py)
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51] Unable to load modeling_iquestcoder.IQuestCoderForCausalLM from IQuestLab/IQuest-Coder-V1-40B-Base on HF Hub.
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51] Traceback (most recent call last):
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/dynamic_module.py", line 33, in try_get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     return get_class_from_dynamic_module(
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 616, in get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     return get_class_in_module(class_name, final_module, force_reload=force_download)
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 311, in get_class_in_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     module_spec.loader.exec_module(module)
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "<frozen importlib._bootstrap_external>", line 999, in exec_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/root/.cache/huggingface/modules/transformers_modules/IQuestLab/IQuest_hyphen_Coder_hyphen_V1_hyphen_40B_hyphen_Base/4e7c31e9eb7976c47b1768eef7296a43a95a97cb/modeling_iquestcoder.py", line 50, in <module>
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     from transformers.utils import (
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51] ImportError: cannot import name 'LossKwargs' from 'transformers.utils' (/usr/local/lib/python3.12/dist-packages/transformers/utils/__init__.py)
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51] Unable to load modeling_iquestcoder.IQuestCoderForSequenceClassification from IQuestLab/IQuest-Coder-V1-40B-Base on HF Hub.
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51] Traceback (most recent call last):
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/dynamic_module.py", line 33, in try_get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     return get_class_from_dynamic_module(
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 616, in get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     return get_class_in_module(class_name, final_module, force_reload=force_download)
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 311, in get_class_in_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     module_spec.loader.exec_module(module)
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "<frozen importlib._bootstrap_external>", line 999, in exec_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/root/.cache/huggingface/modules/transformers_modules/IQuestLab/IQuest_hyphen_Coder_hyphen_V1_hyphen_40B_hyphen_Base/4e7c31e9eb7976c47b1768eef7296a43a95a97cb/modeling_iquestcoder.py", line 50, in <module>
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     from transformers.utils import (
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51] ImportError: cannot import name 'LossKwargs' from 'transformers.utils' (/usr/local/lib/python3.12/dist-packages/transformers/utils/__init__.py)
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51] Unable to load modeling_iquestcoder.IQuestCoderForTokenClassification from IQuestLab/IQuest-Coder-V1-40B-Base on HF Hub.
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51] Traceback (most recent call last):
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/dynamic_module.py", line 33, in try_get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     return get_class_from_dynamic_module(
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 616, in get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     return get_class_in_module(class_name, final_module, force_reload=force_download)
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 311, in get_class_in_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     module_spec.loader.exec_module(module)
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "<frozen importlib._bootstrap_external>", line 999, in exec_module
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]   File "/root/.cache/huggingface/modules/transformers_modules/IQuestLab/IQuest_hyphen_Coder_hyphen_V1_hyphen_40B_hyphen_Base/4e7c31e9eb7976c47b1768eef7296a43a95a97cb/modeling_iquestcoder.py", line 50, in <module>
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51]     from transformers.utils import (
(APIServer pid=1) WARNING 01-06 10:34:09 [dynamic_module.py:51] ImportError: cannot import name 'LossKwargs' from 'transformers.utils' (/usr/local/lib/python3.12/dist-packages/transformers/utils/__init__.py)
(APIServer pid=1) Traceback (most recent call last):
(APIServer pid=1)   File "/usr/local/bin/vllm", line 10, in <module>
(APIServer pid=1)     sys.exit(main())
(APIServer pid=1)              ^^^^^^
(APIServer pid=1)   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/cli/main.py", line 73, in main
(APIServer pid=1)     args.dispatch_function(args)
(APIServer pid=1)   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/cli/serve.py", line 60, in cmd
(APIServer pid=1)     uvloop.run(run_server(args))
(APIServer pid=1)   File "/usr/local/lib/python3.12/dist-packages/uvloop/__init__.py", line 96, in run
(APIServer pid=1)     return __asyncio.run(
(APIServer pid=1)            ^^^^^^^^^^^^^^
(APIServer pid=1)   File "/usr/lib/python3.12/asyncio/runners.py", line 195, in run
(APIServer pid=1)     return runner.run(main)
(APIServer pid=1)            ^^^^^^^^^^^^^^^^
(APIServer pid=1)   File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
(APIServer pid=1)     return self._loop.run_until_complete(task)
(APIServer pid=1)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1)   File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=1)   File "/usr/local/lib/python3.12/dist-packages/uvloop/__init__.py", line 48, in wrapper
(APIServer pid=1)     return await main
(APIServer pid=1)            ^^^^^^^^^^
(APIServer pid=1)   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 1324, in run_server
(APIServer pid=1) WARNING 01-06 10:34:10 [dynamic_module.py:51] Unable to load modeling_iquestcoder.IQuestCoderForQuestionAnswering from IQuestLab/IQuest-Coder-V1-40B-Base on HF Hub.
(APIServer pid=1) WARNING 01-06 10:34:10 [dynamic_module.py:51] Traceback (most recent call last):
(APIServer pid=1) WARNING 01-06 10:34:10 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/vllm/transformers_utils/dynamic_module.py", line 33, in try_get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:10 [dynamic_module.py:51]     return get_class_from_dynamic_module(
(APIServer pid=1) WARNING 01-06 10:34:10 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:10 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 616, in get_class_from_dynamic_module
(APIServer pid=1) WARNING 01-06 10:34:10 [dynamic_module.py:51]     return get_class_in_module(class_name, final_module, force_reload=force_download)
(APIServer pid=1) WARNING 01-06 10:34:10 [dynamic_module.py:51]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1) WARNING 01-06 10:34:10 [dynamic_module.py:51]   File "/usr/local/lib/python3.12/dist-packages/transformers/dynamic_module_utils.py", line 311, in get_class_in_module
(APIServer pid=1) WARNING 01-06 10:34:10 [dynamic_module.py:51]     module_spec.loader.exec_module(module)
(APIServer pid=1) WARNING 01-06 10:34:10 [dynamic_module.py:51]   File "<frozen importlib._bootstrap_external>", line 999, in exec_module
(APIServer pid=1) WARNING 01-06 10:34:10 [dynamic_module.py:51]   File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
(APIServer pid=1) WARNING 01-06 10:34:10 [dynamic_module.py:51]   File "/root/.cache/huggingface/modules/transformers_modules/IQuestLab/IQuest_hyphen_Coder_hyphen_V1_hyphen_40B_hyphen_Base/4e7c31e9eb7976c47b1768eef7296a43a95a97cb/modeling_iquestcoder.py", line 50, in <module>
(APIServer pid=1) WARNING 01-06 10:34:10 [dynamic_module.py:51]     from transformers.utils import (
(APIServer pid=1) WARNING 01-06 10:34:10 [dynamic_module.py:51] ImportError: cannot import name 'LossKwargs' from 'transformers.utils' (/usr/local/lib/python3.12/dist-packages/transformers/utils/__init__.py)
(APIServer pid=1)     await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=1)   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 1343, in run_server_worker
(APIServer pid=1)     async with build_async_engine_client(
(APIServer pid=1)                ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1)   File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=1)     return await anext(self.gen)
(APIServer pid=1)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1)   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 171, in build_async_engine_client
(APIServer pid=1)     async with build_async_engine_client_from_engine_args(
(APIServer pid=1)                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1)   File "/usr/lib/python3.12/contextlib.py", line 210, in __aenter__
(APIServer pid=1)     return await anext(self.gen)
(APIServer pid=1)            ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1)   File "/usr/local/lib/python3.12/dist-packages/vllm/entrypoints/openai/api_server.py", line 197, in build_async_engine_client_from_engine_args
(APIServer pid=1)     vllm_config = engine_args.create_engine_config(usage_context=usage_context)
(APIServer pid=1)                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1)   File "/usr/local/lib/python3.12/dist-packages/vllm/engine/arg_utils.py", line 1348, in create_engine_config
(APIServer pid=1)     model_config = self.create_model_config()
(APIServer pid=1)                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1)   File "/usr/local/lib/python3.12/dist-packages/vllm/engine/arg_utils.py", line 1205, in create_model_config
(APIServer pid=1)     return ModelConfig(
(APIServer pid=1)            ^^^^^^^^^^^^
(APIServer pid=1)   File "/usr/local/lib/python3.12/dist-packages/pydantic/_internal/_dataclasses.py", line 121, in __init__
(APIServer pid=1)     s.__pydantic_validator__.validate_python(ArgsKwargs(args, kwargs), self_instance=s)
(APIServer pid=1) pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
(APIServer pid=1)   Value error, Model architectures ['IQuestCoderForCausalLM'] are not supported for now. Supported architectures: dict_keys(['AfmoeForCausalLM', 'ApertusForCausalLM', 'AquilaModel', 'AquilaForCausalLM', 'ArceeForCausalLM', 'ArcticForCausalLM', 'BaiChuanForCausalLM', 'BaichuanForCausalLM', 'BailingMoeForCausalLM', 'BailingMoeV2ForCausalLM', 'BambaForCausalLM', 'BloomForCausalLM', 'ChatGLMModel', 'ChatGLMForConditionalGeneration', 'CohereForCausalLM', 'Cohere2ForCausalLM', 'CwmForCausalLM', 'DbrxForCausalLM', 'DeciLMForCausalLM', 'DeepseekForCausalLM', 'DeepseekV2ForCausalLM', 'DeepseekV3ForCausalLM', 'DeepseekV32ForCausalLM', 'Dots1ForCausalLM', 'Ernie4_5ForCausalLM', 'Ernie4_5_MoeForCausalLM', 'ExaoneForCausalLM', 'Exaone4ForCausalLM', 'Fairseq2LlamaForCausalLM', 'FalconForCausalLM', 'FalconMambaForCausalLM', 'FalconH1ForCausalLM', 'FlexOlmoForCausalLM', 'GemmaForCausalLM', 'Gemma2ForCausalLM', 'Gemma3ForCausalLM', 'Gemma3nForCausalLM', 'Qwen3NextForCausalLM', 'GlmForCausalLM', 'Glm4ForCausalLM', 'Glm4MoeForCausalLM', 'GptOssForCausalLM', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTJForCausalLM', 'GPTNeoXForCausalLM', 'GraniteForCausalLM', 'GraniteMoeForCausalLM', 'GraniteMoeHybridForCausalLM', 'GraniteMoeSharedForCausalLM', 'GritLM', 'Grok1ModelForCausalLM', 'HunYuanMoEV1ForCausalLM', 'HunYuanDenseV1ForCausalLM', 'HCXVisionForCausalLM', 'InternLMForCausalLM', 'InternLM2ForCausalLM', 'InternLM2VEForCausalLM', 'InternLM3ForCausalLM', 'JAISLMHeadModel', 'Jais2ForCausalLM', 'JambaForCausalLM', 'KimiLinearForCausalLM', 'Lfm2ForCausalLM', 'Lfm2MoeForCausalLM', 'LlamaForCausalLM', 'Llama4ForCausalLM', 'LLaMAForCausalLM', 'LongcatFlashForCausalLM', 'MambaForCausalLM', 'Mamba2ForCausalLM', 'MiniCPMForCausalLM', 'MiniCPM3ForCausalLM', 'MiniMaxForCausalLM', 'MiniMaxText01ForCausalLM', 'MiniMaxM1ForCausalLM', 'MiniMaxM2ForCausalLM', 'MistralForCausalLM', 'MistralLarge3ForCausalLM', 'MixtralForCausalLM', 'MptForCausalLM', 'MPTForCausalLM', 'MiMoForCausalLM', 'MiMoV2FlashForCausalLM', 'NemotronForCausalLM', 'NemotronHForCausalLM', 'OlmoForCausalLM', 'Olmo2ForCausalLM', 'Olmo3ForCausalLM', 'OlmoeForCausalLM', 'OPTForCausalLM', 'OrionForCausalLM', 'OuroForCausalLM', 'PanguEmbeddedForCausalLM', 'PanguProMoEV2ForCausalLM', 'PanguUltraMoEForCausalLM', 'PersimmonForCausalLM', 'PhiForCausalLM', 'Phi3ForCausalLM', 'PhiMoEForCausalLM', 'Plamo2ForCausalLM', 'Plamo3ForCausalLM', 'QWenLMHeadModel', 'Qwen2ForCausalLM', 'Qwen2MoeForCausalLM', 'Qwen3ForCausalLM', 'Qwen3MoeForCausalLM', 'RWForCausalLM', 'SeedOssForCausalLM', 'Step3TextForCausalLM', 'StableLMEpochForCausalLM', 'StableLmForCausalLM', 'Starcoder2ForCausalLM', 'SolarForCausalLM', 'TeleChatForCausalLM', 'TeleChat2ForCausalLM', 'TeleFLMForCausalLM', 'XverseForCausalLM', 'Zamba2ForCausalLM', 'BertModel', 'BertSpladeSparseEmbeddingModel', 'Gemma2Model', 'Gemma3TextModel', 'GPT2ForSequenceClassification', 'GteModel', 'GteNewModel', 'InternLM2ForRewardModel', 'JambaForSequenceClassification', 'LlamaBidirectionalModel', 'LlamaModel', 'MistralModel', 'ModernBertModel', 'NomicBertModel', 'Qwen2Model', 'Qwen2ForRewardModel', 'Qwen2ForProcessRewardModel', 'RobertaForMaskedLM', 'RobertaModel', 'XLMRobertaModel', 'CLIPModel', 'LlavaNextForConditionalGeneration', 'Phi3VForCausalLM', 'Qwen2VLForConditionalGeneration', 'SiglipModel', 'PrithviGeoSpatialMAE', 'Terratorch', 'BertForSequenceClassification', 'BertForTokenClassification', 'GteNewForSequenceClassification', 'JinaVLForRanking', 'LlamaBidirectionalForSequenceClassification', 'ModernBertForSequenceClassification', 'ModernBertForTokenClassification', 'RobertaForSequenceClassification', 'XLMRobertaForSequenceClassification', 'AriaForConditionalGeneration', 'AudioFlamingo3ForConditionalGeneration', 'AyaVisionForConditionalGeneration', 'BagelForConditionalGeneration', 'BeeForConditionalGeneration', 'Blip2ForConditionalGeneration', 'ChameleonForConditionalGeneration', 'Cohere2VisionForConditionalGeneration', 'DeepseekVLV2ForCausalLM', 'DeepseekOCRForCausalLM', 'DotsOCRForCausalLM', 'Ernie4_5_VLMoeForConditionalGeneration', 'FuyuForCausalLM', 'Gemma3ForConditionalGeneration', 'Gemma3nForConditionalGeneration', 'GlmAsrForConditionalGeneration', 'GLM4VForCausalLM', 'Glm4vForConditionalGeneration', 'Glm4vMoeForConditionalGeneration', 'GraniteSpeechForConditionalGeneration', 'H2OVLChatModel', 'HunYuanVLForConditionalGeneration', 'InternVLChatModel', 'NemotronH_Nano_VL_V2', 'OpenCUAForConditionalGeneration', 'InternS1ForConditionalGeneration', 'InternVLForConditionalGeneration', 'Idefics3ForConditionalGeneration', 'IsaacForConditionalGeneration', 'SmolVLMForConditionalGeneration', 'KeyeForConditionalGeneration', 'KeyeVL1_5ForConditionalGeneration', 'RForConditionalGeneration', 'KimiVLForConditionalGeneration', 'LightOnOCRForConditionalGeneration', 'Llama_Nemotron_Nano_VL', 'Llama4ForConditionalGeneration', 'LlavaForConditionalGeneration', 'LlavaNextVideoForConditionalGeneration', 'LlavaOnevisionForConditionalGeneration', 'MantisForConditionalGeneration', 'MiDashengLMModel', 'MiniMaxVL01ForConditionalGeneration', 'MiniCPMO', 'MiniCPMV', 'Mistral3ForConditionalGeneration', 'MolmoForCausalLM', 'NVLM_D', 'Ovis', 'Ovis2_5', 'PaddleOCRVLForConditionalGeneration', 'PaliGemmaForConditionalGeneration', 'Phi4MMForCausalLM', 'PixtralForConditionalGeneration', 'QwenVLForConditionalGeneration', 'Qwen2_5_VLForConditionalGeneration', 'Qwen2AudioForConditionalGeneration', 'Qwen2_5OmniModel', 'Qwen2_5OmniForConditionalGeneration', 'Qwen3OmniMoeForConditionalGeneration', 'Qwen3VLForConditionalGeneration', 'Qwen3VLMoeForConditionalGeneration', 'SkyworkR1VChatModel', 'Step3VLForConditionalGeneration', 'TarsierForConditionalGeneration', 'Tarsier2ForConditionalGeneration', 'UltravoxModel', 'VoxtralForConditionalGeneration', 'VoxtralStreamingGeneration', 'WhisperForConditionalGeneration', 'MiMoMTPModel', 'EagleLlamaForCausalLM', 'EagleLlama4ForCausalLM', 'EagleMiniCPMForCausalLM', 'Eagle3LlamaForCausalLM', 'LlamaForCausalLMEagle3', 'Eagle3Qwen2_5vlForCausalLM', 'Eagle3Qwen3vlForCausalLM', 'EagleMistralLarge3ForCausalLM', 'EagleDeepSeekMTPModel', 'DeepSeekMTPModel', 'ErnieMTPModel', 'LongCatFlashMTPModel', 'Glm4MoeMTPModel', 'MedusaModel', 'OpenPanguMTPModel', 'Qwen3NextMTP', 'SmolLM3ForCausalLM', 'Emu3ForConditionalGeneration', 'TransformersForCausalLM', 'TransformersMoEForCausalLM', 'TransformersMultiModalForCausalLM', 'TransformersMultiModalMoEForCausalLM', 'TransformersEmbeddingModel', 'TransformersMoEEmbeddingModel', 'TransformersMultiModalEmbeddingModel', 'TransformersForSequenceClassification', 'TransformersMoEForSequenceClassification', 'TransformersMultiModalForSequenceClassification']) [type=value_error, input_value=ArgsKwargs((), {'model': ...rocessor_plugin': None}), input_type=ArgsKwargs]
(APIServer pid=1)     For further information visit https://errors.pydantic.dev/2.12/v/value_error

[another1s]

same problem here. I followed some advices given by other LLM which told me that the modeling_iquestcoder.py is conflict with transformers. i guess they have a edited version of both VLLM and transformers with both "LossKwargs" and "IQuestCoderForCausalLM"