Can the IQuest-Coder-V1-40B-Instruct and IQuest-Coder-V1-40B-Loop-Instruct models run on the SGLang framework?
Here is the information about my SGLang framework: registry.cn-hangzhou.aliyuncs.com/wpf19911118/sglang:0.5.4.post2.cu128
However, when I perform inference with the IQuest-Coder-V1-40B-Instruct model, I encounter a version conflict between the model and the SGLang framework regarding the transformers library.
When you downgraded transformers to version 4.36.2, you resolved the LossKwargs import error, but a new Qwen2_5_VLProcessor import error was triggered instead. This is because SGLang version 0.5.4.post2 has a hard dependency on transformers==4.57.1 (clearly indicated by the pip error message: sglang 0.5.4.post2 requires transformers==4.57.1). I will help you resolve this core "version conflict" issue and provide an ultimate solution that balances SGLang compatibility and model code adaptation.
I. Root Cause of the New Error
1. Version Dependency Chain Conflict
ImportError: cannot import name 'Qwen2_5_VLProcessor' from 'transformers'
Qwen2_5_VLProcessor is a class newly added in transformers version 4.38+ (corresponding to the Qwen2.5 multimodal model).- The 4.36.2 version you installed does not include this class, but the source code of SGLang 0.5.4.post2 hardcodes the import of this class, causing the startup script to crash immediately.
- Meanwhile, pip explicitly warns that:
sglang 0.5.4.post2 requires transformers==4.57.1, which means your SGLang version is completely incompatible with 4.36.2.
2. Summary of the Underlying Conflict
| Requester |
Required transformers Version |
Core Requirement |
| IQuest-Coder Model Code |
≤ 4.36.x |
Able to import LossKwargs |
| SGLang 0.5.4.post2 |
== 4.57.1 |
Able to import Qwen2_5_VLProcessor |
This is a typical case of bidirectional version locking, where simply downgrading or upgrading transformers cannot satisfy the requirements of both parties simultaneously.
Can the IQuest-Coder-V1-40B-Instruct and IQuest-Coder-V1-40B-Loop-Instruct models run on the SGLang framework?
Here is the information about my SGLang framework:
registry.cn-hangzhou.aliyuncs.com/wpf19911118/sglang:0.5.4.post2.cu128However, when I perform inference with the IQuest-Coder-V1-40B-Instruct model, I encounter a version conflict between the model and the SGLang framework regarding the
transformerslibrary.When you downgraded
transformersto version 4.36.2, you resolved theLossKwargsimport error, but a newQwen2_5_VLProcessorimport error was triggered instead. This is because SGLang version 0.5.4.post2 has a hard dependency ontransformers==4.57.1(clearly indicated by the pip error message:sglang 0.5.4.post2 requires transformers==4.57.1). I will help you resolve this core "version conflict" issue and provide an ultimate solution that balances SGLang compatibility and model code adaptation.I. Root Cause of the New Error
1. Version Dependency Chain Conflict
Qwen2_5_VLProcessoris a class newly added intransformersversion 4.38+ (corresponding to the Qwen2.5 multimodal model).sglang 0.5.4.post2 requires transformers==4.57.1, which means your SGLang version is completely incompatible with 4.36.2.2. Summary of the Underlying Conflict
transformersVersionLossKwargsQwen2_5_VLProcessorThis is a typical case of bidirectional version locking, where simply downgrading or upgrading
transformerscannot satisfy the requirements of both parties simultaneously.