Skip to content

[EXPERIMENT][WIP] Add Qwen3.5 support to OpenVINO export#1746

Draft
mlukasze wants to merge 1 commit into
huggingface:mainfrom
mlukasze:enable/Qwen-Qwen3.5-27B
Draft

[EXPERIMENT][WIP] Add Qwen3.5 support to OpenVINO export#1746
mlukasze wants to merge 1 commit into
huggingface:mainfrom
mlukasze:enable/Qwen-Qwen3.5-27B

Conversation

@mlukasze
Copy link
Copy Markdown

⚠️ AUTOMATICALLY GENERATED BY OMEGA AGENT — REQUIRES HUMAN REVIEW ⚠️
This PR was created by an AI agent as part of automated model enablement.
A human maintainer must review and approve it before it can be considered for merge.
Do NOT merge without human review and sign-off.

What does this PR do?

Adds and validates Qwen3.5 / Qwen3.5-MoE OpenVINO export support against modern Transformers snapshots. The patch fixes Qwen3.5 dynamic cache mask sizing during traced export, relaxes the supported Transformers upper bound to cover the current upstream Qwen3.5 implementation, and updates the related OpenVINO tests/docs gates. Validation performed on the tiny internal HF checkpoints: optimum-cli export openvino --model optimum-intel-internal-testing/tiny-random-qwen3.5 --task image-text-to-text ... now succeeds, OVModelForVisualCausalLM loads the export on CPU, and pytest tests/openvino/test_export.py -k qwen3_5 -x -v --no-header passes.

Installation instructions

source /opt/home/mlukasze/dev/Qwen-Qwen3.5-27B/venv/bin/activate
pip install -e /opt/home/mlukasze/dev/Qwen-Qwen3.5-27B/optimum-intel --no-build-isolation --no-deps
pip install -U 'git+https://github.com/huggingface/transformers.git'
pip install pytest sentence-transformers kokoro

Exporting cmd-line

optimum-cli export openvino --model optimum-intel-internal-testing/tiny-random-qwen3.5 --task image-text-to-text /opt/home/mlukasze/dev/Qwen-Qwen3.5-27B/tiny-qwen35-ov --trust-remote-code

Inference script

from optimum.intel import OVModelForVisualCausalLM

model = OVModelForVisualCausalLM.from_pretrained(
    '/opt/home/mlukasze/dev/Qwen-Qwen3.5-27B/tiny-qwen35-ov',
    device='CPU',
)
print(type(model).__name__)

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

Adds export support fixes for Qwen/Qwen3.5-27B and related architectures:\n- fix qwen3_5 dynamic cache mask sizing during traced export\n- extend supported transformers window for qwen3_5/qwen3_5_moe\n- update related docs and OpenVINO test gates\n\nRequires transformers >= 5.2.0\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants