[OpenVINO] Fix long context inference for Phi3.5 and Phi4#1744
Merged
Conversation
- Explicitly disable torch.autocast to ensure float32 precision - Add sources for adapted code - Use self.attention_scaling instead of manual computation - Save and restore original _orig_max_position_embeddings - Modify F32_CONFIG to use EXECUTION_MODE_HINT
Exclude longrope for phi3-moe with _disable_longrope
- Add more comments - Remove superfluous select_ext_factor function - Rename long_rope to _phi3_longrope_forward for clarity
- rename tiny model to phi3 - add test for cumulative context - revert F32_CONFIG change
- Set MIN_TRANSFORMERS_VERSION to 4.49 for Phi3 - Remove code specific for transformers<4.49 - Disable trust-remote-code for Phi3
- self.original_inv_freq is the same as self.inv_freq because self.inv_freq does not get updated during model export
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds coverage and version/CLI guardrails to ensure Phi3 LongRoPE (long-context) models can be exported and used for inference reliably with OpenVINO, especially for contexts beyond the original max position embeddings.
Changes:
- Add a new internal tiny test model entry and integrate a
phi3-longropearchitecture into the OpenVINO decoder test matrix. - Introduce a dedicated integration test that exercises Phi3 LongRoPE generation with contexts exceeding
original_max_position_embeddings. - Enforce a minimum Transformers version for Phi3 OpenVINO export config and disable
trust_remote_codeforphi3in the CLI exporter for with-past exports using default configs.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
tests/openvino/utils_tests.py |
Registers the phi3-longrope tiny model id in the shared test model mapping. |
tests/openvino/test_decoder.py |
Adds phi3-longrope to supported architectures, sets expected SDPA count, and adds a LongRoPE long-context generation test. |
optimum/exporters/openvino/model_configs.py |
Sets Phi3OpenVINOConfig.MIN_TRANSFORMERS_VERSION = 4.49.0 to prevent unsupported exports on older Transformers. |
optimum/exporters/openvino/__main__.py |
Extends the CLI’s trust_remote_code disabling logic to include phi3 for with-past/default export configs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
echarlaix
reviewed
May 21, 2026
echarlaix
left a comment
echarlaix
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for the addition!
echarlaix
reviewed
Jun 1, 2026
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
rkazants
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fix for the export was already done in this PR by providing the dedicated patch in the common patcher class: #1589
Fixes 170599
Before submitting