Skip to content

Fix Gemma4 shared KV export compatibility for transformer 5.12.0 with older support as well#1794

Open
dhandhalyabhavik wants to merge 2 commits into
huggingface:mainfrom
dhandhalyabhavik:gemma4-fix
Open

Fix Gemma4 shared KV export compatibility for transformer 5.12.0 with older support as well#1794
dhandhalyabhavik wants to merge 2 commits into
huggingface:mainfrom
dhandhalyabhavik:gemma4-fix

Conversation

@dhandhalyabhavik

@dhandhalyabhavik dhandhalyabhavik commented Jun 16, 2026

Copy link
Copy Markdown

What does this PR do?

ov export for gemma4 models failed with recent transformers version 5.12.0 because it has changed internal model implementation of Gemma4. They started using shared_kv_states along with past_key_values. Older 5.5.0 implementation of gemma4 had only past_key_values with shared_kv_states inside it.

This patch adds support for latest transformers as well as old transformers.

Fixes # (issue)
#1785

Before submitting

  • [NA] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • [NA] Did you make sure to update the documentation with your changes?
  • [NA] Did you write any new necessary tests?

Reference links

https://github.com/huggingface/transformers/blob/v5.12.1/src/transformers/models/gemma4/modeling_gemma4.py#L1234

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@rkazants rkazants left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please set MAX_TRANSFORMERS_VERSION=5.12.99 here https://github.com/huggingface/optimum-intel/blob/main/optimum/exporters/openvino/model_configs.py#L1289

the same we need here https://github.com/huggingface/optimum-intel/blob/main/optimum/exporters/openvino/model_configs.py#L3844 both min and max version. It helps to avoid bug issue report from the community

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants