add HyperClovaX Vision

docs/source/en/_toctree.yml

@@ -1263,6 +1263,8 @@
         title: Grounding DINO
       - local: model_doc/groupvit
         title: GroupViT
+      - local: model_doc/hyperclovax_vision_v2
+        title: HyperCLOVAX Vision V2
       - local: model_doc/idefics
         title: IDEFICS
       - local: model_doc/idefics2

docs/source/en/model_doc/hyperclovax_vision_v2.md

@@ -0,0 +1,265 @@
+<!--Copyright 2026 NAVER Corp. and The HuggingFace Inc. team. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+specific language governing permissions and limitations under the License.
+
+⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+
+-->
+*This model was released on {release_date} and added to Hugging Face Transformers on 2026-05-26.*
+
+<div style="float: right;">
+    <div class="flex flex-wrap space-x-1">
+        <img alt="FlashAttention" src="https://img.shields.io/badge/%E2%9A%A1%EF%B8%8E%20FlashAttention-eae0c8?style=flat">
+        <img alt="SDPA" src="https://img.shields.io/badge/SDPA-DE3412?style=flat&logo=pytorch&logoColor=white">
+    </div>
+</div>
+
+# HyperCLOVAX Vision V2
+
+HyperCLOVAX Vision V2 is a multimodal vision-language model developed by NAVER. It combines the [HyperClovaX](./hyperclovax.md) language model backbone with a [Qwen2.5-VL](./qwen2_5_vl) vision encoder. The model supports text, image, and video inputs and is capable of chain-of-thought reasoning via built-in thinking tokens (`<think>...</think>`).
+
+You can find the original HyperCLOVAX-SEED-Think-32B checkpoint on the [naver-hyperclovax/HyperCLOVAX-SEED-Think-32B](https://huggingface.co/naver-hyperclovax/HyperCLOVAX-SEED-Think-32B) page.
+
+The example below demonstrates how to generate text based on an image with [`HyperCLOVAXVisionV2ForConditionalGeneration`].
+
+<hfoptions id="usage">
+<hfoption id="Image input">
+
+```python
+from transformers import HyperCLOVAXVisionV2ForConditionalGeneration, HyperCLOVAXVisionV2Processor
+
+model = HyperCLOVAXVisionV2ForConditionalGeneration.from_pretrained(
+    "naver-hyperclovax/HyperCLOVAX-SEED-Think-32B",
+    device_map="auto",
+)
+processor = HyperCLOVAXVisionV2Processor.from_pretrained("naver-hyperclovax/HyperCLOVAX-SEED-Think-32B")
+
+messages = [
+    {
+        "role": "system",
+        "content": "You are a helpful assistant.",
+    },
+    {
+        "role": "user",
+        "content": [
+            {
+                "type": "image_url",
+                "image_url": {"url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"},
+            },
+            {"type": "text", "text": "Describe this image."},
+        ],
+    },
+]
+
+inputs = processor.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    tokenize=True,
+    return_dict=True,
+    return_tensors="pt",
+).to(model.device)
+
+generated_ids = model.generate(**inputs, max_new_tokens=256)
+generated_ids_trimmed = [
+    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
+]
+output_text = processor.batch_decode(
+    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
+)
+print(output_text)
+```
+
+</hfoption>
+<hfoption id="Video input">
+
+```python
+from transformers import HyperCLOVAXVisionV2ForConditionalGeneration, HyperCLOVAXVisionV2Processor
+
+model = HyperCLOVAXVisionV2ForConditionalGeneration.from_pretrained(
+    "naver-hyperclovax/HyperCLOVAX-SEED-Think-32B",
+    device_map="auto",
+)
+processor = HyperCLOVAXVisionV2Processor.from_pretrained("naver-hyperclovax/HyperCLOVAX-SEED-Think-32B")
+
+messages = [
+    {
+        "role": "system",
+        "content": "You are a helpful assistant.",
+    },
+    {
+        "role": "user",
+        "content": [
+            {
+                "type": "image_url",
+                "image_url": {"url": "/path/to/video.mp4"},
+            },
+            {"type": "text", "text": "Describe this video."},
+        ],
+    },
+]
+
+# Use processor.tokenizer.apply_chat_template for video inputs.
+# processor.apply_chat_template rewrites image_url to image before the
+# template runs, which breaks HCX's extension-based video detection.
+text = processor.tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    tokenize=False,
+)
+inputs = processor(
+    text=text,
+    videos=["/path/to/video.mp4"],
+    return_tensors="pt",
+).to(model.device)
+
+generated_ids = model.generate(**inputs, max_new_tokens=256)
+generated_ids_trimmed = [
+    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
+]
+output_text = processor.batch_decode(
+    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
+)
+print(output_text)
+```
+
+</hfoption>
+</hfoptions>
+
+Quantization reduces the memory burden of large models by representing the weights in a lower precision. Refer to the [Quantization](../quantization/overview) overview for more available quantization backends.
+
+The example below uses [bitsandbytes](../quantization/bitsandbytes) to load the model in 4-bit.
+
+```python
+from transformers import BitsAndBytesConfig, HyperCLOVAXVisionV2ForConditionalGeneration, HyperCLOVAXVisionV2Processor
+
+quantization_config = BitsAndBytesConfig(load_in_4bit=True)
+model = HyperCLOVAXVisionV2ForConditionalGeneration.from_pretrained(
+    "naver-hyperclovax/HyperCLOVAX-SEED-Think-32B",
+    device_map="auto",
+    quantization_config=quantization_config,
+)
+processor = HyperCLOVAXVisionV2Processor.from_pretrained("naver-hyperclovax/HyperCLOVAX-SEED-Think-32B")
+```
+
+## Notes
+
+- HyperCLOVAX Vision V2 uses a unique media input format. Both images and videos are specified using `{"type": "image_url", "image_url": {"url": "..."}}`. The processor and chat template distinguish images from videos by file extension (`.mp4`, `.avi`, `.mov`, `.mkv`, `.webm`, `.flv`, `.wmv`, `.m4v` are treated as video; everything else is treated as image).
+
+    ```python
+    # Image input
+    {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}}
+
+    # Video input (identified by file extension)
+    {"type": "image_url", "image_url": {"url": "/path/to/video.mp4"}}
+    ```
+
+    > [!WARNING]
+    > Video input via `processor.apply_chat_template` is currently broken. Recent Transformers versions rewrite `image_url` entries to `image` before the chat template runs, so the video-detection branch in HCX's template never triggers and video inputs are silently dropped. As a workaround, use `processor.tokenizer.apply_chat_template` to render the prompt text, then pass the video path separately to `processor(...)`. See [this review comment](https://github.com/huggingface/transformers/pull/44314#discussion_r3008382827) for details.
+
+- The model supports chain-of-thought reasoning. By default, the generation prompt prepends an empty `<think>\n\n</think>` block. To generate an explicit reasoning trace inside `<think>...</think>` tags, pass `thinking=True` to `apply_chat_template` (image/text inputs only):
+
+    ```python
+    inputs = processor.apply_chat_template(
+        messages,
+        add_generation_prompt=True,
+        tokenize=True,
+        return_dict=True,
+        return_tensors="pt",
+        thinking=True,
+    ).to(model.device)
+    ```
+
+- The model supports multi-turn conversations with mixed media. Images and videos can appear across multiple turns. For turns containing video, use the `processor.tokenizer.apply_chat_template` workaround described above.
+
+    ```python
+    messages = [
+        {
+            "role": "user",
+            "content": [
+                {"type": "image_url", "image_url": {"url": "https://example.com/image1.jpg"}},
+                {"type": "text", "text": "What do you see in this image?"},
+            ],
+        },
+        {
+            "role": "assistant",
+            "content": "I see a cat sitting on a couch.",
+        },
+        {
+            "role": "user",
+            "content": [
+                {"type": "image_url", "image_url": {"url": "https://example.com/image2.jpg"}},
+                {"type": "text", "text": "How does this compare to the first image?"},
+            ],
+        },
+    ]
+    ```
+
+- The model supports function/tool calling. Pass tools using the `tools` parameter in `apply_chat_template`:
+
+    ```python
+    tools = [
+        {
+            "type": "function",
+            "function": {
+                "name": "get_weather",
+                "description": "Get the current weather for a location.",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "location": {"type": "string", "description": "City name"},
+                    },
+                    "required": ["location"],
+                },
+            },
+        }
+    ]
+
+    messages = [
+        {"role": "user", "content": "What is the weather in Seoul?"}
+    ]
+
+    inputs = processor.apply_chat_template(
+        messages,
+        tools=tools,
+        add_generation_prompt=True,
+        tokenize=True,
+        return_dict=True,
+        return_tensors="pt",
+    ).to(model.device)
+    ```
+
+## HyperCLOVAXVisionV2Config
+
+[[autodoc]] HyperCLOVAXVisionV2Config
+
+## HyperCLOVAXVisionV2Processor
+
+[[autodoc]] HyperCLOVAXVisionV2Processor
+    - __call__
+
+## HyperCLOVAXVisionV2Model
+
+[[autodoc]] HyperCLOVAXVisionV2Model
+    - forward
+    - get_image_features
+    - get_video_features
+
+## HyperCLOVAXVisionV2ForConditionalGeneration
+
+[[autodoc]] HyperCLOVAXVisionV2ForConditionalGeneration
+    - forward
+    - get_image_features
+    - get_video_features
+
+## HyperCLOVAXVisionV2ForSequenceClassification
+
+[[autodoc]] HyperCLOVAXVisionV2ForSequenceClassification
+    - forward

docs/source/en/model_doc/qwen2_5_vl.md

@@ -141,7 +141,7 @@ model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
             ],
         }
     ]
-    
+
     inputs = processor.apply_chat_template(
         conversation,
         fps=1,
@@ -150,7 +150,7 @@ model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
         return_dict=True,
         return_tensors="pt"
     ).to(model.device)
-    
+
     # Inference: Generation of the output
     output_ids = model.generate(**inputs, max_new_tokens=128)
     generated_ids = [output_ids[len(input_ids):] for input_ids, output_ids in zip(inputs.input_ids, output_ids)]
@@ -164,7 +164,7 @@ model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
     ```python
     import torch
     from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
-    
+
     model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
         "Qwen/Qwen2.5-VL-7B-Instruct",
         device_map="auto",
@@ -175,7 +175,7 @@ model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
         {
             "role": "user",
             "content": [
-                {"type": "image"}, 
+                {"type": "image"},
                 {"type": "text", "text": "Hello, how are you?"}
             ]
         },
@@ -186,10 +186,10 @@ model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
         {
             "role": "user",
             "content": [
-                {"type": "text", "text": "Can you describe these images and video?"}, 
-                {"type": "image"}, 
-                {"type": "image"}, 
-                {"type": "video"}, 
+                {"type": "text", "text": "Can you describe these images and video?"},
+                {"type": "image"},
+                {"type": "image"},
+                {"type": "video"},
                 {"type": "text", "text": "These are from my vacation."}
             ]
         },
@@ -202,12 +202,12 @@ model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
             "content": "It was a trip to the mountains. Can you see the details in the images and video?"
         }
     ]
-    
+
     # default:
     prompt_without_id = processor.apply_chat_template(conversation, add_generation_prompt=True)
     # Excepted output: '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\n<|vision_start|><|image_pad|><|vision_end|>Hello, how are you?<|im_end|>\n<|im_start|>assistant\nI'm doing well, thank you for asking. How can I assist you today?<|im_end|>\n<|im_start|>user\nCan you describe these images and video?<|vision_start|><|image_pad|><|vision_end|><|vision_start|><|image_pad|><|vision_end|><|vision_start|><|video_pad|><|vision_end|>These are from my vacation.<|im_end|>\n<|im_start|>assistant\nI'd be happy to describe the images and video for you. Could you please provide more context about your vacation?<|im_end|>\n<|im_start|>user\nIt was a trip to the mountains. Can you see the details in the images and video?<|im_end|>\n<|im_start|>assistant\n'
-    
-    
+
+
     # add ids
     prompt_with_id = processor.apply_chat_template(conversation, add_generation_prompt=True, add_vision_id=True)
     # Excepted output: '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nPicture 1: <|vision_start|><|image_pad|><|vision_end|>Hello, how are you?<|im_end|>\n<|im_start|>assistant\nI'm doing well, thank you for asking. How can I assist you today?<|im_end|>\n<|im_start|>user\nCan you describe these images and video?Picture 2: <|vision_start|><|image_pad|><|vision_end|>Picture 3: <|vision_start|><|image_pad|><|vision_end|>Video 1: <|vision_start|><|video_pad|><|vision_end|>These are from my vacation.<|im_end|>\n<|im_start|>assistant\nI'd be happy to describe the images and video for you. Could you please provide more context about your vacation?<|im_end|>\n<|im_start|>user\nIt was a trip to the mountains. Can you see the details in the images and video?<|im_end|>\n<|im_start|>assistant\n'
@@ -225,7 +225,7 @@ model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
 
     ```python
     min_pixels = 256*28*28
-    max_pixels = 1024*28*28 
+    max_pixels = 1024*28*28
     processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-7B-Instruct", min_pixels=min_pixels, max_pixels=max_pixels)
     ```
 
@@ -241,6 +241,10 @@ model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
 
 [[autodoc]] Qwen2_5_VLTextConfig
 
+## Qwen2_5_VLVisionConfig
+
+[[autodoc]] Qwen2_5_VLVisionConfig
+
 ## Qwen2_5_VLProcessor
 
 [[autodoc]] Qwen2_5_VLProcessor
@@ -251,6 +255,11 @@ model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
 [[autodoc]] Qwen2_5_VLTextModel
     - forward
 
+## Qwen2_5_VisionTransformerPretrainedModel
+
+[[autodoc]] Qwen2_5_VisionTransformerPretrainedModel
+    - forward
+
 ## Qwen2_5_VLModel
 
 [[autodoc]] Qwen2_5_VLModel

docs/source/ko/_toctree.yml

@@ -1073,6 +1073,8 @@
         title: Grounding DINO
       - local: in_translation
         title: GroupViT
+      - local: model_doc/hyperclovax_vision_v2
+        title: HyperCLOVAX Vision V2
       - local: in_translation
         title: IDEFICS
       - local: in_translation