support added for HF SmolLM3-3B by AshutoshSinghIntel · Pull Request #1715 · huggingface/optimum-intel

AshutoshSinghIntel · 2026-05-04T09:55:55Z

What does this PR do?

OpenVINO export:

optimum-cli export openvino -m HuggingFaceTB/SmolLM3-3B ./SmolLM3-3B --task text-generation-with-past

Inference Script:

import argparse
from transformers import AutoTokenizer
from optimum.intel.openvino import OVModelForCausalLM

model_id = "HuggingFaceTB/SmolLM3-3B"

def main():
    parser = argparse.ArgumentParser(description="SmolLM3-3B inference with OpenVINO")
    parser.add_argument("--model", type=str, default=model_id, help="Path to exported OV model or HF model ID")
    parser.add_argument("--max-new-tokens", type=int, default=100)
    parser.add_argument("--device", type=str, default="CPU")
    args = parser.parse_args()

    model = OVModelForCausalLM.from_pretrained(args.model, device=args.device)
    tokenizer = AutoTokenizer.from_pretrained(args.model)

    messages = [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of France?"},
    ]

    inputs = tokenizer.apply_chat_template(messages, tokenize=True, return_dict=True, return_tensors="pt", add_generation_prompt=True)
    output = model.generate(**inputs, max_new_tokens=args.max_new_tokens)
    print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

if __name__ == "__main__":
    main()

Fixes # CVS-183437

Before submitting

[N/A] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests?

HuggingFaceDocBuilderDev · 2026-05-04T10:02:33Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

rkazants

please provide proper PR description with code snippets for export and inference. See reference #1688

The other part looks good to me

Copilot

Pull request overview

Adds OpenVINO export support for the HuggingFace Transformers smollm3 architecture (SmolLM3-3B), wiring it into the OpenVINO TasksManager configs, test matrices, and the supported-models documentation.

Changes:

Register a new SmolLM3OpenVINOConfig (Llama-based) for multiple tasks with a minimum transformers version of 4.53.0.
Extend OpenVINO exporter and GenAI/decoder/CLI test coverage to include smollm3, and add a tiny internal test model id.
Update OpenVINO supported-architectures documentation to list SmolLM3, and remove smollm3 from the “ONNX supported but untested” warning set.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
`optimum/exporters/openvino/model_configs.py`	Registers `smollm3` in the OpenVINO TasksManager with an OpenVINO config class and version gate.
`optimum/exporters/openvino/utils.py`	Removes `smollm3` from `ONNX_SUPPORTED_ARCHITECTURES` so it’s no longer treated as “untested / export at your own risk”.
`tests/openvino/utils_tests.py`	Adds an internal tiny test model mapping for `smollm3`.
`tests/openvino/test_genai.py`	Includes `smollm3` in the GenAI LLM pipeline supported-architecture matrix for `transformers>=4.53.0`.
`tests/openvino/test_exporters_cli.py`	Adds CLI export test coverage and expected tokenizer artifact counts for `smollm3`.
`tests/openvino/test_export.py`	Adds `smollm3` to the export integration test architecture mapping.
`tests/openvino/test_decoder.py`	Adds `smollm3` to the decoder integration supported-architecture matrix for `transformers>=4.53.0`.
`docs/source/openvino/models.mdx`	Documents SmolLM3 as a supported architecture.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

regisss · 2026-05-05T08:05:27Z

LGTM.
I agree with @rkazants' comment, let's stay consistent with previous PRs please.

AshutoshSinghIntel · 2026-05-05T09:25:50Z

Thanks @rkazants and @regisss , I have updated the PR description.

AshutoshSinghIntel · 2026-05-05T09:28:26Z

@IlyasMoutawwakil Could you please provide me access to https://huggingface.co/optimum-intel-internal-testing for uploading the tiny-random-smollm3 ? I ran it locally and tests are passing.

rkazants · 2026-05-05T11:52:11Z

@popovaan, please take a look:

check real model work
ask for wwb metrics

AshutoshSinghIntel · 2026-05-07T12:57:14Z

The WWB similarity score is 1.0 (using CPU, fp16, and the default number of samples, which was 27).

Similarity evaluation:  96%|#########6| 
26/27 [00:05<00:00,  5.61it/s]
Similarity evaluation: 100%|##########| 
27/27 [00:05<00:00,  5.67it/s]
Similarity evaluation: 100%|##########| 
27/27 [00:05<00:00,  4.74it/s]
INFO:whowhatbench.wwb:Metrics for model: SmolLM3-3B
INFO:whowhatbench.wwb:   similarity
0         1.0

AshutoshSinghIntel · 2026-05-07T12:59:15Z

Below is the script for creating tiny-model (I do not have the access to publish yet):

from transformers import (
    AutoTokenizer,
    SmolLM3Config,
    SmolLM3ForCausalLM,
)

def create_tiny_random_smollm3():

    config = SmolLM3Config(
        vocab_size=128256,
        hidden_size=32,
        intermediate_size=64,
        num_hidden_layers=2,
        num_attention_heads=4,
        num_key_value_heads=2,
        max_position_embeddings=256,
        hidden_act="silu",
        rms_norm_eps=1e-6,
        tie_word_embeddings=True,
        use_cache=True,
        attention_bias=False,
        mlp_bias=False,
        use_sliding_window=False,
        pad_token_id=128004,
        bos_token_id=128000,
        eos_token_id=128012,
    )

    model = SmolLM3ForCausalLM(config)
    print(f"Model parameters: {sum(p.numel() for p in model.parameters()):,}")

    # Use SmolLM3-3B tokenizer
    tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/SmolLM3-3B")

    output_dir = "./tiny-random-smollm3"
    model.save_pretrained(output_dir)
    tokenizer.save_pretrained(output_dir)
    print(f"Saved to {output_dir}")

if __name__ == "__main__":
    create_tiny_random_smollm3()

popovaan · 2026-05-07T13:02:37Z

Please add quantization tests.

popovaan · 2026-05-07T13:05:08Z

There was a warning during conversion of the model "The OpenVINO export of smollm3 models is not officially supported by optimum-intel, export at your own risks.", was it fixed?

AshutoshSinghIntel · 2026-05-07T13:51:31Z

There was a warning during conversion of the model "The OpenVINO export of smollm3 models is not officially supported by optimum-intel, export at your own risks.", was it fixed?

With the dedicated config added by this PR, the warning does not appear.

AshutoshSinghIntel · 2026-05-07T16:10:34Z

Please add quantization tests.

added, kindly check.

popovaan · 2026-05-12T08:20:02Z

@rkazants @echarlaix @regisss please review this PR.

regisss

LGTM

AshutoshSinghIntel · 2026-05-26T10:20:31Z

Hi @regisss and @rkazants , kindly help to re-review. I updated code to take care of different int8 count expectation based on different task in a single model.

e.g. in SmolLM3-3B,
text-generation-with-past: 30
feature-extraction: 30
text-classification: 32

echarlaix

LGTM, thanks for the addition @AshutoshSinghIntel !

…s_cli.py

echarlaix

thanks for iterating @AshutoshSinghIntel

echarlaix · 2026-06-02T15:33:24Z

+    ],
+    library_name="transformers",
+)
+class SmolLM3OpenVINOConfig(LlamaOnnxConfig):


since #1753 was merged and onnx dependency removed, you can now either inherit from BitnetOpenVINOConfig or TextDecoderWithPositionIdsOpenVINOConfig

Suggested change

class SmolLM3OpenVINOConfig(LlamaOnnxConfig):

class SmolLM3OpenVINOConfig(BitnetOpenVINOConfig):

used TextDecoderWithPositionIdsOpenVINOConfig, as BitnetOpenVINOConfig has MAX_TRANSFORMERS_VERSION defined.

echarlaix · 2026-06-02T15:34:19Z

+            expected_int8 = (
+                _ARCHITECTURES_INT8_TASK_OVERRIDES.get(model_type, {}).get(task)
+                or _ARCHITECTURES_TO_EXPECTED_INT8[model_type]
+            )


Suggested change

expected_int8 = (

_ARCHITECTURES_INT8_TASK_OVERRIDES.get(model_type, {}).get(task)

or _ARCHITECTURES_TO_EXPECTED_INT8[model_type]

)

expected_int8 = _ARCHITECTURES_TO_EXPECTED_INT8[model_type]

no need for _ARCHITECTURES_INT8_TASK_OVERRIDES now no ?

yes, it can be removed now. Done, Thanks!

rkazants · 2026-06-03T06:07:38Z

@AshutoshSinghIntel,
what is wwb similarity for int8 and int4 compressed models?

AshutoshSinghIntel · 2026-06-03T09:08:01Z

@AshutoshSinghIntel, what is wwb similarity for int8 and int4 compressed models?

Hi @rkazants, similarity score for INT8 and INT4-sym(using optimum and on CPU) :

Format	Similarity	Samples
INT4 sym	0.9434	28
INT8	0.9689	28

popovaan · 2026-06-05T08:06:33Z

@AshutoshSinghIntel could you please fix code style?
https://github.com/huggingface/optimum-intel/actions/runs/26871258289/job/79261237775?pr=1715

Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>

echarlaix · 2026-06-05T09:38:38Z

thanks a lot for iterating @AshutoshSinghIntel, looks like there is one test failing still

optimum-intel/tests/openvino/test_decoder.py

Line 706 in e6f612a

def test_beam_search(self, model_arch):

   AssertionError: False is not true : generation config : GenerationConfig {
     "do_sample": true,
     "max_new_tokens": 10,
     "min_new_tokens": 10,
     "num_beams": 4
   }
   , transformers output tensor([[ 15724,    374,    264,   6555,   1938,    323,    358,   1097,   5129,
              6949,  70642, 111802,  10580, 126543,  40272,  60953,  93724,  93347,
              9295],
           [  2028,    374,    757, 128012, 128012, 128012, 128012, 128012, 128012,
              1479,   8037,   8037, 116715,  44083,  83853,  83853,  84373,  22747,
             95195]]), ov_model_stateful output tensor([[ 15724,    374,    264,   6555,   1938,    323,    358,   1097,   5129,
              6949,  70642, 111802,  93012,  74455, 116672, 103100, 116672,  89810,
            121411],
           [  2028,    374,    757, 128012, 128012, 128012, 128012, 128012, 128012,
              1479,   8037,   8037, 116715,  44083,  83853,  83853,  84373,  22747,
             95195]])

would you mind taking a look so that we can merge?

AshutoshSinghIntel · 2026-06-08T07:08:02Z

thanks a lot for iterating @AshutoshSinghIntel, looks like there is one test failing still

optimum-intel/tests/openvino/test_decoder.py

Line 706 in e6f612a

def test_beam_search(self, model_arch):

   AssertionError: False is not true : generation config : GenerationConfig {
     "do_sample": true,
     "max_new_tokens": 10,
     "min_new_tokens": 10,
     "num_beams": 4
   }
   , transformers output tensor([[ 15724,    374,    264,   6555,   1938,    323,    358,   1097,   5129,
              6949,  70642, 111802,  10580, 126543,  40272,  60953,  93724,  93347,
              9295],
           [  2028,    374,    757, 128012, 128012, 128012, 128012, 128012, 128012,
              1479,   8037,   8037, 116715,  44083,  83853,  83853,  84373,  22747,
             95195]]), ov_model_stateful output tensor([[ 15724,    374,    264,   6555,   1938,    323,    358,   1097,   5129,
              6949,  70642, 111802,  93012,  74455, 116672, 103100, 116672,  89810,
            121411],
           [  2028,    374,    757, 128012, 128012, 128012, 128012, 128012, 128012,
              1479,   8037,   8037, 116715,  44083,  83853,  83853,  84373,  22747,
             95195]])

would you mind taking a look so that we can merge?

Hi @echarlaix,

I tried to run it locally and it seems to be passing for me.

$ RUN_SLOW=1 python -m pytest tests/openvino/test_decoder.py -k "test_beam_search and smollm3" -xvs

============================= test session starts =============================
platform win32 -- Python 3.13.3, pytest-9.0.3, pluggy-1.6.0
rootdir: optimum-intel
configfile: pyproject.toml
plugins: anyio-4.13.0
collected 180 items / 179 deselected / 1 selected

tests/openvino/test_decoder.py::OVModelForCausalLMIntegrationTest::test_beam_search_51_smollm3 PASSED

=============== 1 passed, 179 deselected, 36 warnings in 55.26s ===============

AshutoshSinghIntel · 2026-06-08T15:05:46Z

Hi @echarlaix ,

smollm3 test_beam_search is passing when we re-ran it, kindly check and let me know if we are good to merge:

build (ubuntu-22.04, latest): https://github.com/huggingface/optimum-intel/actions/runs/27123591855/job/80046491183?pr=1715#step:12:5506
build (windows-2022, latest): https://github.com/huggingface/optimum-intel/actions/runs/27123591855/job/80046491209?pr=1715#step:12:2435
build (main, ubuntu-22.04): https://github.com/huggingface/optimum-intel/actions/runs/27123591855/job/80046491221?pr=1715#step:12:4992

AshutoshSinghIntel added 2 commits May 4, 2026 15:11

support added for HF SmolLM3-3B

2c8c7a6

added smollm3 model to docs

f50a848

AshutoshSinghIntel marked this pull request as ready for review May 4, 2026 14:50

rkazants requested review from Copilot, echarlaix and popovaan May 5, 2026 05:03

Copilot started reviewing on behalf of rkazants May 5, 2026 05:03 View session

rkazants reviewed May 5, 2026

View reviewed changes

rkazants requested a review from regisss May 5, 2026 05:04

Copilot AI reviewed May 5, 2026

View reviewed changes

popovaan reviewed May 7, 2026

View reviewed changes

Comment thread optimum/exporters/openvino/utils.py

Comment thread optimum/exporters/openvino/model_configs.py

added tests for quantization, feature-extraction and text-classification

fdb3e0b

popovaan approved these changes May 12, 2026

View reviewed changes

rkazants approved these changes May 12, 2026

View reviewed changes

regisss approved these changes May 12, 2026

View reviewed changes

AshutoshSinghIntel added 3 commits May 15, 2026 14:01

Merge branch 'main' into support-SmolLM3-3B

db672e0

added missing smollm3 entry in EXPECTED_NUM_SDPA

77da88f

handle task based int8 count

1e9fd30

AshutoshSinghIntel added 3 commits May 20, 2026 14:35

missing import entry

242a39b

Merge branch 'main' into support-SmolLM3-3B

9d64c8e

black tool file reformat

5a6e33e

AshutoshSinghIntel requested review from popovaan, regisss and rkazants May 20, 2026 14:02

popovaan approved these changes May 26, 2026

View reviewed changes

rkazants approved these changes Jun 1, 2026

View reviewed changes

echarlaix approved these changes Jun 1, 2026

View reviewed changes

Comment thread tests/openvino/test_exporters_cli.py Outdated

removed text-classification and feature-extraction from test_exporter…

43d4139

…s_cli.py

rkazants added the openvino-slow Runs OpenVINO slow tests with different versions of transformers label Jun 2, 2026

echarlaix approved these changes Jun 2, 2026

View reviewed changes

using BitnetOpenVINOConfig and removed int8 overrides

d05b5d4

rkazants reviewed Jun 3, 2026

View reviewed changes

Comment thread tests/openvino/test_exporters_cli.py Outdated

AshutoshSinghIntel added 2 commits June 3, 2026 11:41

Merge branch 'main' into support-SmolLM3-3B

2d04ff0

used TextDecoderWithPositionIdsOpenVINOConfig

2daa7bb

rkazants removed the openvino-slow Runs OpenVINO slow tests with different versions of transformers label Jun 3, 2026

Apply suggestion from @rkazants

45fb7aa

Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>

popovaan added the openvino-slow Runs OpenVINO slow tests with different versions of transformers label Jun 8, 2026

echarlaix merged commit 0f41831 into huggingface:main Jun 10, 2026
63 of 150 checks passed

mlukasze mentioned this pull request Jun 10, 2026

[EXPERIMENT][WIP] Add SmolLM3 support for OpenVINO export and fix position_ids double-append #1761

Closed

	class SmolLM3OpenVINOConfig(LlamaOnnxConfig):
	class SmolLM3OpenVINOConfig(BitnetOpenVINOConfig):

Uh oh!

Conversation

AshutoshSinghIntel commented May 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Before submitting

Uh oh!

HuggingFaceDocBuilderDev commented May 4, 2026

Uh oh!

rkazants left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

regisss commented May 5, 2026

Uh oh!

AshutoshSinghIntel commented May 5, 2026

Uh oh!

AshutoshSinghIntel commented May 5, 2026

Uh oh!

rkazants commented May 5, 2026

Uh oh!

AshutoshSinghIntel commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AshutoshSinghIntel commented May 7, 2026

Uh oh!

Uh oh!

Uh oh!

popovaan commented May 7, 2026

Uh oh!

popovaan commented May 7, 2026

Uh oh!

AshutoshSinghIntel commented May 7, 2026

Uh oh!

AshutoshSinghIntel commented May 7, 2026

Uh oh!

popovaan commented May 12, 2026

Uh oh!

regisss left a comment

Choose a reason for hiding this comment

Uh oh!

AshutoshSinghIntel commented May 26, 2026

Uh oh!

echarlaix left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

echarlaix left a comment

Choose a reason for hiding this comment

Uh oh!

echarlaix Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

AshutoshSinghIntel Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

echarlaix Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

AshutoshSinghIntel Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

rkazants commented Jun 3, 2026

Uh oh!

Uh oh!

AshutoshSinghIntel commented Jun 3, 2026

Uh oh!

popovaan commented Jun 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

echarlaix commented Jun 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AshutoshSinghIntel commented Jun 8, 2026

Uh oh!

AshutoshSinghIntel commented Jun 8, 2026

AshutoshSinghIntel commented May 4, 2026 •

edited

Loading

AshutoshSinghIntel commented May 7, 2026 •

edited

Loading

popovaan commented Jun 5, 2026 •

edited

Loading

echarlaix commented Jun 5, 2026 •

edited

Loading