Skip to content

Commit c995192

Browse files
authored
Arm backend: Add DeepSeek-R1-Distill-Qwen layer tests (#21010)
This patch adds Arm backend layer tests for DeepSeek-R1-Distill-Qwen-1.5B. The tests use the checkpoint configuration from the Hugging Face model and the upstream Qwen2 layer implementations that back this distilled model. The test structure follows the existing Qwen3-VL layer-test approach. The config helper constructs the checkpoint-sized model configuration directly, the test file uses layer-level wrappers with dataclass-driven test cases, and coverage includes TOSA FP, TOSA BF16 reference-model, VGF no-quant, and VGF BF16 no-quant runtime paths. Token embedding is excluded because the full checkpoint embedding allocation is too large for regular CI. VGF quant coverage is also left out of this patch so CI resources stay focused on the requested BF16 path and export coverage. The covered layers include rotary embedding, rotary application, KV repetition, attention, RMSNorm, MLP, decoder layer, and final norm. cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani --------- Signed-off-by: Baris Demir <baris.demir@arm.com>
1 parent b532ec1 commit c995192

3 files changed

Lines changed: 461 additions & 0 deletions

File tree

backends/arm/MODELS.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
- Conformer
66
- Deep AutoEncoder
77
- Deit Tiny
8+
- DeepSeek-R1-Distill-Qwen-1.5B
89
- DeepLab v3 (DL3)
910
- DS CNN
1011
- Inception v3 (IC3)
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Copyright 2026 Arm Limited and/or its affiliates.
2+
#
3+
# This source code is licensed under the BSD-style license found in the
4+
# LICENSE file in the root directory of this source tree.
5+
6+
from transformers.models.qwen2.configuration_qwen2 import Qwen2Config
7+
8+
9+
def get_deepseek_r1_distill_qwen_1_5b_checkpoint_config() -> Qwen2Config:
10+
config = Qwen2Config(
11+
architectures=["Qwen2ForCausalLM"],
12+
attention_dropout=0.0,
13+
bos_token_id=151643,
14+
eos_token_id=151643,
15+
hidden_act="silu",
16+
hidden_size=1536,
17+
initializer_range=0.02,
18+
intermediate_size=8960,
19+
max_position_embeddings=131072,
20+
max_window_layers=21,
21+
num_attention_heads=12,
22+
num_hidden_layers=28,
23+
num_key_value_heads=2,
24+
rms_norm_eps=1e-6, # type: ignore[arg-type]
25+
rope_parameters={
26+
"rope_type": "default",
27+
"rope_theta": 10000.0,
28+
},
29+
sliding_window=4096,
30+
tie_word_embeddings=False,
31+
transformers_version="4.44.0",
32+
use_cache=True,
33+
use_sliding_window=False,
34+
vocab_size=151936,
35+
)
36+
config.update(
37+
{
38+
"torch_dtype": "bfloat16",
39+
"use_mrope": False,
40+
}
41+
)
42+
return config

0 commit comments

Comments
 (0)