Skip to content

Commit 9210625

Browse files
committed
[None][test] AutoDeploy: enable gpt-oss registry accuracy tests
- Uncomment gpt-oss-{20b,120b} in models.yaml; registry-driven accuracy tests raised ValueError ("not found in model registry") without them. - gpt-oss world sizes: 20b=1, 120b=2; test_mxfp4_gsm8k now requires 2+ Blackwell GPUs. - gpt_oss.yaml: vocab-parallel lm_head (shard_layers+=lm_head) + AUTO allreduce for TP. - Gotcha: re-exposes these models to the external AutoDeploy dashboard. Signed-off-by: Taylor Yeonbok Lee <249374542+taylor-yb-lee@users.noreply.github.com>
1 parent 02bda4a commit 9210625

3 files changed

Lines changed: 11 additions & 12 deletions

File tree

examples/auto_deploy/model_registry/configs/gpt_oss.yaml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,10 @@ transforms:
3030
apply_sharding_hints:
3131
enabled: true
3232
requires_shape_prop: true
33-
shard_layers: ["mha", "moe"]
33+
shard_layers: ["mha", "moe", "lm_head"] # V3: vocab-parallel lm_head (colwise + all_gather)
34+
# TP2 trial: AUTO -> tunable_allreduce picks oneshot-lamport fused AR.
35+
# NCCL has no fused residual+rmsnorm kernel -> RING_LL + separate rmsnorm.
36+
allreduce_strategy: AUTO
3437
quantize_mxfp4_moe:
3538
backend: trtllm
3639
trtllm_quant_act: mxfp8

examples/auto_deploy/model_registry/models.yaml

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -168,10 +168,9 @@ models:
168168
- name: nvidia/Mistral-NeMo-Minitron-8B-Base
169169
config_id: default_ws_2
170170
yaml_extra: ['dashboard_default.yaml', 'world_size_2.yaml']
171-
# OOM during AutoDeploy run.
172-
# - name: openai/gpt-oss-20b
173-
# config_id: gpt_oss
174-
# yaml_extra: ['gpt_oss.yaml', 'world_size_1.yaml']
171+
- name: openai/gpt-oss-20b
172+
config_id: gpt_oss
173+
yaml_extra: ['gpt_oss.yaml', 'world_size_1.yaml']
175174
- name: ibm-granite/granite-3.0-8b-instruct
176175
config_id: default_ws_1
177176
yaml_extra: ['dashboard_default.yaml', 'world_size_1.yaml']
@@ -333,10 +332,9 @@ models:
333332
# - name: meta-llama/Llama-3.2-90B-Vision-Instruct
334333
# config_id: multimodal
335334
# yaml_extra: ['dashboard_default.yaml', 'world_size_8.yaml', 'multimodal.yaml']
336-
# torch.distributed.DistStoreError: Timed out after 601 seconds waiting for clients. 1/4 clients joined.
337-
# - name: openai/gpt-oss-120b
338-
# config_id: gpt_oss
339-
# yaml_extra: ['gpt_oss.yaml', 'world_size_1.yaml']
335+
- name: openai/gpt-oss-120b
336+
config_id: gpt_oss
337+
yaml_extra: ['gpt_oss.yaml', 'world_size_2.yaml']
340338
# [RANK 3] Error querying confidential compute state: Function Not Found
341339
# - name: meta-llama/Llama-4-Scout-17B-16E-Instruct
342340
# config_id: multimodal__llama4_scout

tests/integration/defs/accuracy/test_llm_api_autodeploy.py

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1276,7 +1276,6 @@ class TestGPTOSS(LlmapiAccuracyTestHarness):
12761276
"openai/gpt-oss-20b",
12771277
None,
12781278
None,
1279-
marks=pytest.mark.skip_less_device(2),
12801279
id="20b",
12811280
),
12821281
pytest.param(
@@ -1291,20 +1290,19 @@ class TestGPTOSS(LlmapiAccuracyTestHarness):
12911290
"openai/gpt-oss-120b",
12921291
2,
12931292
"tp",
1294-
marks=pytest.mark.skip_less_device(2),
12951293
id="120b-tp2",
12961294
),
12971295
pytest.param(
12981296
"120b",
12991297
"openai/gpt-oss-120b",
13001298
2,
13011299
"ep",
1302-
marks=pytest.mark.skip_less_device(2),
13031300
id="120b-ep2",
13041301
),
13051302
]
13061303

13071304
@skip_pre_blackwell
1305+
@pytest.mark.skip_less_device(2)
13081306
@pytest.mark.parametrize(
13091307
"model_id,model_name,world_size_override,moe_topology", MODEL_PARAMS)
13101308
def test_mxfp4_gsm8k(self, model_id, model_name, world_size_override,

0 commit comments

Comments
 (0)