Skip to content

Commit fc3b69e

Browse files
[https://nvbugs/6185173][fix] Set mamba ssm cache to fp32 for NemotronV2 (#14448)
Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com> Signed-off-by: Chenghao Zhang <211069071+nvchenghaoz@users.noreply.github.com> Co-authored-by: Chenghao Zhang <211069071+nvchenghaoz@users.noreply.github.com>
1 parent 89c90eb commit fc3b69e

2 files changed

Lines changed: 3 additions & 3 deletions

File tree

examples/auto_deploy/model_registry/configs/nemotron-nano-9b-v2.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,9 @@ trust_remote_code: true
77
kv_cache_config:
88
enable_block_reuse: false
99
free_gpu_memory_fraction: 0.7
10+
# H20 shows an MMLU accuracy drop with the default BF16 recurrent SSM cache.
11+
# Keep SSM state in FP32 for stable Triton SSM generation.
12+
mamba_ssm_cache_dtype: float32
1013

1114
# Keep max_batch_size as in the PyTorch test to avoid OOM
1215
max_batch_size: 128

tests/integration/test_lists/waives.txt

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -241,9 +241,6 @@ full:GH200/unittest/trt/model_api/test_model_quantization.py SKIP (https://nvbug
241241
full:H100/disaggregated/test_disaggregated.py::test_disaggregated_stress_test[input8k-output1k-conc512-gpt_oss_120b_cutlass_stress] SKIP (https://nvbugs/6222480)
242242
full:H100/disaggregated/test_disaggregated.py::test_disaggregated_stress_test[input8k-output1k-conc512-gpt_oss_120b_eagle_stress] SKIP (https://nvbugs/6222480)
243243
full:H100_PCIe/unittest/llmapi/test_llm_pytorch.py::test_llama_7b_multi_lora_evict_and_reload_lora_gpu_cache SKIP (https://nvbugs/5682551)
244-
full:H20/accuracy/test_llm_api_autodeploy.py::TestNemotronV2::test_auto_dtype[False] SKIP (https://nvbugs/6185173)
245-
full:H20/accuracy/test_llm_api_autodeploy.py::TestNemotronV2::test_auto_dtype[True] SKIP (https://nvbugs/6185173)
246-
full:H20/accuracy/test_llm_api_autodeploy.py::TestNemotronV2::test_fp8[True] SKIP (https://nvbugs/6185173)
247244
full:H20/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_chunked_prefill[triton-auto] SKIP (https://nvbugs/6026676)
248245
full:RTX/accuracy/test_llm_api_pytorch.py::TestGemma3_1BInstruct::test_auto_dtype SKIP (https://nvbugs/5569696)
249246
full:RTXPro6000D/accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_nvfp4_4gpus[moe_backend=CUTLASS-mtp_nextn=0-ep4-fp8kv=True-attention_dp=True-cuda_graph=True-overlap_scheduler=True-low_precision_combine=False-torch_compile=False] SKIP (https://nvbugs/5948435)

0 commit comments

Comments
 (0)