[None][feat] DSv4: model, tokenizer, and integration coverage

[lfr-0531]

Description

Draft PR for PR-14751 split PR-9. This lands the DeepSeek-V4 model/API/tokenizer/parser wiring plus DSv4 docs, examples, CI stage wiring, integration test-list entries, attribution metadata, and focused unit coverage.

This PR intentionally depends on the earlier split PRs:

• PR-1 runtime/KV foundations: [None][feat] DSv4 prep: runtime cache foundations #15378
• PR-2 compressor/mHC primitives: [None][feat] DSv4 prep: compressor and mHC primitives #15379
• PR-3 IndexerTopK/TopK primitives: [None][feat] DSv4 prep: IndexerTopK and TopK primitives #15381
• PR-4 attention op plumbing: [None][feat] DSv4 prep: attention op plumbing #15384
• PR-5 attention fusion custom ops: [None][perf] DSv4 prep: attention fusion custom ops #15390
• PR-6 DSv4 sparse cache manager: [None][feat] DSv4: sparse cache manager adapter #15394
• PR-7 sparse MLA backend: [None][feat] DSv4: sparse MLA attention backend #15409
• PR-8 MoE/routing stack: [None][feat] DSv4 prep: MoE routing and backend support #15402

Notable scope notes:

• tensorrt_llm/_torch/models/modeling_gemma4.py is intentionally absent; the unnecessary PR8 Gemma4 compatibility hunk was removed.
• Earlier primitive/cache/sparse-attention/MoE implementation files are not part of PR-9's own commits.
• base_worker.py is not part of the latest PR-9 extraction; DSv4 disaggregated coverage is represented by DS stage/test-list wiring and should be validated in CI or on a model-share host.

Test plan

• Built wheel with ccache:
  • CCACHE_DIR=/home/scratch.fanrongl_coreai/ccache_trtllm python ./scripts/build_wheel.py --trt_root /usr/local/tensorrt --benchmarks --use_ccache --cuda_architectures "90-real;100-real" --configure_cmake
  • Installed build/tensorrt_llm-1.3.0rc19-cp312-cp312-linux_x86_64.whl with pip install --force-reinstall --no-deps.
  • Build exited 0. Attribution generation warned ninja -t inputs returned no results for wheel targets, but wheel creation succeeded.
• Import/path smoke with PYTHONPATH=$PWD confirmed tensorrt_llm and tensorrt_llm.bindings resolve from the PR-9 worktree, and verified DSv4 sparse config, MEGAMOE_DEEPGEMM, and the deepseek_v4 tokenizer alias.
• After checking nvidia-smi, ran on idle CUDA_VISIBLE_DEVICES=4:
  • pytest tests/unittest/_torch/modeling/test_modeling_deepseekv4.py tests/unittest/_torch/modules/test_engram.py
  • Result: 62 passed, 2 warnings.
• Ran:
  • pytest tests/unittest/_torch/test_model_config.py tests/unittest/llmapi/test_deepseek_v4_tokenizer.py tests/unittest/llmapi/test_reasoning_parser.py tests/unittest/llmapi/apps/test_tool_parsers.py tests/unittest/llmapi/test_llm_args.py tests/unittest/api_stability
  • Result: 617 passed, 3 skipped, 4 warnings.
• git diff --check HEAD passed.
• Scope-leakage grep passed; no modeling_gemma4.py, earlier C++ primitive/cache/sparse-attention/MoE implementation files, or base_worker.py in the PR-9 own manifest.
• python -m py_compile examples/llm-api/quickstart_advanced.py scripts/test_to_stage_mapping.py passed.
• python scripts/test_to_stage_mapping.py --tests "tests/integration/defs/accuracy/test_llm_api_pytorch.py::TestDeepSeekV4Flash::test_auto_dtype tests/integration/defs/accuracy/test_llm_api_pytorch.py::TestDeepSeekV4Flash::test_nvfp4_4gpus_static_eplb tests/integration/defs/accuracy/test_llm_api_pytorch.py::TestDeepSeekV4FlashBase::test_auto_dtype" returned:
  • DGX_B200-4_GPUs-PyTorch-DS-1
  • DGX_B300-4_GPUs-PyTorch-1
  • DGX_B300-4_GPUs-PyTorch-DS-1
• Targeted pre-commit on changed files passed with SKIP=type-check; the local type-check hook was attempted but exceeded 5 minutes while mypy was checking tensorrt_llm/_torch/pyexecutor/sampler.py, so it was interrupted to avoid a hang. All other targeted hooks passed, including test-list validation.

Local waivers / caveats

• DSv4 4-GPU accuracy nodeids were not run locally because LLM_MODELS_ROOT is unset and no local DeepSeek-V4-Flash / DeepSeek-V4-Flash-Base model directory was found under common model roots. These should be validated in the DS CI stages or on a host with the model share mounted.
• Full Sphinx docs build was not run because the active venv does not have sphinx installed.

[lfr-0531]

/bot run --extra-stage "DGX_B200-4_GPUs-PyTorch-DS-1,DGX_B300-4_GPUs-PyTorch-DS-1,GB200-4_GPUs-PyTorch-DS-1"

[tensorrt-cicd]

PR_Github #54577 [ run ] triggered by Bot. Commit: 2f33f8b Link to invocation

[tensorrt-cicd]

PR_Github #54577 [ run ] completed with state SUCCESS. Commit: 2f33f8b
/LLM/main/L0_MergeRequest_PR pipeline #43620 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation