#1967)

Combines the NemotronH MoE PTQ launcher example with the MMLU
expert-parallel sharding fix it exercises. The example is the end-to-end
test for the fix — quantize + **MMLU EP=4** + export + vLLM smoke on
Nano-30B-A3B — so they ship together.

## Fix — `megatron_mmlu.py` shards over the expert-DP group for MoE
models

`megatron_mmlu` shards whole batches across the **dense data-parallel
group** and runs `megatron_prefill` per-rank on a disjoint subset.
Correct for dense models, but for MoE with **EP>1** the prefill forward
runs an **expert all-to-all across the EP group**. When EP overlaps the
dense-DP group (e.g. `EP=4,TP=1,PP=1` on 4 GPUs), each rank is on a
different batch, so the all-to-alls desync (uneven batch counts +
differing padded seq-lengths) → trailing ranks block at NCCL
communicator creation until the c10d store times out (600 s). Reproduced
on Nano-30B-A3B at `EP=4`: ranks 2 & 3 wait on rank 0's `ncclUniqueId`.

The fix shards over the **expert-data-parallel** group when `EP>1`, so
EP peers evaluate every batch in lockstep and only true expert-DP
replicas take disjoint batches. Dense models (`EP==1`) are byte-for-byte
unchanged.

## Example —
`examples/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16/megatron_lm_ptq.yaml`

Quantize (EP=4) + MMLU gate (EP=4) → export (PP=4) → vLLM smoke, modeled
on the Super-120B example. Uses `NVFP4_DEFAULT_CFG` (no Nano-30B recipe)
and sets `modelopt_install_path` to the nemo-container venv so the
mounted modelopt overrides the container copy. Passes
`test_examples_resolve.py`.

## Test plan

- [x] MoE MMLU (`EP>1`) completes instead of hanging — validated
end-to-end via this example on Nano-30B-A3B `EP=4` (nmm-sandbox CI).
- [x] `test_examples_resolve.py` green (example parses/resolves).
- [ ] Dense MMLU sharding unchanged (`EP==1` path identical).

Supersedes #1964 (example) and #1966 (fix).

🤖 Generated with [Claude Code](https://claude.com/claude-code)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Improved MMLU evaluation for expert-parallel MoE models by aligning
batch sharding with expert-parallel execution for more consistent
results.
* **Bug Fixes**
* Dense (non-MoE) models retain standard data-parallel batch sharding,
while MoE handling is corrected.
* **Documentation**
* Updated the Nemotron-3-Nano BF16 PTQ example to clarify the quantize →
MMLU gate → export flow and why the vLLM smoke is intentionally omitted.
* Adjusted the Llama-3.2-1B-Instruct PTQ/MMLU gate lower-bound threshold
(0.40 → 0.36).
* **Tests**
* Marked the homogeneous compressed sharded state-dict test to skip on
Blackwell to avoid a known flaky issue.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Chenhan Yu <chenhany@nvidia.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>