feat: adding PP and CP for nemotron v3 models#2316

Merged

adil-a merged 25 commits into

mainfrom

Jun 4, 2026

Commits on May 25, 2026

feat(nemotron_v3): pipeline parallel + MTP support, plus THD collator fix
adil-a
committed
chore(datasets): drop pre_rendered_chat_dataset.py from branch
adil-a
committed

Commits on May 26, 2026

fix(thd_utils): compute max_seqlen from final cu_seqlens to honor TE contract

adil-a
and
claude
committed
chore(thd_utils): drop dead cu_seqlens_padded fallback in emit

adil-a
and
claude
committed
docs(thd_utils): update docstrings to match post-fix cu_seqlens semantics

adil-a
and
claude
committed
chore(thd_utils): trim verbose inline comments

adil-a
and
claude
committed
comments
adil-a
committed
fix(seq_idx): use searchsorted(right=True) to classify boundary tokens

adil-a
and
claude
committed
chore: remove debug env-var hooks from THD packing investigation

adil-a
and
claude
committed
test(mtp): pin MTPModule cumulative left-rolling of input_ids and position_ids

adil-a
and
claude
committed
chore(mtp): trim verbose comments in calculate_mtp_loss

adil-a
and
claude
committed
docs(nemotron_v3): correct stale 'PP cannot chunk' comments in model.forward

adil-a
and
claude
committed
chore(nemotron_v3): trim verbose comments in model.py

adil-a
and
claude
committed

Commits on May 28, 2026

Commits on May 31, 2026