Fetch from nvidia Megatron-LM by RaymondLi0 · Pull Request #5 · ElementAI/Megatron-LM

RaymondLi0 · 2022-08-03T20:32:25Z

No description provided.

Co-authored-by: William Dykas <wdykas@oci-hsg-cs-001-vscode-03.cm.cluster> Co-authored-by: root <root@nvl72077-T12.cm.cluster> Co-authored-by: root <root@nvl72098-T17.cm.cluster> Co-authored-by: root <root@nvl72160-T18.cm.cluster> Co-authored-by: root <root@nvl72006-T11.cm.cluster> Co-authored-by: root <root@nvl72074-T12.cm.cluster>

…yer for DiT (diffusion transformer) (#4134) Co-authored-by: Huy Vu2 <huvu@login-eos02.eos.clusters.nvidia.com>

Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>

…lel (#4259) Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Signed-off-by: Santosh Bhavani <santosh.bhavani@live.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

#4270)

Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

Co-authored-by: Li Tao <lit@nvidia.com>

Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

…r__ (#3753) Signed-off-by: Maxime Grenu <maxime.grenu@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: gautham-kollu <gkollu@nvidia.com>

Signed-off-by: Hollow Man <hollowman@opensuse.org>

#4014) Co-authored-by: gautham-kollu <gkollu@nvidia.com>

…4281) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Co-authored-by: gautham-kollu <gkollu@nvidia.com>

Signed-off-by: Xin Yao <xiny@nvidia.com>

Signed-off-by: qiyuw <qiyuw@nvidia.com>

Signed-off-by: Deyu Fu <deyuf@nvidia.com> Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Signed-off-by: Xin Yao <xiny@nvidia.com>

…rlap for FlexDispatcher (#2207) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Signed-off-by: Shivanjan Chakravorty <shivanjanc@nvidia.com>

Signed-off-by: Ajay Balasa <abalasa@nvidia.com> Co-authored-by: Cursor <cursoragent@cursor.com>

Signed-off-by: Maanu Grover <maanug@nvidia.com>

Signed-off-by: Chen Cui <chcui@nvidia.com> Co-authored-by: Tuomas Rintamaki <trintamaki@nvidia.com> Co-authored-by: Tyler Poon <tylerpoon@gmail.com> Co-authored-by: Collin McCarthy <cmccarthy@nvidia.com> Co-authored-by: Matthieu Le <matthieul@nvidia.com> Co-authored-by: Piotr Zelasko <pzelasko@nvidia.com> Co-authored-by: Ehsan Hosseini Asl <ehosseiniasl@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-authored-by: Siddharth Singh <sidsingh@nvidia.com>

Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

…parallel_size * expert_tensor_parallel_size (#4678) Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

Co-authored-by: Jorge Albericio <jalbericiola@nvidia.com>

#4509) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Signed-off-by: Maanu Grover <maanug@nvidia.com>

#4492) Signed-off-by: Xiaowei Ren <xren@nvidia.com>

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Co-authored-by: William Dykas <wdykas@oci-hsg-cs-001-vscode-03.cm.cluster>

…parallelism correctly (#4687)

…2645) Signed-off-by: yuzhongw <yuzhongw@nvidia.com> Co-authored-by: kunlunl <kunlunl@nvidia.com> Co-authored-by: Xuesong Ye <xuesongyey@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: Daisy Gao <daisyg@nvidia.com>

…4511) Co-authored-by: Antoni-Joan Solergibert <asolergibert@nvidia.com>

Co-authored-by: gautham-kollu <gkollu@nvidia.com> Co-authored-by: Yashaswi Karnati <144376261+yashaswikarnati@users.noreply.github.com> Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>

Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>

…running tests (#4787)

ko3n1g force-pushed the main branch from ad5a627 to cd71d4c Compare February 18, 2026 09:18

wdykas and others added 29 commits April 10, 2026 19:08

Add conditions_embeddings argument to TransformerBlock, TransformerLa…

1daa19f

…yer for DiT (diffusion transformer) (#4134) Co-authored-by: Huy Vu2 <huvu@login-eos02.eos.clusters.nvidia.com>

Update copy-pr-bot.yaml [skip ci]

705d8ed

Fix build_sequences_per_dataset output path arg usage (#4144)

59fc894

Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>

ci: Flush pending CUDA work before the barrier in destroy_model_paral…

7cbc68c

…lel (#4259) Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Update oncall schedule (#4257)

e3d1204

docs(moe): Update MoE README (#3664)

41fcaa4

Signed-off-by: Santosh Bhavani <santosh.bhavani@live.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Revert "Add conditions_embeddings argument to TransformerBlock, Trans… (

cc4cb01

#4270)

reduce the number of shared expert streams (#3752)

6da6267

remove legacy Bert code (#4204)

7f8f37e

Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

[Main] Feat(moe): Gated delta net context parallel (CP) (#2642)

20ba03f

Co-authored-by: Li Tao <lit@nvidia.com>

remove t5 legacy code (#4203)

81b8b5a

Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

fix: handle list-typed process groups in ProcessGroupCollection.__rep…

dda6901

…r__ (#3753) Signed-off-by: Maxime Grenu <maxime.grenu@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: gautham-kollu <gkollu@nvidia.com>

Fix Context Parallelism documentation link (#4149)

10f5fbd

[MLA] fix: Pad V when Q/V head dims differ for THD (#3003)

5dcda19

Signed-off-by: Hollow Man <hollowman@opensuse.org>

Allow the evaluation batch size to differ from the training batch size (

d85365b

#4014) Co-authored-by: gautham-kollu <gkollu@nvidia.com>

fix(megatron-fsdp): build expt_device_mesh only for MoE models (#3831)

25129bf

Add @NVIDIA/transformer review group to megatron/core/transformer/ (#…

a1595b8

…4281) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Reset AG_pipeline bucket status after validation step. (#3155)

5f80f0a

Co-authored-by: gautham-kollu <gkollu@nvidia.com>

Update copy-pr-bot.yaml [skip ci]

e32e323

Enhance and fix NVTX for training (#3642)

df929f5

Signed-off-by: Xin Yao <xiny@nvidia.com>

NVFP4 native weights for DDP (#4005)

e1db4a0

Signed-off-by: qiyuw <qiyuw@nvidia.com>

Remove unnecessary arguments for layerwise distributed optimizer (#4272)

1ca250c

Signed-off-by: Deyu Fu <deyuf@nvidia.com> Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>

reuse grad buffer for layer-wise param allgather (#3751)

28e13c4

feat(ci): add strict review mode to Claude review workflow (#4197)

e9e513a

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Fix stale approvals (#4280)

cd03c3e

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

[MoE] Add a new score function to the router (#3673)

eb80b74

Signed-off-by: Xin Yao <xiny@nvidia.com>

[MoE] Improvement of shared expert overlap, support shared expert ove…

ebfa138

…rlap for FlexDispatcher (#2207) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

build: bump DeepEP to 34152ae (#4228)

123645b

Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Glitchfix and others added 30 commits May 11, 2026 22:10

feat(gpt): add output postprocess hook (#4686)

97f3bce

Signed-off-by: Shivanjan Chakravorty <shivanjanc@nvidia.com>

Add bump-base-image skill and update golden value comparison (#4733)

f744215

Signed-off-by: Ajay Balasa <abalasa@nvidia.com> Co-authored-by: Cursor <cursoragent@cursor.com>

Guard omegaconf imports (#4685)

6486d52

Signed-off-by: Maanu Grover <maanug@nvidia.com>

Fix a regression introduced by #4625 for nightly runs (#4734)

7d24b28

Support transfomers 5.x.x for text generation server (#4732)

86bf476

Update transformer-engine dependency to version 2.15.0 (#4682)

815c83d

Increase CG cover from max_requests to max_tokens (#4214)

72dd053

Co-authored-by: Siddharth Singh <sidsingh@nvidia.com>

chore: rotate oncall schedule

f8c942b

fully remove legacy code (#4759)

10b514b

Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

fix legacy torch save when tensor_model_parallel_size > expert_model_…

fc41581

…parallel_size * expert_tensor_parallel_size (#4678) Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

Wire --rl-inference-parsers into MRL (#4768)

d802f09

Co-authored-by: Jorge Albericio <jalbericiola@nvidia.com>

Integrate LayerWiseDistributedOptimizer with DDP buffer infrastructure (

c1e938b

#4509) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

[training migration] Migrate mamba builder (#4550)

0dc36df

Signed-off-by: Maanu Grover <maanug@nvidia.com>

NCCL UB fix: reduce memory cost and correctly deregister NCCL mem pool (

e35d4e5

#4492) Signed-off-by: Xiaowei Ren <xren@nvidia.com>

fix: use no_mask in local ViT layer spec (#4395)

1ba0aa9

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

refit clean up and refactoring (#4762)

20bf831

Co-authored-by: William Dykas <wdykas@oci-hsg-cs-001-vscode-03.cm.cluster>

Make weight and optimizer memory estimation take into account expert …

a7c9e8c

…parallelism correctly (#4687)

Support recomputing in HybridModel (#4496)

118933a

One single flag that determines if we are in inference (#4617)

925422c

remove dead manual_release_grads code path in 1F1B overlap schedule (#…

58d45bf

…4511) Co-authored-by: Antoni-Joan Solergibert <asolergibert@nvidia.com>

Fix recompute checkpointing + training CGs (#3919)

ada8dfe

Use Protocols to type-check linear_proj submodules of Attention (#3434)

dbfc96b

Co-authored-by: gautham-kollu <gkollu@nvidia.com> Co-authored-by: Yashaswi Karnati <144376261+yashaswikarnati@users.noreply.github.com> Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>

fix tokenizers in respect to newer transformers (#4608)

d167123

Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

Bump nvidia-modelopt>=0.44.0 (#4803)

98031e1

Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>

Update owners (#4794)

266562f

ci: Update workflow to use same commit for building docker image and …

e9a0930

…running tests (#4787)

chore: Update nightly tests golden values (#4805)

a53107c

Inference: Optimize Prefill Engine Steps for Nemotron (#4764)

9b4074b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fetch from nvidia Megatron-LM#5

Fetch from nvidia Megatron-LM#5
RaymondLi0 wants to merge 7286 commits into
ElementAI:load-iterfrom
NVIDIA:main

RaymondLi0 commented Aug 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

RaymondLi0 commented Aug 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants