Skip to content

Enable TransformerEngine-backed Tensor Parallelism with Llama3.#1483

Open
cspades wants to merge 10 commits into
NVIDIA-BioNeMo:mainfrom
cspades:cye/llama3-te-tp
Open

Enable TransformerEngine-backed Tensor Parallelism with Llama3.#1483
cspades wants to merge 10 commits into
NVIDIA-BioNeMo:mainfrom
cspades:cye/llama3-te-tp

Add weight tying logic to LM head, i.e. Lingua does not tie weights.

1a6f2b4
Select commit
Loading
Failed to load commit list.
DCO / DCO succeeded Mar 20, 2026 in 1s

DCO

All commits are signed off!

View more details on DCO