Pytorch ddp

[RobJY]

I found a README with this tutorial in it with the other code examples, so I modified it to work with Greene using our overlay files. I see now that it's part of a repo from Princeton. Maybe it would be better to make a fork of the original repo? This would make it clear where it comes from and why we are modifying it. We could then make markdown versions of the other 4 PyTorch tutorials in the repo as well.

How does that sound?

[github-actions]

PR Preview Action v1.6.2
Preview removed because the pull request was closed.
2025-11-04 21:23 UTC

[s-sajid-ali]

Maybe it would be better to make a fork of the original repo?

Sure. Or if you're only using a handful of scripts from there they can go into https://github.com/NYU-RTS/rts-docs-examples/ as well.

This PR looks great! The only change I'd request is to fill out the intro page, we currently have two empty intro pages:

• docs/hpc/ml_ai_hpc/intro/
• docs/hpc/ml_ai_hpc/pytorch_intro/

Unless there is a need for both, should we remove the duplicate?

[RobJY]

Thanks for the review @s-sajid-ali. Great points.
I was thinking the pytorch intro would be the single GPU example from the Princeton tutorial. I'll get to work on adding it and we can include them both in this PR.
I'm not sure what to put for the general intro to the section. Please let me know if you have any suggestions.

[RobJY]

I added a first pass on the intro page and the simple single GPU PyTorch example. Please let me know if you have any suggestions @s-sajid-ali. The main intro page needs more content, but I'm not sure what to add.

Reading through it I wonder if the ML and AI pages should be separated into their own sections. What do you think?

I can remove the empty pages for pytorch_lightning, pytorch_fsdp and tensorflow if you'd like to merge this soon. Or put a 'coming soon' message in those pages as placeholders.

[s-sajid-ali]

This looks great!

Reading through it I wonder if the ML and AI pages should be separated into their own sections. What do you think?

By the workflow stack, they're all PyTorch at the end of the day! Right now only the two LLM pages are GenAI right now. If that number grows, we can give GenAI it's own section.

Merge soon is better, we can always add more material in follow up PRs!

[RobJY]

sounds good @s-sajid-ali . I've removed the placeholder pages, so I think it's ready to go now. Please let me know if you see anything that needs to be changed. Thanks!