We welcome the community to submit reproducible benchmarking results.
A submission should be a file / files including the following information
- Entity, which could be your name, GitHub username, company, university, team, etc.
- The model or theme of benchmarking, e.g. Llama 3.1, Async TP.
- The hardware setup, including the types of GPUs, interconnections, etc.
- The actual performance report with training configs, e.g. via
- Python config files / commandline arguments
- complete configs, which can be found in the log with
--print_configturned on (preferred as the default value not shown in config files or specified in commandline could change from time to time)
- The versions and date/time of
torchtitan,torch,torchao, or any relevant dependencies. - Other notes which could help reproduce the results.
The name of the file should follow the format of
[model/theme]_[hardware]_[date/time]_[entity].md
For example, llama3.1_h100_202412_pytorch.md, asynctp_256xh100_20250613_alice+bob.md.
An example can be found at llama3_h100_202412_torchtitan.md.