feat: PyTorch PT2 Model Generation#8547
Closed
whoisj wants to merge 14 commits into
Closed
Conversation
yinggeh
reviewed
Nov 25, 2025
| # Use a different model name for the non-batching variant | ||
| model_name = tu.get_model_name( | ||
| "libtorch_nobatch" if max_batch == 0 else "libtorch", | ||
| "libtorch_nobatch" if max_batch == 0 else "libtorch2" if torch2 else "libtorch", |
Contributor
There was a problem hiding this comment.
Please refactor and make the statement clear
Contributor
Author
There was a problem hiding this comment.
done. also, apologies for the force push, hadn't realized that anyone had reviewed the changes already. won't happen again. thanks.
| with open(f"{config_path}/config.pbtxt", "w") as f: | ||
| f.write( | ||
| """ | ||
| backend: "pytorch" |
Contributor
Author
There was a problem hiding this comment.
trying to keep them as the same backend.
Contributor
There was a problem hiding this comment.
I see. They differs by "platform"?
Contributor
Author
There was a problem hiding this comment.
Correct. That's the plan. Ideally we don't have competing "pytorch" and "pytorch_inductor" backends.
Contributor
Author
There was a problem hiding this comment.
renamed the platform to torch_aoti to better reflect what it is and does.
whoisj
force-pushed
the
jwyman/pt2/gen-models
branch
from
November 25, 2025 23:27
55ebafa to
558c361
Compare
whoisj
force-pushed
the
jwyman/pt2/gen-models
branch
from
December 4, 2025 18:41
558c361 to
78f755f
Compare
mc-nv
reviewed
Dec 8, 2025
yinggeh
reviewed
Dec 15, 2025
| TEST_RESULT_FILE="./test_results.txt" | ||
| CLIENT_PY="./python_based_backends_test.py" | ||
| GEN_PYTORCH_MODEL_PY="../../common/gen_qa_pytorch_model.py" | ||
| GEN_TORCHAOTI_MODEL_PY="../../common/gen_qa_torch_aoti_model.py" |
Contributor
There was a problem hiding this comment.
Where is this variable used?
| @@ -0,0 +1,124 @@ | |||
| # Copyright 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved. | |||
Contributor
There was a problem hiding this comment.
Where is this file being used?
whoisj
force-pushed
the
jwyman/pt2/gen-models
branch
2 times, most recently
from
December 17, 2025 16:31
5dd1300 to
53b28e7
Compare
whoisj
force-pushed
the
jwyman/pt2/gen-models
branch
9 times, most recently
from
December 17, 2025 22:35
dd98938 to
a8a2415
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
whoisj
force-pushed
the
jwyman/pt2/gen-models
branch
5 times, most recently
from
December 17, 2025 23:43
2610725 to
f00cef1
Compare
whoisj
force-pushed
the
jwyman/pt2/gen-models
branch
2 times, most recently
from
December 22, 2025 18:37
b94a33c to
f302116
Compare
whoisj
force-pushed
the
jwyman/pt2/gen-models
branch
from
January 6, 2026 20:30
f302116 to
82df242
Compare
This change adds the generation of PT2 format serialized PyTorch model files to the QA model generation scripts. add backend: pytorch to model config generation fix pedantic style complaints and very valid copyright mistakes pedantic style cop fix fix variable name error after rebase rename model add nvcc to pytorch path remove unnecessary changes add torch.export.export generation support to libtorch remove set -x from generated scripts improving gen_qa_models.py print statements fix color add print based on flags improve color visibility remove IDE help (caused errors) pedantic style cop fixes specify cuda fix libtorch pt2 generation add torchvision generation
whoisj
force-pushed
the
jwyman/pt2/gen-models
branch
from
January 6, 2026 21:31
82df242 to
dafcf53
Compare
Contributor
|
Please mark this PR Draft if not ready. |
whoisj
marked this pull request as draft
January 7, 2026 15:55
whoisj
force-pushed
the
jwyman/pt2/gen-models
branch
3 times, most recently
from
January 13, 2026 18:43
171eeed to
62909e0
Compare
whoisj
force-pushed
the
jwyman/pt2/gen-models
branch
from
January 13, 2026 18:44
62909e0 to
64f7855
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change adds the generation of PT2 format serialized PyTorch model files to the QA model generation scripts.
Includes significant changes to the gen_qa_model_repository flow to improve debugability.
TRI-172