Skip to content

test: PyTorch PT2 Model Generation - #8666

Merged
whoisj merged 4 commits into
mainfrom
jwyman/pt2/gen-models
Mar 13, 2026
Merged

test: PyTorch PT2 Model Generation#8666
whoisj merged 4 commits into
mainfrom
jwyman/pt2/gen-models

Conversation

@whoisj

@whoisj whoisj commented Feb 19, 2026

Copy link
Copy Markdown
Contributor

This change can be conceptually separated into three parts:

  1. Generation of test models using PyTorch AOTI compile and package.
  2. Addition of colorized output to the generation script
  3. "Unhiding" of generation output to avoid the "nothing is happening" moments when the script is running.

Commit Type:

  • Test

Where should the reviewer start?

  1. qa/common/gen_qa_models.py@L1302
  2. qa/common/gen_qa_models.py@L1623
  3. qa/common/gen_qa_models.py@L2118
  4. qa/common/test_utils.py@L269
  5. qa/common/gen_qa_model_repository@L265
  6. qa/common/gen_qa_model_repository@L268

CI Pipeline ID:

44310672

Caveats:

Depends on nvcr.io/nvidia/pytorch:26.02-py3 and later containers.

Comment thread qa/common/gen_qa_models.py Fixed
Comment thread qa/common/gen_qa_models.py Fixed
@yinggeh

yinggeh commented Feb 27, 2026

Copy link
Copy Markdown
Contributor

Label hould be test instead of feat

@yinggeh

yinggeh commented Feb 27, 2026

Copy link
Copy Markdown
Contributor

Please fill out template for non-trivial PRs for easier tracking
https://github.com/triton-inference-server/server/blob/main/.github/PULL_REQUEST_TEMPLATE/pull_request_template_internal_contrib.md

@whoisj whoisj changed the title feat: PyTorch PT2 Model Generation test: PyTorch PT2 Model Generation Mar 2, 2026
@whoisj

whoisj commented Mar 2, 2026

Copy link
Copy Markdown
Contributor Author

Please fill out template for non-trivial PRs for easier tracking

Updated.

@@ -0,0 +1,1000 @@
tench

@yinggeh yinggeh Mar 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not curl download this file?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why "not"? Because that would just add overhead and an additional point of failure to the CI.

The file in only 11KB in the workspace, and less than 5KB in the repository.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for generating additional QA model artifacts using PyTorch PT2 AOTInductor (AOTI), alongside improved visibility/colorization of generation output to make long-running model generation less opaque in CI.

Changes:

  • Add PT2/AOTI model generation + config generation paths to qa/common/gen_qa_models.py (including TorchVision ResNet50 AOTI).
  • Add colorized/less-silent logging across the Python generator and the gen_qa_model_repository wrapper script.
  • Add a ResNet50 label file used by TorchVision/TorchTRT-style model configs; adjust model naming helper to allow a single-output model name format.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 10 comments.

File Description
qa/common/test_util.py Allow get_model_name() to omit output1_dtype for single-output model naming.
qa/common/resnet50_labels.txt Add a 1000-class label list intended for ResNet50 output labeling.
qa/common/gen_qa_models.py Introduce PT2/AOTI model+config generation and add colorized progress logging.
qa/common/gen_qa_model_repository Improve logging/output visibility, add PT2/AOTI invocation, and adjust docker-run verbosity/behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread qa/common/gen_qa_model_repository
Comment thread qa/common/gen_qa_models.py
Comment thread qa/common/gen_qa_models.py
Comment thread qa/common/gen_qa_model_repository
Comment thread qa/common/gen_qa_model_repository
Comment thread qa/common/gen_qa_models.py
Comment thread qa/common/gen_qa_models.py
Comment thread qa/common/gen_qa_models.py
Comment thread qa/common/gen_qa_models.py
Comment thread qa/common/gen_qa_models.py
whoisj and others added 3 commits March 11, 2026 16:47
This change adds the generation of PT2 format serialized PyTorch model files to the QA model generation scripts.

add backend: pytorch to model config generation

fix pedantic style complaints and very valid copyright mistakes

pedantic style cop fix

fix variable name error after rebase

rename model

add nvcc to pytorch path

remove unnecessary changes

add torch.export.export generation support to libtorch

remove set -x from generated scripts

improving gen_qa_models.py print statements

fix color

add print based on flags

improve color visibility

remove IDE help (caused errors)

pedantic style cop fixes

specify cuda

fix libtorch pt2 generation

add torchvision generation

fix aoti model generation output types
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
@whoisj
whoisj force-pushed the jwyman/pt2/gen-models branch from 698eff3 to f9fdaea Compare March 11, 2026 20:48
@whoisj
whoisj requested a review from yinggeh March 11, 2026 22:39
@whoisj whoisj added PR: test Adding missing tests or correcting existing test labels Mar 11, 2026
@whoisj
whoisj merged commit d110600 into main Mar 13, 2026
3 checks passed
@whoisj
whoisj deleted the jwyman/pt2/gen-models branch March 13, 2026 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

PR: test Adding missing tests or correcting existing test

Development

Successfully merging this pull request may close these issues.

4 participants