Skip to content

Enhancement: Add rocprofv2 trace support for AMD GPUs#817

Open
shcho wants to merge 5 commits into
microsoft:mainfrom
shcho:dev/choshenghsun/reliaguard-tracegen-mi300x
Open

Enhancement: Add rocprofv2 trace support for AMD GPUs#817
shcho wants to merge 5 commits into
microsoft:mainfrom
shcho:dev/choshenghsun/reliaguard-tracegen-mi300x

Conversation

@shcho

@shcho shcho commented May 19, 2026

Copy link
Copy Markdown

Description
Extend trace generation to support AMD GPUs using rocprofv2

  • runner.py: Add SB_ENABLE_ROCPROF/SB_ROCPROF_TRACE_DIR env vars to enable rocprofv2 profiling (--hip-trace --kernel-trace --plugin json) in local, torch.distributed, and mpi modes
  • pytorch_base.py: Extend GPU guard to support ROCm (torch.version.hip) so PyTorch profiler works on AMD GPUs

- runner.py: Add SB_ENABLE_ROCPROF/SB_ROCPROF_TRACE_DIR env vars to enable
  rocprofv2 profiling (--hip-trace --kernel-trace --plugin json) in local,
  torch.distributed, and mpi modes
- pytorch_base.py: Extend GPU guard to support ROCm (torch.version.hip)
  so PyTorch profiler works on AMD GPUs
Copilot AI review requested due to automatic review settings May 19, 2026 18:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds ROCm profiling support alongside existing Nsight Systems tracing and updates model benchmark GPU detection to include AMD (HIP) builds.

Changes:

  • Add rocprofv2 command injection (gated by env vars) for local/distributed/mpi runner modes.
  • Introduce ROCm trace directory env var support (SB_ROCPROF_TRACE_DIR).
  • Expand PyTorch GPU check to treat HIP builds as GPU-capable.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.

File Description
superbench/runner/runner.py Adds optional rocprofv2 profiling prefixes/trace commands controlled by environment variables.
superbench/benchmarks/model_benchmarks/pytorch_base.py Updates GPU detection to include ROCm/HIP PyTorch builds.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread superbench/runner/runner.py Outdated
Comment thread superbench/runner/runner.py
Comment thread superbench/runner/runner.py
Comment thread superbench/runner/runner.py Outdated
Comment thread superbench/benchmarks/model_benchmarks/pytorch_base.py Outdated
Shenghsun Cho added 2 commits May 19, 2026 18:46
Address PR review: wrap all interpolated path/name segments in
shlex.quote() to prevent command injection or broken commands when
paths contain whitespace or shell metacharacters. Applied to both
nsys and rocprofv2 trace commands across all three execution modes
(local, torch.distributed, mpi).
Address PR review: the variable holds either an nsys or rocprofv2
prefix, so rename to trace_prefix to avoid implying Nsight-only
behavior.
@shcho shcho marked this pull request as ready for review May 20, 2026 02:35
@shcho shcho requested a review from a team as a code owner May 20, 2026 02:35
Copilot AI review requested due to automatic review settings May 20, 2026 02:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

Comment thread superbench/benchmarks/model_benchmarks/pytorch_base.py Outdated
Comment thread superbench/runner/runner.py
Comment thread superbench/runner/runner.py
Comment thread superbench/runner/runner.py
Shenghsun Cho added 2 commits July 7, 2026 18:19
Address PR review: torch.version.hip (and .cuda) may not exist on all
PyTorch builds/versions, raising AttributeError. Use getattr with a
None default to make the check compatible across all PyTorch builds.
Address PR review: add the conventional -- end-of-options separator
after rocprofv2 flags so that benchmark command arguments starting
with - are not misinterpreted as rocprofv2 options. Applied to all
three execution modes (local, torch.distributed, mpi).
@shcho

shcho commented Jul 7, 2026

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree company="Microsoft"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants