Skip to content

[None][test] CI bisect nvbugs/6280721: probe b4d44d3 (unwaive GPT-OSS w4 DP4 CUTLASS #14884)#15261

Closed
tensorrt-cicd wants to merge 2 commits into
chenfeiz/bisect-6280721-baseline-2-basefrom
chenfeiz/bisect-6280721-baseline-2-head
Closed

[None][test] CI bisect nvbugs/6280721: probe b4d44d3 (unwaive GPT-OSS w4 DP4 CUTLASS #14884)#15261
tensorrt-cicd wants to merge 2 commits into
chenfeiz/bisect-6280721-baseline-2-basefrom
chenfeiz/bisect-6280721-baseline-2-head

Conversation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

Purpose

Pre-merge CI bisect probe for nvbugs/6280721. This PR isolates the wider-window probe at commit a8c4007 ([None][fix] Fix AutoDeploy accuracy tests #13925) — the commit immediately before the parent baseline 910826b — by basing the PR on its parent 33b0a32.

Background

a8c4007 is one step further back from the existing baseline probe (#15242). Combined with the other two probes (#15241 culprit 8e5d9e2, #15242 parent 910826b), this widens the bisect window to confirm the regression at 8e5d9e2 is not pre-existing earlier in the chain.

Expected Result

This test should pass here.

Test Under Observation

perf/test_perf_sanity.py::test_e2e[disagg_upload-gen_only-gb200_deepseek-v32-fp4_1k1k_con2048_ctx1_dep4_gen1_dep4_eplb0_mtp1_ccb-NIXL]

Waiver Note

The test case is not present in tests/integration/test_lists/waives.txt at this commit, so no waiver removal is needed.

Signed-off-by: Dongfeng Yu <dongfengy@nvidia.com>
@tensorrt-cicd tensorrt-cicd requested review from a team as code owners June 11, 2026 11:47
@tensorrt-cicd tensorrt-cicd requested review from suyoggupta and removed request for a team June 11, 2026 11:47
@tensorrt-cicd tensorrt-cicd marked this pull request as draft June 11, 2026 11:48
@chenfeiz0326

Copy link
Copy Markdown
Collaborator

/bot run --disable-fail-fast --stage-list "GB200-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE1-GPU4-Post-Merge-4"

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #53564 [ run ] triggered by Bot. Commit: 3d7d88d Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #53564 [ run ] completed with state FAILURE. Commit: 3d7d88d
/LLM/main/L0_MergeRequest_PR pipeline #42713 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@tensorrt-cicd tensorrt-cicd force-pushed the chenfeiz/bisect-6280721-baseline-2-head branch from 3d7d88d to b4d44d3 Compare June 15, 2026 06:49
@tensorrt-cicd tensorrt-cicd changed the title [None][test] CI bisect nvbugs/6280721: probe baseline-2 a8c4007 (Fix AutoDeploy accuracy tests #13925) [None][test] CI bisect nvbugs/6280721: probe b4d44d3 (unwaive GPT-OSS w4 DP4 CUTLASS #14884) Jun 15, 2026
@chenfeiz0326

Copy link
Copy Markdown
Collaborator

/bot run --disable-fail-fast --stage-list "GB200-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-7,GB200-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-5"

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54244 [ run ] triggered by Bot. Commit: b4d44d3 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54244 [ run ] completed with state SUCCESS. Commit: b4d44d3
/LLM/main/L0_MergeRequest_PR pipeline #43318 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

@chenfeiz0326

Copy link
Copy Markdown
Collaborator

/bot run --disable-fail-fast --stage-list "GB200-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-7,GB200-12_GPUs-3_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE2-GPU8-Post-Merge-5"

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54305 [ run ] triggered by Bot. Commit: 91684c4 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator Author

PR_Github #54305 [ run ] completed with state SUCCESS. Commit: 91684c4
/LLM/main/L0_MergeRequest_PR pipeline #43376 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

@chzblych chzblych closed this Jun 16, 2026
@chzblych chzblych deleted the chenfeiz/bisect-6280721-baseline-2-head branch June 16, 2026 06:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants