[None][test] Waive 23 failed cases for main in QA CI by tensorrt-cicd · Pull Request #15337 · NVIDIA/TensorRT-LLM

tensorrt-cicd · 2026-06-13T11:58:33Z

Auto-generated Waive PR

Created by: TensorRT LLM CI Report (requested by qa@nvidia.com)
Target branch: main
Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_TEST/2243/
Bug(s): 6223530, 6313076, 6316152

Waive entries added

full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_eagle3_4gpus[v1_kv_cache-trtllm-one_model-no_overlap_scheduler] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_eagle3_4gpus[v1_kv_cache-trtllm-one_model-overlap_scheduler] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_eagle3_4gpus[v1_kv_cache-trtllm-two_model-no_overlap_scheduler] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_eagle3_4gpus[v1_kv_cache-trtllm-two_model-overlap_scheduler] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_eagle3_4gpus[v2_kv_cache-cutlass-two_model-no_overlap_scheduler] SKIP (https://nvbugs/6223530)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_eagle3_4gpus[v2_kv_cache-trtllm-one_model-no_overlap_scheduler] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_eagle3_4gpus[v2_kv_cache-trtllm-one_model-overlap_scheduler] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_eagle3_4gpus[v2_kv_cache-trtllm-two_model-no_overlap_scheduler] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v1_kv_cache-dp4-trtllm-auto] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v1_kv_cache-dp4-trtllm-fp8] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v1_kv_cache-ep4-trtllm-auto] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v1_kv_cache-ep4-trtllm-fp8] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v1_kv_cache-tp4-trtllm-auto] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v1_kv_cache-tp4-trtllm-fp8] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v2_kv_cache-dp4-trtllm-auto] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v2_kv_cache-dp4-trtllm-fp8] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v2_kv_cache-ep4-trtllm-auto] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v2_kv_cache-ep4-trtllm-fp8] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v2_kv_cache-tp4-trtllm-auto] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v2_kv_cache_no_reuse-tp4-trtllm-auto] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_4gpus[v2_kv_cache_no_reuse-tp4-trtllm-fp8] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestGPTOSS::test_w4_chunked_prefill[trtllm-fp8] SKIP (https://nvbugs/6316152)
full:RTX_6000D/accuracy/test_llm_api_pytorch.py::TestQwen3_5_35B_A3B::test_fp8[enable_block_reuse=False] SKIP (https://nvbugs/6313076)

This PR was auto-generated by TensorRT LLM CI Report. Please review the waive entries before merging.

Summary by CodeRabbit

Tests
- Updated test waiver configurations for GPU test compatibility across multiple test scenarios and hardware variants.

jieli-matrix · 2026-06-16T03:02:08Z

/bot run --stage-list ""

coderabbitai · 2026-06-16T03:04:44Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e1e9d530-544d-4253-ad09-53d0128b5216

📥 Commits

Reviewing files that changed from the base of the PR and between f49d09f and 9ba5c71.

📒 Files selected for processing (1)

tests/integration/test_lists/waives.txt

📝 Walkthrough

Walkthrough

Adds 23 SKIP/waive entries to tests/integration/test_lists/waives.txt for the RTX_6000D hardware target. New entries cover TestGPTOSS::test_eagle3_4gpus (kv-cache v1/v2, one_model/two_model, overlap scheduler variants), TestGPTOSS::test_w4_4gpus (parallelism/kv-cache reuse combinations), TestGPTOSS::test_w4_chunked_prefill[trtllm-fp8], and TestQwen3_5_35B_A3B::test_fp8[enable_block_reuse=False].

Changes

RTX_6000D Integration Test Waiver Additions

Layer / File(s)	Summary
RTX_6000D waive entries for GPTOSS Eagle3, W4, and Qwen3 FP8 tests `tests/integration/test_lists/waives.txt`	Adds SKIP entries for `TestGPTOSS::test_eagle3_4gpus` with all kv-cache/model/scheduler combinations, multiple `TestGPTOSS::test_w4_4gpus` parallelism and kv-cache reuse permutations, `TestGPTOSS::test_w4_chunked_prefill[trtllm-fp8]`, and `TestQwen3_5_35B_A3B::test_fp8[enable_block_reuse=False]`.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

NVIDIA/TensorRT-LLM#15358: Modifies the same tests/integration/test_lists/waives.txt file, adding different SKIP/waive entries for failing integration test cases.
NVIDIA/TensorRT-LLM#15373: Also edits the same test waiver list file by adding/changing SKIP entries for integration and perf test cases.
NVIDIA/TensorRT-LLM#15315: Directly modifies tests/integration/test_lists/waives.txt by adding SKIP entries for different test cases and devices using the same waiver mechanism.

Suggested reviewers

crazydemo
jieli-matrix
StanleySun639
LarryXFly
xinhe-nv

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly indicates this is a test waiver PR for QA CI with a count of failed cases being waived, which matches the changeset content.
Description check	✅ Passed	The description provides context (auto-generated waive PR), links to CI/bugs, and lists all 23 waived entries with bug references, but does not follow the template sections.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

tensorrt-cicd · 2026-06-16T03:10:03Z

PR_Github #54440 [ run ] triggered by Bot. Commit: 9ba5c71 Link to invocation

tensorrt-cicd · 2026-06-16T04:19:30Z

PR_Github #54440 [ run ] completed with state FAILURE. Commit: 9ba5c71
/LLM/main/L0_MergeRequest_PR pipeline #43505 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

jieli-matrix · 2026-06-16T06:33:17Z

/bot run --stage-list ""

tensorrt-cicd · 2026-06-16T06:39:45Z

PR_Github #54511 [ run ] triggered by Bot. Commit: 681eeaa Link to invocation

tensorrt-cicd · 2026-06-16T09:27:47Z

PR_Github #54511 [ run ] completed with state SUCCESS. Commit: 681eeaa
/LLM/main/L0_MergeRequest_PR pipeline #43568 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

jieli-matrix · 2026-06-17T02:25:16Z

/bot run --stage-list ""

tensorrt-cicd · 2026-06-17T02:32:31Z

PR_Github #54717 [ run ] triggered by Bot. Commit: 53190c0 Link to invocation

tensorrt-cicd · 2026-06-17T04:28:35Z

PR_Github #54717 [ run ] completed with state FAILURE. Commit: 53190c0
/LLM/main/L0_MergeRequest_PR pipeline #43743 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

jieli-matrix · 2026-06-18T06:58:45Z

/bot run --stage-list ""

tensorrt-cicd · 2026-06-18T07:04:15Z

PR_Github #54910 [ run ] triggered by Bot. Commit: 48da6f4 Link to invocation

tensorrt-cicd · 2026-06-18T11:34:17Z

PR_Github #54910 [ run ] completed with state SUCCESS. Commit: 48da6f4
/LLM/main/L0_MergeRequest_PR pipeline #43913 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

jieli-matrix · 2026-06-19T06:22:35Z

/bot run --stage-list ""

tensorrt-cicd · 2026-06-19T06:27:59Z

PR_Github #54961 [ run ] triggered by Bot. Commit: c9d3f4e Link to invocation

tensorrt-cicd · 2026-06-19T10:56:44Z

PR_Github #54961 [ run ] completed with state SUCCESS. Commit: c9d3f4e
/LLM/main/L0_MergeRequest_PR pipeline #43961 (Partly Tested) completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

Bug(s): 6223530, 6313076, 6316152 Requested by: qa@nvidia.com Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_TEST/2243/ Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com>

jieli-matrix · 2026-06-20T04:37:59Z

/bot skip --comment "Only waive tests"

tensorrt-cicd · 2026-06-20T04:43:13Z

PR_Github #54976 [ skip ] triggered by Bot. Commit: e2c7b5b Link to invocation

tensorrt-cicd · 2026-06-20T04:48:18Z

PR_Github #54976 [ skip ] completed with state SUCCESS. Commit: e2c7b5b
Skipping testing for commit e2c7b5b

Link to invocation

Signed-off-by: tensorrt-cicd <90828364+tensorrt-cicd@users.noreply.github.com> Signed-off-by: GitLab CI Bot <gitlab-ci@nvidia.com>

tensorrt-cicd requested review from LarryXFly, StanleySun639, crazydemo, jieli-matrix and xinhe-nv June 13, 2026 11:58

github-actions Bot assigned tensorrt-cicd Jun 13, 2026

jieli-matrix marked this pull request as ready for review June 16, 2026 03:01

jieli-matrix force-pushed the trtllm-ci-report/waive-20260613-120253 branch from e0816c0 to 9ba5c71 Compare June 16, 2026 03:01

jieli-matrix approved these changes Jun 16, 2026

View reviewed changes

jieli-matrix enabled auto-merge (squash) June 16, 2026 03:02

jieli-matrix force-pushed the trtllm-ci-report/waive-20260613-120253 branch from 9ba5c71 to 681eeaa Compare June 16, 2026 06:33

jieli-matrix force-pushed the trtllm-ci-report/waive-20260613-120253 branch from 681eeaa to 53190c0 Compare June 17, 2026 02:25

jieli-matrix force-pushed the trtllm-ci-report/waive-20260613-120253 branch from 53190c0 to 48da6f4 Compare June 18, 2026 06:58

jieli-matrix force-pushed the trtllm-ci-report/waive-20260613-120253 branch from 48da6f4 to c9d3f4e Compare June 19, 2026 06:22

jieli-matrix force-pushed the trtllm-ci-report/waive-20260613-120253 branch from c9d3f4e to ee61567 Compare June 19, 2026 12:48

jieli-matrix force-pushed the trtllm-ci-report/waive-20260613-120253 branch from ee61567 to e2c7b5b Compare June 20, 2026 04:34

jieli-matrix merged commit d3d1b11 into NVIDIA:main Jun 20, 2026
7 checks passed

coderabbitai Bot mentioned this pull request Jun 22, 2026

[https://nvbugs/6276842][test] Loosen rtol/atol on encoder CUDA graph logits parity check #15527

Merged

1 task

This was referenced Jun 23, 2026

[None][test] Waive 11 failed cases for main in QA CI #15506

Merged

[None][test] Waive 4 failed cases for main in QA CI #15505

Merged

[None][test] Waive 10 failed cases for main in post-merge #15535

Merged

Uh oh!

Conversation

tensorrt-cicd commented Jun 13, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Auto-generated Waive PR

Waive entries added

Summary by CodeRabbit

Uh oh!

jieli-matrix commented Jun 16, 2026

Uh oh!

coderabbitai Bot commented Jun 16, 2026

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested reviewers

Uh oh!

tensorrt-cicd commented Jun 16, 2026

Uh oh!

tensorrt-cicd commented Jun 16, 2026

Uh oh!

jieli-matrix commented Jun 16, 2026

Uh oh!

tensorrt-cicd commented Jun 16, 2026

Uh oh!

tensorrt-cicd commented Jun 16, 2026

Uh oh!

jieli-matrix commented Jun 17, 2026

Uh oh!

tensorrt-cicd commented Jun 17, 2026

Uh oh!

tensorrt-cicd commented Jun 17, 2026

Uh oh!

jieli-matrix commented Jun 18, 2026

Uh oh!

tensorrt-cicd commented Jun 18, 2026

Uh oh!

tensorrt-cicd commented Jun 18, 2026

Uh oh!

jieli-matrix commented Jun 19, 2026

Uh oh!

tensorrt-cicd commented Jun 19, 2026

Uh oh!

tensorrt-cicd commented Jun 19, 2026

Uh oh!

jieli-matrix commented Jun 20, 2026

Uh oh!

tensorrt-cicd commented Jun 20, 2026

Uh oh!

tensorrt-cicd commented Jun 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

tensorrt-cicd commented Jun 13, 2026 •

edited by coderabbitai Bot

Loading