Skip to content

[TRI-978][fix] Pass explicit streaming=False in base_metrics_verification_tests#5

Closed
mc-nv wants to merge 241 commits into
mainfrom
mchornyi/TRI-978/fix-test
Closed

[TRI-978][fix] Pass explicit streaming=False in base_metrics_verification_tests#5
mc-nv wants to merge 241 commits into
mainfrom
mchornyi/TRI-978/fix-test

Conversation

@mc-nv

@mc-nv mc-nv commented Apr 20, 2026

Copy link
Copy Markdown
Collaborator

@coderabbitai summary

Description

tensorrt_llm.bindings.executor.Request.__init__() no longer accepts streaming=None — the parameter type changed from nullable to bool (default False). The base_metrics_verification_tests.py _run_infer helper was omitting the streaming tensor entirely, causing get_input_scalar_by_name() to return None downstream, which was then passed directly to the executor constructor and crashed every request through the BLS path.

Fix: add streaming=False as an explicit input tensor in _run_infer so the backend always receives a valid bool.

Exposed by CI job L0_backend_trtllm--base #300267705 (pipeline 48790749, branch 26.04-devel):

  • TritonModelException: __init__(): incompatible function arguments … streaming: NoneType
  • Segfault (exit 139) in TRTLLMBaseMetricsTest.test_end_to_end

Test Coverage

  • L0_backend_trtllm--baseTRTLLMBaseMetricsTest.test_end_to_end and test_end_to_end_beam_width (1-GPU and 2-GPU)
  • custom_metrics_verification_tests.py overall gate

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

Resolves: TRI-978

yiqingy0 and others added 30 commits January 15, 2026 15:01
Signed-off-by: Yiqing Yan <yiqingy@nvidia.com>
NVIDIA#10715)

Signed-off-by: Yukun He <23156053+hyukn@users.noreply.github.com>
… H100 stage (to avoid potential OOM) (pick NVIDIA#10703) (NVIDIA#10743)

Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
…ts (pick NVIDIA#10698) (NVIDIA#10742)

Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>
Signed-off-by: Mike Iovine <6158008+mikeiovine@users.noreply.github.com>
Signed-off-by: Faraz Khoubsirat <58580514+farazkh80@users.noreply.github.com>
Signed-off-by: qqiao <qqiao@nvidia.com>
Signed-off-by: Yanchao Lu <yanchaol@nvidia.com>
Co-authored-by: Yanchao Lu <yanchaol@nvidia.com>
Signed-off-by: Mingyang Jiang <13463932+jmydurant@users.noreply.github.com>
…#10741)

Signed-off-by: Yukun He <23156053+hyukn@users.noreply.github.com>
Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
…ssue for MNNVL Allreduce under PP (NVIDIA#10750)

Signed-off-by: Yukun He <23156053+hyukn@users.noreply.github.com>
Signed-off-by: Ruodi Lu <ruodil@users.noreply.github.com>
Co-authored-by: Ruodi Lu <ruodil@users.noreply.github.com>
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
…DIA#10764)

Signed-off-by: Patrice Castonguay <55748270+pcastonguay@users.noreply.github.com>
…ling_from_logits (NVIDIA#10713)

Signed-off-by: ixlmar <206748156+ixlmar@users.noreply.github.com>
… kernel (NVIDIA#10654)

Signed-off-by: Yukun He <23156053+hyukn@users.noreply.github.com>
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: Yanchao Lu <yanchaol@nvidia.com>
Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com>
…n ranks. (NVIDIA#10822)

Signed-off-by: Yukun He <23156053+hyukn@users.noreply.github.com>
Signed-off-by: Yanchao Lu <yanchaol@nvidia.com>
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
…s test. (NVIDIA#10756)

Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>
tensorrt-cicd and others added 28 commits March 20, 2026 04:23
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
…DIA#12804)

Signed-off-by: Tyler Burt <195370667+tburt-nv@users.noreply.github.com>
Signed-off-by: Yiteng Niu <6831097+niukuo@users.noreply.github.com>
Co-authored-by: tburt-nv <195370667+tburt-nv@users.noreply.github.com>
Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com>
Signed-off-by: Yuanjing Xue <197832395+yuanjingx87@users.noreply.github.com>
Signed-off-by: Yuanjing Xue <197832395+yuanjingx87@users.noreply.github.com>
Signed-off-by: Yuanjing Xue <197832395+yuanjingx87@users.noreply.github.com>
Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
Co-authored-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
…VIDIA#12872)

Signed-off-by: Yuanjing Xue <197832395+yuanjingx87@users.noreply.github.com>
Signed-off-by: Yiteng Niu <6831097+niukuo@users.noreply.github.com>
Co-authored-by: yuanjingx87 <197832395+yuanjingx87@users.noreply.github.com>
…12894)

Signed-off-by: Yiteng Niu <6831097+niukuo@users.noreply.github.com>
Signed-off-by: Yuanjing Xue <197832395+yuanjingx87@users.noreply.github.com>
…release/1.2.1 (NVIDIA#12770)

Signed-off-by: thorjohnsen <41591019+thorjohnsen@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: yuanjingx87 <197832395+yuanjingx87@users.noreply.github.com>

No need to be blocked by the 3 Nemotron Super test failures due to the discussion over slack.
Signed-off-by: Yuanjing Xue <197832395+yuanjingx87@users.noreply.github.com>
Signed-off-by: TensorRT LLM <90828364+tensorrt-cicd@users.noreply.github.com>
…VIDIA#13037)

Signed-off-by: nv-guomingz <137257613+nv-guomingz@users.noreply.github.com>
…DIA#13048)

Signed-off-by: Yuanjing Xue <197832395+yuanjingx87@users.noreply.github.com>
…_tests

tensorrt_llm.bindings.executor.Request.__init__() no longer accepts
streaming=None; the test was omitting the streaming tensor, causing
get_input_scalar_by_name() to return None and the executor to crash.

Relates-to: TRI-978
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.