-
Notifications
You must be signed in to change notification settings - Fork 2.5k
- #15044 · laikhtewari opened
on Jun 6, 2026 1 - #3148 · juney-nvidia opened
on Mar 29, 2025 5 - #3124 · juney-nvidia opened
on Mar 27, 2025 11
Issues
is:issue state:open
is:issue state:open
Issue creation is restricted in this repository
Search results
[Bug]: Gemma4 INT4 AWQ quantization fails during export: Gemma4Config has no attribute vocab_size
bugSomething isn't workingSomething isn't workingCustomized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.Status: Open.#16171 In NVIDIA/TensorRT-LLM;[Feature]: Expose SmoothQuant alpha for int8_sq quantization in quantize.py
feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality supportModel optimization<NV>Model-specific performance optimizations and tuning<NV>Model-specific performance optimizations and tuningStatus: Open.#16151 In NVIDIA/TensorRT-LLM;[Bug]: 142 GiB/rank host memory leak (leading to host OOM if KV cache offloading is enabled) when serving Kimi-K2.6-NVFP4 TP4 on GB300 (aarch64), due to MetaInitException on aten.fill_.Scalar (nn.LayerNorm init) silently falling back to whole-model CPU construction
Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.Pytorch<NV>Pytorch backend related issues<NV>Pytorch backend related issuesStatus: Open.#16086 In NVIDIA/TensorRT-LLM;[AutoDeploy]: Migrate btk to support llmc
AutoDeploy/llmc-blocker<NV> Tag for issues that are blocking AutoDeploy standalone repo<NV> Tag for issues that are blocking AutoDeploy standalone repofeature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality supportStatus: Open.#16076 In NVIDIA/TensorRT-LLM;[Bug]: IndexError: tuple index out of range when loading Qwen 3.6 35B during Quantizing
bugSomething isn't workingSomething isn't workingCustomized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.Status: Open.#16075 In NVIDIA/TensorRT-LLM;[Feature]: Paged / KV-manager-resident storage for the DFlash draft context, replacing the dense buffer
feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality supportSpeculative Decoding<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafterStatus: Open.#16005 In NVIDIA/TensorRT-LLM;[Performance]: Compact pseudo-KV/topological sparse attention benchmark evidence and backend gap
General perf<NV>Broad performance issues not specific to a particular component<NV>Broad performance issues not specific to a particular componentPytorch<NV>Pytorch backend related issues<NV>Pytorch backend related issuesStatus: Open.#15989 In NVIDIA/TensorRT-LLM;[Feature]: Support multiple input types
feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality supportStatus: Open.#15885 In NVIDIA/TensorRT-LLM;[Bug]: B12x NVFP4 MoE JIT fails on SM120/SM121 with CUDA 12 CuTe DSL payload
Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.Pytorch<NV>Pytorch backend related issues<NV>Pytorch backend related issuesStatus: Open.#15853 In NVIDIA/TensorRT-LLM;[Bug]: Redundant KV host cache tier on DGX Spark unified-memory systems
KV-Cache Managementkv-cache management for efficient LLM inferencekv-cache management for efficient LLM inferenceStatus: Open.#15793 In NVIDIA/TensorRT-LLM;SM120 skip-softmax FMHA support probe misses bridge path
Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.Status: Open.#15791 In NVIDIA/TensorRT-LLM;[Feature]: General cross-instance KV-cache transfer/staging (kv_transfer)
Disaggregated serving<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.Status: Open.#15735 In NVIDIA/TensorRT-LLM;