Skip to content

[https://nvbugs/6330273][fix] Reserve KV cache slots for concurrent decode in V2#15462

Open
Kevin-Li-2025 wants to merge 3 commits into
NVIDIA:mainfrom
Kevin-Li-2025:kevin/fix-kv-cache-windowed-min-slots
Open

[https://nvbugs/6330273][fix] Reserve KV cache slots for concurrent decode in V2#15462
Kevin-Li-2025 wants to merge 3 commits into
NVIDIA:mainfrom
Kevin-Li-2025:kevin/fix-kv-cache-windowed-min-slots

Bound decode slot constraint by token budget

207015d
Select commit
Loading
Failed to load commit list.
DCO / DCO succeeded Jun 24, 2026 in 1s

DCO

All commits are signed off!

View more details on DCO