[https://nvbugs/6330273][fix] In StorageManager.__init__, when typical_batch is supplied, append a synthetic…

[tensorrt-cicd]

Summary

• Root cause: typical_batch concurrency was never used to floor _min_slots, so windowed pool groups (window_size < tokens_per_block) collapsed to non_stale=1 per request and the absolute floor of 1 slot per pool group caused scheduler deadlock at concurrency > 1.
• Fix: In StorageManager.init, when typical_batch is supplied, append a synthetic BatchDesc of len(typical_batch.kv_caches) decode requests with capacity=tokens_per_block, history_length=tokens_per_block-1 (yielding non_stale=1 in every PG) before computing _min_slots — flooring every pool group at len(typical_batch.kv_caches).
• Automated fix generated by repair-bot

Test plan

• Verify fix on the same GPU type as the original failure
• Check for regressions in related tests

Links

• Bug: https://nvbugs/6330273

Summary by CodeRabbit

• Refactor
  • Enhanced runtime resource allocation efficiency through improved constraint computation logic.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 577c21a7-dd0a-4224-a923-bde5ffbb63fc

📥 Commits

Reviewing files that changed from the base of the PR and between 42a3e55 and 5740b31.

📒 Files selected for processing (1)

• tensorrt_llm/runtime/kv_cache_manager_v2/_storage_manager.py

---

📝 Walkthrough

Walkthrough

In StorageManager.__init__, a synthetic BatchDesc is now derived from typical_batch.kv_caches when present, using a single-decode KVCacheDesc per cache (capacity=tokens_per_block, history_length=tokens_per_block - 1). This synthetic constraint is appended to form effective_constraints, which is then passed to _compute_min_slots_from_constraints instead of the raw constraints or [].

Changes

StorageManager min-slots constraint synthesis

Layer / File(s)	Summary
Effective constraints computation in StorageManager.__init__ tensorrt_llm/runtime/kv_cache_manager_v2/_storage_manager.py	Adds logic to build effective_constraints by appending a synthesized BatchDesc (one decode-step KVCacheDesc per KV cache from typical_batch) to the provided constraints, then passes effective_constraints to _compute_min_slots_from_constraints instead of constraints or [].

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related issues

• [Bug]: KVCacheManagerV2: windowed pool min_slots underflow causes scheduler deadlock at high concurrency #15401: The synthetic KVCacheDesc(capacity=tokens_per_block, history_length=tokens_per_block - 1) constraint introduced here directly implements the fix proposed in that issue to prevent scheduler deadlock caused by underflowed _min_slots in windowed KV cache pools.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title references the NVBugs ticket and fix type, and describes the core change: appending a synthetic constraint in StorageManager.init when typical_batch is supplied.
Description check	✅ Passed	The description provides root cause analysis, the specific fix implemented, test verification, and links to the bug. However, it lacks some template sections like explicit PR Checklist confirmation.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[lowsfer]

Similar fix is already included in #15462