[Core][Metrics] Expose scheduler queue pressure by waiting reason

[mukeshbaphna]

Add a Prometheus gauge for normalized scheduler queue pressure so control planes can compare replicas with different capacities.

• sglang:scheduler_queue_pressure with reason=capacity|deferred
• normalizes queue depth by max_running_requests
• surfaces both the main waiting queue and deferred/disaggregation queues
• adds metrics coverage for the new series

---

CI States

Latest PR Test (Base): ❌ Missing run-ci label — add it to run CI tests.
Latest PR Test (Extra): ❌ Blocked — run-ci is required first.

[gemini-code-assist]

Code Review

This pull request introduces normalized scheduler queue pressure metrics to track both main waiting queue capacity and secondary deferred queues. It adds a new Prometheus gauge sglang:scheduler_queue_pressure with 'capacity' and 'deferred' labels, along with helper functions to compute and log these values across the scheduler. Feedback indicates that these metrics should also be updated within report_decode_stats to avoid stale data during decode-only periods, and suggests refactoring the conditional logic used to calculate deferred request counts for improved readability.

[gemini-code-assist]

The new scheduler queue pressure metrics (scheduler_queue_pressure_capacity and scheduler_queue_pressure_deferred) are updated in report_prefill_stats and _maybe_log_idle_metrics, but they are missing from report_decode_stats (which starts around line 637). This will result in these Prometheus metrics remaining stale during periods where only decode iterations are running. Please ensure these stats are also computed and updated in report_decode_stats to maintain accuracy under load.

[mukeshbaphna]

Fixed. Queue pressure now refreshes in report_decode_stats as well as the other scheduler paths, so the metric does not go stale during decode-only periods

[gemini-code-assist]

The logic for determining deferred_queue_reqs uses multiple independent if blocks and a membership check. This can be refactored into an if/elif/else structure for better clarity and consistency with the implementation in report_prefill_stats. Additionally, initializing deferred_queue_reqs = 0 at the start of the block would simplify the logic further.

[mukeshbaphna]

Done, deferred queue accounting was factored into a small helper for readability and reuse.

[mukeshbaphna]

Addressed the review feedback in the latest push:

• queue pressure now refreshes in report_decode_stats as well as the other scheduler paths, so the metric does not go stale during decode-only periods
• deferred queue accounting was factored into a small helper for readability and reuse

Pushed in 645079e.