[Bugfix][Helm]: update Grafana dashboards to current vLLM metric names#989
[Bugfix][Helm]: update Grafana dashboards to current vLLM metric names#989bitnik wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the vLLM Grafana dashboards to use current metric names, replacing time_per_output_token with request_time_per_output_token and gpu_cache_usage_perc with kv_cache_usage_perc, while removing panels and queries related to swapped requests and CPU cache usage. The reviewer feedback highlights several key improvement opportunities: adding the percentunit formatting to panels displaying the new KV cache usage metric, updating deprecated prefix cache metrics, adjusting panel widths to fill empty layout spaces, fixing non-sequential query refIds, and removing outdated references to swapped requests in panel descriptions.
| "title": "GPU KV Cache Hit Rate", | ||
| "type": "timeseries" | ||
| }, | ||
| { |
There was a problem hiding this comment.
There are two improvement opportunities here:
- Deprecated Metrics: The preceding panel "GPU KV Cache Hit Rate" (lines 800-836) uses the deprecated metrics
vllm:gpu_prefix_cache_hits_totalandvllm:gpu_prefix_cache_queries_total(at line 825). Since this PR updates the dashboard queries to current vLLM metric names, these should be updated to the current non-deprecated metric names:vllm:prefix_cache_hits_totalandvllm:prefix_cache_queries_totalrespectively. - Panel Layout: Since the "Number of Swapped Requests" panel (which was positioned at
x: 12, y: 39with width12) has been removed, the "GPU KV Cache Hit Rate" panel (positioned atx: 0, y: 39with width12) now has an empty space of width12next to it. Consider increasing its width (w) to24(at line 802) to utilize the full width of the dashboard.
vLLM renamed several metrics and dropped the swap-related ones (V1 no longer swaps between GPU and CPU), so the shipped dashboards queried metrics that no longer exist and rendered empty panels. Update both dashboards accordingly: - vllm:time_per_output_token_seconds_* -> vllm:request_time_per_output_token_seconds_* - vllm:gpu_cache_usage_perc -> vllm:kv_cache_usage_perc - remove the "Number of Swapped Requests" panel and the vllm:num_requests_swapped / vllm:cpu_cache_usage_perc queries Signed-off-by: Kenan Erdogan <kenanerdogan@gmail.com>
ac12b30 to
9392413
Compare
-swhen doinggit commit[Bugfix],[Feat], and[CI].The bundled Grafana dashboards (
helm/dashboards/vllm-dashboard.jsonandhelm/dashboards/vllm-model-metrics-dashboard.json) referenced vLLM metrics that have since been renamed or removed. As a result, several panels showed No data on current vLLM versions.This PR updates the dashboard queries to the current metric names and drops panels for metrics that no longer exist.
Renamed metrics:
vllm:time_per_output_token_seconds_{sum,count,bucket}vllm:request_time_per_output_token_seconds_{sum,count,bucket}vllm:gpu_cache_usage_percvllm:kv_cache_usage_percRemoved panels / queries (metrics no longer emitted — vLLM V1 does not swap requests between GPU and CPU):
vllm-dashboard.json: removed the "Number of Swapped Requests" stat panel (vllm:num_requests_swapped).vllm-model-metrics-dashboard.json: removed thevllm:num_requests_swappedquery and the "CPU Cache Usage" query (vllm:cpu_cache_usage_perc) from the Cache Utilization panel.