Skip to content

kv-cache-collector plugin to fetch KV utilization from inference pools #44

Description

@Mohammad-nassar10

A new datalayer Collector plugin (kv-cache-collector) that periodically scrapes Prometheus metrics from each inference pool and exposes KV-cache utilization and queue depth as per-model attributes in the IPP datastore.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions