Skip to content

Add Redis backend guide for HA metrics aggregation#113

Open
noalimoy wants to merge 1 commit into
llm-d:mainfrom
noalimoy:config/redis-ha-metrics-aggregation
Open

Add Redis backend guide for HA metrics aggregation#113
noalimoy wants to merge 1 commit into
llm-d:mainfrom
noalimoy:config/redis-ha-metrics-aggregation

Conversation

@noalimoy

Copy link
Copy Markdown
Contributor

What does this PR do?

Add config/redis/README.md — a deployment, configuration, and verification guide for the Redis-backed DataStore, enabling HA metrics aggregation across multiple IPP replicas.

Why is this change needed?

With a single replica, the in-memory DataStore holds the complete picture. Under HA (multiple replicas), each sees only a subset of requests — Scorers make routing decisions on partial data (#79, #85).

This guide documents the Redis backend approach — per-replica keys with TTL, heartbeat/refresh goroutines, and zero network calls on the hot path.

How was this tested?

  • Unit tests added/updated
  • Integration/e2e tests added/updated
  • Manual testing performed

Documentation-only change. Verified that Redis YAML manifests are valid, code references (AttributeMap, InflightRequestsCount, Get()/Put()) match merged code on main, and Helm values align with existing values.yaml conventions.

Checklist

  • Commits are signed off (git commit -s) per DCO
  • Code follows project contributing guidelines
  • Tests pass locally (make test)
  • Linters pass (make lint)
  • Documentation updated (if applicable)

Related Issues

Refs: #79
Refs: #85

@github-actions github-actions Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label May 19, 2026
@noalimoy noalimoy force-pushed the config/redis-ha-metrics-aggregation branch from bf57680 to 0c0f743 Compare May 19, 2026 10:24
Comment thread config/redis/README.md Outdated
…A metrics aggregation

Signed-off-by: noalimoy <nlimoy@redhat.com>
@noalimoy noalimoy force-pushed the config/redis-ha-metrics-aggregation branch from 0c0f743 to 0e1396d Compare May 19, 2026 21:58
@github-actions

Copy link
Copy Markdown

This PR is marked as stale after 21d of inactivity. After an additional 14d of inactivity (7d to become rotten, then 7d more), it will be closed. To prevent this PR from being closed, add a comment or remove the lifecycle/stale label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lifecycle/rotten size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants