docs: document the coordination redis#517
Open
yassin-berriai wants to merge 3 commits into
Open
Conversation
Add a canonical Coordination Redis page covering the five consumers of the proxy's shared Redis, the config block, resolution precedence, REDIS_* env fallback, cluster/sentinel, coexistence with a semantic response cache, the Admin UI tab, and migration off the cache: true + supported_call_types: [] workaround. Rewrite the pages that previously told users to enable a response cache purely to obtain a Redis client, and clarify router_settings Redis vs the coordination Redis.
…M pages Multi-region already tells operators to set REDIS_* per region; say that those variables alone build the coordination Redis client, and name coordination Redis as what enforces key/team/user rate limits. ITPM/OTPM counters are router state, so distinguish them from the key, user, and team limits that use the coordination Redis.
…mplementation List the virtual key auth cache as a consumer, move the max_connections pool-tuning snippet onto the coordination_redis block that owns it, and note that the router only inherits the coordination Redis when router_settings has no Redis of its own. Also correct the env fallback: REDIS_CLUSTER_NODES and REDIS_SENTINEL_NODES build cluster and sentinel clients, and an explicit coordination_redis.host now outranks a REDIS_URL in the environment, which is a behavior change for operators carrying a global REDIS_URL. Document the settings endpoints.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documents
general_settings.coordination_redis, the explicit config block for the proxy's coordination Redis: the instance behind cross-pod tpm/rpm rate limits, parallel request limits, the Redis spend transaction buffer, the pod lock manager, shared health checks, and the opt-in virtual key auth cache. Until now that Redis was only populated by configuring a plain-Redis response cache, so selecting a semantic cache (or no cache at all) silently downgraded every one of those to per-pod in-memory state.Depends on BerriAI/litellm#32635 and BerriAI/litellm#32661; merge this after both of those ship.
The new page at
docs/proxy/coordination_redis.mdcovers what the coordination Redis does, the config block, the resolution precedence (explicit block, then a borrowed plain-Redis response cache, then a client built fromREDIS_*environment variables), TLS/cluster/sentinel, coexistence with a semantic response cache, the Admin UI tab, and migration off thecache: trueplussupported_call_types: []workaround that existed solely to obtain a Redis client. It is registered in the sidebar immediately beforeproxy/caching.Two behavior notes called out explicitly on the page: the environment fallback is new, so a proxy running a semantic response cache with
REDIS_HOSTexported previously had no coordination Redis and now gets one; and an explicitcoordination_redis.hostoutranks aREDIS_URLin the environment, where the url used to win and silently discard the host and port.The rest of the diff rewrites the pages that told readers to enable a response cache purely as a means of getting a Redis client (
db_deadlocks,shared_health_check,spend_logs_deletion,deploy,prod), and clarifies the split between the router's own Redis for load-balancing state and the coordination Redis for key, user, and team limits (load_balancing,io_token_rate_limits,multi_region). Theenable_redis_auth_cacherequirement incaching.mdandconfig_settings.mdis reworded so it stays true for existing deployments; the old config forms all keep working and the page says so.npm run buildpasses with no broken links or anchors on any page touched here.