BerriAI · yassin-berriai · Jul 8, 2026 · Jul 8, 2026 · Jul 8, 2026 · Jul 8, 2026
diff --git a/docs/benchmarks.md b/docs/benchmarks.md
@@ -179,10 +179,6 @@ litellm_settings:
     password: os.environ/REDIS_PASSWORD
 ```
 
-:::tip
-Use `redis_host`, `redis_port`, and `redis_password` instead of `redis_url` for ~80 RPS better performance.
-:::
-
 **Scaling:** DB connections scale linearly with instances. Consider PostgreSQL read replicas beyond 5K RPS.
 
 See [Production Configuration](./proxy/prod) for detailed best practices.

diff --git a/docs/index.md b/docs/index.md
@@ -29,11 +29,7 @@ import Image from '@theme/IdealImage';
 uv add litellm
 ```
 
-To run the full Proxy Server (LLM Gateway):
-
-```shell
-uv tool install 'litellm[proxy]'
-```
+To deploy the full AI Gateway (Proxy) with the Admin UI, follow the [Docker Quickstart](./proxy/docker_quick_start.md); it runs as a container and needs no Python setup. To run it from the CLI instead, see the [Gateway Quickstart](./learn/gateway_quickstart.md).
 
 ---
 

diff --git a/docs/providers/azure/azure.md b/docs/providers/azure/azure.md
@@ -1032,7 +1032,7 @@ print("list_batches_response=", list_batches_response)
 </TabItem>
 </Tabs>
 
-### [Health Check Azure Batch models](../../proxy/health.md#batch-models-azure-only)
+### [Health Check Azure Batch models](../../proxy/health.md#model-modes)
 
 
 ### [BETA] Loadbalance Multiple Azure Deployments 

diff --git a/docs/providers/vertex.md b/docs/providers/vertex.md
@@ -3203,7 +3203,7 @@ def load_vertex_ai_credentials():
 
 :::info
 
-Trying to deploy LiteLLM on Google Cloud Run? Tutorial [here](https://docs.litellm.ai/docs/proxy/deploy#deploy-on-google-cloud-run)
+Trying to deploy LiteLLM on Google Cloud Run? Tutorial [here](https://docs.litellm.ai/docs/proxy/deploy#deploy-with-terraform-aws-and-gcp)
 
 :::
 

diff --git a/docs/proxy/access_control.md b/docs/proxy/access_control.md
@@ -1,4 +1,4 @@
-import Image from '@theme/IdealImage';
+import { TenancyDiagram } from '@site/src/components/CloudArchitecture';
 
 # Role-based Access Controls (RBAC)
 
@@ -8,7 +8,7 @@ Role-based access control (RBAC) is based on Organizations, Teams and Internal U
 
 <iframe width="100%" height="415" src="https://www.loom.com/embed/a980e25027ad4ecc9e8db1af2777b2a2" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
 
-<Image img={require('../../img/litellm_user_heirarchy.png')} style={{ width: '100%', maxWidth: '4000px' }} />
+<TenancyDiagram />
 
 
 - `Organizations` are the top-level entities that contain Teams.

diff --git a/docs/proxy/architecture.md b/docs/proxy/architecture.md
@@ -1,23 +1,21 @@
-import Image from '@theme/IdealImage';
-import Tabs from '@theme/Tabs';
-import TabItem from '@theme/TabItem';
+import { RequestFlowDiagram, RouterFlowDiagram, ImageFlowDiagram } from '@site/src/components/CloudArchitecture';
 
 # Life of a Request
 
 ## High Level architecture
 
-<Image img={require('../../img/litellm_gateway.png')} style={{ width: '100%', maxWidth: '4000px' }} />
+<RequestFlowDiagram />
 
 
 ### Request Flow 
 
 1. **User Sends Request**: The process begins when a user sends a request to the LiteLLM Proxy Server (Gateway).
 
-2. [**Virtual Keys**](../virtual_keys): At this stage the `Bearer` token in the request is checked to ensure it is valid and under it's budget. [Here is the list of checks that run for each request](https://github.com/BerriAI/litellm/blob/ba41a72f92a9abf1d659a87ec880e8e319f87481/litellm/proxy/auth/auth_checks.py#L43)
+2. [**Virtual Keys**](../virtual_keys): At this stage the `Bearer` token in the request is checked to ensure it is valid and under its budget. [Here is the list of checks that run for each request](https://github.com/BerriAI/litellm/blob/main/litellm/proxy/auth/auth_checks.py)
     - 2.1 Check if the Virtual Key exists in Redis Cache or In Memory Cache
     - 2.2 **If not in Cache**, Lookup Virtual Key in DB
 
-3. **Rate Limiting**: The [MaxParallelRequestsHandler](https://github.com/BerriAI/litellm/blob/main/litellm/proxy/hooks/parallel_request_limiter.py) checks the **rate limit (rpm/tpm)** for the the following components:
+3. **Rate Limiting**: The [parallel request limiter](https://github.com/BerriAI/litellm/blob/main/litellm/proxy/hooks/parallel_request_limiter_v3.py) checks the **rate limit (rpm/tpm)** for the following components:
     - Global Server Rate Limit
     - Virtual Key Rate Limit
     - User Rate Limit
@@ -31,16 +29,32 @@ import TabItem from '@theme/TabItem';
 
 7. **Post-Request Processing**: After the response is sent back to the client, the following **asynchronous** tasks are performed:
    - [Logging to Lunary, MLflow, LangFuse or other logging destinations](./logging)
-   - The [MaxParallelRequestsHandler](https://github.com/BerriAI/litellm/blob/main/litellm/proxy/hooks/parallel_request_limiter.py) updates the rpm/tpm usage for the 
+   - The [parallel request limiter](https://github.com/BerriAI/litellm/blob/main/litellm/proxy/hooks/parallel_request_limiter_v3.py) updates the rpm/tpm usage for the 
         - Global Server Rate Limit
         - Virtual Key Rate Limit
         - User Rate Limit
         - Team Limit
-    - The `_ProxyDBLogger` updates spend / usage in the LiteLLM database. [Here is everything tracked in the DB per request](https://github.com/BerriAI/litellm/blob/ba41a72f92a9abf1d659a87ec880e8e319f87481/schema.prisma#L172)
+    - The `_ProxyDBLogger` updates spend / usage in the LiteLLM database. [Here is everything tracked in the DB per request](https://github.com/BerriAI/litellm/blob/main/schema.prisma)
+
+## The router: fallbacks and retries
+
+<RouterFlowDiagram />
+
+Step 5 above hands the request to the LiteLLM Router, which owns load balancing, fallbacks, and retries. All unified endpoints (`.completion`, `.embeddings`, and so on) flow through it the same way.
+
+The request first enters `function_with_fallbacks`, which wraps the call in a try-except so it can fall back to another deployment if the primary one fails. From there it passes to `function_with_retries`, which wraps the call again and retries on an available deployment within the same model group when a request fails. Finally `function_with_retries` calls a base litellm unified function such as `litellm.completion` or `litellm.embeddings`, which makes the actual request to the LLM API.
+
+A **model_group** is a set of LLM API deployments that share the same `model_name` and can be load balanced across.
+
+## Image URL handling
+
+<ImageFlowDiagram />
+
+Some LLM APIs don't accept image URLs but do accept base64 strings. For those, LiteLLM detects a URL in the request, checks whether the target API supports URLs, and if not, downloads the image and sends the provider a base64 string instead. Up to 10 converted images are cached in memory to reduce latency on repeated calls, and individual downloads are capped at 50MB (configurable with `MAX_IMAGE_URL_DOWNLOAD_SIZE_MB`).
 
 ## Frequently Asked Questions
 
 1. Is a db transaction tied to the lifecycle of request?
     - No, a db transaction is not tied to the lifecycle of a request.
     - The check if a virtual key is valid relies on a DB read if it's not in cache.
-    - All other DB transactions are async in background tasks
+    - All other DB transactions are async in background tasks
diff --git a/docs/proxy/config_settings.md b/docs/proxy/config_settings.md
@@ -261,7 +261,7 @@ router_settings:
 | database_socket_timeout | float | Maps to the Prisma [`socket_timeout`](https://www.prisma.io/docs/orm/overview/databases/postgresql) URL param (seconds). When set, an idle or slow connection that has not produced data within this window is closed. **Use this to cap idle Prisma connections from LiteLLM.** |
 | database_extra_connection_params | object | Escape hatch — extra key/value pairs appended verbatim to the Prisma `DATABASE_URL` / `DIRECT_URL` query string (e.g. `sslmode`, `pgbouncer`, `statement_cache_size`). Keys here override any default LiteLLM sets. |
 | database_disable_prepared_statements | boolean | Appends `pgbouncer=true` to the Prisma connection URL, disabling reuse of server-side prepared statements. Use behind PgBouncer transaction pooling, or to avoid `cached plan must not change result type` errors during rolling schema migrations. An explicit `pgbouncer` key in `database_extra_connection_params` takes precedence. [Disable Server-Side Prepared Statements](configs#disable-server-side-prepared-statements) |
-| allow_requests_on_db_unavailable | boolean | If true, allows requests to succeed even if DB is unreachable. **Only use this if running LiteLLM in your VPC** This will allow requests to work even when LiteLLM cannot connect to the DB to verify a Virtual Key [Doc on graceful db unavailability](prod#5-if-running-litellm-on-vpc-gracefully-handle-db-unavailability) |
+| allow_requests_on_db_unavailable | boolean | If true, allows requests to succeed even if DB is unreachable. **Only use this if running LiteLLM in your VPC** This will allow requests to work even when LiteLLM cannot connect to the DB to verify a Virtual Key [Doc on graceful db unavailability](prod#gracefully-handle-db-unavailability) |
 | fail_closed_budget_enforcement | boolean | Default `false`. When `true`, budget checks validate spend against the authoritative database for every budgeted request (key, team, user, organization, end-user, tag, and per-window budgets) instead of trusting only the cross-pod Redis counter, and a request is rejected with a `503` when current spend can be verified against neither Redis nor the database. Use this when a configured budget must be a hard ceiling even while Redis is degraded or restarting; leave it off to keep healthy under-budget traffic off the database. [Doc on budget enforcement](./users#hard-budget-enforcement-fail-closed) |
 | custom_auth | string | Write your own custom authentication logic [Doc Custom Auth](./custom_auth) |
 | max_parallel_requests | integer | The max parallel requests allowed per deployment |

diff --git a/docs/proxy/credential_routing.md b/docs/proxy/credential_routing.md
@@ -9,7 +9,7 @@ Route the same model to different LLM provider endpoints (e.g. different Azure i
 
 In multi-tenant deployments, different teams often need the same model name (e.g., `gpt-4`) to hit different provider endpoints — for example, separate Azure OpenAI instances per business unit for cost isolation, data residency, or rate limit separation.
 
-**Credential routing** lets you configure this in team/project metadata using the existing [credentials table](./ui_credentials.md), without duplicating model definitions or creating separate model groups per team.
+**Credential routing** lets you configure this in team/project metadata using the existing [credentials table](./model_management.md#reusable-provider-credentials), without duplicating model definitions or creating separate model groups per team.
 
 ```
 Hotel Team → gpt-4 → https://hotel-eastus.openai.azure.com/
@@ -31,7 +31,7 @@ When a request comes in, the system walks this precedence chain (first match win
 
 ### Step 1: Create Credentials
 
-Store your Azure endpoint credentials in the credentials table. You can do this via the [UI](./ui_credentials.md) or API:
+Store your Azure endpoint credentials in the credentials table. You can do this via the [UI](./model_management.md#reusable-provider-credentials) or API:
 
 ```bash showLineNumbers
 # Create credential for Hotel team's Azure endpoint
@@ -222,7 +222,7 @@ The `model_config` key is a JSON object in team/project `metadata`:
 | `defaultconfig` | Fallback credential for any model not explicitly listed |
 | `<model-name>` | Model-specific override — must match the LiteLLM model group name |
 | `<provider>` | Provider key (e.g. `azure`, `openai`, `bedrock`). When the model name includes a provider prefix (e.g. `azure/gpt-4`), the system prefers the matching provider key |
-| `litellm_credentials` | Name of a credential in the [credentials table](./ui_credentials.md) |
+| `litellm_credentials` | Name of a credential in the [credentials table](./model_management.md#reusable-provider-credentials) |
 
 ### Credential Values
 
@@ -267,7 +267,7 @@ The feature flag must be enabled before `model_config` entries in team/project m
 
 ## Related Documentation
 
-- [Adding LLM Credentials](./ui_credentials.md) — Create and manage reusable credentials
+- [Adding LLM Credentials](./model_management.md#reusable-provider-credentials) — Create and manage reusable credentials
 - [Project Management](./project_management.md) — Project hierarchy and API
 - [Team Budgets](./team_budgets.md) — Team-level budget management
 - [Clientside LLM Credentials](./clientside_auth.md) — Passing credentials in the request body

diff --git a/docs/proxy/credential_usage_tracking.md b/docs/proxy/credential_usage_tracking.md
@@ -1,6 +1,6 @@
 # Credential Usage Tracking
 
-When a model is attached to a [reusable credential](./ui_credentials.md), LiteLLM automatically injects the credential name as a tag on every request that uses that model. This means credential-level spend and usage are tracked with zero extra configuration.
+When a model is attached to a [reusable credential](./model_management.md#reusable-provider-credentials), LiteLLM automatically injects the credential name as a tag on every request that uses that model. This means credential-level spend and usage are tracked with zero extra configuration.
 
 ## How It Works
 
@@ -14,6 +14,6 @@ In the Admin UI, go to **Usage → Tag** and look for tags with the `Credential:
 
 ## Related Documentation
 
-- [Adding LLM Credentials](./ui_credentials.md) - How to create and attach reusable credentials to models
+- [Adding LLM Credentials](./model_management.md#reusable-provider-credentials) - How to create and attach reusable credentials to models
 - [Tag Budgets](./tag_budgets.md) - Setting spend limits on tags
 - [Tag Routing](./tag_routing.md) - Routing requests based on tags
diff --git a/docs/proxy/custom_root_ui.md b/docs/proxy/custom_root_ui.md
@@ -13,7 +13,7 @@ Requires v1.72.3 or higher.
 :::
 
 Limitations:
-- This does not work in [litellm non-root](./deploy#non-root---without-internet-connection) images, as it requires write access to the UI files.
+- This does not work in [litellm non-root](./docker_image_security) images, as it requires write access to the UI files.
 
 ## Usage
 

diff --git a/docs/proxy/db_deadlocks.md b/docs/proxy/db_deadlocks.md
@@ -2,7 +2,7 @@ import Image from '@theme/IdealImage';
 import Tabs from '@theme/Tabs';
 import TabItem from '@theme/TabItem';
 
-# High Availability Setup (Resolve DB Deadlocks)
+# Resolve DB Deadlocks (Redis Transaction Buffer)
 
 :::tip Essential for Production
 

diff --git a/docs/proxy/db_info.md b/docs/proxy/db_info.md
@@ -47,27 +47,12 @@ You can see the full DB Schema [here](https://github.com/BerriAI/litellm/blob/ma
 | Table Name | Description | Row Insert Frequency |
 |------------|-------------|---------------------|
 | LiteLLM_SpendLogs | Detailed logs of all API requests. Records token usage, spend, and timing information. Tracks which models and keys were used. | **Medium - this is a batch process that runs on an interval.** |
+| LiteLLM_DailyUserSpend and siblings (DailyTeamSpend, DailyOrgSpend, DailyTagSpend, DailyEndUserSpend, DailyAgentSpend) | Pre-aggregated daily spend rollups per user, team, org, tag, end user, and agent; the Admin UI Usage views read these aggregates rather than scanning SpendLogs. | Low - one row per entity per day, updated in batches |
 | LiteLLM_AuditLog | Tracks changes to system configuration. Records who made changes and what was modified. Maintains history of updates to teams, users, and models. | **Off by default**, **High - Runs on every change to an entity** |
 
 ## Disable `LiteLLM_SpendLogs`
 
-You can disable spend_logs and error_logs by setting `disable_spend_logs` and `disable_error_logs` to `True` on the `general_settings` section of your proxy_config.yaml file.
-
-```yaml
-general_settings:
-  disable_spend_logs: True   # Disable writing spend logs to DB
-  disable_error_logs: True   # Only disable writing error logs to DB, regular spend logs will still be written unless `disable_spend_logs: True`
-```
-
-### What is the impact of disabling these logs?
-
-When disabling spend logs (`disable_spend_logs: True`):
-- You **will not** be able to view Usage on the LiteLLM UI
-- You **will** continue seeing cost metrics on s3, Prometheus, Langfuse (any other Logging integration you are using)
-
-When disabling error logs (`disable_error_logs: True`):
-- You **will not** be able to view Errors on the LiteLLM UI
-- You **will** continue seeing error logs in your application logs and any other logging integrations you are using
+Set `disable_spend_logs: True` or `disable_error_logs: True` under `general_settings` to stop writing those tables. With spend logs disabled you lose per-request log detail in the UI but keep cost metrics in your logging integrations (s3, Prometheus, Langfuse); with error logs disabled you lose the Errors view in the UI but keep errors in application logs and other logging integrations. See [keeping error logs out of the database](./prod.md#keep-error-logs-out-of-the-database) in the production checklist.
 
 
 ## Migrating Databases 

diff --git a/docs/proxy/db_read_replica.md b/docs/proxy/db_read_replica.md
@@ -58,11 +58,11 @@ worst it degrades to single-database performance.
 ## RDS IAM authentication
 
 When `IAM_TOKEN_DB_AUTH=True`, both the writer and the reader refresh their
-IAM tokens independently on the same ~12-minute cadence. The reader does not
-need parallel `DATABASE_HOST_READ_REPLICA` / `DATABASE_USER_READ_REPLICA`
-env vars — host, port, user, and database name are parsed once from
-`DATABASE_URL_READ_REPLICA` at startup, and only the IAM token rotates after
-that.
+IAM tokens independently on the same ~12-minute cadence. You can supply the
+reader either as a single `DATABASE_URL_READ_REPLICA` or assembled from the
+component vars (`DATABASE_HOST_READ_REPLICA`, `DATABASE_USER_READ_REPLICA`,
+and friends); the token-refresh path re-parses the resulting reader URL,
+so only the IAM token rotates after startup.
 
 This pairs naturally with Aurora's reader endpoint, which resolves to the
 reader instances in the cluster.
-Original file line number
+Diff line change
@@ Expand Up / @@ -3203,7 +3203,7 @@ def load_vertex_ai_credentials(): @@
     :::info
-    Trying to deploy LiteLLM on Google Cloud Run? Tutorial [here](https://docs.litellm.ai/docs/proxy/deploy#deploy-on-google-cloud-run)
+    Trying to deploy LiteLLM on Google Cloud Run? Tutorial [here](https://docs.litellm.ai/docs/proxy/deploy#deploy-with-terraform-aws-and-gcp)
     :::
@@ Expand Down @@