@@ -95,6 +95,27 @@ Some caveats:
9595- Tasks can consume resources. Make sure your worker has enough resources to run ``worker_concurrency `` tasks
9696- Queue names are limited to 256 characters, but each broker backend might have its own restrictions
9797
98+ Redis maintenance
99+ -----------------
100+
101+ When Redis is used as the Celery broker, Airflow uses it for transient Celery messages: queued task
102+ commands, task acknowledgement data, and other broker-side state. Airflow task history, Dag runs,
103+ connections, Variables, and XCom records are stored in the Airflow metadata database and should be
104+ maintained with database tooling such as ``airflow db clean ``. Cleaning the metadata database does not
105+ clean Redis, and cleaning Redis does not clean Airflow's metadata database.
106+
107+ Do not flush or delete keys from the Redis database used by Airflow while schedulers or Celery workers
108+ are running. Removing broker keys while tasks are queued, running, or being acknowledged can lose
109+ pending work or make task state temporarily inconsistent. If you need to discard stale broker data
110+ after an outage, first stop Airflow schedulers and Celery workers, make sure there are no tasks that
111+ you intend to preserve in ``queued `` or ``running `` state, back up the Redis database if it is
112+ persistent, and then remove only the Redis database or keyspace configured in ``[celery] broker_url ``.
113+
114+ If you configured Redis as the Celery ``result_backend ``, the same maintenance window should also
115+ account for result backend keys. Deleting those keys can remove Celery task result information that the
116+ scheduler may still query. For production deployments, Airflow recommends a database-backed result
117+ backend so broker cleanup and task-result storage are handled separately.
118+
98119See :doc: `apache-airflow:administration-and-deployment/modules_management ` for details on how Python and Airflow manage modules.
99120
100121Architecture
0 commit comments