Skip to content

Commit b8071eb

Browse files
committed
Add max_retries and retry_interval to [(api_)database] conf
Nova's default database connection retry settings (max_retries=10, retry_interval=10) mean nova-api can spend up to 100 seconds retrying a connection to an unavailable database. This exceeds the Kubernetes startup probe window (60 seconds), so the pod gets killed before nova finishes retrying, leading to a CrashLoopBackOff when a cell database is down. In RHOSO, Kubernetes provides its own higher-level retry mechanism by killing and recreating pods that fail to start. This is preferable to Nova retrying internally because it reports the situation clearly via CR status fields and events, and allows Kubernetes to reschedule the pod to another worker if needed. Set max_retries to 3 and retry_interval to 1 second for both [database] and [api_database] so that Nova gives up on an unreachable database quickly and lets Kubernetes handle the recovery. Resolves: OSPRH-30130 Signed-off-by: melanie witt <melwittt@gmail.com>
1 parent 7ba93ca commit b8071eb

1 file changed

Lines changed: 4 additions & 0 deletions

File tree

templates/nova/nova.conf

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -241,12 +241,16 @@ cpu_power_management=false
241241

242242
{{if (index . "cell_db_address")}}
243243
[database]
244+
max_retries = 3
245+
retry_interval = 1
244246
connection = mysql+pymysql://{{ .cell_db_user }}:{{ .cell_db_password}}@{{ .cell_db_address }}/{{ .cell_db_name }}?read_default_file=/etc/my.cnf
245247
{{end}}
246248

247249

248250
{{if (index . "api_db_address")}}
249251
[api_database]
252+
max_retries = 3
253+
retry_interval = 1
250254
connection = mysql+pymysql://{{ .api_db_user }}:{{ .api_db_password }}@{{ .api_db_address }}/{{ .api_db_name }}?read_default_file=/etc/my.cnf
251255
{{end}}
252256

0 commit comments

Comments
 (0)