mariadb_copy: replace sleep workarounds with persistent pod retry pattern

Replace the temporary mariadb_client_timeout sleep and bgp wait pauses
with a proper retry loop that polls the podified MariaDB via the
already-running mariadb-copy-data pod, eliminating the race condition
where commands were executed before the network was fully programmed.
Also align pre_checks.bash with the docs by using oc rsh mariadb-copy-data
instead of the ephemeral mariadb-client pod, and add a corresponding
wait step to the docs procedure.

Resolves: OSPRH-27386
Made-with: Cursor

Author: ciecierski

docs_user/modules/proc_migrating-databases-to-mariadb-instances.adoc

@@ -223,6 +223,22 @@ for CELL in $(echo $CELLS); do
 done
 ----
 
+. Wait for the `mariadb-copy-data` pod to reach the podified MariaDB:
++
+----
+$ oc rsh mariadb-copy-data mysql -rsh "${PODIFIED_MARIADB_IP['super']}" \
+  -uroot -p"${PODIFIED_DB_ROOT_PASSWORD['super']}" -e 'select 1;'
+----
++
+[NOTE]
+====
+For BGP-enabled environments, this command might take a few moments to succeed while BGP routes are advertised and propagated through the network. The `mariadb-copy-data` pod needs to receive the route to the podified MariaDB IP address through BGP before it can establish a connection. If the command fails, wait a few seconds and retry. The connection should succeed once the BGP route advertisement is complete.
+
+For IPv6 environments, this command might take a few moments to succeed while the network IPv6 stack completes its setup. If the command fails, wait a few seconds and retry.
+
+For standard deployments, such as non-BGP deployments or IPv4 deployments, this command should succeed immediately.
+====
+
 . Test the connection to the control plane upcall and cells databases:
 +
 ----

tests/roles/common_defaults/defaults/main.yaml

@@ -228,9 +228,6 @@ octavia_adoption: true
 # dataplane_adoption and development_environment
 prelaunch_octavia_workload: false
 
-# MariaDB client connection timeout in seconds
-# Related to OSPRH-18618
-mariadb_client_timeout: 0
 mariadbcheck: true
 
 # Whether bgp is configured

tests/roles/mariadb_copy/tasks/main.yaml

@@ -73,16 +73,18 @@
   retries: 25
   delay: "{{ mariadb_retry_delay }}"
 
-# MariaDB client connection timeout in seconds
-# Related to OSPRH-18618
-- name: Add sleep time
-  ansible.builtin.pause:
-    seconds: "{{ mariadb_client_timeout }}"
-
-- name: wait bgp
-  when: bgp
-  ansible.builtin.pause:
-    seconds: 60
+- name: wait until podified MariaDB is reachable
+  no_log: "{{ use_no_log }}"
+  ansible.builtin.shell: |
+    {{ oc_header }}
+    {{ mariadb_copy_shell_vars_dst }}
+    oc rsh mariadb-copy-data mysql -rsh "${PODIFIED_MARIADB_IP['super']}" \
+      -uroot -p"${PODIFIED_DB_ROOT_PASSWORD['super']}" -e 'select 1;'
+  register: _podified_db_check
+  retries: 60
+  delay: 3
+  until: _podified_db_check.rc == 0
+  changed_when: false
 
 - name: check that the Galera database cluster(s) members are online and synced, for all cells
   no_log: "{{ use_no_log }}"

tests/roles/mariadb_copy/templates/pre_checks.bash

@@ -3,6 +3,6 @@
 
 # Test the connection to the control plane "upcall" and cells' databases
 for CELL in $(echo "super $RENAMED_CELLS"); do
-  oc run mariadb-client --image $MARIADB_IMAGE -i --rm --restart=Never -- \
-    bash -c "sleep {{ mariadb_client_timeout }} && mysql -rsh \"${PODIFIED_MARIADB_IP[$CELL]}\" -uroot -p\"${PODIFIED_DB_ROOT_PASSWORD[$CELL]}\" -e 'SHOW databases;'"
+  oc rsh mariadb-copy-data mysql -rsh "${PODIFIED_MARIADB_IP[$CELL]}" \
+    -uroot -p"${PODIFIED_DB_ROOT_PASSWORD[$CELL]}" -e 'SHOW databases;'
 done