K8SPG-991: fix cleaning up outdated backups

[nmarukovich]

CHANGE DESCRIPTION

Problem:
GetReadyInstancePod reads pod status from the controller-runtime informer cache, which may lag behind actual pod state. During rolling restarts or image updates, a pod can appear as PodReady=True in cache while its containers are already being restarted. The subsequent exec into the "database" container then fails with container not found.

Cause:
Short explanation of the root cause of the issue if applicable.

Solution:
Short explanation of the solution we are providing with this PR.

CHECKLIST

Jira

• Is the Jira ticket created and referenced properly?
• Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
• Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

• Is an E2E test/test case added for the new feature/change?
• Are unit tests added where appropriate?

Config/Logging/Testability

• Are all needed new/changed options added to default YAML files?
• Are all needed new/changed options added to the Helm Chart?
• Did we add proper logging messages for operator actions?
• Did we ensure compatibility with the previous version or cluster upgrade process?
• Does the change support oldest and newest supported PG version?
• Does the change support oldest and newest supported Kubernetes version?

[JNKPercona]

Test Name	Result	Time
backup-enable-disable	passed	00:13:25
builtin-extensions	passed	00:05:40
cert-manager-tls	passed	00:06:24
custom-envs	passed	00:18:34
custom-extensions	passed	00:15:33
custom-tls	passed	00:07:59
database-init-sql	passed	00:02:05
demand-backup	passed	00:27:12
demand-backup-offline-snapshot	passed	00:13:18
dynamic-configuration	passed	00:02:54
finalizers	passed	00:03:26
init-deploy	passed	00:02:30
huge-pages	passed	00:02:39
major-upgrade-13-to-14	passed	00:10:29
major-upgrade-14-to-15	passed	00:11:18
major-upgrade-15-to-16	passed	00:09:21
major-upgrade-16-to-17	passed	00:10:07
major-upgrade-17-to-18	passed	00:09:30
ldap	passed	00:03:21
ldap-tls	passed	00:05:22
monitoring	passed	00:07:30
monitoring-pmm3	passed	00:07:54
one-pod	passed	00:05:39
operator-self-healing	passed	00:10:33
pitr	passed	00:11:37
scaling	passed	00:04:42
scheduled-backup	passed	00:29:15
self-healing	passed	00:08:30
sidecars	passed	00:02:34
standby-pgbackrest	passed	00:18:10
standby-streaming	passed	00:12:34
start-from-backup	passed	00:10:16
tablespaces	passed	00:06:10
telemetry-transfer	passed	00:04:11
upgrade-consistency	passed	00:05:03
upgrade-minor	passed	00:05:29
users	passed	00:04:00

Summary	Value
Tests Run	37/37
Job Duration	01:59:40
Total Test Time	05:35:32

commit: cb0d782
image: perconalab/percona-postgresql-operator:PR-1526-cb0d78289