SolrBackup may never finish if REQUESTSTATUS returns notfound for an accepted backup

[oyjhl]

What happened

A SolrBackup can remain stuck in InProgress forever if:

1. the backup request is accepted (operator backup submit, stable backup async ID),
2. the operator suddenly become unavailable,
3. the Solr async tracking entry for that backup is deleted (DELETESTATUS API, tracker deleteSingleAsyncId),
4. and the operator become available again.

After that:

• the backup files already exist in Solr,
• REQUESTSTATUS for the backup request ID returns notfound (REQUESTSTATUS API, tracker getAsyncTaskRequestStatus),
• but the SolrBackup CR still stays in inProgress=true,
• and the operator keeps polling instead of reaching a terminal state (backup reconcile loop, requeue after 5s).

Environment

• macOS
• local kind cluster
• Kubernetes / kind node version: v1.32.1
• solr-operator version: v0.10.0-orerekease
• solr-operator built from master on March 22, 2026 (ca9d3c5c37a59f29570a6b49a8da5dc614aba75e)
• Solr version: 9.10.0

Steps to reproduce

1. Deploy solr-operator on a local kind cluster.
2. Create a 1-node SolrCloud with a local backup repository, then create a
   collection and start a SolrBackup for it.
3. As soon as the backup first shows inProgress=true, scale the
   solr-operator deployment down to 0.
4. While the operator is down, wait for the Solr async request to finish, then
   delete only that async status entry with DELETESTATUS.
5. Confirm the backup data still exists, but REQUESTSTATUS for that same
   request ID now returns notfound.
6. Scale the operator back up to 1 and observe that the SolrBackup CR never
   reaches a terminal state.

The stuck status looks like:

status:
  collectionBackupStatuses:
  - asyncBackupStatus: notfound
    inProgress: true

and it never sets finished: true or successful: true.

Expected behavior

Once an accepted backup later becomes notfound, the operator should not leave the CR in InProgress forever.

It should eventually either:

• recover, or
• mark the backup failed with a clear reason.

[Soumya8898]

Hi @oyjhl , can you please assign this issue to me.