fix(qwp): treat all-replica window as transient for durable-ack senders

A durable-ack (request_durable_ack=on) sender that walked an endpoint list
where every reachable node was a REPLICA synthesized a terminal
QwpDurableAckMismatchException (-> PROTOCOL_VIOLATION/HALT) in
QwpWebSocketSender.buildAndConnect. That turned a transient failover window
(a replica can be promoted; a primary will reappear) into a permanent
hard-fail, violating the store-and-forward contract whose only terminal
condition is SF exhaustion. HA senders never recovered even after a replica
was promoted:

  server rejected batch: PROTOCOL_VIOLATION - durable-ack-mismatch:
    WebSocket upgrade failed: server does not support durable ack [role=REPLICA]

Fix: when a connect round exhausts with only role-rejects, throw the
retriable QwpRoleMismatchException regardless of request_durable_ack. Both
connect paths (connectWithRetry sync, connectLoop async/reconnect) already
retry role-mismatch within reconnect_max_duration_millis, so the sender
keeps rows in SF and recovers on promotion. A genuine capability gap (an
endpoint that upgrades but does not advertise durable ack) stays terminal
via terminalUpgradeError.

BackgroundDrainer: widen its connect-retry catch to the role-reject types so
an orphaned-slot drainer keeps giving the cluster a budget to settle instead
of quarantining on the first all-replica sweep (behaviour unchanged from
when this surfaced as QwpDurableAckMismatchException).

Author: bluestreak01

core/src/main/java/io/questdb/client/cutlass/qwp/client/QwpWebSocketSender.java

@@ -2567,21 +2567,23 @@ private synchronized WebSocketClient buildAndConnect(ReconnectSupplier ctx) {
             throw terminalUpgradeError;
         }
         if (lastRoleReject != null) {
-            // When the client opted into durable ack but every endpoint
-            // role-rejected the /write/v4 upgrade (typically a misconfigured
-            // address list pointing at replicas only), a primary that can
-            // serve durable ack will not appear by retrying. Throw the typed
-            // QwpDurableAckMismatchException -- the cursor send loop's terminal
-            // classifier recognises it by instanceof and suppresses retry, so
-            // the SYNC/ASYNC connect paths fail fast instead of burning the
-            // full reconnect_max_duration_millis budget walking the same
-            // replicas.
-            if (requestDurableAck) {
-                QwpDurableAckMismatchException ackErr = new QwpDurableAckMismatchException(
-                        lastRoleReject.getHost(), lastRoleReject.getPort(), lastRoleReject.getRole());
-                ackErr.initCause(lastRoleReject);
-                throw ackErr;
-            }
+            // Every endpoint role-rejected the /write/v4 upgrade: right now the
+            // reachable nodes are all replicas (or primary-catchup). That is a
+            // TRANSIENT failover window, not a terminal condition -- a replica
+            // can be promoted and a primary will reappear. Surface it as a
+            // retriable QwpRoleMismatchException so the SYNC/ASYNC connect and
+            // reconnect loops keep the rows in store-and-forward and retry
+            // within reconnect_max_duration_millis (for an SF sender the only
+            // terminal condition is SF exhaustion).
+            //
+            // This holds even when durable ack was requested: a replica that
+            // gets promoted serves durable ack, so an all-replica window must
+            // NOT be reported as a durable-ack mismatch. Doing so conflated a
+            // transient role state with a permanent capability gap and hard-
+            // failed HA senders that should have recovered on promotion. A
+            // genuine capability gap -- an endpoint that upgrades but does not
+            // advertise durable ack -- is still terminal: it is raised as
+            // terminalUpgradeError above, before this block.
             QwpRoleMismatchException ex = new QwpRoleMismatchException(
                     QwpIngressRoleRejectedException.ROLE_PRIMARY,
                     null,

core/src/main/java/io/questdb/client/cutlass/qwp/client/sf/cursor/BackgroundDrainer.java

@@ -26,6 +26,8 @@
 
 import io.questdb.client.cutlass.http.client.WebSocketClient;
 import io.questdb.client.cutlass.qwp.client.QwpDurableAckMismatchException;
+import io.questdb.client.cutlass.qwp.client.QwpIngressRoleRejectedException;
+import io.questdb.client.cutlass.qwp.client.QwpRoleMismatchException;
 import org.jetbrains.annotations.TestOnly;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -163,7 +165,15 @@ public WebSocketClient connectWithDurableAckRetry() {
         while (!stopRequested) {
             try {
                 return clientFactory.reconnect();
-            } catch (QwpDurableAckMismatchException e) {
+            } catch (QwpDurableAckMismatchException | QwpRoleMismatchException
+                     | QwpIngressRoleRejectedException e) {
+                // An all-replica window (every endpoint role-rejected the
+                // upgrade) is handled exactly like a durable-ack-unavailable
+                // cluster: give it a budget to settle, then quarantine. A
+                // replica can be promoted, so the drainer must not quarantine an
+                // orphaned slot on the first sweep just because no primary is
+                // reachable yet. Behaviour is unchanged from when buildAndConnect
+                // surfaced this same condition as QwpDurableAckMismatchException.
                 mismatchAttempts++;
                 long now = System.nanoTime();
                 long elapsedMs = (now - startNanos) / 1_000_000L;