Add ROLE_CHANGE (4001) close code for the role-change handoff

The server's role-change CLOSE now carries the private-use code 4001
(RFC 6455 s7.4.2) instead of NORMAL_CLOSURE, so the client's verbatim
CLOSE echo proves it actually read the server's CLOSE frame. Classify
ROLE_CHANGE as orderly in the send loop (strike-exempt, paced recycle,
reconnect-eligible), matching the treatment the handoff got as 1000.
Pin the classification with a poison-frame test: repeated demote closes
at an unadvanced head FSN must never latch a PROTOCOL_VIOLATION
terminal.

Author: bluestreak01

core/src/main/java/io/questdb/client/cutlass/qwp/client/sf/cursor/CursorWebSocketSendLoop.java

@@ -321,7 +321,7 @@ public final class CursorWebSocketSendLoop implements QuietCloseable {
     private long highestOkFsn = -1L;
     // Zero-progress recycle pacer (I/O thread only; survives reconnects).
     // Counts consecutive strike-EXEMPT recycles -- orderly closes
-    // (NORMAL_CLOSURE/GOING_AWAY), non-orderly closes before any send on the
+    // (NORMAL_CLOSURE/GOING_AWAY/ROLE_CHANGE), non-orderly closes before any send on the
     // connection, and pre-send RETRIABLE_OTHER rejections -- with no
     // acceptance progress in between. These paths deliberately carry no
     // poison strike (they are not a verdict on the bytes), which also exempts
@@ -1527,7 +1527,7 @@ private void failPaced(Throwable initial) {
 
     /**
      * Recycle path for strike-exempt wire events: orderly closes
-     * (NORMAL_CLOSURE / GOING_AWAY), non-orderly closes before any send on
+     * (NORMAL_CLOSURE / GOING_AWAY / ROLE_CHANGE), non-orderly closes before any send on
      * the connection, and RETRIABLE_OTHER rejections (pre- and post-send:
      * NOT_WRITABLE is a node-state verdict, not a frame verdict). None of
      * these implicate the head frame, so they carry no poison strike -- but that
@@ -2172,11 +2172,12 @@ public void onClose(int code, String reason) {
             // after this connection already sent the head frame counts a poison
             // strike; maxHeadFrameRejections consecutive strikes at the same
             // head FSN with no ack progress escalate to a typed terminal. Orderly
-            // closes (NORMAL_CLOSURE role-change handoff, GOING_AWAY restart
-            // drain) never count strikes — they are the server asking us to go
-            // elsewhere, not a verdict on the bytes.
+            // closes (ROLE_CHANGE role-change handoff, NORMAL_CLOSURE, GOING_AWAY
+            // restart drain) never count strikes — they are the server asking us
+            // to go elsewhere, not a verdict on the bytes.
             boolean orderly = code == WebSocketCloseCode.NORMAL_CLOSURE
-                    || code == WebSocketCloseCode.GOING_AWAY;
+                    || code == WebSocketCloseCode.GOING_AWAY
+                    || code == WebSocketCloseCode.ROLE_CHANGE;
             LineSenderException cause = new LineSenderException(
                     "WebSocket closed by server: code=" + code + " reason=" + reason);
             if (!orderly && nextWireSeq > 0) {

core/src/main/java/io/questdb/client/cutlass/qwp/websocket/WebSocketCloseCode.java

@@ -84,6 +84,16 @@ public final class WebSocketCloseCode {
      * Reserved for future use.
      */
     public static final int RESERVED = 1004;
+    /**
+     * Role-change close (4001). QWP application-defined code in the RFC 6455
+     * Section 7.4.2 private-use range: the server closed because its role
+     * changed (primary demoted); reconnect-eligible, not a verdict on the
+     * bytes. Deliberately distinct from {@link #NORMAL_CLOSURE} so the
+     * client's verbatim CLOSE echo proves to the server that the client
+     * received the server's CLOSE (and, by TCP ordering, everything before
+     * it -- the final durable ack included).
+     */
+    public static final int ROLE_CHANGE = 4001;
     /**
      * TLS handshake (1015).
      * Reserved value. MUST NOT be sent in a Close frame.
@@ -134,6 +144,8 @@ public static String describe(int code) {
                 return "Internal Error";
             case TLS_HANDSHAKE:
                 return "TLS Handshake";
+            case ROLE_CHANGE:
+                return "Role Change (QWP)";
             default:
                 if (code >= 3000 && code < 4000) {
                     return "Library/Framework Code (" + code + ")";

core/src/test/java/io/questdb/client/test/cutlass/qwp/client/sf/cursor/CursorWebSocketSendLoopPoisonFrameTest.java

@@ -32,6 +32,7 @@
 import io.questdb.client.cutlass.qwp.client.WebSocketResponse;
 import io.questdb.client.cutlass.qwp.client.sf.cursor.CursorSendEngine;
 import io.questdb.client.cutlass.qwp.client.sf.cursor.CursorWebSocketSendLoop;
+import io.questdb.client.cutlass.qwp.websocket.WebSocketCloseCode;
 import io.questdb.client.network.PlainSocketFactory;
 import io.questdb.client.std.Files;
 import io.questdb.client.std.MemoryTag;
@@ -244,6 +245,43 @@ public void testNonOrderlyClosePoisonKeysOnOkLevelHeadOfLine() throws Exception
         });
     }
 
+    @Test
+    public void testRoleChangeCloseIsOrderlyNeverCountsStrikes() throws Exception {
+        // The server's role-change close (ROLE_CHANGE, 4001) is the demote
+        // handoff: "go elsewhere", not a verdict on the bytes. Like
+        // NORMAL_CLOSURE and GOING_AWAY it must never count a poison strike.
+        // Dropping ROLE_CHANGE from the orderly set would pass the rest of
+        // this suite while latching a PROTOCOL_VIOLATION terminal after
+        // maxHeadFrameRejections demote closes at an unadvanced head FSN --
+        // turning a routine failover into a false data-poison verdict.
+        // Mirrors testNonOrderlyClosePoisonKeysOnOkLevelHeadOfLine with the
+        // opposite expectation.
+        TestUtils.assertMemoryLeak(() -> {
+            List<WebSocketClient> clients = new ArrayList<>();
+            try (CursorSendEngine engine = newEngine()) {
+                appendFrames(engine, 2);
+                CursorWebSocketSendLoop loop = newDurableLoop(engine, clients);
+                setSentCount(loop, 2);
+
+                deliverOk(loop, 0, names("trades"), txns(7L));
+                for (int i = 0; i < MAX_REJECTIONS + 1; i++) {
+                    // Role-change close after at least one send on this
+                    // connection: strike-exempt, so no number of repeats may
+                    // escalate. Restore the sent count after each recycle,
+                    // exactly like the non-orderly variant.
+                    setSentCount(loop, 2);
+                    deliverRoleChangeClose(loop);
+                }
+
+                // Must NOT throw: orderly closes never escalate to a typed
+                // terminal, however many accumulate at the same head FSN.
+                loop.checkError();
+            } finally {
+                closeAll(clients);
+            }
+        });
+    }
+
     @Test
     public void testNackRecycleIsPacedAgainstHealthyServer() throws Exception {
         // A reachable, healthy server that NACKs the head frame (RETRIABLE)
@@ -907,6 +945,16 @@ private static void deliverOrderlyClose(CursorWebSocketSendLoop loop) throws Exc
         m.invoke(handler, 1001, "server draining"); // GOING_AWAY: orderly, strike-exempt
     }
 
+    private static void deliverRoleChangeClose(CursorWebSocketSendLoop loop) throws Exception {
+        Field f = CursorWebSocketSendLoop.class.getDeclaredField("responseHandler");
+        f.setAccessible(true);
+        Object handler = f.get(loop);
+        Method m = handler.getClass().getDeclaredMethod("onClose", int.class, String.class);
+        m.setAccessible(true);
+        // ROLE_CHANGE (4001): the demote handoff, orderly, strike-exempt
+        m.invoke(handler, WebSocketCloseCode.ROLE_CHANGE, "role change: primary demoted");
+    }
+
     private static void deliverNonOrderlyClose(CursorWebSocketSendLoop loop) throws Exception {
         Field f = CursorWebSocketSendLoop.class.getDeclaredField("responseHandler");
         f.setAccessible(true);

design/qwp-nack-policy-v2.md

@@ -66,8 +66,9 @@ is caught *behaviorally*:
 > in durable-ack mode the trim watermark advances only on durable coverage,
 > so every post-NACK recycle replays from the durable watermark and re-OKs
 > frames *behind* the suspect — those re-OKs say nothing about the poisoned
-> bytes and must not launder the count. Orderly closes (`NORMAL_CLOSURE`
-> role-change handoff, `GOING_AWAY` restart drain) never count strikes.
+> bytes and must not launder the count. Orderly closes (`ROLE_CHANGE` (4001)
+> role-change handoff, `NORMAL_CLOSURE`, `GOING_AWAY` restart drain) never
+> count strikes.
 
 Below the escalation threshold, a RETRIABLE NACK's recycle is **paced**: the
 server is reachable (it just answered), so the reconnect succeeds immediately
@@ -99,8 +100,11 @@ diagnostics only.
 
 The server already handles it at the right layer (Invariant B work): the
 read-only gate and the commit-path authorization refusal both set
-`roleChangeClosePending` and close with a reconnect-eligible `NORMAL_CLOSURE`
-instead of NACKing `SECURITY_ERROR` (`QwpIngressProcessorState`). The client
+`roleChangeClosePending` and close with a reconnect-eligible `ROLE_CHANGE`
+(4001, private-use range; distinct from `NORMAL_CLOSURE` so the client's
+verbatim CLOSE echo is distinguishable from a voluntary client CLOSE that
+crossed the server's CLOSE on the wire) instead of NACKing `SECURITY_ERROR`
+(`QwpIngressProcessorState`). The client
 reconnects, hits the 421 role reject on the now-replica, and retries from SF
 until a primary is reachable. Consequently: