Skip to content

feat(ilp): QWiP durable ack#14

Merged
bluestreak01 merged 20 commits into
mainfrom
ia_wal_upload_ack
Apr 25, 2026
Merged

feat(ilp): QWiP durable ack#14
bluestreak01 merged 20 commits into
mainfrom
ia_wal_upload_ack

Conversation

@glasstiger
Copy link
Copy Markdown
Contributor

@glasstiger glasstiger commented Apr 21, 2026

Summary

  • Add opt-in STATUS_DURABLE_ACK (0x02) frame support to the QWP client. When a connection is opened with requestDurableAck(true) (builder API or request_durable_ack=on config string), the client sends an X-QWP-Request-Durable-Ack: true upgrade header. Servers with primary replication enabled then emit per-table durable-upload watermarks as WAL data reaches the object store.
  • Both STATUS_OK and STATUS_DURABLE_ACK frames now carry per-table entries (tableCount + [nameLen + name + seqTxn] repeating). STATUS_OK includes a batch sequence number; STATUS_DURABLE_ACK does not (it is not tied to a specific client batch).
  • Expose per-table progress on QwpWebSocketSender:
    • getHighestDurableSeqTxn(tableName) — highest seqTxn durably uploaded to object store
    • getHighestAckedSeqTxn(tableName) — highest seqTxn committed (written to WAL)
  • Add ping() method that sends a WebSocket PING and drains pending frames (including durable ACKs and STATUS_OK) until the PONG arrives. The server flushes pending durable ACKs before replying, so after ping() returns the durable progress is up to date. Works in both sync and async (send-queue) modes.
  • New CharSequenceLongHashMap collection for sync-mode per-table tracking.
  • New WebSocketResponse.durableAck() factory and isDurableAck() / getStatus() / getTableEntryCount() / getTableName(i) / getTableSeqTxn(i) accessors on the response model.

Wire format changes

STATUS_OK (was: status + sequence; now includes table entries):

+--------+----------+------------+--------------------------------------+
| status | sequence | tableCount | table entries                         |
| 1 byte | 8 bytes  | 2 bytes    | [nameLen(2)+name(N)+seqTxn(8)] × cnt |
+--------+----------+------------+--------------------------------------+

STATUS_DURABLE_ACK (new, no batch sequence):

+--------+------------+--------------------------------------+
| status | tableCount | table entries                         |
| 1 byte | 2 bytes    | [nameLen(2)+name(N)+seqTxn(8)] × cnt |
+--------+------------+--------------------------------------+

Changed files

Area Files
Builder / config SenderrequestDurableAck(boolean) builder method, request_durable_ack=on|off config string parsing
WebSocket upgrade WebSocketClientqwpRequestDurableAck flag → X-QWP-Request-Durable-Ack: true header
Sender QwpWebSocketSendersetRequestDurableAck, per-table getHighestDurableSeqTxn / getHighestAckedSeqTxn, ping(), sync ping loop, durable-ack + table-entry handling in waitForAck
Response model WebSocketResponseSTATUS_DURABLE_ACK constant, per-table entry parsing/writing (readTableEntries / writeTableEntries / validateTableEntries), updated structural validation
Async I/O WebSocketSendQueue — per-table ConcurrentHashMap<String, SeqTxn> tracking for both committed and durable seqTxns, ping() / pingAndDrain / PONG handling in I/O loop
In-flight window InFlightWindowgetHighestAckedSequence() accessor
Collections CharSequenceLongHashMap — new hash map for sync-mode per-table seqTxn tracking

Test plan

  • WebSocketResponseTest — durable-ack factory, round-trip serialization, per-table entries on STATUS_OK and STATUS_DURABLE_ACK, structural validity, truncated entries, trailing garbage, empty table names, unknown status bytes
  • QwpWebSocketSenderStateTest — default values for per-table durable/acked seqTxn, setRequestDurableAck lifecycle guards, sync ping processing (durable ACK, STATUS_OK, bare PONG), ping() after close
  • QwpWebSocketAckIntegrationTest — upgrade header sent/not-sent, sync durable ACK during waitForAck, STATUS_OK with table entries updates committed seqTxns
  • InFlightWindowTestgetHighestAckedSequence initial value, cumulative advancement, monotonicity
  • WebSocketSendQueueTest — STATUS_OK with table entries updates committed seqTxn, durable-ack updates per-table seqTxn, monotonicity under out-of-order delivery, interleaving with STATUS_OK, ping blocks until PONG, ping with in-flight batches, ping timeout, ping transport error, initial value check
  • CharSequenceLongHashMapTest — put/get, update existing key, rehash, clear, contains, custom no-entry value, valueAt with keyIndex

🤖 Generated with Claude Code

@glasstiger glasstiger changed the title Ia wal upload ack feat(qwp): QWiP durable ack Apr 21, 2026
@glasstiger glasstiger changed the title feat(qwp): QWiP durable ack feat(qwp): durable upload acknowledgment support Apr 21, 2026
@glasstiger glasstiger changed the title feat(qwp): durable upload acknowledgment support feat(ilp): QWiP durable ack Apr 21, 2026
glasstiger and others added 18 commits April 22, 2026 11:15
syncPing previously branched only on isDurableAck() and isSuccess().
Any error frame (parse, schema mismatch, security, internal, write
error) arriving between PING and PONG was parsed into ackResponse,
neither branch fired, and the error was silently discarded. A caller
using ping() to confirm "all my batches landed" could get a false
affirmative; the failure only surfaced on the next flush's
waitForAck.

Route error frames through inFlightWindow.fail so the next
waitForAck / flush raises them, matching the normal waitForAck
error-handling path. syncPing itself does not throw, so earlier
durable/committed progress in the same ping round still reaches
the caller.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@mtopolnik
Copy link
Copy Markdown
Contributor

[PR Coverage check]

😍 pass : 299 / 338 (88.46%)

file detail

path covered line new line coverage
🔵 io/questdb/client/Sender.java 2 15 13.33%
🔵 io/questdb/client/cutlass/qwp/client/WebSocketSendQueue.java 64 72 88.89%
🔵 io/questdb/client/cutlass/qwp/client/QwpWebSocketSender.java 61 69 88.41%
🔵 io/questdb/client/cutlass/qwp/client/WebSocketResponse.java 120 130 92.31%
🔵 io/questdb/client/std/CharSequenceLongHashMap.java 46 46 100.00%
🔵 io/questdb/client/cutlass/qwp/client/InFlightWindow.java 2 2 100.00%
🔵 io/questdb/client/cutlass/http/client/WebSocketClient.java 4 4 100.00%

@bluestreak01 bluestreak01 merged commit d54f218 into main Apr 25, 2026
12 checks passed
@bluestreak01 bluestreak01 deleted the ia_wal_upload_ack branch April 25, 2026 18:59
bluestreak01 added a commit that referenced this pull request Apr 27, 2026
Every frame written through the cursor SF path now carries its full
schema definition and the complete symbol-dictionary delta starting at
id 0. No schema-by-id refs, no incremental delta-dicts.

The bytes persist on disk and get replayed against fresh server
connections — post-reconnect, post-restart, or via background drainers
adopting orphan slots. A frame whose schema ref points at an ID the new
connection has never seen is unrecoverable; the spec's
dedup-by-messageSequence assumption fixes duplicates, not stale refs.
The previous testReplayResendsUnackedFramesAcrossReconnect only covered
single-batch replay (first batch always carries full schemas) so the
gap wasn't caught.

Implementation: encode pass forces confirmedMaxId=-1 (full symbol delta
from 0) and useSchemaRef=false (full schema definition, never a ref).
Producer-side maxSentSchemaId / maxSentSymbolId tracking is now
effectively dead state — left in place as a no-op safety net.

Cost: bytes per batch grow vs the prior delta encoding. Acceptable for
correctness across the entire recovery story.

Test: SelfSufficientFramesTest sends two batches with distinct symbol
values over the same connection and verifies batch 2 carries
deltaStart=0 with deltaCount ≥ 2 — i.e. it redefines the prior batch's
symbols too, instead of starting from where the prior delta left off.

Spec updated with decision #14.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants