Skip to content

Commit 8fd9ad1

Browse files
glasstigerclaude
andcommitted
Tidy catch-up comments and harden two edges
Four minor cleanups on the delta symbol-dictionary catch-up, all behaviour-preserving on every reachable path: - The sentDict* field comment said the catch-up mirror is memory-mode only; it is also seeded and used in disk mode on a recovered / orphan-drained slot. Corrected. - positionCursorAt's javadoc said it runs after nextWireSeq was reset to 0, but the catch-up path leaves nextWireSeq past the frames it emitted. Corrected to describe setWireBaselineWithCatchUp anchoring the wire baseline; the method only moves the byte cursor. - The recovery-seed constructor set sentDictCount = pd.size() outside the loadedEntriesLen > 0 block. A recovered slot always has entries when size > 0, so the result is unchanged, but coupling the count to the mirror bytes stops sentDictCount ever claiming symbols the mirror does not hold. - sendDictCatchUp used Integer.MAX_VALUE as the no-cap per-frame budget, so sendCatchUpChunk's int frameLen could overflow on a multi-GB dictionary. Bound it by MAX_SENT_DICT_BYTES, the same ceiling ensureSentDictCapacity enforces. Unreachable at real cardinality (~200M+ symbols); defensive. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent ff79b72 commit 8fd9ad1

1 file changed

Lines changed: 20 additions & 6 deletions

File tree

core/src/main/java/io/questdb/client/cutlass/qwp/client/sf/cursor/CursorWebSocketSendLoop.java

Lines changed: 20 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -186,7 +186,9 @@ public final class CursorWebSocketSendLoop implements QuietCloseable {
186186
private final long reconnectMaxDurationMillis;
187187
private final WebSocketResponse response = new WebSocketResponse();
188188
private final ResponseHandler responseHandler = new ResponseHandler();
189-
// Delta symbol dictionary catch-up state (memory-mode only; see swapClient).
189+
// Delta symbol dictionary catch-up state (see swapClient). Active in memory
190+
// mode, and in disk mode on a recovered / orphan-drained slot (the constructor
191+
// seeds sentDict* from the persisted dictionary there).
190192
// deltaDictEnabled gates all of it. The loop mirrors, in sentDict*, every
191193
// symbol it has ever sent -- the concatenated [len varint][utf8] bytes in
192194
// global-id order (sentDictBytes*) plus the count (sentDictCount) -- so that
@@ -532,8 +534,12 @@ public CursorWebSocketSendLoop(WebSocketClient client, CursorSendEngine engine,
532534
ensureSentDictCapacity(len);
533535
Unsafe.getUnsafe().copyMemory(pd.loadedEntriesAddr(), sentDictBytesAddr, len);
534536
sentDictBytesLen = len;
537+
// Set the count only alongside the bytes so sentDictCount can
538+
// never claim symbols the mirror does not hold. A recovered slot
539+
// always has loadedEntriesLen > 0 when size > 0, so this is the
540+
// same result -- it just makes the coupling explicit.
541+
sentDictCount = pd.size();
535542
}
536-
sentDictCount = pd.size();
537543
}
538544
}
539545
this.fsnAtZero = fsnAtZero;
@@ -1784,8 +1790,11 @@ private void ioLoop() {
17841790
/**
17851791
* Walk the engine's segments to find the one containing {@code targetFsn},
17861792
* and set {@code sendOffset} to the byte offset of that frame within it.
1787-
* This is called at startup and after every reconnect, after fsnAtZero has
1788-
* already been reset to {@code targetFsn} and nextWireSeq to 0.
1793+
* This is called at startup and after every reconnect, once
1794+
* {@link #setWireBaselineWithCatchUp} has anchored the wire baseline
1795+
* ({@code fsnAtZero} / {@code nextWireSeq}) -- which may leave {@code nextWireSeq}
1796+
* past the catch-up frames it emitted. This method only positions the byte
1797+
* cursor at {@code targetFsn}; it does not touch the wire mapping.
17891798
* <p>
17901799
* If {@code targetFsn} is already published, the method positions the byte
17911800
* cursor exactly at that frame. If {@code targetFsn} is not published yet,
@@ -2133,8 +2142,13 @@ private int sendDictCatchUp() {
21332142
int cap = client.getServerMaxBatchSize();
21342143
// Symbol-bytes budget per frame, leaving room for the 12-byte header and
21352144
// the two delta-section varints. cap <= 0 means the server advertised no
2136-
// limit -> send the whole dictionary in one frame.
2137-
int budget = cap > 0 ? Math.max(1, cap - QwpConstants.HEADER_SIZE - 16) : Integer.MAX_VALUE;
2145+
// limit -- send the whole dictionary in one frame, but still bound the
2146+
// budget by MAX_SENT_DICT_BYTES so sendCatchUpChunk's int frameLen
2147+
// (HEADER_SIZE + varints + symbolsLen) cannot overflow on a pathological
2148+
// multi-GB dictionary (unreachable at real cardinality; defensive).
2149+
int budget = cap > 0
2150+
? Math.max(1, cap - QwpConstants.HEADER_SIZE - 16)
2151+
: MAX_SENT_DICT_BYTES - QwpConstants.HEADER_SIZE - 16;
21382152
int framesSent = 0;
21392153
int chunkStartId = 0;
21402154
long chunkStartAddr = sentDictBytesAddr;

0 commit comments

Comments
 (0)