Skip to content

Commit de8ba19

Browse files
bluestreak01claude
andcommitted
Add async progress callback, fast-close opt-out, and test coverage
Pair the existing SenderError async dispatch with a SenderProgressHandler sibling so users can react to ack-watermark advances off the I/O thread. SenderProgressDispatcher mirrors SenderErrorDispatcher: bounded inbox, lazy-started daemon, volatile handler swap. Watermarks are monotonic, so a full inbox evicts the oldest entry rather than the newest -- drops compress, they don't lose information. CursorWebSocketSendLoop dispatches on every advance the engine actually accepts; CursorSendEngine.acknowledge and SegmentRing.acknowledge now return a boolean to give the loop a no-cost way to skip dispatch on idempotent re-acks. Make the close() drain genuinely opt-out. close_flush_timeout_millis<=0 now skips both drainOnClose AND the pre-drain checkError(). Previously checkError ran unconditionally, so a HALT-policy rejection always escaped through close() even when the user explicitly asked for a fast close. Users opting out are expected to observe outcomes via the async progress or error callback instead, which is exactly the surface this change adds. setProgressHandler and setErrorHandler are post-connect-effective: they forward to the live dispatcher's volatile handler field so builders, tests, and reconfigurable apps can install a new handler without tearing down the dispatcher thread. Drop the unused 1-arg SenderProgressDispatcher constructor and the dead capacity field; the LinkedBlockingDeque enforces its own bound, so the field had no readers. Cover the public surface with focused unit tests: - CursorSendEngineTest gains coverage for wasRecoveredFromDisk(), sealedSegments(), sealedSegmentsSnapshot(), and segmentSizeBytes(). Snapshot tests assert empty-buffer, sized-fit, and undersize-buffer paths; recovery tests cover memory-mode, fresh-disk, and reopen. - CursorWebSocketSendLoopDurableAckTest gains coverage for getTotalDurableAcks() and getTotalDurableTrimAdvances(), including the on-enqueue drain path that bumps trim advances even when the durable-ack frame itself ran against an empty queue. - SegmentRingTest gains coverage for maxBytesPerSegment(), including the openExisting recovery path which forwards the configured value. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 5ae7149 commit de8ba19

11 files changed

Lines changed: 1380 additions & 519 deletions

File tree

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
/*+*****************************************************************************
2+
* ___ _ ____ ____
3+
* / _ \ _ _ ___ ___| |_| _ \| __ )
4+
* | | | | | | |/ _ \/ __| __| | | | _ \
5+
* | |_| | |_| | __/\__ \ |_| |_| | |_) |
6+
* \__\_\\__,_|\___||___/\__|____/|____/
7+
*
8+
* Copyright (c) 2014-2019 Appsicle
9+
* Copyright (c) 2019-2026 QuestDB
10+
*
11+
* Licensed under the Apache License, Version 2.0 (the "License");
12+
* you may not use this file except in compliance with the License.
13+
* You may obtain a copy of the License at
14+
*
15+
* http://www.apache.org/licenses/LICENSE-2.0
16+
*
17+
* Unless required by applicable law or agreed to in writing, software
18+
* distributed under the License is distributed on an "AS IS" BASIS,
19+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
20+
* See the License for the specific language governing permissions and
21+
* limitations under the License.
22+
*
23+
******************************************************************************/
24+
25+
package io.questdb.client;
26+
27+
/**
28+
* User-supplied callback invoked when the asynchronous SF send loop observes a
29+
* server acknowledgement that advances the per-sender ack watermark. Registered
30+
* on {@code QwpWebSocketSender} via {@code setProgressHandler(...)} or on the
31+
* builder via {@code LineSenderBuilder.progressHandler(...)}.
32+
*
33+
* <h2>Watermark semantics</h2>
34+
* {@code ackedFsn} is the highest FSN whose batch is now durable on the server
35+
* side (committed to WAL, or skipped past on a {@code DROP_AND_CONTINUE}
36+
* rejection). The handler fires only when the watermark <em>advances</em>:
37+
* <ul>
38+
* <li>delivered values are strictly increasing,</li>
39+
* <li>the handler may be called many times during the lifetime of a sender,</li>
40+
* <li>a single call may skip multiple FSNs if the server batches several
41+
* frames into one OK frame.</li>
42+
* </ul>
43+
*
44+
* Callers polling for "is everything up to FSN N durable?" should compare
45+
* {@code ackedFsn} against their target and act once the inequality is satisfied,
46+
* not assume one call per sent batch.
47+
*
48+
* <h2>Threading</h2>
49+
* Implementations are invoked on a dedicated daemon dispatcher thread, never on
50+
* the I/O thread or the producer thread. Slow handlers cannot stall publishing.
51+
* If the bounded inbox fills, surplus notifications are dropped — visible via
52+
* {@code QwpWebSocketSender.getDroppedProgressNotifications()}. Drops are
53+
* tolerable because the next delivered call carries an equal-or-higher FSN, so
54+
* watchers comparing against a target threshold catch up automatically.
55+
*
56+
* <h2>Exceptions</h2>
57+
* Any {@link Throwable} thrown by the handler is caught and logged by the
58+
* dispatcher. The dispatcher and the sender continue running.
59+
*
60+
* <h2>What this callback is for</h2>
61+
* Marking application state durable, releasing producer-side latches, fan-out
62+
* to journals tagged with {@code (fsn, domainContext)} pairs returned by
63+
* {@code flushAndGetSequence()}. For an "is this batch rejected" question on the
64+
* producer thread, see {@link SenderErrorHandler} and
65+
* {@link io.questdb.client.cutlass.line.LineSenderException}.
66+
*
67+
* @see SenderErrorHandler
68+
*/
69+
@FunctionalInterface
70+
public interface SenderProgressHandler {
71+
/**
72+
* Called when the server-acked watermark advances. Strictly monotonic:
73+
* {@code ackedFsn} on call N+1 is greater than on call N.
74+
*/
75+
void onAcked(long ackedFsn);
76+
}

0 commit comments

Comments
 (0)