Skip to content

Commit 7491d95

Browse files
committed
feat(facade): replace writeOnly with lazy_connect (read-capable tolerant startup)
Drop write-only mode (it permanently disabled reads -- query()/newQuery() threw for the life of the handle) in favour of a read-capable tolerant-startup flag, lazy_connect, reachable from the connect string. lazy_connect=true: - a) starts even when the server is down -- the ingest side connects async and the read pool defaults to query_pool_min=0, so neither side fail-fasts; - b) buffers writes while the server is down (async sender); - c) reads once the server is up -- the read pool stays ENABLED and connects lazily on the first query. Because both sides must start non-blocking, a knob that forces a blocking / fail-fast startup is a configuration conflict, rejected up front with a clear remedy: - initial_connect_retry other than async (off/false/on/true/sync), and - an explicit query_pool_min > 0 (connect string or builder call). Changes: - ConfigSchema: write_only -> lazy_connect (Side.POOL; both clients ignore it). - ConfigView.getBool: accept true/false (and on/off). - QuestDBBuilder: remove writeOnly()/buildWriteOnly(); build() resolves lazy_connect, validates the two conflicts, defaults query_pool_min to 0 and injects initial_connect_retry=async when unset. - QuestDBImpl: remove the write-only constructor/flag and requireQueryEnabled; the query pool is always built (the white-box reflection seam is unchanged). - Tests: QuestDBWriteOnlyTest -> QuestDBLazyConnectTest (start+write while down, reads stay enabled, both conflicts via string and builder, true/on parsing); QuestDBServerRecoveryTest now dogfoods lazy_connect=true for the full down->write->up->read lifecycle; PoolConfigHonoredTest drift guard skips the flag.
1 parent 0b8c7f9 commit 7491d95

9 files changed

Lines changed: 261 additions & 269 deletions

File tree

core/src/main/java/io/questdb/client/QuestDB.java

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,10 @@
4141
* Configuration: one {@code ws}/{@code wss} string describes the whole cluster
4242
* (a single {@code addr} server list) and both the ingest and query pools
4343
* connect across it. Use {@link #connect(CharSequence)} for the common case, or
44-
* {@link #builder()} for pool sizing, the ingest callbacks, and write-only mode.
44+
* {@link #builder()} for pool sizing and the ingest callbacks. To tolerate the
45+
* server being down at startup, set {@code lazy_connect=true} in the config
46+
* (async ingest + lazy reads; reads stay enabled and connect once the server
47+
* is up).
4548
* <p>
4649
* Thread safety: instances are safe to share. {@link #borrowSender()} and
4750
* {@link #query()} may be called concurrently from any thread; the pool
@@ -51,7 +54,7 @@ public interface QuestDB extends Closeable {
5154

5255
/**
5356
* Builder for advanced configuration (pool sizes, acquisition timeouts,
54-
* ingest callbacks, write-only mode).
57+
* ingest callbacks).
5558
*/
5659
static QuestDBBuilder builder() {
5760
return new QuestDBBuilder();
@@ -64,8 +67,9 @@ static QuestDBBuilder builder() {
6467
* protocol), so one string configures both clients. List every cluster node
6568
* in a single {@code addr} server list and both pools connect across it.
6669
* <p>
67-
* Use {@link #builder()} for pool sizing, the ingest callbacks, and
68-
* write-only mode.
70+
* Use {@link #builder()} for pool sizing and the ingest callbacks. To
71+
* tolerate the server being down at startup, set {@code lazy_connect=true}
72+
* in the config (async ingest + lazy reads, reads still enabled).
6973
*
7074
* @param configurationString a {@code ws}/{@code wss} config string (see
7175
* {@link Sender#fromConfig} or

core/src/main/java/io/questdb/client/QuestDBBuilder.java

Lines changed: 62 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -35,8 +35,13 @@
3535

3636
/**
3737
* Builder for {@link QuestDB}. Most callers use {@link QuestDB#connect(CharSequence)};
38-
* this builder adds pool sizing, idle/lifetime knobs, the acquire timeout, the
39-
* ingest callbacks, and write-only mode.
38+
* this builder adds pool sizing, idle/lifetime knobs, the acquire timeout, and
39+
* the ingest callbacks.
40+
* <p>
41+
* To tolerate the server being down at startup, set {@code lazy_connect=true}
42+
* in the config: the ingest side connects asynchronously (writes buffer until
43+
* the wire is up) and the read pool connects lazily on first use. Reads stay
44+
* fully enabled -- they just connect once the server is available.
4045
* <p>
4146
* One configuration string describes the whole QuestDB cluster (see
4247
* {@link #fromConfig}): list every node in a single {@code addr} server list and
@@ -72,10 +77,6 @@ public final class QuestDBBuilder {
7277
private int queryPoolMin = UNSET;
7378
private int senderPoolMax = UNSET;
7479
private int senderPoolMin = UNSET;
75-
// When true, build() creates only the ingest (Sender) side: no query pool
76-
// is constructed, so the facade starts even when the server / read primary
77-
// is down, and query() is disabled. A query config is not required.
78-
private boolean writeOnly;
7980

8081
QuestDBBuilder() {
8182
}
@@ -93,26 +94,6 @@ public QuestDBBuilder acquireTimeoutMillis(long millis) {
9394
return this;
9495
}
9596

96-
/**
97-
* Builds an ingest-only (write-only) handle: no query/read pool is created,
98-
* so the facade starts even when the server / read primary is unavailable
99-
* (the read side is normally fail-fast and would sink the whole facade).
100-
* {@link QuestDB#query()} / {@link QuestDB#newQuery()} are disabled, and the
101-
* read-side config keys are ignored.
102-
* <p>
103-
* Write-only also defaults the ingest side to a non-blocking async initial
104-
* connect, so {@code build()} returns promptly even with the server down and
105-
* the sender pool warm ({@code sender_pool_min >= 1}); writes buffer until
106-
* the wire comes up. Override by setting {@code initial_connect_retry}
107-
* explicitly in the config.
108-
* <p>
109-
* Equivalent to the {@code write_only=on} connect-string key.
110-
*/
111-
public QuestDBBuilder writeOnly() {
112-
this.writeOnly = true;
113-
return this;
114-
}
115-
11697
/**
11798
* Sets the async connection-event listener applied to every pooled ingest
11899
* {@link Sender}. The listener observes connect / disconnect / failover
@@ -167,11 +148,6 @@ public QuestDB build() {
167148
}
168149
ConfigString cs = ConfigString.parse(config);
169150
ConfigView view = new ConfigView(cs);
170-
// Write-only may be requested via the builder (writeOnly()) or the
171-
// connect string (write_only=on); either skips the read pool.
172-
if (writeOnly || view.getBoolOnOff("write_only", false)) {
173-
return buildWriteOnly(view);
174-
}
175151
// Validate the single cluster config exactly as both pools will, but
176152
// without connecting: the full Sender parse plus validateParameters
177153
// (ingress value keys are registry-STRING, so only the real parse
@@ -182,17 +158,28 @@ public QuestDB build() {
182158
Sender.LineSenderBuilder.validateWsConfigString(config);
183159
QwpQueryClient.validateConfig(view, "wss".equals(cs.schema()));
184160

161+
// lazy_connect: tolerate a down server at startup without disabling
162+
// reads. The ingest side connects asynchronously (writes buffer until the
163+
// wire is up) and the read pool defaults to min=0 -- it connects lazily
164+
// on the first query once the server is up. Reads stay enabled.
165+
boolean lazyConnect = view.getBool("lazy_connect", false);
166+
String ingestConfig = config;
167+
if (lazyConnect) {
168+
ingestConfig = resolveLazyConnect(view);
169+
}
170+
185171
resolvePoolInt(senderPoolMin, "sender_pool_min", view, DEFAULT_POOL_MIN, this::senderPoolMin);
186172
resolvePoolInt(senderPoolMax, "sender_pool_max", view, DEFAULT_POOL_MAX, this::senderPoolMax);
187-
resolvePoolInt(queryPoolMin, "query_pool_min", view, DEFAULT_POOL_MIN, this::queryPoolMin);
173+
// lazy_connect makes the read pool lazy (min=0); without it the default min is 1.
174+
resolvePoolInt(queryPoolMin, "query_pool_min", view, lazyConnect ? 0 : DEFAULT_POOL_MIN, this::queryPoolMin);
188175
resolvePoolInt(queryPoolMax, "query_pool_max", view, DEFAULT_POOL_MAX, this::queryPoolMax);
189176
resolvePoolLong(acquireTimeoutMillis, "acquire_timeout_ms", view, DEFAULT_ACQUIRE_TIMEOUT_MILLIS, this::acquireTimeoutMillis);
190177
resolvePoolLong(idleTimeoutMillis, "idle_timeout_ms", view, DEFAULT_IDLE_TIMEOUT_MILLIS, this::idleTimeoutMillis);
191178
resolvePoolLong(maxLifetimeMillis, "max_lifetime_ms", view, DEFAULT_MAX_LIFETIME_MILLIS, this::maxLifetimeMillis);
192179
resolvePoolLong(housekeeperIntervalMillis, "housekeeper_interval_ms", view, DEFAULT_HOUSEKEEPER_INTERVAL_MILLIS, this::housekeeperIntervalMillis);
193180

194181
return new QuestDBImpl(
195-
config,
182+
ingestConfig,
196183
config,
197184
senderPoolMin,
198185
senderPoolMax,
@@ -207,34 +194,43 @@ public QuestDB build() {
207194
);
208195
}
209196

210-
// Ingest-only build path: validates and resolves the sender + shared pool
211-
// knobs from the ingest config alone and constructs a QuestDBImpl with no
212-
// query pool. Never touches the read side, so a down server cannot fail it.
213-
private QuestDB buildWriteOnly(ConfigView view) {
214-
Sender.LineSenderBuilder.validateWsConfigString(config);
215-
// writeOnly() owns "starts even when the server is down": default the
216-
// ingest side to a non-blocking async initial connect so prewarming a
217-
// sender (sender_pool_min >= 1) against a down server cannot fail-fast
218-
// the build. An explicit initial_connect_retry in the user's string still
219-
// wins (last-write-wins -- see withDefaultAsyncConnect).
220-
String senderConfig = withDefaultAsyncConnect(config);
221-
resolvePoolInt(senderPoolMin, "sender_pool_min", view, DEFAULT_POOL_MIN, this::senderPoolMin);
222-
resolvePoolInt(senderPoolMax, "sender_pool_max", view, DEFAULT_POOL_MAX, this::senderPoolMax);
223-
resolvePoolLong(acquireTimeoutMillis, "acquire_timeout_ms", view, DEFAULT_ACQUIRE_TIMEOUT_MILLIS, this::acquireTimeoutMillis);
224-
resolvePoolLong(idleTimeoutMillis, "idle_timeout_ms", view, DEFAULT_IDLE_TIMEOUT_MILLIS, this::idleTimeoutMillis);
225-
resolvePoolLong(maxLifetimeMillis, "max_lifetime_ms", view, DEFAULT_MAX_LIFETIME_MILLIS, this::maxLifetimeMillis);
226-
resolvePoolLong(housekeeperIntervalMillis, "housekeeper_interval_ms", view, DEFAULT_HOUSEKEEPER_INTERVAL_MILLIS, this::housekeeperIntervalMillis);
227-
return new QuestDBImpl(
228-
senderConfig,
229-
senderPoolMin,
230-
senderPoolMax,
231-
acquireTimeoutMillis,
232-
idleTimeoutMillis,
233-
maxLifetimeMillis,
234-
housekeeperIntervalMillis,
235-
errorHandler,
236-
connectionListener
237-
);
197+
// Validates the lazy_connect contract and returns the ingest config to use:
198+
// the original string with a non-blocking async initial connect injected
199+
// when the user did not set one. lazy_connect requires BOTH sides to start
200+
// non-blocking, so an explicit knob that forces a blocking / fail-fast
201+
// startup is a configuration conflict and is rejected with a clear remedy.
202+
private String resolveLazyConnect(ConfigView view) {
203+
// (1) ingest side: only initial_connect_retry=async is non-blocking;
204+
// off/false/on/true/sync all block or fail-fast at startup.
205+
String mode = view.getStr("initial_connect_retry");
206+
if (mode != null && !"async".equalsIgnoreCase(mode)) {
207+
throw new IllegalArgumentException(
208+
"conflicting configuration: lazy_connect=true needs a non-blocking startup, but "
209+
+ "initial_connect_retry=" + mode + " makes the initial connect block / fail-fast. "
210+
+ "Resolve by removing initial_connect_retry (lazy_connect implies "
211+
+ "initial_connect_retry=async) or setting initial_connect_retry=async.");
212+
}
213+
// (2) read side: lazy_connect requires query_pool_min=0 so the read pool
214+
// does not eagerly fail-fast at startup. An explicit query_pool_min > 0
215+
// (builder call or connect string) contradicts that.
216+
int explicitQueryMin;
217+
if (queryPoolMin != UNSET) {
218+
explicitQueryMin = queryPoolMin; // explicit builder call
219+
} else if (view.has("query_pool_min")) {
220+
explicitQueryMin = view.getInt("query_pool_min", UNSET); // connect string
221+
} else {
222+
explicitQueryMin = 0; // unset -> lazy default of 0
223+
}
224+
if (explicitQueryMin > 0) {
225+
throw new IllegalArgumentException(
226+
"conflicting configuration: lazy_connect=true needs query_pool_min=0 (the read pool "
227+
+ "connects lazily on first use and must not fail-fast at startup), but query_pool_min="
228+
+ explicitQueryMin + " was set. Resolve by removing query_pool_min (lazy_connect "
229+
+ "defaults it to 0) or setting query_pool_min=0.");
230+
}
231+
// No explicit initial_connect_retry -> inject async so the ingest build
232+
// is non-blocking. An explicit async needs no injection.
233+
return mode == null ? withDefaultAsyncConnect(config) : config;
238234
}
239235

240236
/**
@@ -382,11 +378,11 @@ public java.util.Map<String, Object> poolConfigSnapshotForTest() {
382378
return m;
383379
}
384380

385-
// writeOnly() owns "starts even when the server is down". Inject a
386-
// non-blocking async initial connect right after the schema separator so
387-
// build() never blocks or fail-fast on a down server. Placed first so an
388-
// explicit initial_connect_retry later in the user's string overrides it
389-
// (last-write-wins).
381+
// Inject a non-blocking async initial connect right after the schema
382+
// separator so lazy_connect's build never blocks or fail-fast on a down
383+
// server. Only used when the user set no initial_connect_retry of their own
384+
// (resolveLazyConnect rejects an explicit blocking mode rather than silently
385+
// overriding it), so placement is immaterial -- there is no competing value.
390386
private static String withDefaultAsyncConnect(String config) {
391387
int sep = config.indexOf("::");
392388
// sep >= 0: fromConfig() validated a ws/wss schema, so "::" is present.

core/src/main/java/io/questdb/client/impl/ConfigSchema.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,7 @@ public final class ConfigSchema {
112112
longRange("idle_timeout_ms", Side.POOL, OPEN, OPEN_MAX, false, false);
113113
longRange("max_lifetime_ms", Side.POOL, OPEN, OPEN_MAX, false, false);
114114
longRange("housekeeper_interval_ms", Side.POOL, OPEN, OPEN_MAX, false, false);
115-
boolOnOff("write_only", Side.POOL); // facade routing flag: build an ingest-only handle
115+
boolOnOff("lazy_connect", Side.POOL); // facade flag: tolerant non-blocking startup (async ingest + lazy reads)
116116

117117
// RESERVED -- accepted no-op (error-policy keys reserved by the spec).
118118
str("on_internal_error", Side.RESERVED);

core/src/main/java/io/questdb/client/impl/ConfigView.java

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,25 @@ public static String relocatedHint(String key) {
9595
return RELOCATED_HINTS.get(key);
9696
}
9797

98+
/**
99+
* A boolean flag accepting {@code true}/{@code false} (and {@code on}/{@code off}
100+
* for consistency with the rest of the connect-string surface). Returns
101+
* {@code dflt} when the key is absent; throws on any other value.
102+
*/
103+
public boolean getBool(String key, boolean dflt) {
104+
String v = getStr(key);
105+
if (v == null) {
106+
return dflt;
107+
}
108+
if ("true".equals(v) || "on".equals(v)) {
109+
return true;
110+
}
111+
if ("false".equals(v) || "off".equals(v)) {
112+
return false;
113+
}
114+
throw new IllegalArgumentException("invalid " + key + ": " + v + " (expected true, false)");
115+
}
116+
98117
public boolean getBoolOnOff(String key, boolean dflt) {
99118
String v = getStr(key);
100119
if (v == null) {

core/src/main/java/io/questdb/client/impl/QuestDBImpl.java

Lines changed: 7 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -66,26 +66,7 @@ public QuestDBImpl(
6666
) {
6767
this(ingestConfig, queryConfig, senderMin, senderMax, queryMin, queryMax,
6868
acquireTimeoutMillis, idleTimeoutMillis, maxLifetimeMillis,
69-
housekeeperIntervalMillis, null, null, errorHandler, connectionListener, false);
70-
}
71-
72-
// Write-only (ingest-only) handle: no query pool is created, so build()
73-
// never touches the read side and the facade starts even when the
74-
// server / read primary is unavailable. query()/newQuery() are disabled.
75-
public QuestDBImpl(
76-
String ingestConfig,
77-
int senderMin,
78-
int senderMax,
79-
long acquireTimeoutMillis,
80-
long idleTimeoutMillis,
81-
long maxLifetimeMillis,
82-
long housekeeperIntervalMillis,
83-
SenderErrorHandler errorHandler,
84-
SenderConnectionListener connectionListener
85-
) {
86-
this(ingestConfig, null, senderMin, senderMax, 0, 0,
87-
acquireTimeoutMillis, idleTimeoutMillis, maxLifetimeMillis,
88-
housekeeperIntervalMillis, null, null, errorHandler, connectionListener, true);
69+
housekeeperIntervalMillis, null, null, errorHandler, connectionListener);
8970
}
9071

9172
// Package-private constructor exposing the senderFactory and connectHook test
@@ -110,7 +91,7 @@ public QuestDBImpl(
11091
) {
11192
this(ingestConfig, queryConfig, senderMin, senderMax, queryMin, queryMax,
11293
acquireTimeoutMillis, idleTimeoutMillis, maxLifetimeMillis,
113-
housekeeperIntervalMillis, senderFactory, connectHook, null, null, false);
94+
housekeeperIntervalMillis, senderFactory, connectHook, null, null);
11495
}
11596

11697
// Full constructor adding the ingest-side errorHandler/connectionListener,
@@ -131,8 +112,7 @@ public QuestDBImpl(
131112
IntFunction<Sender> senderFactory,
132113
Consumer<QwpQueryClient> connectHook,
133114
SenderErrorHandler errorHandler,
134-
SenderConnectionListener connectionListener,
135-
boolean writeOnly
115+
SenderConnectionListener connectionListener
136116
) {
137117
SenderPool builtSenderPool = null;
138118
QueryClientPool builtQueryPool = null;
@@ -146,13 +126,9 @@ public QuestDBImpl(
146126
// server; the housekeeper drives it via runStartupRecoveryStep().
147127
true,
148128
errorHandler, connectionListener);
149-
// Write-only: skip the read side entirely so a down server / read
150-
// primary cannot fail the facade build.
151-
if (!writeOnly) {
152-
builtQueryPool = new QueryClientPool(
153-
queryConfig, queryMin, queryMax, acquireTimeoutMillis,
154-
idleTimeoutMillis, maxLifetimeMillis, connectHook);
155-
}
129+
builtQueryPool = new QueryClientPool(
130+
queryConfig, queryMin, queryMax, acquireTimeoutMillis,
131+
idleTimeoutMillis, maxLifetimeMillis, connectHook);
156132
builtHousekeeper = new PoolHousekeeper(builtSenderPool, builtQueryPool, housekeeperIntervalMillis);
157133
builtHousekeeper.start();
158134
} catch (Throwable e) {
@@ -182,7 +158,7 @@ public QuestDBImpl(
182158
this.senderPool = builtSenderPool;
183159
this.queryPool = builtQueryPool;
184160
this.housekeeper = builtHousekeeper;
185-
this.queryThreadLocal = writeOnly ? null : ThreadLocal.withInitial(() -> new QueryImpl(queryPool));
161+
this.queryThreadLocal = ThreadLocal.withInitial(() -> new QueryImpl(queryPool));
186162
}
187163

188164
@Override
@@ -243,24 +219,16 @@ public Completion executeSql(CharSequence sql, QwpColumnBatchHandler handler) {
243219

244220
@Override
245221
public Query newQuery() {
246-
requireQueryEnabled();
247222
return new QueryImpl(queryPool);
248223
}
249224

250225
@Override
251226
public Query query() {
252-
requireQueryEnabled();
253227
QueryImpl q = queryThreadLocal.get();
254228
q.resetIfDone();
255229
return q;
256230
}
257231

258-
private void requireQueryEnabled() {
259-
if (queryPool == null) {
260-
throw new IllegalStateException("query/read is disabled on a write-only QuestDB client");
261-
}
262-
}
263-
264232
@Override
265233
public void releaseSender() {
266234
senderPool.releaseCurrentThread();

0 commit comments

Comments
 (0)