Skip to content

Commit b002158

Browse files
clockwork-labs-botclockwork-labs-botbfops
authored
Enable confirmed reads by default (#4390)
## Summary Enable confirmed reads by default for all WebSocket subscriptions and SQL queries. This is a 2.0 breaking change that improves data integrity. ### What changed Previously, subscription updates and SQL results were sent to clients immediately, before the transaction was confirmed durable. A server crash could cause clients to have observed data that was lost. Now the server defaults to `confirmed=true`. Clients receive updates only after durability is confirmed. This adds a small latency cost but guarantees that any data a client receives will survive a server restart. ### Changes **Server (2 files, 2 lines each):** - `subscribe.rs`: `SubscribeQueryParams.confirmed` defaults to `true` - `database.rs`: `SqlQueryParams.confirmed` defaults to `true` **Documentation:** - Migration guide updated with "Confirmed Reads Enabled by Default" section - Added to overview list and quick migration checklist ### Opt-out Clients can opt out by explicitly passing `?confirmed=false` in the WebSocket URL or using `.withConfirmedReads(false)` / `.WithConfirmedReads(false)` / `.with_confirmed_reads(false)` in SDKs. ### Smoketest impact Smoketests that don't explicitly pass `--confirmed` will now get confirmed reads via the server default. This should not cause failures -- confirmed reads only add a small wait for durability confirmation before sending results. The `confirmed_reads.py` smoketest explicitly passes `--confirmed` and continues to work as before. ### SDK impact No SDK changes needed. SDKs only send the `confirmed` query parameter when explicitly set by the user. When not set, the server default applies -- which is now `true`. --------- Signed-off-by: Zeke Foppa <196249+bfops@users.noreply.github.com> Co-authored-by: clockwork-labs-bot <clockwork-labs-bot@users.noreply.github.com> Co-authored-by: clockwork-labs-bot <bot@clockworklabs.com> Co-authored-by: Zeke Foppa <196249+bfops@users.noreply.github.com> Co-authored-by: Zeke Foppa <bfops@users.noreply.github.com>
1 parent e476668 commit b002158

14 files changed

Lines changed: 126 additions & 24 deletions

File tree

crates/cli/src/common_args.rs

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,8 @@ pub fn confirmed() -> Arg {
4141
Arg::new("confirmed")
4242
.required(false)
4343
.long("confirmed")
44-
.action(SetTrue)
44+
.num_args(1)
45+
.value_parser(value_parser!(bool))
4546
.help("Instruct the server to deliver only updates of confirmed transactions")
4647
}
4748

crates/cli/src/subcommands/sql.rs

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -235,12 +235,12 @@ pub async fn exec(config: Config, args: &ArgMatches) -> Result<(), anyhow::Error
235235
}
236236
})?;
237237
let query = resolved.remaining_args.join(" ");
238-
let confirmed = args.get_flag("confirmed");
238+
let confirmed = args.get_one::<bool>("confirmed").copied();
239239

240240
let con = parse_req(config, args, &resolved.database, resolved.server.as_deref()).await?;
241241
let mut api = ClientApi::new(con).sql();
242-
if confirmed {
243-
api = api.query(&[("confirmed", "true")]);
242+
if let Some(confirmed) = confirmed {
243+
api = api.query(&[("confirmed", if confirmed { "true" } else { "false" })]);
244244
}
245245

246246
run_sql(api, &query, false).await?;

crates/cli/src/subcommands/subscribe.rs

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -148,7 +148,7 @@ pub async fn exec(config: Config, args: &ArgMatches) -> Result<(), anyhow::Error
148148
let num = args.get_one::<u32>("num-updates").copied();
149149
let timeout = args.get_one::<u32>("timeout").copied();
150150
let print_initial_update = args.get_flag("print_initial_update");
151-
let confirmed = args.get_flag("confirmed");
151+
let confirmed = args.get_one::<bool>("confirmed").copied();
152152
let resolved_server = server.or(resolved.server.as_deref());
153153

154154
let mut config = config;
@@ -169,8 +169,9 @@ pub async fn exec(config: Config, args: &ArgMatches) -> Result<(), anyhow::Error
169169
unknown => unreachable!("Invalid URL scheme in `Connection::db_uri`: {unknown}"),
170170
})
171171
.unwrap();
172-
if confirmed {
173-
url.query_pairs_mut().append_pair("confirmed", "true");
172+
if let Some(confirmed) = confirmed {
173+
url.query_pairs_mut()
174+
.append_pair("confirmed", if confirmed { "true" } else { "false" });
174175
}
175176

176177
// Create the websocket request.

crates/client-api/src/lib.rs

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,11 @@ pub mod auth;
2727
pub mod routes;
2828
pub mod util;
2929

30+
/// The default value for the `confirmed` reads parameter when the client does
31+
/// not specify it explicitly. When `true`, the server waits for durability
32+
/// confirmation before sending subscription updates and SQL results.
33+
pub const DEFAULT_CONFIRMED_READS: bool = true;
34+
3035
/// Defines the state / environment of a SpacetimeDB node from the PoV of the
3136
/// client API.
3237
///

crates/client-api/src/routes/database.rs

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -496,7 +496,7 @@ pub struct SqlQueryParams {
496496
/// If `true`, return the query result only after its transaction offset
497497
/// is confirmed to be durable.
498498
#[serde(default)]
499-
pub confirmed: bool,
499+
pub confirmed: Option<bool>,
500500
}
501501

502502
pub async fn sql_direct<S>(
@@ -518,7 +518,8 @@ where
518518
.authorize_sql(caller_identity, database.database_identity)
519519
.await?;
520520

521-
host.exec_sql(auth, database, confirmed, sql).await
521+
host.exec_sql(auth, database, confirmed.unwrap_or(crate::DEFAULT_CONFIRMED_READS), sql)
522+
.await
522523
}
523524

524525
pub async fn sql<S>(

crates/client-api/src/routes/subscribe.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ pub struct SubscribeQueryParams {
9292
///
9393
/// If `false`, send them immediately.
9494
#[serde(default)]
95-
pub confirmed: bool,
95+
pub confirmed: Option<bool>,
9696
}
9797

9898
pub fn generate_random_connection_id() -> ConnectionId {
@@ -170,7 +170,7 @@ where
170170
version: negotiated.version,
171171
compression,
172172
tx_update_full: !light,
173-
confirmed_reads: confirmed,
173+
confirmed_reads: confirmed.unwrap_or(crate::DEFAULT_CONFIRMED_READS),
174174
};
175175

176176
// TODO: Should also maybe refactor the code and the protocol to allow a single websocket

crates/pg/src/pg_server.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -161,7 +161,7 @@ where
161161
database::sql_direct(
162162
self.ctx.clone(),
163163
db,
164-
SqlQueryParams { confirmed: true },
164+
SqlQueryParams { confirmed: Some(true) },
165165
params.caller_identity,
166166
query.to_string(),
167167
)

crates/smoketests/src/lib.rs

Lines changed: 16 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1136,6 +1136,7 @@ log = "0.4"
11361136
"--server",
11371137
&self.server_url,
11381138
"--confirmed",
1139+
"true",
11391140
identity.as_str(),
11401141
query,
11411142
])
@@ -1375,16 +1376,26 @@ log = "0.4"
13751376
/// This matches Python's subscribe semantics - start subscription first,
13761377
/// perform actions, then call the handle to collect results.
13771378
pub fn subscribe_background(&self, queries: &[&str], n: usize) -> Result<SubscriptionHandle> {
1378-
self.subscribe_background_opts(queries, n, false)
1379+
self.subscribe_background_opts(queries, n, None)
13791380
}
13801381

13811382
/// Starts a subscription in the background with --confirmed flag.
13821383
pub fn subscribe_background_confirmed(&self, queries: &[&str], n: usize) -> Result<SubscriptionHandle> {
1383-
self.subscribe_background_opts(queries, n, true)
1384+
self.subscribe_background_opts(queries, n, Some(true))
1385+
}
1386+
1387+
/// Starts a subscription in the background with --confirmed flag.
1388+
pub fn subscribe_background_unconfirmed(&self, queries: &[&str], n: usize) -> Result<SubscriptionHandle> {
1389+
self.subscribe_background_opts(queries, n, Some(false))
13841390
}
13851391

13861392
/// Internal helper for background subscribe with options.
1387-
fn subscribe_background_opts(&self, queries: &[&str], n: usize, confirmed: bool) -> Result<SubscriptionHandle> {
1393+
fn subscribe_background_opts(
1394+
&self,
1395+
queries: &[&str],
1396+
n: usize,
1397+
confirmed: Option<bool>,
1398+
) -> Result<SubscriptionHandle> {
13881399
use std::io::{BufRead, BufReader};
13891400

13901401
let identity = self
@@ -1410,8 +1421,9 @@ log = "0.4"
14101421
n.to_string(),
14111422
"--print-initial-update".to_string(),
14121423
];
1413-
if confirmed {
1424+
if let Some(confirmed) = confirmed {
14141425
args.push("--confirmed".to_string());
1426+
args.push(confirmed.to_string());
14151427
}
14161428
args.push("--".to_string());
14171429
cmd.args(&args)

crates/smoketests/tests/auto_migration.rs

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -343,7 +343,13 @@ fn test_add_table_columns() {
343343
// Subscribe to person table changes multiple times to simulate active clients
344344
let mut subs = Vec::with_capacity(NUM_SUBSCRIBERS);
345345
for _ in 0..NUM_SUBSCRIBERS {
346-
subs.push(test.subscribe_background(&["select * from person"], 5).unwrap());
346+
// We need unconfirmed reads for the updates to arrive properly.
347+
// Otherwise, there's a race between module teardown in publish, vs subscribers
348+
// getting the row deletion they expect.
349+
subs.push(
350+
test.subscribe_background_unconfirmed(&["select * from person"], 5)
351+
.unwrap(),
352+
);
347353
}
348354

349355
// Insert under initial schema

docs/docs/00300-resources/00100-how-to/00600-migrating-to-2.0.md

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ SpacetimeDB 2.0 introduces a new WebSocket protocol (v2) and SDK with several br
1818
2. **`light_mode` removed** -- no longer necessary since reducer events are no longer broadcast
1919
3. **`CallReducerFlags` removed** -- `NoSuccessNotify` and `set_reducer_flags()` are gone
2020
4. **Event tables introduced** -- a new table type for publishing transient events to subscribers
21+
5. **Confirmed reads enabled by default** -- subscription updates and SQL results are only sent after the transaction is confirmed durable
2122

2223
## Reducer Callbacks
2324

@@ -1295,6 +1296,69 @@ In 2.0, the success notification is lightweight (just `request_id` and `timestam
12951296
</TabItem>
12961297
</Tabs>
12971298

1299+
## Confirmed Reads Enabled by Default
1300+
1301+
### What changed
1302+
1303+
In 1.0, subscription updates and SQL query results were sent to the client immediately, before the underlying transaction was confirmed to be durable. This meant a client could observe a row that was later lost if the server crashed before persisting it.
1304+
1305+
In 2.0, **confirmed reads are enabled by default**. The server waits until a transaction is confirmed durable before sending updates to clients. This ensures that any data a client receives will survive a server restart.
1306+
1307+
### Impact
1308+
1309+
- **Slightly higher latency**: Subscription updates and SQL results may arrive a few milliseconds later, as the server waits for durability confirmation before sending them.
1310+
- **Stronger consistency**: Clients will never observe data that could be lost due to a crash.
1311+
- **No code changes required**: This is a server-side default change. Existing client code works without modification.
1312+
1313+
### Opting out
1314+
1315+
If your application prioritizes low latency over durability guarantees (for example, a real-time game where occasional data loss on crash is acceptable), you can opt out by passing `confirmed=false` in the connection URL:
1316+
1317+
<Tabs groupId="client-language" queryString>
1318+
<TabItem value="typescript" label="TypeScript">
1319+
1320+
```typescript
1321+
DbConnection.builder()
1322+
.withUri("https://maincloud.spacetimedb.com")
1323+
.withDatabaseName("my-database")
1324+
.withConfirmedReads(false) // opt out of confirmed reads
1325+
.build()
1326+
```
1327+
1328+
</TabItem>
1329+
<TabItem value="csharp" label="C#">
1330+
1331+
```csharp
1332+
DbConnection.Builder()
1333+
.WithUri("https://maincloud.spacetimedb.com")
1334+
.WithDatabaseName("my-database")
1335+
.WithConfirmedReads(false) // opt out of confirmed reads
1336+
.Build();
1337+
```
1338+
1339+
</TabItem>
1340+
<TabItem value="rust" label="Rust">
1341+
1342+
```rust
1343+
DbConnection::builder()
1344+
.with_uri("https://maincloud.spacetimedb.com")
1345+
.with_database_name("my-database")
1346+
.with_confirmed_reads(false) // opt out of confirmed reads
1347+
.build()
1348+
.expect("Failed to connect");
1349+
```
1350+
1351+
</TabItem>
1352+
</Tabs>
1353+
1354+
For the CLI:
1355+
1356+
```bash
1357+
# SQL without confirmed reads
1358+
spacetime sql <database> "SELECT * FROM my_table"
1359+
# The --confirmed flag is no longer needed (it is the default)
1360+
```
1361+
12981362
## Quick Migration Checklist
12991363

13001364
- [ ] Remove all `ctx.reducers.on_<reducer>()` calls
@@ -1320,3 +1384,4 @@ In 2.0, the success notification is lightweight (just `request_id` and `timestam
13201384
- [ ] Remove `with_light_mode()` from `DbConnectionBuilder`
13211385
- [ ] Remove `set_reducer_flags()` calls and `CallReducerFlags` imports
13221386
- [ ] Remove `unstable::CallReducerFlags` from imports
1387+
- [ ] Note that confirmed reads are now enabled by default (no action needed unless you want to opt out with `.withConfirmedReads(false)`)

0 commit comments

Comments
 (0)