docs updates

Author: stevensJourney

crates/core/src/crud_vtab.rs

@@ -248,11 +248,10 @@ impl SimpleCrudTransactionMode {
     fn record_local_write(&mut self, db: *mut sqlite::sqlite3) -> Result<(), ResultCode> {
         if !self.had_writes {
             // Also clear the seen/applied high-water marks: checkpoint request ids observed before
-            // this write can't acknowledge it, and they may come from an incompatible id namespace
-            // (legacy write checkpoints migrated by v14, or state from before a counter restart).
-            // Keeping them around could open the apply gate for a newly allocated target id that
-            // compares below a stale seen value. The legacy `$local` bookkeeping had the same
-            // behavior by resetting the entire row on local writes.
+            // this write can't acknowledge it, and stale values may predate a request counter
+            // restart. Keeping them around could open the apply gate for a newly allocated target
+            // id that compares below a stale seen value. The legacy `$local` bookkeeping had the
+            // same behavior by resetting the entire row on local writes.
             db.exec_safe(formatcp!(
                 "INSERT OR REPLACE INTO ps_kv(key, value) VALUES('local_target_op', {MAX_OP_ID});
 DELETE FROM ps_kv WHERE key IN ('last_seen_checkpoint_request_id', 'last_applied_checkpoint_request_id')"

crates/core/src/sync/interface.rs

@@ -143,8 +143,11 @@ pub enum Instruction {
         request: StreamingSyncRequest,
         /// The latest checkpoint request id known locally before opening this stream.
         ///
-        /// SDKs can use a missing value as a cue to fetch checkpoint request state from the service
-        /// and report it back with `seed_checkpoint_request_id`.
+        /// This is simply the client's current counter state. SDKs use it on every connection
+        /// attempt to re-affirm checkpoint request state with the service, which may have deleted
+        /// its record. The re-affirmation works bidirectionally: it can restore the service-side
+        /// value from this hint or bump the local counter from the service's response, which is
+        /// reported back with `seed_checkpoint_request_id`.
         last_checkpoint_request_id: Option<i64>,
     },
     FetchCredentials {

dart/test/crud_test.dart

@@ -250,8 +250,8 @@ void main() {
         });
 
         test('updates local target op and updated rows', () {
-          // Stale high-water marks (e.g. migrated legacy write checkpoints) must be cleared by a
-          // local write, so they can't open the apply gate for a smaller new target id.
+          // Stale high-water marks (e.g. from before a request counter restart) must be cleared
+          // by a local write, so they can't open the apply gate for a smaller new target id.
           db.execute('''
 INSERT INTO ps_kv(key, value) VALUES
   ('last_seen_checkpoint_request_id', 6),

dart/test/sync_test.dart

@@ -611,8 +611,8 @@ void _syncTests<T>({
     expect(lastAppliedCheckpointRequestId(), 5);
 
     // A local write can only be acknowledged by a checkpoint request id observed after it. Stale
-    // seen/applied values (which may come from another id namespace, like migrated legacy write
-    // checkpoints) must not remain to open the apply gate for a smaller new target id.
+    // seen/applied values (which may predate a request counter restart) must not remain to open
+    // the apply gate for a smaller new target id.
     db.execute("insert into items (id, col) values ('local', 'data');");
 
     expect(lastAppliedCheckpointRequestId(), isNull);

docs/schema.md

@@ -33,8 +33,11 @@ a checkpoint and that we have validated its checksum).
 Schema version 14 removes the legacy `target_op` column after migrating `$local.target_op` to
 `ps_kv.local_target_op`, and deletes the `$local` row so `ps_buckets` only contains real sync
 buckets. This makes older SDKs fail with a hard SQLite error if they try to keep using the migrated
-database without downgrading. The down migration restores `target_op` and recreates the `$local`
-row from `ps_kv` for older schema versions.
+database without downgrading. That failure is deliberate — including for multi-process deployments
+where processes with mixed SDK versions share one database — because an older SDK silently
+maintaining `$local` state the new implementation no longer reads would be worse than a loud error.
+The down migration restores `target_op` and recreates the `$local` row from `ps_kv` for older
+schema versions.
 
 ## `ps_crud`
 

docs/sync.md

@@ -47,9 +47,14 @@ The following commands are supported:
     ahead, or restore the service-side value when the service has cleared stale state but core still
     has a local hint. Then seed core with the reconciled value. Core stores the seeded value
     verbatim and does not enforce monotonicity; SDKs own the reconciliation and must not seed a
-    stale value. `NULL` means neither side has a record for the client yet; core stores `0` in that
-    case so the state counts as seeded and the first allocation returns `1`. If both the client and
-    service have lost the value, the counter may restart.
+    stale value. A `NULL` payload is accepted for completeness (core stores `0`, marking the state
+    as seeded so the first allocation returns `1`), but SDKs should not need it in practice:
+    posting a checkpoint request with an id of at least `1` during reconciliation and seeding the
+    service's response covers the no-record case and doubles as a probe of the service's
+    checkpoint-request support. Blindly forwarding a raw `NULL` service response while core holds a
+    counter would reset it, since the store is verbatim (see
+    `docs/write-checkpoint-requests.md`). If both the client and service have lost the value, the
+    counter may restart.
 
 When uploads request a write checkpoint, SDKs should call
 `powersync_control('next_checkpoint_request_id', NULL)` inside a transaction to allocate the id to
@@ -78,7 +83,9 @@ only real sync buckets in `ps_buckets`, and drops `ps_buckets.target_op` so
 older SDKs fail hard if they try to keep using the migrated database directly. Downgrading restores
 the column, and restores a `$local` row only when `local_target_op` exists, so older SDKs can keep
 using target-op based blocking without inventing a synthetic local bucket when there was no local
-target state. Because the down migration keeps the `ps_kv` keys around, the up migration clears
+target state. A restored concrete target remains satisfiable after a downgrade because checkpoint
+request ids and legacy write checkpoint ids share one namespace: the service reports accepted
+checkpoint request ids as the `write_checkpoint` values older-protocol clients observe. Because the down migration keeps the `ps_kv` keys around, the up migration clears
 them before copying, so re-upgrading a downgraded database takes the `$local` row (including any
 progress an older SDK made) as the source of truth instead of failing on the existing keys.
 
@@ -124,11 +131,11 @@ interface LogLine {
 }
 
 // Instructs client SDKs to open a connection to the sync service.
-// last_checkpoint_request_id is core's local counter value before this stream request. On connect,
-// SDKs can use it to re-request this client's last checkpoint request state from the service, then
-// call powersync_control('seed_checkpoint_request_id', value) with the actual response for
-// reconciliation. `value` may be null when the service has no checkpoint request state for this
-// client.
+// last_checkpoint_request_id is the client's current counter state before this stream request.
+// On every connect, SDKs use it to re-affirm checkpoint request state with the service (which may
+// have deleted its record). The re-affirmation is bidirectional: the hint can restore the
+// service-side value, or the service's response can bump the local counter via
+// powersync_control('seed_checkpoint_request_id', response).
 interface EstablishSyncStream {
   request: any // The JSON-encoded StreamingSyncRequest to send to the sync service
   last_checkpoint_request_id: null | number

docs/write-checkpoint-requests.md

@@ -49,8 +49,7 @@ WHERE key IN ('last_seen_checkpoint_request_id', 'last_applied_checkpoint_reques
 
 Clearing the high-water marks mirrors how the legacy flow reset the whole `$local` row on local
 writes. A checkpoint request id observed before the write cannot acknowledge it, and a stale seen
-value may even come from an incompatible id namespace — a legacy write checkpoint migrated by v14,
-or state from before a request counter restart. If such a value stayed around, a newly allocated
+value may even predate a request counter restart. If such a value stayed around, a newly allocated
 target id could compare below it and open the apply gate before the service acknowledged the write.
 After a local write, only checkpoint request ids observed from that point on count towards the gate.
 
@@ -154,9 +153,16 @@ local hint can restore the service-side value and keep the counter from moving b
 client lost local state but the service still has a record, the service response restores the seed
 locally. If both sides have lost the value, it is acceptable for the counter to restart; this can
 happen after local state is cleared and stale service state expires, or when multiple user ids share
-the same client id. After reconciliation, `value` may be `NULL` when neither side has a record for
-the client; core stores `0` in that case so the state counts as seeded and the first allocation
-returns `1`. SDKs may also refresh service state when their user/client context changes.
+the same client id. SDKs may also refresh service state when their user/client context changes.
+
+Because seeds are stored verbatim, seeding `NULL` (or a low id) while core holds a higher counter
+resets that counter, and previously allocated ids would be handed out again. SDKs must therefore
+never forward a raw service response without reconciling: the recommended pattern is to always post
+an id of at least `1` (the maximum of the local hint and any concrete local target) during
+reconciliation and seed core with the service's response to that request. In practice SDKs never
+need to seed `NULL` at all — when there is no local record, posting a checkpoint request with id
+`1` works and doubles as a probe of the service's checkpoint-request support. Core still accepts a
+`NULL` seed for completeness.
 
 `powersync_control('next_checkpoint_request_id', NULL)` must be called inside a transaction during
 an active sync iteration after `last_requested_checkpoint_request_id` exists locally. It increments
@@ -171,6 +177,14 @@ RETURNING value;
 
 This command only allocates an id. It does not update `local_target_op`.
 
+Calling it without an active iteration or before seeding raises a state error. This is a normal
+part of the connection lifecycle (for example a `requestCheckpoint` call racing a stream restart),
+not a programming error — SDKs should surface it as a retryable condition.
+
+The increment participates in the caller's transaction. If the transaction rolls back after the id
+was already posted to the service, the same id is allocated and posted again on retry; this is safe
+because the service treats the latest posted id as the effective request state.
+
 Note on sequences: SQLite does not have standalone sequences. The sequence-like alternatives are
 either an `AUTOINCREMENT` table backed by SQLite's internal `sqlite_sequence`, or a dedicated
 single-row counter table like the existing `ps_tx` transaction counter. The checkpoint request
@@ -298,6 +312,12 @@ greater than or equal to any previously requested id and core emits a fresh
   migration/downgrade state and debugging; SDKs should use `CheckpointRequestApplied` instructions
   to resolve `CheckpointRequest` waits.
 
+`powersync_clear` deletes all of these keys in both clear modes (it removes every `ps_kv` entry
+except `client_id`). This is deliberate and mirrors the legacy behavior of deleting the `$local`
+row: pending CRUD is wiped in the same operation, so no apply gate is needed, and the request
+counter is restored by the connect-time reconciliation described above. If the service has also
+lost its record, the counter restarting is acceptable.
+
 ## Migration from `$local`
 
 Migration v14 moves the old `$local` bucket state into `ps_kv`:
@@ -344,3 +364,18 @@ The down migration restores `ps_buckets.target_op` and rebuilds a `$local` row o
 
 This keeps older SDKs able to use the historic target-op gate after a downgrade without inventing a
 synthetic `$local` bucket when there was no local target state.
+
+Two properties make the restored gate safe rather than a potential stall:
+
+- **Shared id namespace.** Client-created checkpoint request ids and legacy write checkpoint ids
+  are one namespace, compatible in both directions — including values migrated from the historic
+  service-generated write checkpoint scheme. The service reports the checkpoint request ids it
+  accepted as the `write_checkpoint` values older-protocol clients observe, so a restored concrete
+  `$local.target_op` is satisfiable by the next write checkpoint the downgraded SDK sees; it does
+  not wait on an incomparable id sequence.
+- **Downgrade fidelity.** `last_seen_checkpoint_request_id` and
+  `last_applied_checkpoint_request_id` only advance on full checkpoint completions — but so did the
+  legacy `$local.last_op`/`last_applied_op` bookkeeping (partial priority applies never updated
+  `$local`, which is not a real bucket). The rebuilt `$local` row therefore matches exactly what a
+  legacy client would have recorded at the same point in the stream; the down migration cannot lag
+  behind legacy behavior.