Self-delivered cross-process change signal causes a permanent watch-query storm with App Group databases

[asiergmorato]

Fix open in PR #166. This issue documents the underlying problem and the reasoning behind the fix.

Summary

When a database is opened at an absolute path (an App Group container, the documented setup for sharing with app extensions), the app process enters a permanent re-query loop: every watch query re-runs and re-emits identical rows every ~65-115 ms indefinitely, with zero application writes. CPU stays pinned (measured ~348% on an iPad) and the battery drains.

The same app with the database in the default sandbox directory does not exhibit this. That matches the code: AsyncConnectionPool only creates a CrossProcessChangeSignal for .atPath locations, so only App Group databases are affected.

Environment

• powersync-swift 1.15.0 (introduced in Multi-process database support: absolute paths, open retry, cross-process change signal #147, shipped in 1.15.0)
• iOS 26 / 27
• Database opened at an absolute App Group path, app connected and syncing
• Reproduces on device and simulator

Evidence

The same watch query emits identical results at a fixed cadence for minutes, with the last real application write more than a minute earlier:

20:57:08.235 [GoalRepository] Emitting 4 active goals: ... (identical values)
20:57:08.350 [GoalRepository] Emitting 4 active goals: ...
20:57:08.477 [GoalRepository] Emitting 4 active goals: ...   <- every ~115 ms, forever
...
20:57:09.942 [StreamingSyncClient] crud upload: notify completion  <- every ~1 s, nothing to upload

The storm persists while the app is idle in the foreground. The crud upload: notify completion line fires roughly every second (the CRUD throttle) with an empty upload queue.

Root cause

The cross-process signal is delivered back to the process that posted it, and that self-delivery is pure redundant amplification.

1. NativeConnectionPool.write runs dispatchWrites in a defer after every write. It calls powersync_update_hooks('get') and, if any tables were affected, invokes handleUpdates.

   • NativeConnectionPool.swift:

     powersync-swift/Sources/PowerSync/Implementation/sqlite3/NativeConnectionPool.swift

     Lines 38 to 69 in 4507e96

     	private func dispatchWrites(lease: NativeConnectionLease) {
     	do {
     	try lease.withIterator(sql: "SELECT powersync_update_hooks('get')", parameters: []) { rows in
     	let affectedTables = try rows.next {
     	let decoder = JSONDecoder()
     	return try decoder.decode(Set<String>.self, from: try $0.getString(index: 0).data(using: .utf8)!)
     	}
     	if let affectedTables, !affectedTables.isEmpty {
     	self.handleUpdates(affectedTables)
     	}
     	}
     	} catch {
     	logger.warning("Could not read affected tables", tag: "NativeConnectionPool")
     	}
     	}
     	func read<T>(onConnection: (NativeConnectionLease) async throws -> T) async throws -> T {
     	// No dedicated readers? Acquire write connection for this then
     	let semaphore = readers ?? writer
     	let connection = try await semaphore.acquire(count: 1)
     	let lease = connection.acquiredItems[0].asLease()
     	return try await onConnection(lease)
     	}
     	func write<T>(onConnection: (NativeConnectionLease) async throws -> T) async throws -> T {
     	let connection = try await writer.acquire(count: 1)
     	let lease = connection.acquiredItems[0].asLease()
     	defer { dispatchWrites(lease: lease) }
     	let result = try await onConnection(lease)
     	return result
     	}

2. handleUpdates (wired in AsyncConnectionPool.PoolOpener.obtainPool) does two things on every local write:

   • dispatches the precise affected tables to tableUpdates (this is all local watchers need), and
   • calls changeSignal.post() to wake other processes.
   • AsyncConnectionPool.swift:

     powersync-swift/Sources/PowerSync/Implementation/AsyncConnectionPool.swift

     Lines 232 to 241 in 4507e96

     	let handleUpdates: @Sendable (_: Set<String>) -> () = { [weak context] updates in
     	context?.tableUpdatesStream.dispatch(event: updates)
     	// Tell other processes sharing this file that tables changed.
     	context?.changeSignal?.post()
     	}
     	context.changeSignal?.start { [weak context] in
     	// Another process (or this one; harmless, throttled downstream) changed
     	// the database outside this pool's update hooks.
     	context?.tableUpdatesStream.dispatch(event: [EXTERNAL_CHANGES_MARKER])
     	}

3. CrossProcessChangeSignal posts a Darwin notification, and deliveries to the posting process itself are deliberately not suppressed (documented in the class comment). The receive handler dispatches EXTERNAL_CHANGES_MARKER.

   • CrossProcessChangeSignal.swift: https://github.com/powersync-ja/powersync-swift/blob/4507e96787f256264de56ce524e70ac91873f369/Sources/PowerSync/Implementation/CrossProcessChangeSignal.swift

4. EXTERNAL_CHANGES_MARKER is by contract treated as "potentially matching every table". It bypasses every per-table watch filter and it triggers the upload client's ps_crud check.

   • watch.swift filter:

     powersync-swift/Sources/PowerSync/Implementation/queries/watch.swift

     Lines 30 to 34 in 4507e96

     	let updateNotifications = pool.tableUpdates.filter { changedTables in
     	changedTables.contains(where: watchedTables.contains)
     	// Another process changed unknown tables: conservatively re-query.
     	|| changedTables.contains(EXTERNAL_CHANGES_MARKER)
     	}.map { _ in () }

   • StreamingSyncClient upload trigger:

     powersync-swift/Sources/PowerSync/Implementation/sync/StreamingSyncClient.swift

     Lines 40 to 42 in 4507e96

     	let updates = db.pool.tableUpdates.filter { updates in
     	updates.contains("ps_crud") || updates.contains(EXTERNAL_CHANGES_MARKER)
     	}.map { _ in () }

There are two distinct problems, and the first holds even without a fully closed loop:

• Redundant amplification (always present). For every local write, the precise per-table update is already delivered via the native update hooks. The self-delivered catch-all marker adds nothing for the writing process except forcing every watcher that does NOT watch the affected tables to re-run, plus a ps_crud check. Cost is N watchers x M writes.

• Self-sustaining loop (the storm). Any bookkeeping commit made as a consequence of a marker wake-up re-enters the same path. EXTERNAL_CHANGES_MARKER -> uploadLoop -> uploadAllCrud -> uploadLocalTarget, which under a pending $local target can UPDATE ps_buckets ... inside a write transaction. That commit goes through NativeConnectionPool.write -> dispatchWrites -> handleUpdates -> changeSignal.post(), which the process re-delivers to itself, re-triggering the marker. The observed cadence matches the internal watch throttle (30 ms default) plus query time.

  • uploadLocalTarget commit path:

    powersync-swift/Sources/PowerSync/Implementation/sync/StreamingSyncClient.swift

    Lines 109 to 139 in 4507e96

    	private func uploadLocalTarget() async throws {
    	guard let _ = try await db.getOptional(
    	sql: "SELECT 1 FROM ps_buckets WHERE name = '$local' AND target_op = ?",
    	parameters: [PowerSyncDatabaseImpl.maxOpId],
    	mapper: { cursor in () }
    	) else {
    	return // Nothing to update
    	}
    	guard let seqBefore = try await db.getOptional("SELECT seq FROM main.sqlite_sequence WHERE name = 'ps_crud'", mapper: { try $0.getInt64(index: 0) }) else {
    	return // Nothing to update
    	}
    	let opId = try await getWriteCheckpoint()
    	try await db.writeTransaction { tx in
    	let anyData = try tx.getOptional(sql: "SELECT 1 FROM ps_crud LIMIT 1", parameters: nil) { cursor in 1 }
    	if anyData != nil {
    	// Additional write after we've obtained the write checkpoint
    	return
    	}
    	let seqAfter = try tx.getOptional(sql: "SELECT seq FROM main.sqlite_sequence WHERE name = 'ps_crud'", parameters: nil, mapper: { try $0.getInt64(index: 0) })
    	if seqBefore != seqAfter {
    	// New crud data may have been uploaded since we got the checkpoint, abort.
    	return
    	}
    	try tx.execute(sql: "UPDATE ps_buckets SET target_op = CAST(? AS INTEGER) WHERE name = '$local'", parameters: [opId])
    	}
    	}

An aggravating interaction: each extension process spawn (for example a widget timeline request) runs powersync_init / powersync_replace_schema, which commits and posts, so routine widget reloads wake every watcher in the app.

Minimal reproduction

1. Open a PowerSyncDatabase at an absolute App Group path.
2. Register one or more watch queries.
3. Connect and let the client sync.
4. Observe the watch callbacks re-firing with identical rows at a fixed sub-second cadence with no application writes, and CPU pinned.

Opening the same database with a plain filename (default sandbox directory) does not reproduce it, because no CrossProcessChangeSignal is created for non-.atPath locations.

Fix

The accompanying PR #166 applies both changes:

• Suppress own-process deliveries. A single sender-stamp scheme is racy: a coalesced external change could be misattributed to self and dropped, which would be a missed cross-process update. Instead each process owns a monotonic counter (one slot in a small file next to the database) and bumps only its own slot on post; a receiver fires the marker only when another process's slot advanced. Coalescing stays safe because the counters are monotonic, and nothing is lost for the posting process, since its own writes already reach its watchers via the native update hooks. It degrades to the previous always-fire behavior on any I/O error or slot exhaustion, so a change is never missed.
• Coalesce external wake-ups to one per interval, so a burst (for example rapid widget process spawns, each committing during powersync_init) cannot amplify into a re-query burst.

Together these break the storm and also remove the steady-state N x M redundant re-queries.

[simolus3]

Shouldn't uploadLocalTarget do nothing starting from the second iteration since the $local bucket is no longer at maxOpId after it finishes?