fix: clean up wasm SQLite driver corruption + federated-object boot errors in pnpm dev#2896
Merged
Merged
Conversation
…efined→null bindings
The wasm SQLite driver is the dev fallback used whenever native better-sqlite3
has an ABI mismatch, so its failure modes surface as `pnpm dev` startup noise.
Three root causes are fixed:
- Persist flushes atomically (temp file + fsync + rename) so a process killed
mid-write no longer leaves a torn file that reads back as "database disk image
is malformed" on the next boot.
- Detect a corrupt on-disk image at open via PRAGMA quick_check, quarantine it
to `<db>.corrupt-<timestamp>`, and boot on a fresh database instead of failing
every query forever.
- Coerce `undefined` bindings to SQL NULL. sql.js throws a raw string for an
undefined bind ("Wrong API use : ... unknown type (undefined)"), which aborted
the write and logged as a garbled char-indexed object; NULL matches the
useNullAsDefault semantics and the native better-sqlite3 path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0111PrLebaDQAsjWmYrrcZmE
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
📓 Docs Drift CheckThis PR changes 2 package(s): 13 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
…ip backfill claimSeedOwnership walks every registered object exposing an owner_id column and re-owns unowned rows to the first admin. owner_id is auto-injected into external (ADR-0015 federated) object schemas, so they passed the filter and the backfill ran `select id from <remote_table> where owner_id is null` against a read-only remote datasource whose table may not be provisioned at boot — e.g. the showcase example logged `Find operation failed … no such table: customers / orders` on startup. External objects are read-only and their ownership is not the platform's to reassign, so skip them alongside managedBy / sys_* objects. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0111PrLebaDQAsjWmYrrcZmE
pnpm dev
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
pnpm devwas spamming database errors at startup. Most trace back to the wasm SQLite driver — the dev fallback that kicks in whenever nativebetter-sqlite3has an ABI mismatch (NODE_MODULE_VERSIONafter a Node upgrade, or never built). In that mode a persistent dev DB runs withpersist: 'on-write', so background dispatchers (notification / http) flush on every tick, which surfaced several latent bugs:update ... - database disk image is malformed(repeated forever)Insert operation failed {"0":"W","1":"r",...}— a garbled char-indexed logWrong API use : tried to bind a value of an unknown type (undefined)Find operation failed … no such table: customers / orders(federated showcase objects)Root causes & fixes
1. Non-atomic flush → torn file →
malformed.flush()persisted via a plainwriteFile(filename, bytes), which truncates the target and streams new bytes in place. A process killed mid-write (dev-server restart, Ctrl-C, crash — probable under per-tickon-writeflushing) left a half-written file that sql.js rejects on the next boot. Now flushes write to a sibling temp file,fsyncit, and atomicallyrename()over the target, so a reader always sees a complete image.2. No recovery from an already-corrupt image. A malformed file made every query fail forever with no way out (deleting the DB by hand was the only escape — which the reporter tried).
open()now runsPRAGMA quick_checkon the loaded bytes, and on corruption quarantines the file to<db>.corrupt-<timestamp>(bytes preserved for post-mortem) and boots on a fresh database with a loud warning.3.
undefinedbinding crash + garbled log. sql.js's binderthrows a raw string (not anError) for anundefinedvalue — that string aborts the write and logs as{"0":"W","1":"r",...}because it has no.message/.stack.formatBindingsnow coercesundefined→null, matching the driver'suseNullAsDefaultconfig and the native better-sqlite3 path.4. Seed-ownership backfill hit federated tables.
claimSeedOwnership(plugin-security) walks every registered object exposing anowner_idcolumn and re-owns unowned rows to the first admin.owner_idis auto-injected into external (ADR-0015 federated) object schemas, soshowcase_ext_customer/showcase_ext_orderpassed the filter and the backfill ranselect id from <remote_table> where owner_id is nullagainst a read-only remote datasource whose table isn't provisioned that early in boot →no such table. External objects are read-only (DDL forbidden, writes double-opt-in) and their ownership isn't the platform's to reassign, so they're now skipped alongsidemanagedBy/sys_*.Changes
driver-sqlite-wasm/src/wasm-connection.ts— atomicatomicWriteFile()(temp + fsync + rename);assertReadable()/quarantineCorruptFile()self-heal inopen().driver-sqlite-wasm/src/knex-wasm-dialect.ts—formatBindingsmapsundefined→null.plugin-security/src/claim-seed-ownership.ts— skip objects with anexternalbinding.driver-sqlite-wasm/src/sqlite-wasm-driver-durability.test.ts(atomic write leaves no temp file & round-trips; corrupt image quarantined + boots fresh;undefinedpersists asNULL);plugin-security/src/claim-seed-ownership.test.ts(external objects are not scanned).Testing
pnpm --filter @objectstack/driver-sqlite-wasm test→ 119 passed (incl. 3 new).pnpm --filter @objectstack/plugin-security test→ 309 passed (incl. 1 new).🤖 Generated with Claude Code