Skip to content

fix: clean up wasm SQLite driver corruption + federated-object boot errors in pnpm dev#2896

Merged
os-zhuang merged 2 commits into
mainfrom
claude/pnpm-dev-db-corruption-7dki0y
Jul 14, 2026
Merged

fix: clean up wasm SQLite driver corruption + federated-object boot errors in pnpm dev#2896
os-zhuang merged 2 commits into
mainfrom
claude/pnpm-dev-db-corruption-7dki0y

Conversation

@os-zhuang

@os-zhuang os-zhuang commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Background

pnpm dev was spamming database errors at startup. Most trace back to the wasm SQLite driver — the dev fallback that kicks in whenever native better-sqlite3 has an ABI mismatch (NODE_MODULE_VERSION after a Node upgrade, or never built). In that mode a persistent dev DB runs with persist: 'on-write', so background dispatchers (notification / http) flush on every tick, which surfaced several latent bugs:

  1. update ... - database disk image is malformed (repeated forever)
  2. Insert operation failed {"0":"W","1":"r",...} — a garbled char-indexed log
  3. Wrong API use : tried to bind a value of an unknown type (undefined)
  4. Find operation failed … no such table: customers / orders (federated showcase objects)

Root causes & fixes

1. Non-atomic flush → torn file → malformed. flush() persisted via a plain writeFile(filename, bytes), which truncates the target and streams new bytes in place. A process killed mid-write (dev-server restart, Ctrl-C, crash — probable under per-tick on-write flushing) left a half-written file that sql.js rejects on the next boot. Now flushes write to a sibling temp file, fsync it, and atomically rename() over the target, so a reader always sees a complete image.

2. No recovery from an already-corrupt image. A malformed file made every query fail forever with no way out (deleting the DB by hand was the only escape — which the reporter tried). open() now runs PRAGMA quick_check on the loaded bytes, and on corruption quarantines the file to <db>.corrupt-<timestamp> (bytes preserved for post-mortem) and boots on a fresh database with a loud warning.

3. undefined binding crash + garbled log. sql.js's binder throws a raw string (not an Error) for an undefined value — that string aborts the write and logs as {"0":"W","1":"r",...} because it has no .message/.stack. formatBindings now coerces undefinednull, matching the driver's useNullAsDefault config and the native better-sqlite3 path.

4. Seed-ownership backfill hit federated tables. claimSeedOwnership (plugin-security) walks every registered object exposing an owner_id column and re-owns unowned rows to the first admin. owner_id is auto-injected into external (ADR-0015 federated) object schemas, so showcase_ext_customer / showcase_ext_order passed the filter and the backfill ran select id from <remote_table> where owner_id is null against a read-only remote datasource whose table isn't provisioned that early in boot → no such table. External objects are read-only (DDL forbidden, writes double-opt-in) and their ownership isn't the platform's to reassign, so they're now skipped alongside managedBy / sys_*.

Changes

  • driver-sqlite-wasm/src/wasm-connection.ts — atomic atomicWriteFile() (temp + fsync + rename); assertReadable() / quarantineCorruptFile() self-heal in open().
  • driver-sqlite-wasm/src/knex-wasm-dialect.tsformatBindings maps undefinednull.
  • plugin-security/src/claim-seed-ownership.ts — skip objects with an external binding.
  • Tests: driver-sqlite-wasm/src/sqlite-wasm-driver-durability.test.ts (atomic write leaves no temp file & round-trips; corrupt image quarantined + boots fresh; undefined persists as NULL); plugin-security/src/claim-seed-ownership.test.ts (external objects are not scanned).

Testing

  • pnpm --filter @objectstack/driver-sqlite-wasm test119 passed (incl. 3 new).
  • pnpm --filter @objectstack/plugin-security test309 passed (incl. 1 new).
  • Package builds (incl. DTS type-check) succeed.

🤖 Generated with Claude Code

…efined→null bindings

The wasm SQLite driver is the dev fallback used whenever native better-sqlite3
has an ABI mismatch, so its failure modes surface as `pnpm dev` startup noise.
Three root causes are fixed:

- Persist flushes atomically (temp file + fsync + rename) so a process killed
  mid-write no longer leaves a torn file that reads back as "database disk image
  is malformed" on the next boot.
- Detect a corrupt on-disk image at open via PRAGMA quick_check, quarantine it
  to `<db>.corrupt-<timestamp>`, and boot on a fresh database instead of failing
  every query forever.
- Coerce `undefined` bindings to SQL NULL. sql.js throws a raw string for an
  undefined bind ("Wrong API use : ... unknown type (undefined)"), which aborted
  the write and logged as a garbled char-indexed object; NULL matches the
  useNullAsDefault semantics and the native better-sqlite3 path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0111PrLebaDQAsjWmYrrcZmE
@vercel

vercel Bot commented Jul 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
spec Ready Ready Preview, Comment Jul 14, 2026 8:06am

Request Review

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling size/m labels Jul 14, 2026
@github-actions

github-actions Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 2 package(s): @objectstack/driver-sqlite-wasm, @objectstack/plugin-security.

13 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/data-modeling/drivers.mdx (via @objectstack/driver-sqlite-wasm)
  • content/docs/getting-started/cli.mdx (via @objectstack/plugin-security)
  • content/docs/getting-started/glossary.mdx (via @objectstack/driver-sqlite-wasm)
  • content/docs/permissions/access-recipes.mdx (via packages/plugins/plugin-security)
  • content/docs/permissions/authorization.mdx (via packages/plugins/plugin-security)
  • content/docs/permissions/explain.mdx (via @objectstack/plugin-security)
  • content/docs/permissions/permissions-matrix.mdx (via packages/plugins/plugin-security)
  • content/docs/permissions/sharing-rules.mdx (via @objectstack/plugin-security)
  • content/docs/plugins/index.mdx (via @objectstack/plugin-security)
  • content/docs/plugins/packages.mdx (via @objectstack/driver-sqlite-wasm, @objectstack/plugin-security)
  • content/docs/releases/implementation-status.mdx (via @objectstack/driver-sqlite-wasm, @objectstack/plugin-security)
  • content/docs/ui/audience-based-interfaces.mdx (via packages/plugins/plugin-security)
  • content/docs/ui/dashboards.mdx (via @objectstack/plugin-security)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

…ip backfill

claimSeedOwnership walks every registered object exposing an owner_id column
and re-owns unowned rows to the first admin. owner_id is auto-injected into
external (ADR-0015 federated) object schemas, so they passed the filter and the
backfill ran `select id from <remote_table> where owner_id is null` against a
read-only remote datasource whose table may not be provisioned at boot — e.g.
the showcase example logged `Find operation failed … no such table: customers /
orders` on startup. External objects are read-only and their ownership is not
the platform's to reassign, so skip them alongside managedBy / sys_* objects.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0111PrLebaDQAsjWmYrrcZmE
@os-zhuang os-zhuang changed the title fix(driver-sqlite-wasm): atomic flushes, corrupt-image self-heal, undefined→null bindings fix: clean up wasm SQLite driver corruption + federated-object boot errors in pnpm dev Jul 14, 2026
@os-zhuang os-zhuang marked this pull request as ready for review July 14, 2026 10:53
@os-zhuang os-zhuang merged commit a199626 into main Jul 14, 2026
16 checks passed
@os-zhuang os-zhuang deleted the claude/pnpm-dev-db-corruption-7dki0y branch July 14, 2026 10:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants