DataConnect.app: bundled personal-server crashes — MODULE_NOT_FOUND on '@opendatalabs/personal-server-ts-core/config' (pkg snapshot can't resolve ESM-only subpath exports)

[natea]

DataConnect.app personal-server crashes: Cannot find module '@opendatalabs/personal-server-ts-core/config' from pkg-snapshotted bundle

Summary

The personal-server binary bundled inside DataConnect.app (/Applications/DataConnect.app/Contents/Resources/personal-server/dist/personal-server) crashes on startup with MODULE_NOT_FOUND when its compiled CJS bundle does require('@opendatalabs/personal-server-ts-core/config'). The binary was produced with Vercel's pkg, and pkg does not resolve subpath exports from an ESM-only package, so the snapshot's CJS loader cannot find /config even though the package files are present on disk next to the binary.

Net effect: DataConnect's "Start personal server on port 8080" step fails immediately, the personal server exits with code=1, and downstream connector flows (e.g. Playwright YouTube) cannot complete.

Environment

• DataConnect.app build dated 2026-04-14 (bundled binary timestamp)
• macOS, Node.js v22.10.0 (embedded in the pkg snapshot)
• Bundled package: @opendatalabs/personal-server-ts-core@0.0.1-canary.92ee4d6

Stack trace

pkg/prelude/bootstrap.js:1872
      throw error;
      ^

Error: Cannot find module '@opendatalabs/personal-server-ts-core/config'
Require stack:
- /snapshot/data-connect/personal-server/dist/bundle.cjs
1) If you want to compile the package/file into executable, please pay attention
   to compilation warnings and specify a literal in 'require' call.
2) If you don't want to compile the package/file into executable and want to
   'require' it from filesystem (likely plugin), specify an absolute path in
   'require' call using process.cwd() or process.execPath.
    at Function.<anonymous> (node:internal/modules/cjs/loader:1253:15)
    at Function._resolveFilename (pkg/prelude/bootstrap.js:1955:46)
    at _M._resolveFilename (/snapshot/data-connect/personal-server/dist/bundle.cjs:1:467)
    at Function._load (node:internal/modules/cjs/loader:1079:27)
    at TracingChannel.traceSync (node:diagnostics_channel:315:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:218:24)
    at Module.<anonymous> (node:internal/modules/cjs/loader:1344:12)
    at Module.require (pkg/prelude/bootstrap.js:1851:31)
    at require (node:internal/modules/helpers:141:16)
    at Object.<anonymous> (/snapshot/data-connect/personal-server/dist/bundle.cjs:34:21) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/snapshot/data-connect/personal-server/dist/bundle.cjs' ],
  pkg: true
}

Node.js v22.10.0

Root cause analysis

1. packages/core/package.json is "type": "module" and declares its public surface via the exports field, including:

   "./config": {
     "types": "./dist/config/index.d.ts",
     "import": "./dist/config/index.js"
   }

   There is no "require" entry and no "default" fallback. The package is therefore ESM-only at the subpath-exports layer.

2. The DataConnect bundle is CJS (dist/bundle.cjs) compiled with pkg. At runtime it does a bare require('@opendatalabs/personal-server-ts-core/config').

3. pkg is archived (https://github.com/vercel/pkg, deprecated 2023) and its snapshot loader does not honor the exports field for subpath resolution — it relies on legacy CJS resolution (looking for package.json#main, index.js, or whatever asset paths were declared at compile time). Because ./config is only declared via the import condition, pkg never copies dist/config/index.js into the snapshot, and the CJS require from inside the snapshot cannot resolve it.

4. The files DO exist outside the snapshot at Contents/Resources/personal-server/dist/node_modules/@opendatalabs/personal-server-ts-core/dist/config/index.js, but pkg's bare-specifier resolver does not fall back to the real filesystem.

So this is a packaging contract mismatch: a CJS executable requiring subpath exports from an ESM-only package, glued together by a bundler that doesn't understand modern exports.

Suggested fixes (any one is sufficient)

On the personal-server side (cleanest):

• Add CJS conditions to every subpath export in packages/core/package.json, e.g.:

  "./config": {
    "types": "./dist/config/index.d.ts",
    "import": "./dist/config/index.js",
    "require": "./dist/config/index.cjs"
  }

  and emit both ESM and CJS builds. This unblocks any CJS consumer (including legacy bundlers) regardless of which packager DataConnect uses downstream.

On the DataConnect bundling side:

• Stop using pkg (archived). Move to one of:
  • @yao-pkg/pkg (active fork) — same UX, better modern Node support, can be told about subpath exports via pkg.assets.
  • bun build --compile — first-class ESM, no snapshot weirdness.
  • Node.js Single Executable Applications (SEA) — official, works with ESM if you ship a small loader.
• If staying on pkg short-term, add the subpath asset explicitly to the pkg.assets array in the build config so the snapshot includes node_modules/@opendatalabs/personal-server-ts-core/dist/config/index.js and any other consumed subpath exports.

Belt-and-suspenders:

• Add a CI smoke test that boots the compiled binary on a clean macOS runner and hits the /health endpoint. Today's failure is a startup crash, so the cheapest test catches it.

Repro

1. Install DataConnect.app (build 2026-04-14).
2. Launch it and start a connector flow that boots the personal server on port 8080.
3. Observe the personal server exits within ~1s with the stack above (visible in the DataConnect logs, source dataconnect_lib::commands::server).

Workarounds tried

• None work from the consumer side — the failing require is inside the frozen pkg snapshot, so patching files on disk under Contents/Resources/personal-server/ has no effect.

Happy to test a candidate fix build.