Skip to content

feat: migrate from Node SEA to Bun compile#185

Closed
bobbyg603 wants to merge 8 commits into
mainfrom
migrate-sea-to-bun-compile
Closed

feat: migrate from Node SEA to Bun compile#185
bobbyg603 wants to merge 8 commits into
mainfrom
migrate-sea-to-bun-compile

Conversation

@bobbyg603
Copy link
Copy Markdown
Member

@bobbyg603 bobbyg603 commented Mar 4, 2026

Summary

  • Replace the 5-tool Node SEA pipeline (ncc, bpkg, sea-config, postject, codesign) with Bun's --compile, which natively embeds N-API .node addons and produces smaller binaries (~68MB vs ~106MB)
  • Remove node:sea from preload.ts and command-line-definitions.ts, simplifying both significantly
  • Rewrite CI workflow to use oven-sh/setup-bun + bun build --compile
  • Add a patch for workerpool to fix an eval()-based require that doesn't work in Bun compiled binaries

Details

New file: bin/compile-entry.ts — Bun compile entry point that embeds the pre-bundled compression worker via import with { type: "file" } and stores its path on globalThis.__embeddedWorkerPath. This file is excluded from tsc so the npm package path is unaffected.

Removed: sea-config.json, sea/macos.sh, sea/linux.sh, sea/windows.ps1, and devDependencies @vercel/ncc, bpkg, postject, npm-run-all, @mapbox/node-pre-gyp

workerpool patch: workerpool uses eval("typeof require !== 'undefined' ? require : ...") to hide require from webpack. In Bun's compiled binary, require isn't available inside eval(), so the patch replaces this with a direct module.exports = require.

Blocked

This PR is blocked by a Bun bug where Bun.file() as a fetch() body segfaults in compiled binaries (works fine with bun run). This affects S3 presigned URL uploads of symbol files.

Test plan

  • tsc --noEmit passes (npm package path unaffected)
  • vitest run — all 43 tests pass
  • bun build --compile produces working binary
  • ./dist/symbol-upload -h shows help with correct version
  • ./dist/symbol-upload -m -l ./tmp/local -d ./spec/support -f "**/*.dSYM" exercises native addon loading + dump_syms + file output
  • CI matrix build (ubuntu, macos, macos-intel, windows)
  • S3 upload (blocked by Bun bug)

🤖 Generated with Claude Code

bobbyg603 and others added 3 commits March 4, 2026 11:51
Replace the 5-tool Node SEA pipeline (ncc, bpkg, sea-config, postject,
codesign) with Bun's --compile which natively embeds N-API addons and
produces smaller binaries (~68MB vs ~106MB).

- Rewrite src/preload.ts: remove node:sea, simplify importNodeDumpSyms
  to a plain import, use globalThis for embedded worker path in compiled mode
- Add bin/compile-entry.ts: Bun compile entry point that embeds the
  pre-bundled worker via import with { type: "file" }
- Simplify bin/command-line-definitions.ts: replace node:sea with JSON import
- Add resolveJsonModule to tsconfig.json
- Replace SEA build scripts with bundle:worker + build:compile
- Remove devDependencies: @vercel/ncc, bpkg, postject, npm-run-all,
  @mapbox/node-pre-gyp
- Delete sea-config.json and sea/ directory
- Rewrite CI workflow for Bun (oven-sh/setup-bun, bun install, bun build)
- Add workerpool patch: replace eval-based requireFoolWebpack with direct
  require for Bun compile compatibility

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bun's fetch doesn't support ReadableStreams created via Node's
ReadStream.toWeb() as PUT request bodies (duplex streaming issue).
Use Bun.file().stream() which produces a proper web-standard
ReadableStream that works with Bun's fetch, while keeping
ReadStream.toWeb() for Node.js. Both paths are streaming — no
buffering the entire file into memory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- compile-entry.ts: use dynamic import so __embeddedWorkerPath is set
  before the app starts (static imports are hoisted before module body)
- preload.ts: check embeddedPath before devPath to avoid finding the
  unbundled source file; use readFileSync for /$bunfs/ virtual filesystem
- worker.ts: return Bun.file() (lazy Blob) instead of .stream() so
  fetch sends proper content-length instead of chunked encoding

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@bobbyg603 bobbyg603 marked this pull request as draft March 26, 2026 19:36
bobbyg603 and others added 2 commits April 17, 2026 22:16
SymbolsApiClient.postSymbols tees the body to verify gzip magic bytes,
which requires a ReadableStream. Returning Bun.file() (a Blob) broke
uploads in the Bun compiled binary with "R.tee is not a function".
Content-Length is already set explicitly in s3-api-client headers,
so Bun.file().stream() is safe and remains lazy/streaming.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bobbyg603
Copy link
Copy Markdown
Member Author

Status update: compiled binary now works end-to-end on Bun canary 1.3.13.

Verified in the compiled binary:

  • .sym upload (direct) — uploads cleanly with tee'd ReadableStream body
  • -m dump_syms → upload — native node-dump-syms module loads and generated sym file uploads successfully

Fix in this round: createFileStream now returns Bun.file(filePath).stream() (a real ReadableStream) instead of a Bun.file() Blob. The js-api-client tees the body to verify gzip magic bytes, so a Blob fails with R.tee is not a function in the minified compiled binary. Content-Length is already set explicitly in the S3 client headers, so the stream path is safe.

Blocker before merging: The fix for oven-sh/bun#20740 (Bun.file() as fetch body segfault in compiled binaries) is currently canary-only. We need to wait for a stable Bun release that includes it, then also pin oven-sh/setup-bun@v2 in .github/workflows/sea.yaml to that version (or leave it unpinned once stable carries the fix).

Tests: 44/44 passing locally after merging main.

bobbyg603 and others added 3 commits April 20, 2026 10:49
Bun 1.3.13 is the first stable release containing the fix for
oven-sh/bun#20740 (Bun.file() as fetch body segfault in compiled
binaries), which this branch depends on.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bun skips install/postinstall lifecycle scripts by default. node-dump-syms
uses its install script to download the prebuilt native .node binary via
node-pre-gyp. Without this, fresh bun install on CI or new machines leaves
the native binary missing and bun build --compile fails to resolve
../native/index.node.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bobbyg603 bobbyg603 marked this pull request as ready for review April 20, 2026 15:19
Copilot AI review requested due to automatic review settings April 20, 2026 15:19
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the project’s single-binary build from Node SEA to Bun’s bun build --compile, simplifying the build pipeline and adding a workerpool patch to support Bun-compiled binaries.

Changes:

  • Replaces the Node SEA toolchain and scripts with a Bun-based compile workflow (bun build --compile) and updated CI.
  • Adds a Bun compile entrypoint that embeds the pre-bundled compression worker and exposes its path via globalThis.__embeddedWorkerPath.
  • Updates upload streaming to support Bun (via Bun.file(...).stream()) and adds a workerpool patch to avoid eval()-based require.

Reviewed changes

Copilot reviewed 10 out of 15 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tsconfig.json Enables resolveJsonModule to support JSON imports used by updated CLI metadata.
src/worker.ts Switches upload file streaming abstraction to support Bun vs Node runtimes.
src/preload.ts Reworks compression worker path resolution to support Bun compiled asset extraction.
spec/worker.spec.ts Updates tests to match the new stream/destroy abstraction.
bin/compile-entry.ts New Bun compile entrypoint embedding the worker bundle and importing the CLI.
bin/command-line-definitions.ts Simplifies version resolution via package.json import (currently broken for dist output).
.github/workflows/sea.yaml Replaces Node SEA workflow steps with Bun install + bun build --compile.
patches/workerpool@6.5.1.patch Patches workerpool to avoid eval() require indirection for Bun compile.
package.json Removes SEA-related scripts/deps; adds Bun compile scripts and patched dependency metadata.
package-lock.json Updates lockfile to reflect dependency changes (SEA toolchain removals, node-dump-syms bump, etc.).
bun.lock Adds Bun lockfile for Bun-based install/compile flow.
sea-config.json, sea/*.sh, sea/windows.ps1 Removes obsolete Node SEA configuration/scripts.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/preload.ts
import { existsSync, mkdirSync, writeFileSync } from "node:fs";
import { mkdir, writeFile } from "node:fs/promises";
import { createRequire } from "node:module";
import { copyFileSync, existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copyFileSync is imported but never used (only referenced in a comment). Please remove it to avoid dead imports and keep the module surface minimal.

Suggested change
import { copyFileSync, existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";

Copilot uses AI. Check for mistakes.
@bobbyg603
Copy link
Copy Markdown
Member Author

made the binaries smaller on macos and linux but made the windows binary bigger. since windows is our most popular platform this change kind of defeats the purpose. closing for now

@bobbyg603 bobbyg603 closed this Apr 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants