feat(bb): migrate WASM toolchain from wasi-sdk to Emscripten#22815
Open
AztecBot wants to merge 16 commits into
Open
feat(bb): migrate WASM toolchain from wasi-sdk to Emscripten#22815AztecBot wants to merge 16 commits into
AztecBot wants to merge 16 commits into
Conversation
AztecBot
added
ci-draft
zac-williamson
added
ci-full
…arding, pthread stack, library main override
….sh $USER # Conflicts: # barretenberg/cpp/src/barretenberg/polynomials/backing_memory.cpp
…asm_bin exec `bb` (built by the wasm-threads preset) links against the `barretenberg` target. Previously under WASM, `barretenberg` was an `add_executable` and the link line emits `-lbarretenberg`, which wasm-ld fails to resolve. Split the target: `barretenberg` is always a static library (so `bb` and other consumers can link it), and the WASM bundle is produced by a new `barretenberg_wasm_bin` executable target with `OUTPUT_NAME barretenberg` (so artifacts on disk remain `barretenberg.js` / `barretenberg.wasm`). Also pin bb / bb-avm WASM link options: -sNODERAWFS=1, -sPROXY_TO_PTHREAD=0, -sALLOW_BLOCKING_ON_MAIN_THREAD=1 — bb has main() and reads CRS from disk; the toolchain default proxies main onto a worker which silently zeros file ops under NODERAWFS (Emscripten #19330).
…meta.url `run_test.sh` runs jest with `--rootDir ./dest/node` from `barretenberg/ts/`, so the test file at runtime lives at `dest/node/barretenberg/clean_shutdown.test.js` and `path.resolve(import.meta.url, '..', '..')` lands in `dest/`, where the harness source does not exist (it stays in `src/`). The test would silently time out at 30s. Use `process.cwd()` (set by run_test.sh to the package root) and join from there.
2 tasks
…m targets preset_cache_paths only matched targets by name + native sibling extensions (.exe, .node, lib*.a). Emscripten emits a .js loader and .worker.mjs pthread worker alongside every .wasm exec target as a unit, so cache_upload missed them and a wasm-threads cache hit restored barretenberg.wasm without the matching barretenberg.js. bb-ts/scripts/copy_wasm.sh then unconditionally copies cpp/build-wasm-threads/ bin/barretenberg.js into dest/<flavor>/barretenberg_wasm/, which fast-mode CI hits before any cache miss can rebuild the cpp side, producing: cp: cannot stat '../cpp/build-wasm-threads/bin/barretenberg.js': No such file or directory Extend preset_cache_paths so any target ending in .wasm also picks up the $stem.js and $stem.worker.mjs siblings from the same bin/ directory.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Applies the diff from
AztecProtocol/barretenberg-claude#2921(wasi-sdk → Emscripten migration) on top ofnext. Source patch (mirror): https://gist.githubusercontent.com/zac-williamson/8eb9ee8a7b0b384b245456dfe34b88bb/raw/fd8eaeeb1f3d66fcfa8127e4d07d257d0171e79b/gistfile1.txtWhat this changes
wasi-sdk(wasmtime runtime) to Emscripten / emsdk + Node runtime..emsdk-version(4.0.7) is the single source of truth, consumed bybootstrap.sh, build-images, and container setup.barretenberg/cpp/scripts/wasm-runNode-based runner replacesbarretenberg/cpp/scripts/wasmtime.sh(deleted).barretenberg/cpp/cmake/toolchains/wasm-emscripten.cmake;wasm32-wasi.cmakedeleted.barretenberg/cpp/src/barretenberg/wasi/*glue (wasi_stubs.cpp,wasm_init.cpp) with the newwasm_threads_tests/harness (memory growth + thread-pool exhaustion tests).barretenberg/ts/src/barretenberg_wasm/**— drops thefactory/browser/,barretenberg_wasm_thread/,fetch_code/,helpers/browser/shims that were specific to the old wasi flow; addsclean_shutdown+reentrytest harnesses.bb_backends/reorganisation (browser / node split,wasm.ts).scripts/setup-container.shupdated; node version floor lowered to 22 (Emscriptenworker_threadsrequirement).Manual fixups vs the upstream patch
3 hunks rejected against
nextdue to context drift, all resolved by hand:barretenberg/cpp/cmake/arch.cmake— drops-fno-exceptionsfrom the WASMadd_compile_optionsline; exception handling is now owned by the new wasm-emscripten toolchain.barretenberg/cpp/scripts/ci_benchmark_ivc_flows.sh—WASMTIME_ALLOWED_DIRS→BB_WASM_ALLOWED_DIRS,scripts/wasmtime.sh→scripts/wasm-run.bootstrap.sh— first hunk's context lineexpected_abs_yarn_version=4.13.0doesn't exist onnext(we have4.5.2). Applied the actual changes (expected_min_node_version→ 22.0.0,expected_abs_wasi_version→expected_abs_emsdk_version) by hand and left the yarn version untouched.Hunks intentionally not applied
barretenberg/.claude/skills/{benchmark-chonk,profile-chonk,remote-bench}/SKILL.md— these are modifications (not new files) but the skills don't exist onnext(onlystdlib-point-at-infinityandsumcheckdo). The diff is delta-only, so I can't reconstruct them. If those skills should land here, push the full SKILL.md content separately.docs/network_versioned_docs/version-v4.2.0/operators/setup/building-from-source.md—version-v4.2.0doesn't exist underdocs/network_versioned_docs/onnext(onlyversion-v2.1.11-ignitionandversion-v4.1.0-rc.2). The matching change to the livedocs/docs-operate/operators/setup/building-from-source.mddid apply.Container-policy adjustment — please move before merging
.github/workflows/wasm-emscripten.yml(320 lines, the newwasm-grep-gate/wasm-threaded-tests/wasm-perf-gate/legacy-toolchain-compatworkflow) was relocated to.github-new/workflows/wasm-emscripten.ymlbecause this session was not started withci-allow. Move it back before this merges:The file content is unchanged.
Suspected scratch files at the repo root — please review
These were in the source diff and look like Claude's own iterative review notes. They were applied as-is (they're not obviously CLAUDE.md-style files Claude would skip), but they don't seem like intended PR content:
Finalise.mdREVIEW_ITER_1.mdREVIEW_ITER_2.mdREVIEW_ITER_3.mdStrip them on the branch if they shouldn't ship.
Not built / not tested
This is a large multi-toolchain migration; the build container has no Docker, no preinstalled emsdk, and the new tests need the new
wasm-threadsEmscripten preset. Verification belongs in CI.Sanity / application notes
Full notes (including totals, every skipped hunk, and what the diff does per area) are in:
https://gist.github.com/AztecBot/5f8c2c2c87f0580383b9ba341bec8ef8
ClaudeBox log: https://claudebox.work/s/a62d1521c38e34c3?run=3