Skip to content

Commit cc587ed

Browse files
lodyai[bot]zxch3n
lodyai[bot]
and
zxch3n
authored
fix: improve loro wasm bundler compatibility (#976)
Co-authored-by: Zixuan Chen <zx@loro.dev>
1 parent 19f709d commit cc587ed

14 files changed

Lines changed: 791 additions & 5 deletions

File tree

‎.changeset/fix-loro-vite-rolldown-wasm.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
"loro-crdt": patch
3+
"loro-crdt-map": patch
4+
---
5+
6+
Add a browser package remapping so Vite/Rolldown production builds load WASM without top-level await or circular wasm wrapper chunks.
7+
8+
Also make the base64 entry easier to bundle with plain esbuild, Rollup, and Next.js Webpack by avoiding static Node builtin `require()` calls and top-level await in browser bundles.

‎crates/loro-wasm-map/package.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
"license": "MIT",
1717
"files": [
1818
"bundler",
19+
"browser",
1920
"nodejs",
2021
"web",
2122
"README.md",

‎crates/loro-wasm/README.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,14 @@ The standard build pipeline (`deno run -A ./scripts/build.ts dev|release`) now k
8080

8181
Load the source map in browser devtools; when devtools fetches the debug companion it can map instructions back to Rust source files and line numbers without inflating the shipped `.wasm`.
8282

83+
## Bundler entries
84+
85+
Bare `import { LoroDoc } from "loro-crdt"` in browser bundlers that respect the package `browser` field remaps the WASM glue to a synchronous browser build, which avoids Vite/Rolldown production chunk cycles around `.wasm` wrappers. Runtimes that need native `.wasm` module imports can still use the `bundler` entry, and apps that prefer explicit async initialization can use `loro-crdt/web`.
86+
87+
Vite and Webpack understand `new URL("./loro_wasm_bg.wasm", import.meta.url)` and emit the WASM asset automatically. Plain esbuild and plain Rollup do not copy that asset by default. For those tools, either import `loro-crdt/base64` to inline the WASM into the JS bundle without top-level await, or keep the default `loro-crdt` import and copy `node_modules/loro-crdt/browser/loro_wasm_bg.wasm` next to the emitted JS bundle as a build step.
88+
89+
Next.js Turbopack can use the default browser entry. If a Next.js Webpack build resolves the `bundler` entry instead of the package `browser` remap, use `loro-crdt/base64`.
90+
8391
# Example
8492

8593
[![Open in StackBlitz](https://developer.stackblitz.com/img/open_in_stackblitz.svg)](https://stackblitz.com/edit/loro-basic-test?file=test%2Floro-sync.test.ts)

‎crates/loro-wasm/package.json‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,10 +15,15 @@
1515
"url": "git+https://github.com/loro-dev/loro.git"
1616
},
1717
"main": "nodejs/index.js",
18+
"browser": {
19+
"./bundler/loro_wasm": "./browser/loro_wasm.js",
20+
"./bundler/loro_wasm.js": "./browser/loro_wasm.js"
21+
},
1822
"module": "bundler/index.js",
1923
"types": "bundler/index.d.ts",
2024
"files": [
2125
"./bundler",
26+
"./browser",
2227
"./nodejs",
2328
"./web",
2429
"./base64",

‎crates/loro-wasm/rollup.config.mjs‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,9 @@ export default [
3434
// ESM for Web
3535
createConfig('es', 'ES2020', 'web'),
3636

37+
// ESM for browser bundlers that do not support top-level await.
38+
createConfig('es', 'ES2020', 'browser'),
39+
3740
// ESM for bundler
3841
createConfig('es', 'ES2020', 'bundler'),
3942
];

‎crates/loro-wasm/scripts/browser_patch.js‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
import * as imports from "./loro_wasm_bg.js";
2+
3+
const WASM_IMPORTS = {
4+
"./loro_wasm_bg.js": imports,
5+
};
6+
7+
const finalize = (exports) => {
8+
imports.__wbg_set_wasm(exports);
9+
tryStart(imports);
10+
};
11+
12+
function tryStart(imports) {
13+
if (typeof imports.__wbindgen_start === "function") {
14+
imports.__wbindgen_start();
15+
}
16+
}
17+
18+
// Keep this entry synchronous without top-level await. Vite/Rolldown can
19+
// otherwise create circular wasm wrapper chunks in production builds.
20+
function loadWasmBytesSync(url) {
21+
if (typeof XMLHttpRequest !== "function") {
22+
throw new Error(
23+
"loro-crdt browser build requires XMLHttpRequest for synchronous WASM loading. Use the nodejs, web, base64, or bundler entry for this runtime.",
24+
);
25+
}
26+
27+
const request = new XMLHttpRequest();
28+
request.open("GET", url, false);
29+
request.responseType = "arraybuffer";
30+
request.send(null);
31+
32+
if (request.status !== 0 && (request.status < 200 || request.status >= 300)) {
33+
throw new Error(
34+
`Failed to load loro-crdt WASM from ${url}: ${request.status} ${request.statusText}`,
35+
);
36+
}
37+
38+
if (!(request.response instanceof ArrayBuffer)) {
39+
throw new Error(
40+
"Failed to load loro-crdt WASM: response is not an ArrayBuffer",
41+
);
42+
}
43+
44+
return request.response;
45+
}
46+
47+
function instantiateSync(bytes, importObject) {
48+
const module = new WebAssembly.Module(bytes);
49+
return new WebAssembly.Instance(module, importObject);
50+
}
51+
52+
const wasmUrl = new URL("./loro_wasm_bg.wasm", import.meta.url);
53+
const instance = instantiateSync(loadWasmBytesSync(wasmUrl.href), WASM_IMPORTS);
54+
55+
finalize(instance.exports);
56+
57+
export * from "./loro_wasm_bg.js";

‎crates/loro-wasm/scripts/build.ts‎

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ const __dirname = path.dirname(path.fromFileUrl(import.meta.url));
1414

1515
// deno run -A build.ts debug
1616
// deno run -A build.ts release
17+
// deno run -A build.ts release browser
1718
// deno run -A build.ts release web
1819
// deno run -A build.ts release nodejs
1920
let profile = "dev";
@@ -22,7 +23,7 @@ if (Deno.args[0] == "release") {
2223
profile = "release";
2324
profileDir = "release";
2425
}
25-
const TARGETS = ["bundler", "nodejs", "web"];
26+
const TARGETS = ["bundler", "browser", "nodejs", "web"];
2627
const startTime = performance.now();
2728
const LoroWasmDir = path.resolve(__dirname, "..");
2829
const WorkspaceCargoToml = path.resolve(__dirname, "../../../Cargo.toml");
@@ -224,8 +225,9 @@ async function buildTarget(target: string) {
224225
}
225226

226227
// TODO: polyfill FinalizationRegistry
228+
const bindgenTarget = target === "browser" ? "bundler" : target;
227229
const cmd =
228-
`wasm-bindgen --keep-debug --weak-refs --target ${target} --out-dir ${target} ${RawWasmPath}`;
230+
`wasm-bindgen --keep-debug --weak-refs --target ${bindgenTarget} --out-dir ${target} ${RawWasmPath}`;
229231
console.log(">", cmd);
230232
await Deno.run({ cmd: cmd.split(" "), cwd: LoroWasmDir }).status();
231233
console.log();
@@ -255,6 +257,16 @@ async function buildTarget(target: string) {
255257
patch,
256258
);
257259
}
260+
if (target === "browser") {
261+
console.log("🔨 Patching browser target");
262+
const patch = await Deno.readTextFile(
263+
path.resolve(__dirname, "./browser_patch.js"),
264+
);
265+
await Deno.writeTextFile(
266+
path.resolve(targetDirPath, "loro_wasm.js"),
267+
patch,
268+
);
269+
}
258270
}
259271

260272
async function stripReferenceTypesFeatureHint() {

‎crates/loro-wasm/scripts/post-rollup.ts‎

Lines changed: 69 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import { walk } from "https://deno.land/std@0.224.0/fs/mod.ts";
22

3-
const DIRS_TO_SCAN = ["./nodejs", "./bundler", "./web"];
3+
const DIRS_TO_SCAN = ["./nodejs", "./bundler", "./browser", "./web"];
44
const FILES_TO_PROCESS = ["index.js", "index.d.ts"];
55

66
async function replaceInFile(filePath: string) {
@@ -73,7 +73,12 @@ async function rollupBase64() {
7373

7474
const base64IndexPath = "./base64/index.js";
7575
const content = await Deno.readTextFile(base64IndexPath);
76-
const nextContent = injectBase64WasmBranch(content, base64IndexPath);
76+
let nextContent = injectBase64WasmBranch(content, base64IndexPath);
77+
nextContent = simplifyBase64WasmInitialization(
78+
nextContent,
79+
base64IndexPath,
80+
);
81+
nextContent = patchBase64NodeRequires(nextContent, base64IndexPath);
7782
await Deno.writeTextFile(base64IndexPath, nextContent);
7883

7984
await Deno.copyFile("./bundler/loro_wasm.d.ts", "./base64/loro_wasm.d.ts");
@@ -109,13 +114,74 @@ function injectBase64WasmBranch(content: string, filePath: string): string {
109114
return content.replace(bunBranchPattern, base64Branch);
110115
}
111116

117+
function simplifyBase64WasmInitialization(
118+
content: string,
119+
filePath: string,
120+
): string {
121+
const startMarker =
122+
"// Normalize how bundlers expose the wasm module/exports.";
123+
const endMarker = `\n\n/**
124+
* @deprecated Please use LoroDoc
125+
*/`;
126+
const start = content.indexOf(startMarker);
127+
const end = start === -1 ? -1 : content.indexOf(endMarker, start);
128+
if (start === -1 || end === -1) {
129+
throw new Error(
130+
`Could not locate wasm initialization block while patching ${filePath}`,
131+
);
132+
}
133+
134+
const replacement = `// Instantiate the inlined base64 wasm synchronously.
135+
const wasmModuleOrInstance = rawWasm.default({
136+
"./loro_wasm_bg.js": imports,
137+
});
138+
const wasmInstance =
139+
wasmModuleOrInstance instanceof WebAssembly.Instance
140+
? wasmModuleOrInstance
141+
: new WebAssembly.Instance(wasmModuleOrInstance, {
142+
"./loro_wasm_bg.js": imports,
143+
});
144+
__wbg_set_wasm(wasmInstance.exports ?? wasmInstance);
145+
if (typeof imports.__wbindgen_start === "function") {
146+
imports.__wbindgen_start();
147+
}`;
148+
149+
return content.slice(0, start) + replacement + content.slice(end);
150+
}
151+
152+
function patchBase64NodeRequires(content: string, filePath: string): string {
153+
const directRequires = `var fs = require("fs");
154+
var path = require("path");`;
155+
const indirectRequires = `var nodeRequire = typeof require === "function" ? require : null;
156+
var fs = nodeRequire && nodeRequire("fs");
157+
var path = nodeRequire && nodeRequire("path");`;
158+
const browserSafeRequires = `var fs = null;
159+
var path = null;`;
160+
161+
if (content.includes(browserSafeRequires)) {
162+
return content;
163+
}
164+
165+
if (content.includes(directRequires)) {
166+
return content.replace(directRequires, browserSafeRequires);
167+
}
168+
169+
if (content.includes(indirectRequires)) {
170+
return content.replace(indirectRequires, browserSafeRequires);
171+
}
172+
173+
throw new Error(
174+
`Could not locate Node require block while patching ${filePath}`,
175+
);
176+
}
177+
112178
async function main() {
113179
for (const dir of DIRS_TO_SCAN) {
114180
await transform(dir);
115181
}
116182

117183
await rollupBase64();
118-
transform("./base64");
184+
await transform("./base64");
119185
}
120186

121187
if (import.meta.main) {

‎examples/bundler-smoke-tests/.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
.tmp/
2+
node_modules/

‎examples/bundler-smoke-tests/README.md‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Loro Bundler Smoke Tests
2+
3+
This package checks that the published `loro-crdt` JavaScript/WASM package can be
4+
imported by common browser bundlers.
5+
6+
The tests create one temporary project per bundler under `.tmp/`, install the
7+
requested bundler version there, build a tiny app, and inspect the output for the
8+
WASM packaging shape that matters for Loro.
9+
10+
## Usage
11+
12+
Build `crates/loro-wasm` first so `bundler/`, `browser/`, `base64/`, `nodejs/`,
13+
and `web/` artifacts exist:
14+
15+
```sh
16+
pnpm release-wasm
17+
```
18+
19+
Then run:
20+
21+
```sh
22+
pnpm --dir examples/bundler-smoke-tests run test:fast
23+
```
24+
25+
Run Next.js/Turbopack separately because it is heavier:
26+
27+
```sh
28+
pnpm --dir examples/bundler-smoke-tests run test:next
29+
```
30+
31+
To test an already-published package instead of the local workspace build:
32+
33+
```sh
34+
LORO_SMOKE_PACKAGE=loro-crdt@1.12.1 pnpm --dir examples/bundler-smoke-tests run test:fast
35+
```
36+
37+
## Matrix
38+
39+
- `vite5`, `vite6`, `vite7`, `vite8`: bare `import "loro-crdt"`.
40+
- `rolldown-vite`: bare `import "loro-crdt"` against Rolldown's Vite package.
41+
- `webpack5`: bare `import "loro-crdt"`.
42+
- `rsbuild2`: bare `import "loro-crdt"`.
43+
- `rspack2`: bare `import "loro-crdt"`.
44+
- `parcel2`: bare `import "loro-crdt"`.
45+
- `esbuild-default-copy`: bare `import "loro-crdt"` plus a post-build copy of
46+
`browser/loro_wasm_bg.wasm` next to the emitted JS bundle.
47+
- `rollup-default-copy`: bare `import "loro-crdt"` plus the same post-build copy.
48+
- `esbuild-base64`: `import "loro-crdt/base64"` with no external WASM asset.
49+
- `rollup-base64`: `import "loro-crdt/base64"` with no external WASM asset.
50+
- `next16-turbopack`: Next 16 default Turbopack production build with bare
51+
`import "loro-crdt"`.
52+
- `next16-webpack`: Next 16 production build with `--webpack` and
53+
`import "loro-crdt/base64"`.
54+
55+
## Notes
56+
57+
Vite and Webpack understand `new URL("./asset", import.meta.url)` as an asset
58+
reference and emit the `.wasm` file automatically. Plain esbuild and plain Rollup
59+
do not do that by themselves, so they should either use `loro-crdt/base64` for a
60+
single-file bundle with no top-level await, or copy `browser/loro_wasm_bg.wasm`
61+
to the output directory as a build step.
62+
63+
Next 16 Turbopack handles the default browser entry in this smoke test. Next 16's
64+
Webpack build was observed to resolve Loro's bundler entry instead of the package
65+
`browser` remap, so this matrix tests the documented `base64` entry for that
66+
mode.

0 commit comments

Comments
 (0)