fix(turbopack): handle Turbopack hashed external module IDs in workerd#1196
fix(turbopack): handle Turbopack hashed external module IDs in workerd#1196Thereallo1026 wants to merge 2 commits intoopennextjs:mainfrom
Conversation
Turbopack externalizes packages using content-hashed IDs like `shiki-43d062b67f27bbdc/core`. In workerd, `await import(id)` fails for these because no module is registered under the hashed name. The actual package IS bundled in node_modules and accessible via `require()`. This patch detects the Turbopack hash pattern at runtime (`<pkg>-<16+hex chars>[/subpath]`) in the `externalImport` default case, strips the hash, and falls back to `require()` instead. This fixes runtime errors of the form: Failed to load external module shiki-43d062b67f27bbdc/core: Error: No such module "shiki-43d062b67f27bbdc/core" The regex also handles scoped packages (`@scope/pkg-<hash>/subpath`). Non-hashed module IDs continue to use `await import()` unchanged. Adds a spec file with tests covering the regex's behaviour across normal IDs, hashed IDs, and scoped package IDs.
|
|
I realized that #1139 was not merged yet. |
Revise the approach from the previous commit. Instead of patching the
`externalImport` function in [turbopack]_runtime.js (which fails because
workerd can't use CJS require() on ESM-only packages), patch the chunk
files themselves before wrangler bundles them.
Turbopack emits hashed module IDs like `shiki-43d062b67f27bbdc/core` in
`a.y("...")` calls across server chunks ([externals]_*.js and
[root-of-the-server]_*.js). Replace these with:
Promise.resolve().then(() => require("shiki/core"))
The static require() is resolved by wrangler at bundle time, inlining the
real package — the same mechanism as other bundled requires. The
Promise wrapper preserves the async contract of the original a.y() call.
The pathFilter targets all `.next/server/chunks/**/*.js` files and the
contentFilter `a.y("` limits application to files that actually contain
externalImport calls.
Adds patchHashedExternalImports as an exported function for unit testing.
|
Hi human here (Opus degrade is real), the previous commit was wrong Patching But the If we replace them with I added a patch in the second commit targeting all The regex strips Turbopack's content hash ( Tested working with a real fumadocs app on |
No worries @Thereallo1026 and thanks for the PR. |
2 participants
Problem
Turbopack externalizes some packages using content-hashed module IDs like
shiki-43d062b67f27bbdc/core. These are generated by Turbopack's module ID hashing and appear asa.y("shiki-43d062b67f27bbdc/core")calls inside the[externals]_*chunks.In workerd,
await import(id)only works for registered named modules. A hashed ID likeshiki-43d062b67f27bbdc/coreis not a valid module name, so the call fails with:The actual package (
shiki) is bundled in the worker's node_modules and is accessible viarequire("shiki/core")— the hash just needs to be stripped.Fix
The
inlineExternalImportRuleAST patch already wrapsexternalImportin a switch. I've extended thedefaultcase to detect Turbopack's hash pattern at runtime:When detected, it strips the hash and falls back to
require():This handles:
shiki-43d062b67f27bbdc→shikishiki-43d062b67f27bbdc/core→shiki/coreshiki-43d062b67f27bbdc/wasm→shiki/wasm@shikijs/core-43d062b67f27bbdc/dist/index.js→@shikijs/core/dist/index.jsreact,next/dist/...) → unchanged, useawait import()as beforeTests
Added
turbopack.spec.tscovering the regex behaviour for all these cases. All 302 existing tests continue to pass.Reproduction
Any Next.js 16 (Turbopack) app that uses
shiki(e.g. viafumadocs-core) will hit this when deployed to Cloudflare Workers.