Skip to content

test(utils): formalize bundle/snapshot module-loader hooks #6002

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
killagu merged 1 commit into from
Jun 27, 2026
+216 −15
Merged

test(utils): formalize bundle/snapshot module-loader hooks #6002

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 46 additions & 14 deletions packages/typings/src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,55 @@
/**
* Module loader for bundled Egg apps. Called with the `importModule()` filepath
* after POSIX path normalization. Return `undefined` to fall through to the
* standard import path.
* Module loader for bundled Egg apps, registered on `globalThis` as
* `__EGG_BUNDLE_MODULE_LOADER__` (use `setBundleModuleLoader()` from
* `@eggjs/utils`). This is the highest-priority hook: it runs before on-disk
* resolution in both `importModule()` / `importResolve()` (`@eggjs/utils`) and
* the loaders in `@eggjs/core` and the tegg loader.
*
* It is called with the `importModule()` filepath (or a virtual specifier)
* after POSIX path normalization, and is meant to return a module already
* inlined into the bundle — typically a lookup into a static bundle map emitted
* by `egg-bundler`. Return `undefined` to fall through to the next hook (the
* snapshot loader registered via `setSnapshotModuleLoader()`, then
* `__EGG_MODULE_IMPORTER__`) or the standard `import()` / `require()` path.
*
* The non-undefined return value follows the same default-export unwrapping
* rules as a native import (double-default `__esModule` compatibility plus the
* caller's `importDefaultOnly` option).
*/
export type BundleModuleLoader = (filepath: string) => unknown;

/**
* Async module importer override for the tegg loader's file loading.
* Async module importer override, registered on `globalThis` as
* `__EGG_MODULE_IMPORTER__`. When set (and neither the bundle loader nor a
* snapshot loader registered via `setSnapshotModuleLoader()` already resolved
* the path), `importModule()` / the loaders delegate module loading to this
* importer instead of the built-in `await import(filePath)`. The return value
* is awaited and mirrors `await import()` (default unwrapping and
* `importDefaultOnly` apply); because it is awaited, a synchronous return value
* such as the result of `require()` is also valid.
*
* The path passed in depends on the caller: `@eggjs/utils` `importModule()`
* passes the resolved module path from `importResolve()` (OS-native separators,
* not normalized), while the tegg loader passes the original loader filepath
* with separators normalized to POSIX. Importers that care about separators
* should normalize defensively.
*
* Two main uses:
*
* When set, the loader delegates module loading to this importer instead of the
* built-in `await import(filePath)`. Its main use is testing with a bundler-based
* test runner (e.g. Vitest): when an app's egg modules are loaded by the loader
* via the native `import()` while the test file imports the same source through
* the runner's module graph, the two resolve to *different* module instances —
* so a class decorated as an egg proto by the loader is not the same class the
* test references, and `ctx.getEggObject(ClassRef)` fails with "can not get proto".
* 1. Testing with a bundler-based test runner (e.g. Vitest): when an app's egg
* modules are loaded by the loader via the native `import()` while the test
* file imports the same source through the runner's module graph, the two
* resolve to *different* module instances — so a class decorated as an egg
* proto by the loader is not the same class the test references, and
* `ctx.getEggObject(ClassRef)` fails with "can not get proto". A test runner
* injects an importer that routes loading through its own module graph
* (e.g. `filePath => import(filePath)` evaluated inside the runner context),
* keeping a single module instance.
*
* A test runner can inject an importer that routes loading through its own module
* graph (e.g. `filePath => import(filePath)` evaluated inside the runner context),
* keeping a single module instance. Return value mirrors `await import()`.
* 2. V8 startup-snapshot restore: the deserialized main function runs without a
* host dynamic-import callback, so native `import()` throws. The snapshot
* entry generated by `egg-bundler` installs a synchronous `require()`-based
* importer (`createRequire()` over the bundle output dir); `require()` can
* load ESM on Node >= 22, so modules resolve without dynamic import.
*/
export type ModuleImporter = (filePath: string) => Promise<unknown>;

@coderabbitai coderabbitai Bot Jun 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -nP --type=ts -C3 '__EGG_MODULE_IMPORTER__' 
ast-grep run --pattern '$X.__EGG_MODULE_IMPORTER__($_)' --lang typescript

Repository: eggjs/egg

Length of output: 223

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo '--- packages/typings/src/index.ts (relevant section) ---'
sed -n '1,120p' packages/typings/src/index.ts

echo
echo '--- search ModuleImporter / __EGG_MODULE_IMPORTER__ ---'
rg -n --hidden --glob '!**/node_modules/**' --glob '!**/dist/**' --glob '!**/build/**' 'ModuleImporter|__EGG_MODULE_IMPORTER__' packages

echo
echo '--- search related importer usage in packages/utils and packages/typings ---'
rg -n --hidden --glob '!**/node_modules/**' --glob '!**/dist/**' --glob '!**/build/**' 'module[- ]importer|importer|require\(' packages/utils packages/typings

Repository: eggjs/egg

Length of output: 10598

Widen ModuleImporter to accept sync returns

packages/typings/src/index.ts:55 still forces Promise<unknown>, but the contract already allows synchronous require()-style importers. Change it to unknown | Promise<unknown> so callers don’t need a cast.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/typings/src/index.ts` at line 55, The ModuleImporter type is too
narrow because it only allows async importers, while the contract also supports
sync require-style importers. Update the ModuleImporter alias in the typings
entrypoint to accept both synchronous and asynchronous return values, using the
ModuleImporter symbol so callers can pass either form without casting.

43 changes: 43 additions & 0 deletions packages/utils/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,49 @@ The bundle loader is stored on `globalThis`, so bundled and external copies of
default export unwrapping rules as `importModule()`, including
`importDefaultOnly`.

### Bundle / snapshot module-loading hooks

`importModule()` resolves a module through the following hooks, in order. The
first one that produces a value wins; otherwise it falls back to the native
`import()` / `require()` path:

1. **`globalThis.__EGG_BUNDLE_MODULE_LOADER__`** — set via
`setBundleModuleLoader()`. Looks up a module already inlined into the bundle
(typically a static bundle map emitted by `egg-bundler`). Runs before on-disk
resolution and receives the POSIX-normalized `importModule()` filepath or a
virtual specifier. Return `undefined` to fall through.
2. **Snapshot module loader** — set via `setSnapshotModuleLoader()`. This is a
module-local hook (not a `globalThis` global) used by the V8 snapshot entry
generator to serve pre-bundled modules synchronously, keyed by the resolved
path. Once registered it handles every load that reaches it, so the importer
below is not consulted while it is active.
3. **`globalThis.__EGG_MODULE_IMPORTER__`** — an async (or sync, since the value
is awaited) importer that receives the resolved file path (the
`importResolve()` result, with OS-native separators — not normalized). When
set, and the two hooks above did not resolve the module, it replaces the
native `await import(filePath)`.

The bundle loader and importer globals are typed in `@eggjs/typings`
(`BundleModuleLoader` / `ModuleImporter`); import `@eggjs/typings/global` to pick
up the `declare global` augmentation. These hooks are the contract that
`egg-bundler`'s generated entry relies on. `@eggjs/core`'s `ManifestLoaderFS`
consults `__EGG_BUNDLE_MODULE_LOADER__` directly; its importer/native fallback is
reached through `@eggjs/loader-fs`, which calls back into `importModule()`. The
tegg loader (`LoaderUtil.loadFile`) consults both globals directly, passing the
loader filepath with separators normalized to POSIX.

`__EGG_MODULE_IMPORTER__` has two main uses:

- **Bundler-based test runners (e.g. Vitest):** route module loading through the
runner's own module graph so the loader and the test file share a single
module instance (otherwise `ctx.getEggObject(ClassRef)` fails with
"can not get proto").
- **V8 startup-snapshot restore:** the deserialized main function runs without a
host dynamic-import callback, so native `import()` throws. The snapshot entry
installs a synchronous `require()`-based importer (`createRequire()` over the
bundle output dir); `require()` can load ESM on Node >= 22, so modules resolve
without dynamic import.

## License

[MIT](LICENSE)
Expand Down
47 changes: 47 additions & 0 deletions packages/utils/test/fixtures/module-importer-require-esm/run.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
import assert from 'node:assert/strict';
import { createRequire } from 'node:module';
import path from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
import vm from 'node:vm';

import { importModule } from '../../../src/import.ts';

// Reproduces the V8 startup-snapshot restore environment, where the deserialized
// main function runs without a host dynamic-import callback. In that environment
// `import()` is unusable, so the snapshot entry installs a synchronous
// `require()`-based `__EGG_MODULE_IMPORTER__` and relies on `importModule()`
// routing through it. This asserts that contract end to end.

const dirname = path.dirname(fileURLToPath(import.meta.url));
const esmPath = path.resolve(dirname, '../esm/index.js');

// 1) Establish the premise: a context with no `importModuleDynamically` callback
// rejects native `import()` with ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING — the
// same failure a snapshot-restore main function would hit.
const ctx = vm.createContext({});
await assert.rejects(
() => vm.runInContext(`import(${JSON.stringify(pathToFileURL(esmPath).href)})`, ctx),
/dynamic import callback/i,
);

// 2) Install a require-based importer, exactly like the snapshot entry generator
// does (createRequire over the bundle output dir). require() loads ESM
// synchronously on Node >= 22, so no dynamic import callback is needed.
const requireFromHere = createRequire(import.meta.url);
let importerCalls = 0;
globalThis.__EGG_MODULE_IMPORTER__ = (filepath) => {
importerCalls++;
return requireFromHere(filepath);
};

try {
const result = await importModule(esmPath);
// The importer short-circuits before importModule's native `import()` branch,
// so a single importer call proves the ESM module was loaded via require().
assert.equal(importerCalls, 1, 'importer must be used instead of native import()');
assert.equal(result.one, 1);
assert.equal(result.default.foo, 'bar');
console.log('IMPORTER_REQUIRE_ESM_OK');
} finally {
globalThis.__EGG_MODULE_IMPORTER__ = undefined;
}
30 changes: 30 additions & 0 deletions packages/utils/test/module-importer.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
import { strict as assert } from 'node:assert';
import { createRequire } from 'node:module';

import type {} from '@eggjs/typings/global';
import coffee from 'coffee';
import { afterEach, describe, it } from 'vitest';

import { importModule } from '../src/import.ts';
Expand Down Expand Up @@ -53,4 +55,32 @@ describe('test/module-importer.test.ts', () => {
const result = await importModule(getFilepath('esm'));
assert.deepEqual(result, { fromImporter: true });
});

it('loads a real ESM module through a synchronous require-based importer', async () => {
// The importer return value is awaited, so a synchronous `require()` (which
// returns the module synchronously) is a valid importer. require() can load
// ESM on Node >= 22, which is what the snapshot entry relies on.
const require = createRequire(import.meta.url);
let calls = 0;
globalThis.__EGG_MODULE_IMPORTER__ = ((filepath: string) => {
calls++;
return require(filepath);
}) as typeof globalThis.__EGG_MODULE_IMPORTER__;

const result = await importModule(getFilepath('esm'));
assert.equal(calls, 1);
assert.equal(result.one, 1);
assert.deepEqual(result.default, { foo: 'bar' });
});

it('uses a require-based importer when no dynamic import callback exists', async () => {
// Reproduces the V8 snapshot-restore environment in a child process: native
// import() has no host callback, so importModule() must route ESM loading
// through the require-based __EGG_MODULE_IMPORTER__. See the fixture.
await coffee
.spawn(process.execPath, ['--experimental-strip-types', getFilepath('module-importer-require-esm/run.mjs')])
.expect('stdout', /IMPORTER_REQUIRE_ESM_OK/)
.expect('code', 0)
.end();
});
Comment on lines +59 to +85

@gemini-code-assist gemini-code-assist Bot Jun 27, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

These tests rely on experimental Node.js features that are not supported or enabled by default on all Node.js versions:

  1. require() of ESM modules (used in both tests) is only supported on Node.js >= 22.0.0, and is only enabled by default starting from Node.js v22.12.0. On Node.js versions between v22.0.0 and v22.11.0, it requires the --experimental-require-module flag. On older Node.js versions (v18/v20), it is not supported at all.
  2. --experimental-strip-types (used in the second test) was introduced in Node.js v22.6.0.

To prevent the test suite from failing when executed on older Node.js versions (e.g., in CI environments running Node 18 or 20), we should:

  • Dynamically detect support for these features and conditionally skip the tests if they are not supported/enabled.
  • Explicitly pass the --experimental-require-module flag to the spawned child process to ensure it works on Node.js versions between v22.6.0 and v22.11.0.
  const [major, minor] = process.versions.node.split('.').map(Number);
  const supportsRequireEsm =
    major > 22 ||
    (major === 22 && minor >= 12) ||
    process.execArgv.includes('--experimental-require-module');
  const supportsStripTypes = major > 22 || (major === 22 && minor >= 6);

  (supportsRequireEsm ? it : it.skip)('loads a real ESM module through a synchronous require-based importer', async () => {
    // The importer return value is awaited, so a synchronous require() (which
    // returns the module synchronously) is a valid importer. require() can load
    // ESM on Node >= 22, which is what the snapshot entry relies on.
    const require = createRequire(import.meta.url);
    let calls = 0;
    globalThis.__EGG_MODULE_IMPORTER__ = ((filepath: string) => {
      calls++;
      return require(filepath);
    }) as typeof globalThis.__EGG_MODULE_IMPORTER__;

    const result = await importModule(getFilepath('esm'));
    assert.equal(calls, 1);
    assert.equal(result.one, 1);
    assert.deepEqual(result.default, { foo: 'bar' });
  });

  (supportsStripTypes ? it : it.skip)('uses a require-based importer when no dynamic import callback exists', async () => {
    // Reproduces the V8 snapshot-restore environment in a child process: native
    // import() has no host callback, so importModule() must route ESM loading
    // through the require-based __EGG_MODULE_IMPORTER__. See the fixture.
    await coffee
      .spawn(process.execPath, [
        '--experimental-strip-types',
        '--experimental-require-module',
        getFilepath('module-importer-require-esm/run.mjs'),
      ])
      .expect('stdout', /IMPORTER_REQUIRE_ESM_OK/)
      .expect('code', 0)
      .end();
  });

@killagu killagu Jun 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the careful read. This is intentional and safe for this repo's supported baseline, so I'm not adding the version guards:

  • Node baseline. Both the root and @eggjs/utils package.json declare engines.node: ">=22.18.0", and the CI matrix runs only Node 22 and 24 (.github/workflows/ci.yml). Node 18/20 and 22.x < 22.12 are not supported targets, so the tests never run there.
  • require(ESM) is enabled by default since Node 22.12 — well below the >=22.18.0 floor — so no --experimental-require-module flag is needed.
  • Type stripping is on by default since Node 22.18, which is exactly the engines floor. The explicit --experimental-strip-types flag is harmless on 22.18+ and matches the existing precedent in the same package: packages/utils/test/bundle-import.test.ts already spawns fixtures with --experimental-strip-types and no version guards.

Adding it.skip guards only here would be inconsistent with that sibling test and would mask a real failure if the supported baseline ever regressed. If the repo later lowers engines below 22.12, the guards (and the spawned-process flag) would be the right move — but that's a repo-wide policy change, out of scope for this PR.

});
6 changes: 6 additions & 0 deletions wiki/log.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,3 +72,9 @@ Dates use the workspace-local Asia/Shanghai calendar date.
- sources touched: `packages/typings/package.json`, `packages/typings/src/index.ts`, `packages/typings/src/global.ts`, `AGENTS.md`, `CLAUDE.md`
- pages updated: `wiki/index.md`, `wiki/log.md`, `wiki/packages/typings.md`
- note: Recorded `@eggjs/typings` as the shared home for cross-package global typing contracts.

## [2026-06-27] api | formalize bundle/snapshot module-loader hooks

- sources touched: `packages/utils/src/import.ts`, `packages/utils/README.md`, `packages/utils/test/module-importer.test.ts`, `packages/utils/test/fixtures/module-importer-require-esm/run.mjs`, `packages/typings/src/index.ts`
- pages updated: `wiki/log.md`, `wiki/packages/utils.md`
- note: Documented the `__EGG_BUNDLE_MODULE_LOADER__` → snapshot loader (`setSnapshotModuleLoader`) → `__EGG_MODULE_IMPORTER__` → native priority as a formal contract (JSDoc on `BundleModuleLoader`/`ModuleImporter` + README). Added regression coverage for the V8 snapshot-restore path where `__EGG_MODULE_IMPORTER__ = require` loads ESM with no dynamic-import callback (inline sync-require test + spawned `node:vm` fixture). No load-semantics change — types/declarations already existed.
45 changes: 44 additions & 1 deletion wiki/packages/utils.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@ source_files:
- packages/utils/README.md
- packages/utils/src/import.ts
- packages/utils/test/bundle-import.test.ts
updated_at: 2026-05-06
- packages/utils/test/module-importer.test.ts
- packages/typings/src/index.ts
- packages/typings/src/global.ts
updated_at: 2026-06-27
status: active
---

Expand Down Expand Up @@ -36,3 +39,43 @@ ESM loading, `packages/utils/src/import.ts` uses an opaque native dynamic import
created with `new Function('specifier', 'return import(specifier);')`. This
prevents bundlers such as Turbopack from rewriting non-static dynamic import
expressions and preserves Node's native fallback for external ESM modules.

## Module-loader hook priority

`importModule()` resolves through three hooks before the native `import()` /
`require()` fallback (see `packages/utils/src/import.ts`):

1. **`__EGG_BUNDLE_MODULE_LOADER__`** (`setBundleModuleLoader`) — synchronous
`globalThis` bundle-map lookup, runs before on-disk resolution and receives
the POSIX-normalized filepath. Returning `undefined` falls through.
2. **Snapshot loader** (`setSnapshotModuleLoader`) — a module-local hook (not a
`globalThis` global) used by the V8 snapshot entry generator, keyed by the
resolved path. Once registered it handles every load that reaches it, so the
importer below is bypassed while it is active.
3. **`__EGG_MODULE_IMPORTER__`** — async (or sync, since awaited) `globalThis`
importer that receives the resolved path (the `importResolve()` result, with
OS-native separators — _not_ normalized) and replaces native `await import()`.

The bundle loader and importer globals are typed in `@eggjs/typings`
(`BundleModuleLoader`, `ModuleImporter`) and augmented onto `globalThis` in
`@eggjs/typings/global`. `@eggjs/core`'s `ManifestLoaderFS` (`#loadBundledModule`)
consults **only** `__EGG_BUNDLE_MODULE_LOADER__`; the importer/native path is
reached transitively through its `@eggjs/loader-fs` fallback, which calls back
into `importModule()`. The tegg loader (`LoaderUtil.loadFile`) consults both
globals directly, but passes the _original_ loader filepath separator-normalized
to POSIX — not the `importResolve()` output — so the two callers differ in both
which path they pass and whether it is normalized.

`__EGG_MODULE_IMPORTER__` has two contract uses:

- **Bundler-based test runners (Vitest):** route loading through the runner's
module graph so loader and test file share one module instance (otherwise
`ctx.getEggObject(ClassRef)` throws "can not get proto").
- **V8 startup-snapshot restore:** the deserialized main function has no host
dynamic-import callback, so native `import()` throws
(`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING`). The snapshot entry installs a
synchronous `require()`-based importer (`createRequire()` over the output
dir); `require()` loads ESM on Node >= 22. Covered by
`test/module-importer.test.ts` (inline require importer +
`fixtures/module-importer-require-esm/run.mjs`, which asserts the no-callback
premise via `node:vm`).
Loading