fix(NODE-7603): emit dynamic import('os') via tsconfig #4992
Conversation
etc/ isn't covered by check:eslint, so these were never CI-enforced, but manually running eslint against them surfaced real issues: bare `process` global (no-restricted-globals — repo convention imports it explicitly), a TSDoc-malformed-tag warning from an unbacktick'd `@__PURE__` in a doc comment, and a few prettier formatting deviations. Fixed all three by hand rather than expanding lint scope to etc/.
…cation Adds etc/lib-emit-classify.mjs: parses every old/new lib/ file pair with the TypeScript compiler API and checks two things instead of trusting a sampled classification. Check 1 (hard gate): the set of exported names is identical between pipelines per file — this is what would catch esbuild silently dropping or renaming an export. Check 2 (informational): after stripping only statement shapes that are provably safe plumbing (esbuild's interop preamble, the export wiring block, tsc's TDZ/self-assign export wiring, barrel-file getter re-exports, void 0/undefined), reports the residual diff honestly rather than forcing a false "clean". Also fixes a real bug surfaced along the way: .diff-gate/ is gitignored, so Prettier's default ignore rules were silently skipping normalization the whole time. Fixing that (via --ignore-path) shrank the raw baseline diff from 51,602 to 32,119 lines before classification even runs. Real result: 131/131 files pass the export-set check; residual after stripping known-safe transformations is 7,610 lines (down from 32,119). Both checks verified with deliberate mutations (a dropped export, an injected unexplained statement) to confirm they actually catch regressions rather than always reporting clean.
The previous commit's residual-comparison design wrote all 131 files' stripped-and-printed output to a scratch directory under .diff-gate/ and normalized them with one batched `npx prettier --write` subprocess call. Repeated runs proved this nondeterministic: prettier's own glob-matching intermittently failed to see files that had just been written and confirmed present via readdir, producing a stale/partial read on the next pass and flipping allExportChecksPass between true and false on byte-identical input. A classifier whose entire purpose is mechanical proof cannot have that failure mode. Fixes it by removing the temp-file/subprocess step entirely: prettier's own formatting API is called in-process on in-memory strings instead. No new dependency (prettier is already a devDependency used elsewhere in this exact script). Verified deterministic across many repeated full rebuilds and two independently-run negative-test mutations (a dropped export, an injected unexplained statement), both still caught correctly. Also documents a real 8th diff class found along the way (tsc's `foo_1` vs esbuild's `import_foo` require-binding naming) that dominates the residual and is deliberately left unstripped, and hardens the gate's top-level directory cleanup with the same ENOTEMPTY retry already used elsewhere in this file, after observing the failure directly.
…ble size
The classifier's residual was ~7,600 lines, dominated by consistent
require-binding renames (tsc's `foo_1` vs esbuild's `import_foo` and the
local-shadowing renames those force). A regex over the diff text cannot
discount renames safely — `value` vs `value2` is textually indistinguishable
from a real change — so this does it soundly at the AST level instead:
- Alpha-renaming: every file-locally declared binding is renamed, on both
sides, to positional canonical names ($1, $2, ... in document order —
interleaved across declaration kinds, since tsc declares imports with
`const` and esbuild with `var`, and phase-ordered numbering would misalign
every file). Property names, exported names, globals, labels, and literal
contents are never renamed, so real changes still surface. Resolution
failures fail visible: they grow the residual, never shrink it.
- tsc's binding-less value exports (`exports.NAME = expr` + `exports.NAME`
reads) are rewritten to the local-const form esbuild uses — sound because
the export-set check has already proven the names identical.
- String/template/numeric literals are compared by cooked value ('\x00' vs
'\0', 1e4 vs 10000); object shorthand is expanded; `__toESM(require(x))`
is unwrapped as a counted, reported rule rather than silently.
Result: 151 residual lines across 11 files (was 7,610; raw diff 32,119),
with 120/131 files byte-identical after normalization. The remainder is
small enough to read exhaustively and falls into three documented
semantics-preserving shapes (esbuild string-constant folding, bare-`var`
declaration-position cascades, `(0, exports.fn)()` call style). Verified
with five mutation tests: a changed property name, an inconsistent rename,
a changed string value, a changed numeric value, and a dropped export all
surface exactly; two full fresh rebuilds reproduce identical output.
Also ignores the .spike/ scratch directory from the earlier Rollup
emitter spike.
041de65 to
3fe931d
Compare
There was a problem hiding this comment.
Pull request overview
This PR fixes ESM-bundled consumption of the driver by switching the default os runtime adapter resolution from require('os') to a dynamic import('os') that survives downstream ESM bundling, and updates the build pipeline to emit lib/ via an ESM intermediate (tsc → ESM, then esbuild → CJS) so that dynamic imports are preserved in the shipped artifact.
Changes:
- Make runtime-adapter resolution asynchronous and load the default
osadapter via dynamicimport('os')to avoidrequirein bundled ESM output. - Replace the
build:tspipeline with a tsc-to-ESM intermediate followed by an esbuild ESM→CJS transform, plus add artifact/diff-gate verification scripts. - Add/adjust unit tests to validate adapter async resolution and prevent regressions when downstream bundlers produce ESM.
Reviewed changes
Copilot reviewed 15 out of 16 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
tsconfig.build.json |
New build tsconfig to emit ESM-shaped JS into an intermediate directory while sending declarations to lib/. |
src/runtime_adapters.ts |
Switch default os adapter loading to dynamic import() and make adapter resolution async. |
src/mongo_client.ts |
Update internal MongoOptions.runtime type to a Promise<Runtime>. |
src/cmap/handshake/client_metadata.ts |
Await runtime before reading the os adapter for client metadata. |
src/cmap/connection.ts |
Update internal connection options type to accept runtime: Promise<Runtime>. |
src/cmap/auth/gssapi.ts |
Await runtime before using os in the GSSAPI auth flow. |
test/unit/runtime_adapters.test.ts |
Update tests to await runtime resolution and validate adapter behavior. |
test/unit/bundling.test.ts |
New unit test that reproduces downstream ESM bundling and verifies no global require is needed. |
test/tools/utils.ts |
Update test runtime helper to match async runtime resolution while keeping a stub-friendly os. |
test/tools/runner/vm_context_helper.ts |
Configure vm harness to support dynamic import() when loading the bundled driver. |
package.json |
Route build:ts through the new build script and add lib artifact/diff-gate checks. |
etc/build-lib.mjs |
New build script: tsc emits ESM intermediate + declarations, esbuild transforms to shipped CJS. |
etc/check-lib-artifact.mjs |
New smoke test validating shipped lib/ loads under both loaders and preserves import('os'). |
etc/diff-lib-emit.mjs |
New migration diff gate to compare old vs new lib/ emits and report residual diffs. |
etc/lib-emit-classify.mjs |
New AST-based classifier to mechanically validate export-surface equivalence across emits. |
.gitignore |
Ignore lib-intermediate/ and .diff-gate/ build artifacts. |
…de-mongodb-native into esm-build-migration-node-7603
import('os') via tsconfig
tadjik1
left a comment
There was a problem hiding this comment.
Great work, thanks @PavelSafronov! It looks really solid.
The only my comments are about testing strategy, I might misunderstood the approach here, but it feels we can leave just a unit test which reads the actual tsconfig, and drop check-lib-artifact entirely.
nbbeeken
left a comment
There was a problem hiding this comment.
Can't comment on it, but this line in connection_string.ts
mongoOptions.runtime = resolveRuntimeAdapters(options);Should mark the promise as handled to prevent the process from crashing if there's a failure
mongoOptions.runtime = resolveRuntimeAdapters(options);
mongoOptions.runtime.then(undefined, squashError);
Good point, this follows our existing approach on line 87 of the same file. Updating. |
Description
Summary of Changes
Load
osruntime adapter via dynamic import, so ESM bundle works again.Notes for Reviewers
This PR supersedes #4979, which was an attempt to solve the same problem.
This PR follows the approach discussed offline: update tsc instructions and verify that we get the correct output from the build.
Verifying changes
The build-affecting change in this PR is three lines of tsconfig (
module: node16,moduleResolution: node16,esModuleInterop: falsepinned to its previous effective value).Its entire effect on the shipped
lib/output is one line in one file:All other 130 emitted files are byte-identical (modulo nothing — same helpers, same export
wiring, same comments). To reproduce this yourself:
Permanent guards (all wired into CI):
test/unit/bundling.test.ts— the NODE-7603 acceptance test: replicates the emit forruntime_adapters.ts, bundles it to ESM the way a downstream Vite/esbuild build would, andruns it with no
requirein scope. Fails on the oldmodule: commonjsemit.etc/check-lib-artifact.mjs(check:lib-artifact, part ofcheck:lint) — loads the builtlib/via bothrequireandimport()and statically asserts the shippedlib/runtime_adapters.jsretainsimport("os")and contains no downleveledrequire("os"). This is the guard wired to the real build output: if tsconfig everregresses to
module: commonjs, this fails even though the bundling test (which mirrors thenode16 settings itself) would not.
test/unit/nodeless.test.ts— audits every dynamic-import specifier in the driver testbundle against the sandbox allowlist (currently just
os), so a future dynamic import of aNode built-in can't silently bypass the nodeless sandbox's restricted
require.Full check status on this change: unit suite 3178/0,
check:ts,check:lint(api-extractor,tsd, eslint),
mongodb.d.tsbyte-identical.What is the motivation for this change?
Slack convo: https://mongodb.slack.com/archives/GGWBN4ZNK/p1782927460341129
Release Highlight
Bundling the driver into ESM no longer throws
ReferenceError: require is not definedv7.2.0 introduced the experimental
runtimeAdaptersoption and, as part of it, replaced the driver’s static import of Node’sosmodule with a runtimerequire('os'). That works in a CommonJS build, but when the driver is bundled into ESM output (e.g. a Vite/esbuild/rollup server build with"type": "module"), there is norequirein module scope, so constructing a client threwReferenceError: require is not defined. The driver now loads the defaultosadapter through a dynamicimport()that survives bundling, sonew MongoClient()works in ESM bundles. CommonJS usage is unchanged, and supplying your ownruntimeAdapters.oscontinues to work.Double check the following
npm run check:lint)type(NODE-xxxx)[!]: descriptionfeat(NODE-1234)!: rewriting everything in coffeescript