Skip to content

fix(NODE-7603): emit dynamic import('os') via tsconfig #4992

Merged
tadjik1 merged 27 commits into
mainfrom
esm-build-migration-node-7603
Jul 10, 2026
Merged

fix(NODE-7603): emit dynamic import('os') via tsconfig #4992
tadjik1 merged 27 commits into
mainfrom
esm-build-migration-node-7603

Conversation

@PavelSafronov

@PavelSafronov PavelSafronov commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Description

Summary of Changes

Load os runtime adapter via dynamic import, so ESM bundle works again.

Notes for Reviewers

This PR supersedes #4979, which was an attempt to solve the same problem.

This PR follows the approach discussed offline: update tsc instructions and verify that we get the correct output from the build.

Verifying changes

The build-affecting change in this PR is three lines of tsconfig (module: node16,
moduleResolution: node16, esModuleInterop: false pinned to its previous effective value).
Its entire effect on the shipped lib/ output is one line in one file:

--- lib/runtime_adapters.js (module: commonjs)
+++ lib/runtime_adapters.js (module: node16)
-        os: options.runtimeAdapters?.os ?? (await Promise.resolve().then(() => require('os')))
+        os: options.runtimeAdapters?.os ?? (await import('os'))

All other 130 emitted files are byte-identical (modulo nothing — same helpers, same export
wiring, same comments). To reproduce this yourself:

npx tsc -p tsconfig.json --module commonjs --moduleResolution node --noEmitHelpers true \
  --outDir /tmp/old-emit --declaration false --declarationMap false --sourceMap false
npm run build:ts
diff -r -x '*.map' -x '*.d.ts' /tmp/old-emit lib   # one hunk, plus sourceMappingURL lines

Permanent guards (all wired into CI):

  • test/unit/bundling.test.ts — the NODE-7603 acceptance test: replicates the emit for
    runtime_adapters.ts, bundles it to ESM the way a downstream Vite/esbuild build would, and
    runs it with no require in scope. Fails on the old module: commonjs emit.
  • etc/check-lib-artifact.mjs (check:lib-artifact, part of check:lint) — loads the built
    lib/ via both require and import() and statically asserts the shipped
    lib/runtime_adapters.js retains import("os") and contains no downleveled
    require("os"). This is the guard wired to the real build output: if tsconfig ever
    regresses to module: commonjs, this fails even though the bundling test (which mirrors the
    node16 settings itself) would not.
  • test/unit/nodeless.test.ts — audits every dynamic-import specifier in the driver test
    bundle against the sandbox allowlist (currently just os), so a future dynamic import of a
    Node built-in can't silently bypass the nodeless sandbox's restricted require.

Full check status on this change: unit suite 3178/0, check:ts, check:lint (api-extractor,
tsd, eslint), mongodb.d.ts byte-identical.

What is the motivation for this change?

Slack convo: https://mongodb.slack.com/archives/GGWBN4ZNK/p1782927460341129

Release Highlight

Bundling the driver into ESM no longer throws ReferenceError: require is not defined

v7.2.0 introduced the experimental runtimeAdapters option and, as part of it, replaced the driver’s static import of Node’s os module with a runtime require('os'). That works in a CommonJS build, but when the driver is bundled into ESM output (e.g. a Vite/esbuild/rollup server build with "type": "module"), there is no require in module scope, so constructing a client threw ReferenceError: require is not defined. The driver now loads the default os adapter through a dynamic import() that survives bundling, so new MongoClient() works in ESM bundles. CommonJS usage is unchanged, and supplying your own runtimeAdapters.os continues to work.

Double check the following

  • Lint is passing (npm run check:lint)
  • Self-review completed using the steps outlined here
  • PR title follows the correct format: type(NODE-xxxx)[!]: description
    • Example: feat(NODE-1234)!: rewriting everything in coffeescript
  • Changes are covered by tests
  • New TODOs have a related JIRA ticket

@PavelSafronov PavelSafronov changed the title Esm build migration node 7603 fix(NODE-7603): build lib via ESM intermediate so os loads via dynamic import Jul 6, 2026
etc/ isn't covered by check:eslint, so these were never CI-enforced, but
manually running eslint against them surfaced real issues: bare `process`
global (no-restricted-globals — repo convention imports it explicitly), a
TSDoc-malformed-tag warning from an unbacktick'd `@__PURE__` in a doc
comment, and a few prettier formatting deviations. Fixed all three by hand
rather than expanding lint scope to etc/.
…cation

Adds etc/lib-emit-classify.mjs: parses every old/new lib/ file pair with the
TypeScript compiler API and checks two things instead of trusting a sampled
classification. Check 1 (hard gate): the set of exported names is identical
between pipelines per file — this is what would catch esbuild silently
dropping or renaming an export. Check 2 (informational): after stripping only
statement shapes that are provably safe plumbing (esbuild's interop preamble,
the export wiring block, tsc's TDZ/self-assign export wiring, barrel-file
getter re-exports, void 0/undefined), reports the residual diff honestly
rather than forcing a false "clean".

Also fixes a real bug surfaced along the way: .diff-gate/ is gitignored, so
Prettier's default ignore rules were silently skipping normalization the
whole time. Fixing that (via --ignore-path) shrank the raw baseline diff from
51,602 to 32,119 lines before classification even runs.

Real result: 131/131 files pass the export-set check; residual after
stripping known-safe transformations is 7,610 lines (down from 32,119).
Both checks verified with deliberate mutations (a dropped export, an
injected unexplained statement) to confirm they actually catch regressions
rather than always reporting clean.
The previous commit's residual-comparison design wrote all 131 files'
stripped-and-printed output to a scratch directory under .diff-gate/ and
normalized them with one batched `npx prettier --write` subprocess call.
Repeated runs proved this nondeterministic: prettier's own glob-matching
intermittently failed to see files that had just been written and confirmed
present via readdir, producing a stale/partial read on the next pass and
flipping allExportChecksPass between true and false on byte-identical input.
A classifier whose entire purpose is mechanical proof cannot have that
failure mode.

Fixes it by removing the temp-file/subprocess step entirely: prettier's own
formatting API is called in-process on in-memory strings instead. No new
dependency (prettier is already a devDependency used elsewhere in this exact
script). Verified deterministic across many repeated full rebuilds and two
independently-run negative-test mutations (a dropped export, an injected
unexplained statement), both still caught correctly.

Also documents a real 8th diff class found along the way (tsc's `foo_1` vs
esbuild's `import_foo` require-binding naming) that dominates the residual
and is deliberately left unstripped, and hardens the gate's top-level
directory cleanup with the same ENOTEMPTY retry already used elsewhere in
this file, after observing the failure directly.
…ble size

The classifier's residual was ~7,600 lines, dominated by consistent
require-binding renames (tsc's `foo_1` vs esbuild's `import_foo` and the
local-shadowing renames those force). A regex over the diff text cannot
discount renames safely — `value` vs `value2` is textually indistinguishable
from a real change — so this does it soundly at the AST level instead:

- Alpha-renaming: every file-locally declared binding is renamed, on both
  sides, to positional canonical names ($1, $2, ... in document order —
  interleaved across declaration kinds, since tsc declares imports with
  `const` and esbuild with `var`, and phase-ordered numbering would misalign
  every file). Property names, exported names, globals, labels, and literal
  contents are never renamed, so real changes still surface. Resolution
  failures fail visible: they grow the residual, never shrink it.
- tsc's binding-less value exports (`exports.NAME = expr` + `exports.NAME`
  reads) are rewritten to the local-const form esbuild uses — sound because
  the export-set check has already proven the names identical.
- String/template/numeric literals are compared by cooked value ('\x00' vs
  '\0', 1e4 vs 10000); object shorthand is expanded; `__toESM(require(x))`
  is unwrapped as a counted, reported rule rather than silently.

Result: 151 residual lines across 11 files (was 7,610; raw diff 32,119),
with 120/131 files byte-identical after normalization. The remainder is
small enough to read exhaustively and falls into three documented
semantics-preserving shapes (esbuild string-constant folding, bare-`var`
declaration-position cascades, `(0, exports.fn)()` call style). Verified
with five mutation tests: a changed property name, an inconsistent rename,
a changed string value, a changed numeric value, and a dropped export all
surface exactly; two full fresh rebuilds reproduce identical output.

Also ignores the .spike/ scratch directory from the earlier Rollup
emitter spike.
@PavelSafronov PavelSafronov force-pushed the esm-build-migration-node-7603 branch from 041de65 to 3fe931d Compare July 6, 2026 21:21
@PavelSafronov PavelSafronov marked this pull request as ready for review July 6, 2026 21:45
@PavelSafronov PavelSafronov requested a review from a team as a code owner July 6, 2026 21:45
Copilot AI review requested due to automatic review settings July 6, 2026 21:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes ESM-bundled consumption of the driver by switching the default os runtime adapter resolution from require('os') to a dynamic import('os') that survives downstream ESM bundling, and updates the build pipeline to emit lib/ via an ESM intermediate (tsc → ESM, then esbuild → CJS) so that dynamic imports are preserved in the shipped artifact.

Changes:

  • Make runtime-adapter resolution asynchronous and load the default os adapter via dynamic import('os') to avoid require in bundled ESM output.
  • Replace the build:ts pipeline with a tsc-to-ESM intermediate followed by an esbuild ESM→CJS transform, plus add artifact/diff-gate verification scripts.
  • Add/adjust unit tests to validate adapter async resolution and prevent regressions when downstream bundlers produce ESM.

Reviewed changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tsconfig.build.json New build tsconfig to emit ESM-shaped JS into an intermediate directory while sending declarations to lib/.
src/runtime_adapters.ts Switch default os adapter loading to dynamic import() and make adapter resolution async.
src/mongo_client.ts Update internal MongoOptions.runtime type to a Promise<Runtime>.
src/cmap/handshake/client_metadata.ts Await runtime before reading the os adapter for client metadata.
src/cmap/connection.ts Update internal connection options type to accept runtime: Promise<Runtime>.
src/cmap/auth/gssapi.ts Await runtime before using os in the GSSAPI auth flow.
test/unit/runtime_adapters.test.ts Update tests to await runtime resolution and validate adapter behavior.
test/unit/bundling.test.ts New unit test that reproduces downstream ESM bundling and verifies no global require is needed.
test/tools/utils.ts Update test runtime helper to match async runtime resolution while keeping a stub-friendly os.
test/tools/runner/vm_context_helper.ts Configure vm harness to support dynamic import() when loading the bundled driver.
package.json Route build:ts through the new build script and add lib artifact/diff-gate checks.
etc/build-lib.mjs New build script: tsc emits ESM intermediate + declarations, esbuild transforms to shipped CJS.
etc/check-lib-artifact.mjs New smoke test validating shipped lib/ loads under both loaders and preserves import('os').
etc/diff-lib-emit.mjs New migration diff gate to compare old vs new lib/ emits and report residual diffs.
etc/lib-emit-classify.mjs New AST-based classifier to mechanically validate export-surface equivalence across emits.
.gitignore Ignore lib-intermediate/ and .diff-gate/ build artifacts.

Comment thread etc/lib-emit-classify.mjs Outdated
Comment thread etc/lib-emit-classify.mjs Outdated
Comment thread test/tools/runner/vm_context_helper.ts
@johnmtll johnmtll added the tracked-in-jira Ticket filed in MongoDB's Jira system label Jul 7, 2026
@PavelSafronov PavelSafronov requested a review from tadjik1 July 7, 2026 22:08
@tadjik1 tadjik1 self-assigned this Jul 8, 2026
@tadjik1 tadjik1 added the Primary Review In Review with primary reviewer, not yet ready for team's eyes label Jul 8, 2026
@tadjik1 tadjik1 changed the title fix(NODE-7603): build lib via ESM intermediate so os loads via dynamic import fix(NODE-7603): emit dynamic import('os') via tsconfig Jul 9, 2026
Comment thread test/unit/bundling.test.ts Outdated
Comment thread etc/check-lib-artifact.mjs Outdated

@tadjik1 tadjik1 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, thanks @PavelSafronov! It looks really solid.
The only my comments are about testing strategy, I might misunderstood the approach here, but it feels we can leave just a unit test which reads the actual tsconfig, and drop check-lib-artifact entirely.

@tadjik1 tadjik1 requested a review from nbbeeken July 9, 2026 10:07

@nbbeeken nbbeeken left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't comment on it, but this line in connection_string.ts

mongoOptions.runtime = resolveRuntimeAdapters(options);

Should mark the promise as handled to prevent the process from crashing if there's a failure

mongoOptions.runtime = resolveRuntimeAdapters(options);
mongoOptions.runtime.then(undefined, squashError);

@PavelSafronov

Copy link
Copy Markdown
Contributor Author

Can't comment on it, but this line in connection_string.ts

mongoOptions.runtime = resolveRuntimeAdapters(options);

Should mark the promise as handled to prevent the process from crashing if there's a failure

mongoOptions.runtime = resolveRuntimeAdapters(options);
mongoOptions.runtime.then(undefined, squashError);

Good point, this follows our existing approach on line 87 of the same file. Updating.

Comment thread tsconfig.json
@tadjik1 tadjik1 merged commit 52f5e61 into main Jul 10, 2026
31 of 33 checks passed
@tadjik1 tadjik1 deleted the esm-build-migration-node-7603 branch July 10, 2026 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Primary Review In Review with primary reviewer, not yet ready for team's eyes tracked-in-jira Ticket filed in MongoDB's Jira system

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants