Skip to content

loader: add experimental text import#62300

Open
efekrskl wants to merge 7 commits intonodejs:mainfrom
efekrskl:loader/experimental-raw-imports
Open

loader: add experimental text import#62300
efekrskl wants to merge 7 commits intonodejs:mainfrom
efekrskl:loader/experimental-raw-imports

Conversation

@efekrskl
Copy link
Copy Markdown
Contributor

@efekrskl efekrskl commented Mar 17, 2026

Closes #62082

Adds support for text import under --experimental-import-text flag.

Example:

import text from './file.txt' with { type: 'text' };

Import text is a Stage 3 TC39 proposal

Deno supports text imports with --unstable-raw-imports.
Bun supports text imports directly.

Not entirely sure what the current stance is on not-yet-landed proposals, but putting this up for discussion and feedback.

@nodejs-github-bot
Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/config
  • @nodejs/loaders

@nodejs-github-bot nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Mar 17, 2026
@bakkot
Copy link
Copy Markdown
Contributor

bakkot commented Mar 18, 2026

Note that the import-bytes proposal requires producing Uint8Arrays backed by immutable ArrayBuffers, which are not yet supported in V8. I would be hesitant to ship this with mutable backing buffers, even flagged as experimental, lest people come to rely on that.

Text imports don't have the same problem.

@efekrskl efekrskl changed the title loader: add experimental text and bytes import loader: add experimental text import Mar 20, 2026
@efekrskl efekrskl marked this pull request as ready for review March 20, 2026 18:40
@efekrskl
Copy link
Copy Markdown
Contributor Author

Note that the import-bytes proposal requires producing Uint8Arrays backed by immutable ArrayBuffers, which are not yet supported in V8. I would be hesitant to ship this with mutable backing buffers, even flagged as experimental, lest people come to rely on that.

Text imports don't have the same problem.

That's a great point, looks like I've missed that. I've removed bytes related part of the code. Thank you!

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 20, 2026

Codecov Report

❌ Patch coverage is 94.33962% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.70%. Comparing base (06a8240) to head (4a5953c).
⚠️ Report is 76 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/modules/esm/load.js 78.57% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #62300      +/-   ##
==========================================
+ Coverage   89.68%   89.70%   +0.01%     
==========================================
  Files         676      692      +16     
  Lines      206578   214212    +7634     
  Branches    39555    41123    +1568     
==========================================
+ Hits       185267   192154    +6887     
- Misses      13446    14112     +666     
- Partials     7865     7946      +81     
Files with missing lines Coverage Δ
lib/internal/modules/esm/assert.js 100.00% <100.00%> (ø)
lib/internal/modules/esm/resolve.js 99.05% <100.00%> (+<0.01%) ⬆️
lib/internal/modules/esm/translators.js 97.70% <100.00%> (+0.03%) ⬆️
src/node_options.cc 76.49% <100.00%> (+0.04%) ⬆️
src/node_options.h 97.96% <100.00%> (+0.03%) ⬆️
lib/internal/modules/esm/load.js 90.67% <78.57%> (-0.81%) ⬇️

... and 110 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@GeoffreyBooth
Copy link
Copy Markdown
Member

Note that the import-bytes proposal requires producing Uint8Arrays backed by immutable ArrayBuffers, which are not yet supported in V8. I would be hesitant to ship this with mutable backing buffers, even flagged as experimental, lest people come to rely on that.
Text imports don't have the same problem.

That's a great point, looks like I've missed that. I've removed bytes related part of the code. Thank you!

This is really good knowledge that would be good to preserve somewhere. Maybe add a comment near the most appropriate spot in the new code, such as where the bytes support would be, to note that we shouldn’t add bytes support before immutable ArrayBuffers exist in V8? That way if someone later on comes by to add bytes, they'll stumble upon the comment in the code.

Comment on lines +81 to +98
function getFormatFromImportAttributes(importAttributes) {
if (
!importAttributes ||
!ObjectPrototypeHasOwnProperty(importAttributes, 'type') ||
typeof importAttributes.type !== 'string'
) {
return undefined;
}

if (
getOptionValue('--experimental-import-text') &&
importAttributes.type === 'text'
) {
return 'text';
}

return undefined;
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already support with { type: 'json' }. This seems to introduce a new pattern, rather than following the existing one?

Copy link
Copy Markdown
Contributor Author

@efekrskl efekrskl Mar 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, the reason is JSON imports are format led (.json/MIME), whereas for text imports anything can be imported as text (regardless of .txt/MIME). So we needed to have a separate mapping

I'm open for ideas though. Maybe we can make this more clear with a comment

Copy link
Copy Markdown
Member

@GeoffreyBooth GeoffreyBooth Mar 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that it should have a new mapping, but in general we want to follow the existing patterns of the cocebase. Don’t create a new code path for handling modules of varying formats; we already have a path for with { type: 'json' } so text should work the same way, only diverging where it needs to because of the type. For example, JSON didn’t require getFormatFromImportAttributes so this function shouldn’t be needed to add support for text.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, thanks for the guidance.

I've updated the patch, wdyt?

Copy link
Copy Markdown

@pdeveltere pdeveltere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in loader.js, you can add:

/**
 * @typedef {'builtin'|'commonjs'|'json'|'module'|'text'|'wasm'} ModuleFormat
 */

function getFileProtocolModuleFormat(url, context = { __proto__: null }, ignoreErrors) {
const { source } = context;
const ext = extname(url);

Copy link
Copy Markdown

@pdeveltere pdeveltere Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// If the caller explicitly requests text format via import attributes, honour it regardless of file extension.
if (context.importAttributes?.type === 'text') {
return 'text';
}

) !== null
) { return 'module'; }
if (mime === 'application/json') { return 'json'; }
if (mime === 'application/wasm') { return 'wasm'; }
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (mime === 'application/wasm') { return 'wasm'; }
if (RegExpPrototypeExec(/^\s*text\/plain\s*(;\s*charset=utf-?8\s*)?$/i, mime) !== null) { return 'text'; }

@efekrskl
Copy link
Copy Markdown
Contributor Author

Note that the import-bytes proposal requires producing Uint8Arrays backed by immutable ArrayBuffers, which are not yet supported in V8. I would be hesitant to ship this with mutable backing buffers, even flagged as experimental, lest people come to rely on that.
Text imports don't have the same problem.

That's a great point, looks like I've missed that. I've removed bytes related part of the code. Thank you!

This is really good knowledge that would be good to preserve somewhere. Maybe add a comment near the most appropriate spot in the new code, such as where the bytes support would be, to note that we shouldn’t add bytes support before immutable ArrayBuffers exist in V8? That way if someone later on comes by to add bytes, they'll stumble upon the comment in the code.

Yeah makes sense, I've added a comment right below formatTypeMap

await assert.rejects(
import('../fixtures/file-to-read-without-bom.txt'),
{ code: 'ERR_UNKNOWN_FILE_EXTENSION' },
);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a test for what happens when you try to import a binary data file such as an image.png or mod.wasm.

I encourage Node.js to take a strict view here. The following code should throw an exception

import text from "image.png" with { type: "text" };

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I'm not so sure. I would just allow any file to be read as text. You never know it might have a use case. And implementing a list of potential extensions which don't make sense could be a never ending process?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm really not sure why it should - if there's a text representation of a file, it should give it. If it's nonsense, that's the developer's problem to solve.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I'm not so sure. I would just allow any file to be read as text. You never know it might have a use case.

Replacement characters render the text form of a non-text file unusable and unrecoverable, so it's hard to think of a use case.

And implementing a list of potential extensions which don't make sense could be a never ending process?

I don't suggest filtering by extension. I suggest throwing an exception if any non-text byte is encountered when reading the file.

@bakkot
Copy link
Copy Markdown
Contributor

bakkot commented Apr 2, 2026

Does this actually need to be behind an experimental flag? The proposal is stage 3, and pretty simple.

@ShogunPanda
Copy link
Copy Markdown
Contributor

Does this actually need to be behind an experimental flag? The proposal is stage 3, and pretty simple.

I agree. We don't have to use experimental for anything, especially since it will require additional burden about promoting to stable at some point.

Since the feature is low risk and unlikely to change in the future (which I how I envision "experimental" is meant to use), I would directly insert as stable.

@GeoffreyBooth
Copy link
Copy Markdown
Member

I think this should be introduced as experimental. I can’t think of any modules features we’ve introduced directly as stable. Modules features have compatibility concerns with other runtimes; we want to see how import-type-text behaves in Deno and Bun and browsers before we promote our version to stable. There’s also the issue that as soon as our version is stable, people start relying on it in published public npm packages; we want to make sure we have ours fully correct before that happens.

Comment on lines 32 to 33
'module': kImplicitTypeAttribute,
'wasm': kImplicitTypeAttribute, // It's unclear whether the HTML spec will require an type attribute or not for Wasm; see https://github.com/WebAssembly/esm-integration/issues/42
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this map is never exported, I think it can have experimental formats? Then we won’t need to override it or check in so many places.

Suggested change
'module': kImplicitTypeAttribute,
'wasm': kImplicitTypeAttribute, // It's unclear whether the HTML spec will require an type attribute or not for Wasm; see https://github.com/WebAssembly/esm-integration/issues/42
'module': kImplicitTypeAttribute,
'text': 'text',
'wasm': kImplicitTypeAttribute, // It's unclear whether the HTML spec will require an type attribute or not for Wasm; see https://github.com/WebAssembly/esm-integration/issues/42

Comment on lines +35 to +45
// NOTE: Don't add bytes support yet as it requires Uint8Arrays backed by immutable ArrayBuffers,
// which V8 does not support yet.
// see: https://github.com/nodejs/node/pull/62300#issuecomment-4079163816

function getFormatType(format) {
if (format === 'text' && getOptionValue('--experimental-import-text')) {
return 'text';
}

return formatTypeMap[format];
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you accept my previous suggestion, then this function is unnecessary . . .

}
}
const validType = formatTypeMap[format];
const validType = getFormatType(format);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

. . . and this change can be reverted . . .

Suggested change
const validType = getFormatType(format);
const validType = formatTypeMap[format];

Comment on lines +115 to +116
if (!ArrayPrototypeIncludes(supportedTypeAttributes, type) &&
!(type === 'text' && getOptionValue('--experimental-import-text'))) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

. . . and this change can be reverted . . .

Suggested change
if (!ArrayPrototypeIncludes(supportedTypeAttributes, type) &&
!(type === 'text' && getOptionValue('--experimental-import-text'))) {
if (!ArrayPrototypeIncludes(supportedTypeAttributes, type)) {

Comment on lines +98 to +103
if (isExperimentalTextImport(importAttributes)) {
format = 'text';
}

// Now that we have the source for the module, run `defaultGetFormat` to detect its format.
format = defaultGetFormat(urlInstance, context);
format ??= defaultGetFormat(urlInstance, context);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this if added here? We don’t do this for JSON. This deviates from the pattern and seems suspicious, like you’re adding something here that should be somewhere else; perhaps within defaultGetFormat?

Comment on lines +166 to +168
if (format == null && isExperimentalTextImport(importAttributes)) {
format = 'text';
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto here.

Comment on lines +1009 to +1020
let format = defaultGetFormatWithoutErrors(url, context);
if (getOptionValue('--experimental-import-text') &&
importAttributes?.type === 'text') {
format = 'text';
}

return {
__proto__: null,
// Do NOT cast `url` to a string: that will work even when there are real
// problems, silencing them
url: url.href,
format: defaultGetFormatWithoutErrors(url, context),
format,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we now determining format within defaultResolve? The previous pattern did it within defaultGetFormatWithoutErrors; perhaps that’s a better place for it?

});

// Strategy for loading source as text.
translators.set('text', function textStrategy(url, translateContext) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When strip-types was experimental, how were these handled? Were the translators just never set at all if the flag wasn’t passed, or would they throw within the function if the flag wasn’t passed, or something else?

@ShogunPanda
Copy link
Copy Markdown
Contributor

I think this should be introduced as experimental. I can’t think of any modules features we’ve introduced directly as stable. Modules features have compatibility concerns with other runtimes; we want to see how import-type-text behaves in Deno and Bun and browsers before we promote our version to stable. There’s also the issue that as soon as our version is stable, people start relying on it in published public npm packages; we want to make sure we have ours fully correct before that happens.

I'm honestly fine with this as well.
My only note here is that we need to establish a policy to continously review experimental modules and don't leave them in that state forever.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support for importing text modules

8 participants