Conversation
|
Review requested:
|
|
Note that the import-bytes proposal requires producing Uint8Arrays backed by immutable ArrayBuffers, which are not yet supported in V8. I would be hesitant to ship this with mutable backing buffers, even flagged as experimental, lest people come to rely on that. Text imports don't have the same problem. |
That's a great point, looks like I've missed that. I've removed |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #62300 +/- ##
==========================================
+ Coverage 89.68% 89.70% +0.01%
==========================================
Files 676 692 +16
Lines 206578 214212 +7634
Branches 39555 41123 +1568
==========================================
+ Hits 185267 192154 +6887
- Misses 13446 14112 +666
- Partials 7865 7946 +81
🚀 New features to boost your workflow:
|
This is really good knowledge that would be good to preserve somewhere. Maybe add a comment near the most appropriate spot in the new code, such as where the |
| function getFormatFromImportAttributes(importAttributes) { | ||
| if ( | ||
| !importAttributes || | ||
| !ObjectPrototypeHasOwnProperty(importAttributes, 'type') || | ||
| typeof importAttributes.type !== 'string' | ||
| ) { | ||
| return undefined; | ||
| } | ||
|
|
||
| if ( | ||
| getOptionValue('--experimental-import-text') && | ||
| importAttributes.type === 'text' | ||
| ) { | ||
| return 'text'; | ||
| } | ||
|
|
||
| return undefined; | ||
| } |
There was a problem hiding this comment.
We already support with { type: 'json' }. This seems to introduce a new pattern, rather than following the existing one?
There was a problem hiding this comment.
Good catch, the reason is JSON imports are format led (.json/MIME), whereas for text imports anything can be imported as text (regardless of .txt/MIME). So we needed to have a separate mapping
I'm open for ideas though. Maybe we can make this more clear with a comment
There was a problem hiding this comment.
Agreed that it should have a new mapping, but in general we want to follow the existing patterns of the cocebase. Don’t create a new code path for handling modules of varying formats; we already have a path for with { type: 'json' } so text should work the same way, only diverging where it needs to because of the type. For example, JSON didn’t require getFormatFromImportAttributes so this function shouldn’t be needed to add support for text.
There was a problem hiding this comment.
Makes sense, thanks for the guidance.
I've updated the patch, wdyt?
pdeveltere
left a comment
There was a problem hiding this comment.
in loader.js, you can add:
/**
* @typedef {'builtin'|'commonjs'|'json'|'module'|'text'|'wasm'} ModuleFormat
*/
| function getFileProtocolModuleFormat(url, context = { __proto__: null }, ignoreErrors) { | ||
| const { source } = context; | ||
| const ext = extname(url); | ||
|
|
There was a problem hiding this comment.
| // If the caller explicitly requests text format via import attributes, honour it regardless of file extension. | |
| if (context.importAttributes?.type === 'text') { | |
| return 'text'; | |
| } |
| ) !== null | ||
| ) { return 'module'; } | ||
| if (mime === 'application/json') { return 'json'; } | ||
| if (mime === 'application/wasm') { return 'wasm'; } |
There was a problem hiding this comment.
| if (mime === 'application/wasm') { return 'wasm'; } | |
| if (RegExpPrototypeExec(/^\s*text\/plain\s*(;\s*charset=utf-?8\s*)?$/i, mime) !== null) { return 'text'; } |
Yeah makes sense, I've added a comment right below |
| await assert.rejects( | ||
| import('../fixtures/file-to-read-without-bom.txt'), | ||
| { code: 'ERR_UNKNOWN_FILE_EXTENSION' }, | ||
| ); |
There was a problem hiding this comment.
Please add a test for what happens when you try to import a binary data file such as an image.png or mod.wasm.
I encourage Node.js to take a strict view here. The following code should throw an exception
import text from "image.png" with { type: "text" };There was a problem hiding this comment.
Hmm, I'm not so sure. I would just allow any file to be read as text. You never know it might have a use case. And implementing a list of potential extensions which don't make sense could be a never ending process?
There was a problem hiding this comment.
I'm really not sure why it should - if there's a text representation of a file, it should give it. If it's nonsense, that's the developer's problem to solve.
There was a problem hiding this comment.
Hmm, I'm not so sure. I would just allow any file to be read as text. You never know it might have a use case.
Replacement characters render the text form of a non-text file unusable and unrecoverable, so it's hard to think of a use case.
And implementing a list of potential extensions which don't make sense could be a never ending process?
I don't suggest filtering by extension. I suggest throwing an exception if any non-text byte is encountered when reading the file.
|
Does this actually need to be behind an experimental flag? The proposal is stage 3, and pretty simple. |
I agree. We don't have to use experimental for anything, especially since it will require additional burden about promoting to stable at some point. Since the feature is low risk and unlikely to change in the future (which I how I envision "experimental" is meant to use), I would directly insert as stable. |
|
I think this should be introduced as experimental. I can’t think of any modules features we’ve introduced directly as stable. Modules features have compatibility concerns with other runtimes; we want to see how import-type-text behaves in Deno and Bun and browsers before we promote our version to stable. There’s also the issue that as soon as our version is stable, people start relying on it in published public npm packages; we want to make sure we have ours fully correct before that happens. |
| 'module': kImplicitTypeAttribute, | ||
| 'wasm': kImplicitTypeAttribute, // It's unclear whether the HTML spec will require an type attribute or not for Wasm; see https://github.com/WebAssembly/esm-integration/issues/42 |
There was a problem hiding this comment.
Since this map is never exported, I think it can have experimental formats? Then we won’t need to override it or check in so many places.
| 'module': kImplicitTypeAttribute, | |
| 'wasm': kImplicitTypeAttribute, // It's unclear whether the HTML spec will require an type attribute or not for Wasm; see https://github.com/WebAssembly/esm-integration/issues/42 | |
| 'module': kImplicitTypeAttribute, | |
| 'text': 'text', | |
| 'wasm': kImplicitTypeAttribute, // It's unclear whether the HTML spec will require an type attribute or not for Wasm; see https://github.com/WebAssembly/esm-integration/issues/42 |
| // NOTE: Don't add bytes support yet as it requires Uint8Arrays backed by immutable ArrayBuffers, | ||
| // which V8 does not support yet. | ||
| // see: https://github.com/nodejs/node/pull/62300#issuecomment-4079163816 | ||
|
|
||
| function getFormatType(format) { | ||
| if (format === 'text' && getOptionValue('--experimental-import-text')) { | ||
| return 'text'; | ||
| } | ||
|
|
||
| return formatTypeMap[format]; | ||
| } |
There was a problem hiding this comment.
If you accept my previous suggestion, then this function is unnecessary . . .
| } | ||
| } | ||
| const validType = formatTypeMap[format]; | ||
| const validType = getFormatType(format); |
There was a problem hiding this comment.
. . . and this change can be reverted . . .
| const validType = getFormatType(format); | |
| const validType = formatTypeMap[format]; |
| if (!ArrayPrototypeIncludes(supportedTypeAttributes, type) && | ||
| !(type === 'text' && getOptionValue('--experimental-import-text'))) { |
There was a problem hiding this comment.
. . . and this change can be reverted . . .
| if (!ArrayPrototypeIncludes(supportedTypeAttributes, type) && | |
| !(type === 'text' && getOptionValue('--experimental-import-text'))) { | |
| if (!ArrayPrototypeIncludes(supportedTypeAttributes, type)) { |
| if (isExperimentalTextImport(importAttributes)) { | ||
| format = 'text'; | ||
| } | ||
|
|
||
| // Now that we have the source for the module, run `defaultGetFormat` to detect its format. | ||
| format = defaultGetFormat(urlInstance, context); | ||
| format ??= defaultGetFormat(urlInstance, context); |
There was a problem hiding this comment.
Why is this if added here? We don’t do this for JSON. This deviates from the pattern and seems suspicious, like you’re adding something here that should be somewhere else; perhaps within defaultGetFormat?
| if (format == null && isExperimentalTextImport(importAttributes)) { | ||
| format = 'text'; | ||
| } |
| let format = defaultGetFormatWithoutErrors(url, context); | ||
| if (getOptionValue('--experimental-import-text') && | ||
| importAttributes?.type === 'text') { | ||
| format = 'text'; | ||
| } | ||
|
|
||
| return { | ||
| __proto__: null, | ||
| // Do NOT cast `url` to a string: that will work even when there are real | ||
| // problems, silencing them | ||
| url: url.href, | ||
| format: defaultGetFormatWithoutErrors(url, context), | ||
| format, |
There was a problem hiding this comment.
Why are we now determining format within defaultResolve? The previous pattern did it within defaultGetFormatWithoutErrors; perhaps that’s a better place for it?
| }); | ||
|
|
||
| // Strategy for loading source as text. | ||
| translators.set('text', function textStrategy(url, translateContext) { |
There was a problem hiding this comment.
When strip-types was experimental, how were these handled? Were the translators just never set at all if the flag wasn’t passed, or would they throw within the function if the flag wasn’t passed, or something else?
I'm honestly fine with this as well. |
Closes #62082
Adds support for text import under
--experimental-import-textflag.Example:
Import text is a Stage 3 TC39 proposal
Deno supports text imports with
--unstable-raw-imports.Bun supports text imports directly.
Not entirely sure what the current stance is on not-yet-landed proposals, but putting this up for discussion and feedback.