Skip to content

Commit f7aa8b7

Browse files
authored
change back to Uint8Array (backed by immutable ArrayBuffer) (#12)
As discussed in 109th Meeting of TC39, we are switching back to Uint8Array (backed by immutable ArrayBuffer). Also bumping to Stage 2 🎉
1 parent f7c16f8 commit f7aa8b7

1 file changed

Lines changed: 33 additions & 29 deletions

File tree

README.md

Lines changed: 33 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,91 +1,93 @@
1-
# Proposal Import Buffer
1+
# Import Bytes
22

33
Champions: [@styfle](https://github.com/styfle)
44

55
Author(s): [@styfle](https://github.com/styfle), Guy Bedford
66

7-
Status: Stage 0.
7+
Status: Stage 2.
88

99
Please leave any feedback in the [issues](https://github.com/styfle/proposal-import-bytes/issues), thanks!
1010

1111
## Synopsis
1212

1313
This proposal is buit on top of [import attributes](https://github.com/tc39/proposal-import-attributes) and [immutable arraybuffer](https://github.com/tc39/proposal-immutable-arraybuffer) to add the ability to import arbitrary bytes in a common way across JavaScript environments.
1414

15-
Developers will then be able to import the buffer as follows:
15+
Developers will then be able to import the bytes as follows:
1616

1717
```js
18-
import buffer from "./photo.png" with { type: "buffer" };
19-
import("photo.png", { with: { type: "buffer" } });
18+
import bytes from "./photo.png" with { type: "bytes" };
19+
import("./photo.png", { with: { type: "bytes" } });
2020
```
2121

22+
The bytes are returned as a `Uint8Array` backed by an [immutable arraybuffer](https://github.com/tc39/proposal-immutable-arraybuffer).
23+
2224
Note: a similar proposal was mentioned in https://github.com/whatwg/html/issues/9444
2325

2426
## Motivation
2527

2628
In a similar manner to why JSON modules are useful, importing raw bytes is useful to extend this behavior to all files. This proposal provides an isomorphic way to read a file, regardless of the JavaScript environment.
2729

28-
For example, a developer may want to read a `.png` file to process an image or `.woff` to process a font and pass the buffer into isomorphic tools like [satori](https://github.com/vercel/satori).
30+
For example, a developer may want to read a `.png` file to process an image or `.woff` to process a font and pass the bytes into isomorphic tools like [satori](https://github.com/vercel/satori).
2931

30-
Today, the developer must detect the platform in order to read the buffer.
32+
Today, the developer must detect the platform in order to read the bytes.
3133

3234
```js
33-
async function getBuffer(path) {
35+
async function getBytes(path) {
3436
if (typeof Deno !== "undefined") {
35-
const result = await Deno.readFile(path);
36-
return result.buffer;
37+
const bytes = await Deno.readFile(path);
38+
return bytes;
3739
}
3840
if (typeof Bun !== "undefined") {
39-
const buffer = await Bun.file(path).arrayBuffer();
40-
return buffer;
41+
const bytes = await Bun.file(path).bytes();
42+
return bytes;
4143
}
4244
if (typeof require !== "undefined") {
4345
const fs = require("fs/promises");
44-
const result = await fs.readFile(path);
45-
return result.buffer;
46+
const bytes = await fs.readFile(path);
47+
return bytes;
4648
}
4749
if (typeof window !== "undefined") {
4850
const response = await fetch(path);
49-
const buffer = await response.arrayBuffer();
50-
return buffer;
51+
const bytes = await response.bytes();
52+
return bytes;
5153
}
5254
throw new Error("Unsupported runtime");
5355
}
5456

55-
const buffer = await getBuffer("./photo.png");
57+
const bytes = await getBytes("./photo.png");
5658
```
5759

5860
We can maximize portability and reduce boilerplate by turning this into a single line of code:
5961

6062
```js
61-
import buffer from "./photo.png" with { type: "buffer" };
63+
import bytes from "./photo.png" with { type: "bytes" };
6264
```
6365

6466
Using an import also provides opportunity for further optimizations when using a bundler. For example, bundlers can statically analyze this import and inline as base64.
6567

6668
```js
67-
const buffer = Uint8Array.fromBase64("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkqAcAAIUAgUW0RjgAAAAASUVORK5CYII=").buffer.transferToImmutable()
69+
const bytes = Uint8Array.fromBase64("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkqAcAAIUAgUW0RjgAAAAASUVORK5CYII=")
6870
```
6971

7072
## Proposed semantics and interoperability
7173

72-
If a module import has an attribute with key `type` and value `buffer`, the host is required to either fail the import, or treat it as an immutable ArrayBuffer. The ArrayBuffer object is the default export of the module (which has no named exports).
74+
If a module import has an attribute with key `type` and value `bytes`, the host is required to either fail the import, or treat it as a Uint8Array backed by an immutable ArrayBuffer. The Uint8Array object is the default export of the module (which has no named exports).
7375

7476
In browser environments, this will be equivalent to `fetch()` such that `sec-fetch-dest` will be empty. The response `content-type` will be ignored.
7577

7678
In "local" desktop/server/embedded, this will be equivalent to a file read. The file extension will be ignored.
7779

78-
All of the import statements in the module graph that address the same module will evaluate to the same immutable ArrayBuffer object.
80+
All of the import statements in the module graph that address the same module will evaluate to the same Uint8Array object.
7981

8082
## FAQ
8183

8284
### How would this proposal work with caching?
8385

8486
The determination of whether the `type` attribute is part of the module cache key is left up to hosts (as it is for all import attributes).
8587

86-
For example, `import "foo"` and `import "foo" with { type: "buffer" }` may return the same module in one host, and different modules with another host. Both are valid implementations.
88+
For example, `import "foo"` and `import "foo" with { type: "bytes" }` may return the same module in one host, and different modules with another host. Both are valid implementations.
8789

88-
However, a dynamic import and a static import with the same `type` will return the same module, regardless of host.
90+
However, a dynamic import and a static import with the same `type` must return the same module, regardless of host.
8991

9092
See discussion in Issue https://github.com/styfle/proposal-import-bytes/issues/4
9193

@@ -131,26 +133,28 @@ let resource = new Resource("logo.bmp");
131133

132134
Mutable can be problematic for several reasons:
133135

134-
- may need multiple copies of the buffer in memory to avoid different underlying bytes for `import(specifier, { type: "json" })` and `import(specifier, { type: "buffer" })`
136+
- may need multiple copies of the buffer in memory to avoid different underlying bytes for `import(specifier, { type: "json" })` and `import(specifier, { type: "bytes" })`
135137
- may cause unexpected behavior when multiple modules import the same buffer and detach (say `postMessage()` or `transferToImmutable()`) in one module which would cause it to become detached in the other module too
136138
- may cause excessive RAM usage for embedded system (immutable could use ROM instead)
137139
- may cause excessive memory when tracking source maps
138140
- may cause an undeniable global communication channel
139141

140142
See discussion in Issue https://github.com/styfle/proposal-import-bytes/issues/2 and https://github.com/styfle/proposal-import-bytes/issues/5
141143

142-
### Why not Uint8Array?
144+
### Why not ArrayBuffer?
145+
146+
ArrayBuffers cannot be read from directly; the developer must create a view, such as a Uint8Array, to read data. Providing a Uint8Array avoids that additional effort.
143147

144-
Uint8Array is one array-like view of an underlying ArrayBuffer, but there are many views (see TypedArray) because there are many different ways that one might want to access the content of the buffer.
148+
This pattern is defined in the [W3C design principals](https://www.w3.org/TR/design-principles/#uint8array).
145149

146-
See discussion in Issue https://github.com/styfle/proposal-import-bytes/issues/5
150+
There was some discussion to switch to ArrayBuffer in Issue https://github.com/styfle/proposal-import-bytes/issues/5 however the plenary meeting reached concensus that Uint8Array is more desireable and the problems described in that issue can still be solved with Uint8Array, as long as it's backed by an Immutable ArrayBuffer.
147151

148152
### Why not Blob?
149153

150154
Blob is part of the W3C [File API](https://www.w3.org/TR/FileAPI/), not part of JavaScript, so it is not a viable solution to include in a TC39 Proposal. Furthermore, Blob typically includes a MIME Type but this proposal ignores the type.
151155

152156
### Why not ReableStream?
153157

154-
ReadableStream is part of the WHATWG [Streams](https://streams.spec.whatwg.org), bot part of JavaScript, so it is not a viable solution to include in a TC39 Proposal. Furthermore, there is [no helper method](https://github.com/whatwg/streams/issues/1019) to turn a stream into a buffer so this won't solve the original motivation of writing isomorphic JavaScript.
158+
ReadableStream is part of the WHATWG [Streams](https://streams.spec.whatwg.org), not part of JavaScript, so it is not a viable solution to include in a TC39 Proposal. Furthermore, there is [no helper method](https://github.com/whatwg/streams/issues/1019) to turn a stream into a buffer so this won't solve the original motivation of writing isomorphic JavaScript.
155159

156-
See discussion in Issue https://github.com/styfle/proposal-import-buffer/issues/3
160+
See discussion in Issue https://github.com/styfle/proposal-import-bytes/issues/3

0 commit comments

Comments
 (0)