streams.md

⏳️ Streams

Node.js streams

Input

import {createReadStream} from 'node:fs';
import {once} from 'node:events';
import {execa} from 'execa';

const readable = createReadStream('input.txt');
await once(readable, 'open');
await execa({stdin: readable})`npm run scaffold`;

Output

import {createWriteStream} from 'node:fs';
import {once} from 'node:events';
import {execa} from 'execa';

const writable = createWriteStream('output.txt');
await once(writable, 'open');
await execa({stdout: writable})`npm run build`;

File descriptors

When passing a Node.js stream to the stdin, stdout or stderr option, that stream must have an underlying file or socket, such as the streams created by the fs, net or http core modules. Otherwise the following error is thrown.

TypeError [ERR_INVALID_ARG_VALUE]: The argument 'stdio' is invalid.

This limitation can be worked around by either:

• Using the input option instead of the stdin option.
• Passing a web stream.
• Passing [nodeStream, 'pipe'] instead of nodeStream.

Web streams

Web streams (ReadableStream or WritableStream) can be used instead of Node.js streams.

const response = await fetch('https://example.com');
await execa({stdin: response.body})`npm run build`;

Iterables as input

const getReplInput = async function * () {
	for await (const replLine of getReplLines()) {
		yield replLine;
	}
};

await execa({stdin: getReplInput()})`npm run scaffold`;

Manual streaming

subprocess.stdin is a Node.js Readable stream and subprocess.stdout/subprocess.stderr/subprocess.all are Node.js Writable streams.

They can be used to stream input/output manually. This is intended for advanced situations. In most cases, the following simpler solutions can be used instead:

• result.stdout, result.stderr or result.stdio.
• The stdin, stdout, stderr or stdio options.
• subprocess.iterable().
• subprocess.pipe().

Converting a subprocess to a stream

Convert

The subprocess.readable(), subprocess.writable() and subprocess.duplex() methods convert the subprocess to a Node.js Readable, Writable and Duplex stream.

This is useful when using a library or API that expects Node.js streams as arguments. In every other situation, the simpler solutions described above can be used instead.

const readable = execa`npm run scaffold`.readable();

const writable = execa`npm run scaffold`.writable();

const duplex = execa`npm run scaffold`.duplex();

Different file descriptor

By default, subprocess.readable(), subprocess.writable() and subprocess.duplex() methods use stdin and stdout. This can be changed using the from and to options.

const readable = execa`npm run scaffold`.readable({from: 'stderr'});

const writable = execa`npm run scaffold`.writable({to: 'fd3'});

const duplex = execa`npm run scaffold`.duplex({from: 'stderr', to: 'fd3'});

Writing to an additional file descriptor like 'fd3' requires marking it as input, since it otherwise defaults to output.

const subprocess = execa({stdio: ['pipe', 'pipe', 'pipe', {value: 'pipe', input: true}]})`npm run scaffold`;

const writable = subprocess.writable({to: 'fd3'});

Error handling

When using subprocess.readable(), subprocess.writable() or subprocess.duplex(), the stream waits for the subprocess to end, and emits an error event if the subprocess fails. This differs from subprocess.stdin, subprocess.stdout and subprocess.stderr's behavior.

This means you do not need to await the subprocess' promise. On the other hand, you (or the library using the stream) do need to both consume the stream, and handle its error event. This can be done by using await finished(stream), await pipeline(..., stream, ...) or await text(stream) which throw an exception when the stream errors.

Converting a subprocess to a web stream

The subprocess.readableStream(), subprocess.writableStream() and subprocess.transformStream() methods are the web streams counterparts of subprocess.readable(), subprocess.writable() and subprocess.duplex(). They behave the same way, including error handling and the from/to options, but return a ReadableStream, a WritableStream and a {readable, writable} pair instead of Node.js streams.

This is useful when using a library or API that expects web streams as arguments. In every other situation, the Node.js stream methods can be used instead.

const readableStream = execa`npm run scaffold`.readableStream();

const writableStream = execa`npm run scaffold`.writableStream();

const {readable, writable} = execa`npm run scaffold`.transformStream();

---

Next: 📞 Inter-process communication
Previous: 🔀 Piping multiple subprocesses
Top: Table of contents