Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .changeset/wasm-memlimit-option.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
'node-liblzma': minor
---

Add `memlimit` option to `LZMAOptions` and wire it through `unxzAsync`/`unxz` (WASM).

Callers can now set a memory usage limit for WASM decompression:

```ts
await unxzAsync(buf, { memlimit: 64 * 1024 * 1024 }); // 64 MiB limit
```

When the compressed stream would require more memory than the limit, the promise rejects with `LZMAMemoryLimitError` (`errno === LZMA_MEMLIMIT_ERROR === 6`).

**Accepted types:** `number | bigint` (both coerced to `bigint` for the WASM C ABI).
**Default:** `BigInt(256 * 1024 * 1024)` (256 MiB — unchanged from existing behaviour).

**Native parity:** The native Node.js binding (`InitializeDecoder`) still hardcodes `UINT64_MAX` and ignores `memlimit`. This is WASM-only for now; native tracking in TODO.md.
10 changes: 8 additions & 2 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,20 @@ _None_

## Pending - MEDIUM

_None_
- [ ] [tar-xz] True streaming for Node `extract()`/`list()` — replace `Buffer.concat` accumulation (extract.ts:59,91 + list.ts:26) with incremental header→content parsing so memory stays O(largest entry) instead of O(archive). Public README v6.0.0 advertises this as a "planned optimization" — Priority: M
- [ ] [Native] Wire `memlimit` in `src/bindings/node-liblzma.cpp` `InitializeDecoder` — currently hardcodes `UINT64_MAX`; should read `opts.memlimit` and call `lzma_stream_decoder(stream, memlimit, flags)`. WASM already supports it. — Priority: M
- [→] [WASM] Wire `memlimit` through `LZMAOptions` and `unxzAsync` — moved to In Progress (2026-04-28) → ✅ completed (2026-04-28)

## Pending - LOW (Nice to Have)

_None_
- [ ] [WASM] `validateMemlimit` symmetry — bigint branch has no UINT64_MAX upper-bound guard (only the `number` branch checks `MAX_SAFE_INTEGER`). Currently benign because native side hardcodes UINT64_MAX and `lzma_stream_decoder` reads `uint64_t` (so `2n ** 65n` would silently wrap to a benign-but-unintended ceiling, no security/leak/crash). Becomes load-bearing once `[Native] Wire memlimit` lands — fix together. Priority: L (defer until native parity work).

## Completed

- [x] ✅ [WASM] PR #111 Round 3 Copilot fixes — C-3-001/2/3 duplicate JSDoc blocks removed from decoderInit/autoDecoderInit/validateMemlimit, C-3-004 stale xzAsync/unxzAsync comment fixed in lzma.ts:370; tsc+memlimit+full suite pass (2026-04-28)
- [x] ✅ [WASM] PR #111 Round 2 Copilot fixes — C-2-001 TSDoc xzAsync removed from honored-by list, C-2-002 stale lzma.ts comment, C-2-003 LZMA_OPTIONS_ERROR constant replaces magic 8, C-2-004 MAX_SAFE_INTEGER guard + TSDoc, C-2-005 validateMemlimit lifted to decoderInit+autoDecoderInit; 12 new tests, 474+99+27=600 tests pass (2026-04-28)
- [x] ✅ [WASM] PR #111 Round 1 review fixes — F-001 memlimit validation (NaN/Inf/frac/neg → LZMAOptionsError), F-002 ResolvedLZMAOptions internal type, C-001/C-002 async callback fixture pattern, C-003 byte-equality assertion, F-003 TSDoc reorder, F-004 stale comment, F-005 fixture comment magnitude, F-006 default-path caveat; 4 new tests (12 total in decompress-memlimit.test.ts), 458+99+27=584 tests pass (2026-04-28)
- [x] ✅ [WASM] Wire `memlimit` through `LZMAOptions` → `unxzAsync`/`unxz` — `LZMAMemoryLimitError` thrown when limit exceeded; 8 new tests in `test/wasm/decompress-memlimit.test.ts`; TSDoc with parity note (2026-04-28)
- [x] ✅ [tar-xz v6] Universal stream-first redesign: `create()`/`extract()`/`list()` with `AsyncIterable<Uint8Array>`, identical Node/Browser signatures, `tar-xz/file` subpath for fs helpers — published as `tar-xz@6.0.0` + `nxz-cli@6.0.0` (2026-04-27)
- [x] ✅ [tar-xz v6] Security hardening: 18 path/symlink TOCTOU vectors audited and closed (leaf check, ENOENT walk, hardlink linkSource, NUL/empty rejection, setuid mask, fd-based fs ops with O_NOFOLLOW, pipeline error propagation) — 8 Copilot review rounds + 1 consolidated audit (2026-04-27)
- [x] ✅ [Infra] Independent versioning per workspace package: `release.yml`/`publish.yml` accept `target_package` input, no cross-package version sync; proven in prod — `tar-xz@6.0.0` published without bumping `node-liblzma` (still at 5.0.0) (2026-04-27)
Expand Down
4 changes: 2 additions & 2 deletions src/errors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,8 @@ export class LZMAFormatError extends LZMAError {
* Options error - thrown when invalid or unsupported options are provided
*/
export class LZMAOptionsError extends LZMAError {
constructor(errno: number) {
super('Invalid or unsupported options', errno);
constructor(errno: number, message?: string) {
super(message ?? 'Invalid or unsupported options', errno);
this.name = 'LZMAOptionsError';
}
}
Expand Down
27 changes: 26 additions & 1 deletion src/lzma.ts
Original file line number Diff line number Diff line change
Expand Up @@ -301,8 +301,29 @@ export type {
*
* Emits `progress` event after each chunk with `{bytesRead, bytesWritten}` info.
*/

/**
* Internal resolved options for XzStream instances.
*
* All fields are required (defaults applied in constructor) EXCEPT memlimit,
* which is genuinely optional: the native binding ignores it (UINT64_MAX
* hardcoded; see TODO "[Native] Wire memlimit in src/bindings/node-liblzma.cpp").
* Only the WASM Buffer API decompression paths (unxz/unxzAsync/streamBufferDecode) honour memlimit; xzAsync is compression-only and ignores this field.
*/
interface ResolvedLZMAOptions {
check: number;
preset: number;
filters: number[];
mode: number;
threads: number;
chunkSize: number;
flushFlag: number;
/** Honoured only by the WASM Buffer API; native streams ignore this field. */
memlimit?: number | bigint;
}

export abstract class XzStream extends Transform {
protected _opts: Required<LZMAOptions>;
protected _opts: ResolvedLZMAOptions;
protected _chunkSize: number;
protected _flushFlag: number;
protected lzma: NativeLZMA;
Expand Down Expand Up @@ -344,6 +365,10 @@ export abstract class XzStream extends Transform {
threads: opts.threads ?? 1,
chunkSize: opts.chunkSize ?? liblzma.BUFSIZ,
flushFlag: opts.flushFlag ?? liblzma.LZMA_RUN,
// memlimit is genuinely optional in ResolvedLZMAOptions: the native binding ignores it
// (UINT64_MAX hardcoded; see TODO "[Native] Wire memlimit in src/bindings/node-liblzma.cpp").
// Only the WASM decompression APIs (unxz/unxzAsync/streamBufferDecode) honour this field. xzAsync is compression-only and ignores it.
memlimit: opts.memlimit,
};

this._chunkSize = this._opts.chunkSize;
Expand Down
26 changes: 26 additions & 0 deletions src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,32 @@ export interface LZMAOptions {
chunkSize?: number;
/** Flush flag to use */
flushFlag?: number;
/**
* Memory usage limit for decompression, in bytes.
*
* **Honored only by `unxz`/`unxzAsync` (WASM Buffer API decompression).**
* `xzAsync` is compression-only and does not read `memlimit`.
* Currently **silently ignored** by native streams (`Xz`/`Unxz`/`createXz`/`createUnxz`) —
* see TODO `[Native] Wire memlimit in src/bindings/node-liblzma.cpp`.
*
* Accepted types: `number` or `bigint` (both are validated then coerced to
* `bigint` before being passed to the WASM C ABI, which maps `uint64_t` to
* `BigInt`). The `number` form must be a finite, non-negative integer; passing
* `NaN`, `Infinity`, a fractional value, or a negative number throws
* `LZMAOptionsError` before any decompression is attempted.
* For values ≥ `Number.MAX_SAFE_INTEGER` (2^53 - 1), use `bigint` to avoid
* precision loss on coercion; passing a `number` above this threshold also
* throws `LZMAOptionsError`.
*
* WASM default: `BigInt(256 * 1024 * 1024)` (256 MiB).
* Native: ignored (UINT64_MAX hardcoded — see follow-up TODO above).
*
* When the compressed stream requires more memory than this limit,
* decompression throws `LZMAMemoryLimitError` with
* `code === LZMA_MEMLIMIT_ERROR` (numeric constant `6`, re-exported from
* `src/errors.ts`).
*/
memlimit?: number | bigint;
}

/**
Expand Down
50 changes: 49 additions & 1 deletion src/wasm/bindings.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
* with proper memory management and error handling.
*/

import { createLZMAError, LZMAError } from '../errors.js';
import { createLZMAError, LZMAError, LZMA_OPTIONS_ERROR, LZMAOptionsError } from '../errors.js';
import { copyFromWasm, copyToWasm, type WasmLzmaStream, wasmAlloc, wasmFree } from './memory.js';
import {
LZMA_BUF_ERROR,
Expand Down Expand Up @@ -116,12 +116,14 @@ export function encoderInit(
*
* @param stream - Allocated WasmLzmaStream
* @param memlimit - Memory limit in bytes (default: 256MB)
* @throws LZMAOptionsError if memlimit is invalid
* @throws LZMAError on initialization failure
*/
export function decoderInit(
stream: WasmLzmaStream,
memlimit: number | bigint = DEFAULT_MEMLIMIT
): void {
validateMemlimit(memlimit);
const module = getModule();
const limit = typeof memlimit === 'number' ? BigInt(memlimit) : memlimit;
const ret = module._lzma_stream_decoder(stream.ptr, limit, 0);
Expand All @@ -137,12 +139,14 @@ export function decoderInit(
*
* @param stream - Allocated WasmLzmaStream
* @param memlimit - Memory limit in bytes (default: 256MB)
* @throws LZMAOptionsError if memlimit is invalid
* @throws LZMAError on initialization failure
*/
export function autoDecoderInit(
stream: WasmLzmaStream,
memlimit: number | bigint = DEFAULT_MEMLIMIT
): void {
validateMemlimit(memlimit);
const module = getModule();
const limit = typeof memlimit === 'number' ? BigInt(memlimit) : memlimit;
const ret = module._lzma_auto_decoder(stream.ptr, limit, 0);
Expand Down Expand Up @@ -244,6 +248,49 @@ export function easyBufferEncode(
* @returns Decompressed data
* @throws LZMAError on decompression failure
*/

/**
* Validate a memlimit value before coercion to BigInt.
*
* BigInt() throws a native RangeError for NaN, Infinity, and non-integer
* numbers (e.g. 1.5). Negative integers produce a huge unsigned value when
* interpreted by the C ABI (uint64_t wrap-around). Numbers above
* Number.MAX_SAFE_INTEGER (2^53 - 1) lose precision on coercion to BigInt.
* All these cases are rejected here with LZMAOptionsError so callers always
* get an LZMAError subclass.
*
* @throws LZMAOptionsError if the value is invalid
*/
function validateMemlimit(memlimit: number | bigint): void {
if (typeof memlimit === 'bigint') {
if (memlimit < 0n) {
throw new LZMAOptionsError(LZMA_OPTIONS_ERROR, 'memlimit must be a non-negative value');
}
return;
}
if (!Number.isFinite(memlimit)) {
throw new LZMAOptionsError(
LZMA_OPTIONS_ERROR,
'memlimit must be a finite number (NaN and Infinity are not allowed)'
Comment on lines +266 to +274
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

validateMemlimit hardcodes errno 8 when throwing LZMAOptionsError. Since src/errors.ts already exports LZMA_OPTIONS_ERROR, using the constant here would avoid a magic number and keep this aligned if the mapping ever changes.

Copilot uses AI. Check for mistakes.
);
}
if (!Number.isInteger(memlimit)) {
throw new LZMAOptionsError(
LZMA_OPTIONS_ERROR,
'memlimit must be an integer (fractional values are not allowed)'
);
}
Comment on lines +271 to +282
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

validateMemlimit accepts any finite integer number, but values above Number.MAX_SAFE_INTEGER can’t be represented precisely and may be coerced to an unintended BigInt. Consider either rejecting non-safe integers (and requiring callers to use bigint for large limits) or explicitly documenting the safe-integer requirement/precision caveat.

Copilot uses AI. Check for mistakes.
if (memlimit < 0) {
throw new LZMAOptionsError(LZMA_OPTIONS_ERROR, 'memlimit must be a non-negative value');
}
if (memlimit > Number.MAX_SAFE_INTEGER) {
throw new LZMAOptionsError(
LZMA_OPTIONS_ERROR,
'memlimit number exceeds MAX_SAFE_INTEGER (use bigint for values >= 2^53)'
);
}
}

export function streamBufferDecode(
input: Uint8Array,
memlimit: number | bigint = DEFAULT_MEMLIMIT
Expand All @@ -261,6 +308,7 @@ export function streamBufferDecode(
let outPtr = wasmAlloc(module, outSize);

try {
validateMemlimit(memlimit);
const limit = typeof memlimit === 'number' ? BigInt(memlimit) : memlimit;
module.setValue(memlimitPtr, limit, 'i64');

Expand Down
5 changes: 2 additions & 3 deletions src/wasm/decompress.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,11 @@ import { toUint8Array } from './utils.js';
*/
export async function unxzAsync(
buffer: Uint8Array | ArrayBuffer | string,
_opts?: LZMAOptions
opts?: LZMAOptions
): Promise<Uint8Array> {
await initModule();
const input = toUint8Array(buffer);
// TODO: pass opts.memlimit when LZMAOptions supports it
return streamBufferDecode(input);
return streamBufferDecode(input, opts?.memlimit);
}

/**
Expand Down
Loading
Loading