Skip to content

fix: bound snappy uncompressed length to avoid OOM on untrusted block#5

Open
moshap-firebolt wants to merge 1 commit into
mainfrom
moshap/bound-snappy-uncompress-oom
Open

fix: bound snappy uncompressed length to avoid OOM on untrusted block#5
moshap-firebolt wants to merge 1 commit into
mainfrom
moshap/bound-snappy-uncompress-oom

Conversation

@moshap-firebolt

Copy link
Copy Markdown

Problem

DataFileReaderBase::readDataBlock() decompresses a snappy-coded OCF block with snappy::Uncompress(), which reads the declared uncompressed length from the varint prefix of the block and resize()s the destination string to that full length before decompressing a single byte. The length is attacker-controlled when the Avro data is untrusted (e.g. Iceberg manifest/data files, or ... FORMAT Avro in PackDB). A tiny crafted block can declare a multi-GiB uncompressed length, triggering a huge allocation — an OOM — before the decoder discovers the compressed payload is far shorter.

Surfaced by PackDB's avro_input libFuzzer harness: a 220-byte snappy OCF drove a single malloc(2415919104) (~2.4 GiB), reported as a libFuzzer out-of-memory.

This is the same class as the decodeString/decodeBytes OOM fixed in #4, but a distinct code path (the snappy block decompress, not the binary decoder). Present verbatim in Apache Avro C++ main — not fixed upstream.

Fix

Read the declared uncompressed length without allocating via snappy::GetUncompressedLength() and reject a block whose declared length implausibly dwarfs the compressed block. A valid snappy stream expands by at most ~64x, so a generous 256x cap blocks the bomb while leaving every well-formed block untouched — allocating proportional to the bytes that are really there, mirroring #4.

Test

  • Reproduced the OOM against the PackDB avro_input fuzzer, then confirmed the crash input runs clean (exit 0, 0 ms) with this change.
  • Well-formed snappy Avro fixtures (weather-snappy, nulls.snappy, datapage_v2.snappy, alltypes_plain.snappy) still decode identically with no regression.

🤖 Generated with Claude Code

## Problem

DataFileReaderBase::readDataBlock() decompresses a snappy-coded OCF block
with snappy::Uncompress(), which reads the declared uncompressed length from
the varint prefix of the block and resize()s the destination string to that
full length before decompressing a single byte. The length is
attacker-controlled when the Avro data is untrusted (e.g. Iceberg manifest /
data files or `... FORMAT Avro` in PackDB). A tiny crafted block can declare a
multi-GiB uncompressed length, triggering a huge allocation -- an OOM -- before
the decoder discovers the compressed payload is far shorter.

Surfaced by PackDB's `avro_input` libFuzzer harness: a 220-byte snappy OCF drove
a single malloc(2415919104) (~2.4 GiB), reported as a libFuzzer out-of-memory.

This is the same class as the decodeString/decodeBytes OOM fixed in #4, but a
distinct code path (the snappy block decompress, not the binary decoder).

## Fix

Read the declared uncompressed length without allocating via
snappy::GetUncompressedLength() and reject a block whose declared length
implausibly dwarfs the compressed block. A valid snappy stream expands by at
most ~64x, so a generous 256x cap blocks the bomb while leaving every
well-formed block untouched -- allocating proportional to the bytes that are
really there, mirroring #4.

## Test

- Reproduced the OOM against the PackDB `avro_input` fuzzer, then confirmed the
  crash input runs clean (exit 0, 0 ms) with this change.
- Well-formed snappy Avro fixtures (weather-snappy, nulls, datapage_v2,
  alltypes_plain) still decode identically with no regression.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant