Skip to content

AVRO-4299: [perl] Bound allocation when decoding length-prefixed values and collections#3864

Open
iemejia wants to merge 24 commits into
apache:mainfrom
iemejia:AVRO-4299-perl-available-bytes
Open

AVRO-4299: [perl] Bound allocation when decoding length-prefixed values and collections#3864
iemejia wants to merge 24 commits into
apache:mainfrom
iemejia:AVRO-4299-perl-available-bytes

Conversation

@iemejia

@iemejia iemejia commented Jul 11, 2026

Copy link
Copy Markdown
Member

What is the purpose of the change

A bytes or string value is encoded as a length prefix followed by that many bytes of data, and an array or map block is encoded as an element count followed by that many items. A malicious or truncated input can declare a very large length or count while carrying little or no actual data, which causes a correspondingly large allocation before the shortfall is noticed.

This applies the equivalent of the Java SDK fix AVRO-4241 to the Perl SDK and extends it to collections. It has two complementary parts.

1. Validate available bytes before allocating

When the source can report how many bytes remain, a declared length (or a collection block count) that exceeds the bytes actually available is rejected before allocating for it. The collection check uses the minimum on-wire size of the element schema, so a zero-byte element type (such as null) is never falsely rejected. Sources that cannot report their remaining size are unaffected.

_bytes_remaining reports the bytes still readable, found by seeking to the end and restoring the position; readers that cannot seek to the end (e.g. a streaming decompressor) yield undef and are simply skipped. decode_bytes rejects an over-large declared length above a threshold, and decode_array/decode_map reject a block whose element count could not be backed by the bytes remaining, using _min_bytes_per_element from the element schema.

2. Cap collection allocation for zero-byte elements

Zero-byte elements (null, a zero-length fixed, or a record with only zero-byte fields) consume no input, so the available-bytes check cannot bound their count: a tiny payload such as {{"type":"array","items":"null"}} declaring a block count of 200,000,000 would otherwise drive an unbounded allocation. In addition to the available-bytes check, _ensure_collection_available caps the cumulative count of zero-byte elements ($MAX_COLLECTION_ITEMS, default 10,000,000) and applies a structural cap to every collection ($MAX_COLLECTION_STRUCTURAL, default Integer.MAX_VALUE - 8) that also covers non-seekable readers. skip_block is now element-aware (bounds zero-byte skips tightly, other skips by the structural cap). Rejections raise the dedicated Avro::BinaryDecoder::Error::CollectionSize. When set, the AVRO_MAX_COLLECTION_ITEMS environment variable caps both limits.

This folds in and supersedes the standalone collection-limit change (AVRO-4280, #3847), so this PR is the single complete fix for collection/length-prefixed allocation DoS in the Perl SDK.

This is a sub-task of AVRO-4292 and resolves AVRO-4299.

Verifying this change

This change added tests and can be verified as follows:

  • Extended lang/perl/t/03_bin_decode.t with over-limit bytes/array/map rejection, an array<null> with a huge block count that must be rejected, and skip bounding.
  • Run: cd lang/perl && prove -Ilib t/

Documentation

  • Does this pull request introduce a new feature? (no — hardening / robustness)
  • If yes, how is the feature documented? (not applicable)

…th-prefixed values and collections

A bytes or string value is a length prefix followed by that many bytes, and an
array or map block is an element count followed by that many items. A malicious
or truncated input can declare a huge length or count with little or no data.

- _bytes_remaining reports the bytes still readable, found by seeking to the end
  and restoring the position; readers that cannot seek to the end (e.g. a
  streaming decompressor) yield undef and are simply skipped.
- decode_bytes rejects an over-large declared length above a threshold before
  allocating.
- decode_array/decode_map reject a block whose element count could not be backed
  by the bytes remaining, using _min_bytes_per_element computed from the element
  schema so a zero-byte element type (e.g. null) is not falsely rejected.

Mirrors the Java SDK's checks (AVRO-4241).

Assisted-by: GitHub Copilot:claude-opus-4.8
@github-actions github-actions Bot added the Perl label Jul 11, 2026
The Lint CI step (perlcritic, default severity) flagged a
Subroutines::ProhibitExplicitReturnUndef violation: 'return undef' in
_bytes_remaining. Use a bare 'return', which yields undef in the scalar
context the callers use, so a reader that cannot report its size still
skips the check.

Assisted-by: GitHub Copilot:claude-opus-4.8

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Hardens the Perl binary decoder against malicious/truncated inputs that advertise extremely large length/count prefixes by validating claimed sizes against the bytes actually remaining on seekable readers, preventing large allocations before short reads are detected.

Changes:

  • Add remaining-bytes checks for large bytes/string length-prefixed reads (seekable readers only).
  • Add remaining-bytes checks for array/map blocks using a computed minimum on-wire element size (with special handling to avoid rejecting zero-byte element types like null).
  • Extend Perl decoder tests to cover oversized/truncated bytes/array/map cases and a large array<null> that must still decode.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
lang/perl/lib/Avro/BinaryDecoder.pm Adds seek-based “bytes remaining” logic and pre-allocation/pre-iteration validation for bytes/strings and array/map blocks.
lang/perl/t/03_bin_decode.t Adds regression tests covering oversized length/count prefixes and ensuring array-of-nulls is not falsely rejected.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lang/perl/lib/Avro/BinaryDecoder.pm Outdated
Comment on lines 132 to 136
my $class = shift;
my $reader = pop;
my $size = decode_long($class, undef, undef, $reader);
_ensure_available($reader, $size);
$reader->read(my $buf, $size);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 92df248: decode_bytes now rejects a negative decoded length before reading.

Comment on lines +152 to +164
sub _bytes_remaining {
my ($reader) = @_;
my $current = eval { $reader->tell };
return if !defined $current || $current < 0;
return eval {
$reader->seek(0, 2) # SEEK_END
or die "seek to end failed\n";
my $end = $reader->tell;
$reader->seek($current, 0); # SEEK_SET, restore position
die "unknown end position\n" if !defined $end || $end < 0;
$end - $current;
};
}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 92df248: _bytes_remaining now always restores the original position after seeking to the end (even if reading the end offset fails) and treats a restore failure as fatal, so the reader is never silently left at EOF.

Comment thread lang/perl/lib/Avro/BinaryDecoder.pm Outdated
Comment on lines +386 to +387
# Map keys are strings (>= 1 byte length prefix) plus the value.
my $min_bytes = 1 + _min_bytes_per_element($writer_values);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 92df248: decode_map now uses 2 bytes as the minimum key size (1-byte length prefix + at least 1 byte of data), since empty keys are rejected, making the pre-check slightly tighter.

… tighter map key min

Review feedback:
- decode_bytes rejects a negative decoded length before reading, since Avro
  bytes/string lengths must be non-negative.
- _bytes_remaining now always restores the original position after seeking to
  the end (even if reading the end offset fails), and treats a restore failure
  as fatal, so the reader is never left at EOF and silently used for subsequent
  decoding.
- decode_map keys are non-empty strings (empty keys are rejected), so the
  minimum per-entry size uses 2 bytes for the key (1-byte length prefix plus at
  least 1 byte of data) instead of 1, making the pre-check slightly tighter.

Assisted-by: GitHub Copilot:claude-opus-4.8

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread lang/perl/lib/Avro/BinaryDecoder.pm Outdated
return if $count <= 0 || $min_bytes <= 0;
my $remaining = _bytes_remaining($reader);
return unless defined $remaining;
if ($count * $min_bytes > $remaining) {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 98aaeb6: _ensure_collection_available compares count against int(remaining / min_bytes) instead of multiplying, avoiding overflow/wrap on 32-bit builds.

Review feedback: $count * $min_bytes could overflow/wrap on some Perl builds
(notably 32-bit), defeating the pre-check for very large declared counts. Compare
$count against int($remaining / $min_bytes) instead.

Assisted-by: GitHub Copilot:claude-opus-4.8

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread lang/perl/lib/Avro/BinaryDecoder.pm Outdated
Comment on lines 136 to 140
throw Avro::Schema::Error::Parse(
"Invalid negative bytes/string length: $size");
}
_ensure_available($reader, $size);
$reader->read(my $buf, $size);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in fa8332b: decode_bytes now verifies read() returned exactly the declared number of bytes (rejecting truncated input), and skip_bytes (used by skip_string during resolution) rejects a negative declared length.

Review feedback:
- decode_bytes now verifies that read() returned exactly the declared number of
  bytes; a short read (truncated input) is rejected instead of silently
  returning an under-sized buffer that would misalign the decoder.
- skip_bytes (used by skip_string during schema resolution) rejects a negative
  declared length, consistent with decode_bytes.

Assisted-by: GitHub Copilot:claude-opus-4.8

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread lang/perl/lib/Avro/BinaryDecoder.pm Outdated
throw Avro::Schema::Error::Parse(
"Invalid negative bytes/string length: $size");
}
$reader->seek($size, 0);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9a4cf0a: skip_bytes() now uses whence 1 (SEEK_CUR) to skip forward relative to the current position, instead of whence 0 (SEEK_SET) which seeks to the absolute offset.

Review feedback: skip_bytes() used seek($size, 0), i.e. SEEK_SET, which seeks
to the absolute offset $size instead of skipping $size bytes forward from the
current position. Use whence 1 (SEEK_CUR) for a relative skip.

Assisted-by: GitHub Copilot:claude-opus-4.8

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread lang/perl/lib/Avro/BinaryDecoder.pm Outdated
Comment on lines 131 to 135
# Skip forward by $size bytes relative to the current position
# (SEEK_CUR); whence 0 (SEEK_SET) would incorrectly seek to the
# absolute offset $size.
$reader->seek($size, 1);
return;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 25ab8ab: skip_bytes now calls _ensure_available before skipping (mirroring decode_bytes), so a declared length that exceeds the remaining bytes is rejected on a size-reporting reader instead of silently seeking past EOF, and a failed seek is treated as fatal.

While here I fixed the same absolute-vs-relative seek bug in two related paths that shared it: skip_fixed used seek(size, 0) (SEEK_SET) — now SEEK_CUR — and skip_block's negative-count branch ignored the block-size long, did an absolute seek of a negative offset, and re-looped on the same count (an infinite loop); it now reads the block size and skips it relatively. Added tests for skipping fixed/bytes writer fields absent from the reader schema and for rejecting an oversized skipped bytes length. Full suite (386 tests) and perlcritic pass.

Review feedback: skip_bytes could silently succeed when the declared length
exceeded the bytes actually available (a seek past EOF is allowed on many
handles), causing subsequent decoding to read EOF as zeros instead of failing
on truncated input.

- skip_bytes now calls _ensure_available before skipping (mirroring
  decode_bytes) and treats a failed seek as fatal.
- skip_fixed used seek(size, 0) (SEEK_SET, absolute); use SEEK_CUR for a
  relative skip and throw on a failed seek.
- skip_block's negative-count branch ignored the block-size long, did an
  absolute seek of a negative offset, and re-looped on the same count (an
  infinite loop). Read the block size and skip it relatively (SEEK_CUR),
  throwing on a failed seek.

Added tests covering skipping fixed/bytes writer fields absent from the reader
schema and rejecting an oversized skipped bytes length.

Assisted-by: GitHub Copilot:claude-opus-4.8

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread lang/perl/lib/Avro/BinaryDecoder.pm Outdated
Comment on lines +347 to +351
my $block_size = decode_long($class, undef, undef, $reader);
unless ($reader->seek($block_size, 1)) {
throw Avro::Schema::Error::Parse(
"Failed to skip block of $block_size bytes");
}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 8af98be: the negative-count path now rejects a negative block size (throwing Invalid negative block size) before seeking, so a corrupt input cannot seek the reader backwards into an infinite loop or mis-decode. It also validates the block size against the bytes remaining via _ensure_available before the relative seek, consistent with skip_bytes.

Review feedback: in skip_block's negative-count path the decoded block-size long
was not validated. A corrupted input with a negative block size would make
seek($block_size, SEEK_CUR) move the reader backwards, risking an infinite loop
or mis-decoding. Reject a negative block size, and validate it against the bytes
remaining (via _ensure_available) before seeking.

Assisted-by: GitHub Copilot:claude-opus-4.8

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread lang/perl/lib/Avro/BinaryDecoder.pm Outdated
Comment on lines +339 to +343
my $block_count = decode_long($class, undef, undef, $reader);
while ($block_count) {
if ($block_count < 0) {
$reader->seek($block_count, 0);
next;
# A negative count is followed by a long block size in bytes, which
# lets the whole block be skipped without decoding each item. Skip

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great catch — this was a genuine pre-existing bug. Fixed in fce9c6a: skip_block no longer shifts an implicit $class; it takes ($reader, $block_content) to match the plain-function call style used by skip_array/skip_map, and passes PACKAGE to decode_long (which ignores its first argument). Previously $class captured the reader handle and $reader captured the coderef, so decode_long would call $coderef->read and die whenever an array/map field was skipped during projection. Added a test that skips writer array and map fields absent from the reader schema (it would have crashed on the old code).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comment thread lang/perl/t/03_bin_decode.t Outdated
## union and enum index bounds
{
my $union = Avro::Schema->parse(q(["string","null"]));
# Index 2 (zig-zag long 0x04) is out of range for a 2-branch union.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — comment now says 'zig-zag int'.

Comment thread lang/perl/t/03_bin_decode.t Outdated
);
} 'Avro::Schema::Error::Parse', "union branch index out of range is rejected";

# Index -1 (zig-zag long 0x01) must not wrap to a valid branch.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — comment now says 'zig-zag int'.

The union branch index is encoded as an Avro int per the spec; update the two
test comments that said "zig-zag long".

Assisted-by: GitHub Copilot:claude-opus-4.8

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comment on lines +374 to +380
my $symbols = $writer_schema->symbols;
## A negative or out-of-range index is malformed; reject it before indexing
## (Perl's negative indexing would otherwise select the wrong symbol).
if ($index < 0 || $index >= scalar @$symbols) {
throw Avro::Schema::Error::Parse(
"Enum symbol index $index out of range for " . scalar(@$symbols) . " symbols");
}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unlike skip_union (where the branch index selects which schema to skip, so a wrong index desyncs the stream), the enum index doesn't affect skipping: an enum is encoded as just the index varint with nothing following it, so skip_enum reads the varint and moves on regardless of the value. An out-of-range index is therefore skipped correctly without desync or allocation, so validating it here would only be for parity with decode_enum, not correctness/safety. I've left skip_enum as skip_int to keep the skip path minimal; happy to add the schema-aware check if you'd prefer strict parity.

Comment on lines 502 to +506
sub skip_map {
my $class = shift;
my ($schema, $reader) = @_;
skip_block($reader, sub {
# Map entries always carry a >= 1 byte key, so pass a positive minimum.
skip_block($reader, 1 + _min_bytes_per_element($schema->values), sub {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly, an empty map key is skipped correctly (a 0-length string = a 1-byte length prefix and no data), so it doesn't desync or over-read during projection — decode_map rejects it for data validity, but skip only needs to advance past it. So there's no correctness/security impact here; it's a decode-vs-skip strictness parity point. Left as-is to keep skip minimal.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment on lines +431 to +437
{
## The reported exploit: a ~6 byte payload declaring 200,000,000 nulls is
## rejected by the default limit, before allocating.
throws_ok {
decode_zero_byte_array('"null"', encode_long(200_000_000) . encode_long(0));
} qr/more than \d+ zero-byte/, "array of 200M nulls rejected by default limit";
}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — the test now does local $Avro::BinaryDecoder::MAX_COLLECTION_ITEMS = $Avro::BinaryDecoder::DEFAULT_MAX_COLLECTION_ITEMS;, so it's deterministic regardless of AVRO_MAX_COLLECTION_ITEMS in the environment (verified passing with it set to 999999999). Unlike some SDKs the Perl limit is a settable package var, so localizing it works cleanly.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comment on lines +423 to +433
my $block_size = decode_long(__PACKAGE__, undef, undef, $reader);
if ($block_size < 0) {
# A negative block size would seek the reader backwards,
# risking an infinite loop or mis-decoding of a corrupt input.
throw Avro::Schema::Error::Parse(
"Invalid negative block size: $block_size");
}
# Reject a block size that exceeds the bytes actually remaining
# before skipping, so a truncated input fails instead of seeking
# past EOF.
_ensure_available($reader, $block_size, 1);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — skip_block's negative-count branch now rejects a block_size too small to hold the declared count at its minimum on-wire size ($count > int($block_size / $min_bytes) when min_bytes > 0), before seeking.

Comment thread lang/perl/t/03_bin_decode.t Outdated
Comment on lines +135 to +136
# A 64-bit value uses at most 10 bytes; an 11th continuation byte is a
# malformed overlong varint and must be rejected.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — reworded to 'an 11th byte (10 continuation bytes 0x80 then a terminator)'.

The "200M nulls rejected by default limit" test relied on the default
MAX_COLLECTION_ITEMS, which BinaryDecoder.pm initializes from
AVRO_MAX_COLLECTION_ITEMS at load. Localize
$Avro::BinaryDecoder::MAX_COLLECTION_ITEMS to DEFAULT_MAX_COLLECTION_ITEMS in the
test so it is deterministic even if that env var is set in the runner.

Assisted-by: GitHub Copilot:claude-opus-4.8

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 10 comments.

Comments suppressed due to low confidence (1)

lang/perl/lib/Avro/BinaryDecoder.pm:493

  • decode_array iterates with for (1..$block_count), which allocates the full range list before the loop body runs. For large counts this can blow memory/CPU independent of the collection caps (notably on non-seekable readers where the bytes-remaining check is skipped). Use an index-based loop instead.
        _ensure_collection_available($reader, scalar(@array), $block_count, $min_bytes);
        for (1..$block_count) {
            push @array, $class->decode(
                writer_schema => $writer_items,
                reader_schema => $reader_items,

Comment on lines 449 to 451
for (1..$block_count) {
$block_content->();
}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a Perl foreach-range optimization: for (1..$n) in a foreach loop is special-cased and iterated lazily — it does NOT build the list. Verified: for (1..200_000_000) {} uses 0 extra RSS (4636 kB before and after). So there's no materialization here; and the block count is already bounded by the cumulative caps before this loop runs.

Comment on lines +548 to 552
_ensure_collection_available($reader, $decoded, $block_count, $min_bytes);
$decoded += $block_count;
for (1..$block_count) {
my $key = decode_string($class, @_);
unless (defined $key && length $key) {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same — for (1..$block_count) in decode_map is the foreach-range form, which Perl iterates lazily without materializing the list (confirmed: no RSS growth for 200M). The count is also bounded by ensure_collection_available before the loop.

Comment on lines +436 to +437
local $Avro::BinaryDecoder::MAX_COLLECTION_ITEMS =
$Avro::BinaryDecoder::DEFAULT_MAX_COLLECTION_ITEMS;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed globally: rather than localize MAX_COLLECTION_STRUCTURAL per test, the top of t/03_bin_decode.t now resets BOTH MAX_COLLECTION_ITEMS and MAX_COLLECTION_STRUCTURAL to their defaults (they're both initialized from AVRO_MAX_COLLECTION_ITEMS at load). Verified the whole suite passes with AVRO_MAX_COLLECTION_ITEMS=5 set.


{
## Within a lowered limit, a legitimate small null array still decodes.
local $Avro::BinaryDecoder::MAX_COLLECTION_ITEMS = 1000;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed globally: rather than localize MAX_COLLECTION_STRUCTURAL per test, the top of t/03_bin_decode.t now resets BOTH MAX_COLLECTION_ITEMS and MAX_COLLECTION_STRUCTURAL to their defaults (they're both initialized from AVRO_MAX_COLLECTION_ITEMS at load). Verified the whole suite passes with AVRO_MAX_COLLECTION_ITEMS=5 set.

}

{
local $Avro::BinaryDecoder::MAX_COLLECTION_ITEMS = 1000;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed globally: rather than localize MAX_COLLECTION_STRUCTURAL per test, the top of t/03_bin_decode.t now resets BOTH MAX_COLLECTION_ITEMS and MAX_COLLECTION_STRUCTURAL to their defaults (they're both initialized from AVRO_MAX_COLLECTION_ITEMS at load). Verified the whole suite passes with AVRO_MAX_COLLECTION_ITEMS=5 set.


{
## Cumulative across blocks: two blocks of 600 (1200 > 1000).
local $Avro::BinaryDecoder::MAX_COLLECTION_ITEMS = 1000;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed globally: rather than localize MAX_COLLECTION_STRUCTURAL per test, the top of t/03_bin_decode.t now resets BOTH MAX_COLLECTION_ITEMS and MAX_COLLECTION_STRUCTURAL to their defaults (they're both initialized from AVRO_MAX_COLLECTION_ITEMS at load). Verified the whole suite passes with AVRO_MAX_COLLECTION_ITEMS=5 set.

{
## A negative block count (abs value + block byte-size) is normalized and
## must still be bounded.
local $Avro::BinaryDecoder::MAX_COLLECTION_ITEMS = 1000;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed globally: rather than localize MAX_COLLECTION_STRUCTURAL per test, the top of t/03_bin_decode.t now resets BOTH MAX_COLLECTION_ITEMS and MAX_COLLECTION_STRUCTURAL to their defaults (they're both initialized from AVRO_MAX_COLLECTION_ITEMS at load). Verified the whole suite passes with AVRO_MAX_COLLECTION_ITEMS=5 set.


{
## A record whose only field is null encodes to zero bytes.
local $Avro::BinaryDecoder::MAX_COLLECTION_ITEMS = 1000;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed globally: rather than localize MAX_COLLECTION_STRUCTURAL per test, the top of t/03_bin_decode.t now resets BOTH MAX_COLLECTION_ITEMS and MAX_COLLECTION_STRUCTURAL to their defaults (they're both initialized from AVRO_MAX_COLLECTION_ITEMS at load). Verified the whole suite passes with AVRO_MAX_COLLECTION_ITEMS=5 set.

{
## Skipping a huge array<null> writer field absent from the reader schema is
## bounded (a CPU exhaustion otherwise), via skip_block.
local $Avro::BinaryDecoder::MAX_COLLECTION_ITEMS = 1000;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed globally: rather than localize MAX_COLLECTION_STRUCTURAL per test, the top of t/03_bin_decode.t now resets BOTH MAX_COLLECTION_ITEMS and MAX_COLLECTION_STRUCTURAL to their defaults (they're both initialized from AVRO_MAX_COLLECTION_ITEMS at load). Verified the whole suite passes with AVRO_MAX_COLLECTION_ITEMS=5 set.


{
## The zero-byte cap raises the dedicated exception class.
local $Avro::BinaryDecoder::MAX_COLLECTION_ITEMS = 1000;

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed globally: rather than localize MAX_COLLECTION_STRUCTURAL per test, the top of t/03_bin_decode.t now resets BOTH MAX_COLLECTION_ITEMS and MAX_COLLECTION_STRUCTURAL to their defaults (they're both initialized from AVRO_MAX_COLLECTION_ITEMS at load). Verified the whole suite passes with AVRO_MAX_COLLECTION_ITEMS=5 set.

…pendent

- skip_block's negative-count branch now also rejects a block_size too small to
  hold the declared element count at its minimum on-wire size, so a malformed
  size can't leave the reader mid-element.
- BinaryDecoder initializes both collection caps from AVRO_MAX_COLLECTION_ITEMS
  at load, which affected the whole test suite (not just the limit tests). Reset
  both caps to their defaults at the top of t/03_bin_decode.t so every test is
  deterministic regardless of the environment (verified with the env var set to
  5); individual tests still local-override where needed.
- Fix the overlong-varint test comment wording.

Assisted-by: GitHub Copilot:claude-opus-4.8

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment thread lang/perl/t/03_bin_decode.t Outdated
Comment on lines +426 to +428
## Zero-byte elements (null, zero-length fixed, all-null records) consume no
## input, so the bytes-remaining check cannot bound them. A huge declared block
## count of such elements is capped instead.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — removed 'zero-length fixed' from the comment and noted that this SDK's parser requires a positive Fixed size, so Fixed is never a zero-byte element here.

Fixed is not a zero-byte element type in Perl: this SDK's schema parser requires
a positive Fixed size. Drop 'zero-length fixed' from the comment.

Assisted-by: GitHub Copilot:claude-opus-4.8

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants