Skip to content

Guard array_resize target size#22296

Open
Sean-Kenneth-Doherty wants to merge 1 commit into
apache:mainfrom
Sean-Kenneth-Doherty:codex/array-resize-overflow
Open

Guard array_resize target size#22296
Sean-Kenneth-Doherty wants to merge 1 commit into
apache:mainfrom
Sean-Kenneth-Doherty:codex/array-resize-overflow

Conversation

@Sean-Kenneth-Doherty
Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

array_resize can panic when a query asks for an impossibly large target size, for example array_resize(make_array(1), 9223372036854775807, 0). The resize path was using the requested count to size fill arrays and mutable array buffers before rejecting values that cannot fit in the output list offsets or allocation limits.

What changes are included in this PR?

  • Validates each non-null target size before using it for resize capacity planning.
  • Rejects target sizes that cannot fit the output list offset type.
  • Adds a conservative value-capacity check before materializing fill arrays or MutableArrayData buffers.
  • Adds a unit regression and an SQL logic regression for the large target-size query.

Scope note: this is limited to array_resize target-size validation for #22227.

Are these changes tested?

Yes.

  • cargo test -p datafusion-functions-nested array_resize_rejects_target_count_overflow
  • cargo test -p datafusion-sqllogictest --test sqllogictests -- array/array_resize.slt
  • cargo test -p datafusion-functions-nested
  • cargo fmt --check
  • cargo clippy -p datafusion-functions-nested --all-targets -- -D warnings
  • git diff --check

Are there any user-facing changes?

Yes. Oversized array_resize target lengths now return a DataFusion execution error instead of panicking the process.

@github-actions github-actions Bot added sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels May 17, 2026
@Sean-Kenneth-Doherty
Copy link
Copy Markdown
Author

Fresh local validation on 4f1bbbdba:

  • cargo test -p datafusion-functions-nested array_resize_rejects_target_count_overflow -> passed
  • cargo test -p datafusion-sqllogictest --test sqllogictests -- array/array_resize.slt -> passed
  • cargo test -p datafusion-functions-nested -> 63 unit tests passed plus 2 doctests passed
  • cargo fmt --all --check -> passed
  • cargo clippy -p datafusion-functions-nested --all-targets -- -D warnings -> passed
  • git diff --check origin/main...HEAD -> clean

GitHub currently only shows the labeler Process check, so this covers the changed array_resize overflow path and the touched SQLLogicTest file locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

panic: array_resize capacity overflow with large target size

1 participant