Skip to content

Converting a sliced FixedSizeList<Struct> Arrow array panics: "end <= self.len()" #8349

@joseph-isaacs

Description

@joseph-isaacs

Found by the Python API Hypothesis fuzzer (vortex-python/test/test_fuzz_file_roundtrip.py).

Converting an Arrow array of type fixed_size_list<struct> that has a non-zero slice offset (e.g. produced by Table.slice) panics with an assertion failure instead of converting or raising a clean error:

import pyarrow as pa
import vortex as vx

typ = pa.list_(pa.struct([("a", pa.int32())]), 2)
t = pa.table({"c0": pa.array([[{"a": 1}, {"a": 2}], [{"a": 3}, {"a": 4}], [{"a": 5}, {"a": 6}]], type=typ)})
t2 = pa.concat_tables([t.slice(0, 1), t.slice(1)])
vx.array(t2)
# pyo3_runtime.PanicException: assertion failed: end <= self.len()

Sliced fixed_size_list of non-struct element types converts fine; the struct element type is required to trigger it. Likely the slice offset is not propagated to (or is double-applied on) the struct child when computing the fixed-size-list window. The fuzzer currently avoids re-chunking tables containing fixed_size_list<struct>; that guard can be removed once this is fixed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions