Variant Support for Arrow and Parquet [DRAFT]#7404

Draft

PinkCrow007 wants to merge 21 commits into

apache:mainfrom

PinkCrow007:variant-clean

Commits on Apr 3, 2025

schema: add initial Variant type as a Canonical Extension Type

Jiaying Li
authored and
PinkCrow007
committed
parquet: initial support for LogicalType and ConvertedType for Variant

Jiaying Li
authored and
PinkCrow007
committed
schema: enforce required value field in Variant

Jiaying Li
authored and
PinkCrow007
committed
parquet: add support for variant extension type conversion

Jiaying Li
authored and
PinkCrow007
committed
turn variant into primitive type, add roundtrip test
PinkCrow007
committed
test variant roundtrip with multiple columns RecordBatch; refine variant
PinkCrow007
committed
update variant roundtrip test
PinkCrow007
committed
add VariantArray and VariantBuilder draft
PinkCrow007
committed
implement VariantArrayReader and encode_variant_array for variant roundtrip
PinkCrow007
committed
update comment
PinkCrow007
committed
implement get_metadata_length
PinkCrow007
committed
modify comments
PinkCrow007
committed
create arrow-variant; implement variant metadata encoding
PinkCrow007
committed

Commits on Apr 7, 2025

Commits on Apr 10, 2025

add json_variant_parquet_roundtrip test; refine variant <-> json
PinkCrow007
committed

Commits on Apr 11, 2025

fix bug
PinkCrow007
committed

Commits on Apr 16, 2025

Merge remote-tracking branch 'upstream/main' into struct_group_type
PinkCrow007
committed

Commits on Apr 17, 2025

Make Variant an ExtensionType over Struct in Arrow;
PinkCrow007
committed

Commits on Apr 21, 2025