Skip to content

Commit 7e68126

Browse files
SteNicholasclaude
andcommitted
fix(variant): support nested variant access in ROW/ARRAY/MAP reads
A variant-access projection carries the variant extension marker, so IsVariantField matched it and BuildNestedVariantPlan planned a nested projection as a full variant: it reassembled struct<value, metadata> into a column declared with the access struct type, and on an unshredded file it built no plan at all. Recognise the access type at every nesting level and share one leaf-plan factory between the top level and nested positions, as Java does in VariantShreddingReadPlanFactory. Descend through LIST and MAP as well. The parquet reader rejects partial projection inside a repeated group, so the file subtree is pushed down verbatim there and only reassembled back; ChildrenLineUp keeps that safe by requiring the read and file children to line up one to one. PruneDataType then has to let such a read through: it fails fast on any projection inside a repeated group, which blocked the whole table read path. Replacing a variant by its access projection drops no field, so it passes through while a real partial projection keeps failing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 4e55a24 commit 7e68126

5 files changed

Lines changed: 955 additions & 132 deletions

File tree

0 commit comments

Comments
 (0)