Commit df2781c
committed
Skip files outside partition structure in hive-partitioned listing tables
When a hive-partitioned listing table contains files in the root
directory (not inside any partition_col=value/ path), these files
have no partition values. Previously they were included with empty
partition_values, causing "Unable to get field named" errors when
queries reference partition columns.
Now try_into_partitioned_file returns None for files that don't
match the partition structure, and the caller skips them.1 parent fb8be01 commit df2781c
2 files changed
Lines changed: 7 additions & 9 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
352 | 352 | | |
353 | 353 | | |
354 | 354 | | |
355 | | - | |
356 | | - | |
357 | | - | |
358 | | - | |
| 355 | + | |
359 | 356 | | |
360 | 357 | | |
361 | 358 | | |
| |||
592 | 589 | | |
593 | 590 | | |
594 | 591 | | |
595 | | - | |
| 592 | + | |
596 | 593 | | |
597 | 594 | | |
598 | 595 | | |
| |||
615 | 612 | | |
616 | 613 | | |
617 | 614 | | |
618 | | - | |
| 615 | + | |
619 | 616 | | |
620 | 617 | | |
621 | 618 | | |
| |||
635 | 632 | | |
636 | 633 | | |
637 | 634 | | |
638 | | - | |
| 635 | + | |
639 | 636 | | |
640 | 637 | | |
641 | 638 | | |
| |||
658 | 655 | | |
659 | 656 | | |
660 | 657 | | |
661 | | - | |
| 658 | + | |
662 | 659 | | |
663 | 660 | | |
664 | 661 | | |
| |||
688 | 685 | | |
689 | 686 | | |
690 | 687 | | |
691 | | - | |
| 688 | + | |
692 | 689 | | |
693 | 690 | | |
694 | 691 | | |
| |||
0 commit comments