Commit 600500d
committed
parquet: seek ahead skipped push decoder row groups
Teach the row-group frontier to seek ahead over queued row groups that
can be proven unreachable before instantiating the row-group builder.
Skip queued row groups when their selection slice is empty, when
offset/limit leaves no rows to read, or when the remaining limit is
already exhausted. Keep predicate-bearing row groups conservative and
stop at the first row group that may still need data.
Add a push decoder regression covering `try_next_reader` with
offset/limit so the frontier path is exercised directly.
Signed-off-by: Hippolyte Barraud <hippolyte.barraud@datadoghq.com>1 parent 7b564c3 commit 600500d
1 file changed
Lines changed: 22 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1121 | 1121 | | |
1122 | 1122 | | |
1123 | 1123 | | |
| 1124 | + | |
| 1125 | + | |
| 1126 | + | |
| 1127 | + | |
| 1128 | + | |
| 1129 | + | |
| 1130 | + | |
| 1131 | + | |
| 1132 | + | |
| 1133 | + | |
| 1134 | + | |
| 1135 | + | |
| 1136 | + | |
| 1137 | + | |
| 1138 | + | |
| 1139 | + | |
| 1140 | + | |
| 1141 | + | |
| 1142 | + | |
| 1143 | + | |
| 1144 | + | |
| 1145 | + | |
1124 | 1146 | | |
1125 | 1147 | | |
1126 | 1148 | | |
| |||
0 commit comments