Search before asking
Description
When pushing down partition filters, both the Spark and Flink connectors stringify everything, predicate literals and partition values alike, before evaluating. That makes equality work fine, but range comparisons fall back to string order.
So with an INT partition column and partitions pt=2, pt=10, a query WHERE pt > 2 lexicographically compares "10" < "2" and drops pt=10. Rows go missing.
We can build the partition row with typed values using the existing PartitionUtils.parseValueOfType and skip stringification for both Flink/Spark.
Willingness to contribute
Search before asking
Description
When pushing down partition filters, both the Spark and Flink connectors stringify everything, predicate literals and partition values alike, before evaluating. That makes equality work fine, but range comparisons fall back to string order.
So with an INT partition column and partitions pt=2, pt=10, a query WHERE pt > 2 lexicographically compares "10" < "2" and drops pt=10. Rows go missing.
We can build the partition row with typed values using the existing PartitionUtils.parseValueOfType and skip stringification for both Flink/Spark.
Willingness to contribute