You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[GLUTEN-12013][VL] Guard optimizer rule against DPP-injected bloom filters
In Spark 4.1, the optimizer processes BloomFilterMightContainJointRewriteRule
after InjectRuntimeFilter, so the rule sees DPP-injected bloom filters
(which use xxhash64(col, seed) as their value expression). Rewriting those
to VeloxBloomFilterMightContain caused FilterExecTransformer to fail Velox
validation and fall back to JVM, changing the physical plan structure and
breaking 19 TPC-DS plan-stability golden files across v1.4, v2.7, and
modified suites (q2, q10, q16, q24a, q24b, q32, q37, q40, q59, q69, q80,
q82, q85, q92, q94, q95, q10a, q64, q80a).
Fix: guard the first arm of transformAllExpressions with `v: Attribute`.
DPP bloom filters always hash the join key — xxhash64(col, seed) — so
their value is never a bare Attribute and the guard skips them cleanly.
User-facing bloom filters (might_contain(subquery, col)) pass a plain
column reference and continue to be rewritten correctly.
Also guard the catch-all arm against ScalarSubquery to be consistent.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
0 commit comments