fix(query): skip inactive rows in vectorized LIKE - #20211
Conversation
sundy-li
left a comment
There was a problem hiding this comment.
The validity short-circuit logic looks sound: inactive rows produce false without invoking pattern conversion/matching, active-row results remain equivalent to dense evaluation, and the specialized SurroundByPercent path is preserved. Focused tests pass on the PR head.\n\nThis PR now conflicts with current main after #20207 merged, in the shared comparison test module. Please rebase/update the branch and retain both the escaped-LIKE/VARIANT coverage from #20207 and this PR's sparse-validity coverage. I resolved that conflict locally and all 13 comparison tests passed, so it appears mechanical, but I am withholding approval until the final rebased head is available for verification.
|
One additional validity-contract concern: Please add an evaluator-level regression with nullable input and a non-filter wrapper/conditional, for example combinations using |
I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/
Summary
ANDevaluation passes rows selected by earlier predicates throughEvalContext.validity. Thecustom vectorized string
LIKEimplementation ignored this bitmap and still ran the matcher forevery row in the input column. This is especially expensive when FUSE Parquet Prewhere combines a
selective predicate with one or more LIKE predicates.
For example:
If
worker = 'worker-7'selects 1/64 of the rows, the two LIKE predicates previously stillprocessed the complete columns. They now process only rows whose validity bit is true.
The change covers
column LIKE scalar,scalar LIKE column, andcolumn LIKE column. It keeps theexisting full-column path when validity is absent or all true, and retains the specialized
%literal%substring search path.On an S3-backed synthetic FUSE table with 10 million rows and approximately 2 KiB strings, the
query above improved from a 13.50 s median to 1.60-1.74 s with default settings. Both plans read the
same 10 million rows; LIKE evaluations dropped from 10 million to 156,250 per predicate. Aggregate
results and the Top-50 result fingerprint were identical before and after the change.
Tests
Added tests for:
Type of change
This change is