Skip to content

Commit 18f0cf2

Browse files
committed
OnPair: drop Like pushdown for now, keep Compare token-aware path
The byte-streaming `prefix%` and per-row decode + memmem `%contains%` implementations were not consistently faster than canonicalize + scalar LIKE: the bulk 4×-unrolled decoder is hard to beat with per-row work. Drop Like from PARENT_KERNELS so the system falls through to canonicalize + scalar LIKE. Compare stays pushed: LPM-tokenise the literal once, then `&[u16]` equality on every row's `codes[lo..hi]` — no decode at all, ~7 ns/row. Tests still pass via the canonicalize fallback. A token-DFA implementation (FSST-style, EQSearch / PrefixAutomaton on tokens) is tracked for the next iteration. Signed-off-by: claude <claude@anthropic.com>
1 parent 53c3ea4 commit 18f0cf2

1 file changed

Lines changed: 6 additions & 2 deletions

File tree

encodings/onpair/src/kernel.rs

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,17 @@ use vortex_array::arrays::filter::FilterExecuteAdaptor;
55
use vortex_array::kernel::ParentKernelSet;
66
use vortex_array::scalar_fn::fns::binary::CompareExecuteAdaptor;
77
use vortex_array::scalar_fn::fns::cast::CastExecuteAdaptor;
8-
use vortex_array::scalar_fn::fns::like::LikeExecuteAdaptor;
98

109
use crate::OnPair;
1110

11+
// Compare is pushed: LPM-tokenise the literal once, compare the row's
12+
// `codes[lo..hi]` against the token sequence as `&[u16]` — no decode.
13+
// Like is currently *not* registered: the per-row byte-streaming /
14+
// `memmem`-on-decoded-row implementations are slower than letting the
15+
// canonicalize + scalar `LIKE` path run. A token-DFA pushdown (FSST-
16+
// style) is the right replacement and tracked as future work.
1217
pub(super) const PARENT_KERNELS: ParentKernelSet<OnPair> = ParentKernelSet::new(&[
1318
ParentKernelSet::lift(&CastExecuteAdaptor(OnPair)),
1419
ParentKernelSet::lift(&CompareExecuteAdaptor(OnPair)),
1520
ParentKernelSet::lift(&FilterExecuteAdaptor(OnPair)),
16-
ParentKernelSet::lift(&LikeExecuteAdaptor(OnPair)),
1721
]);

0 commit comments

Comments
 (0)