Description
InplaceSort() in pkg/sql/colexec/hashbuild/build.go (lines 366, 370) sorts UniqueJoinKeys vectors using sort.Slice, which reorders data but does NOT reorder the null bitmap (nsp). This corrupts the null bitmap when the vector contains NULLs.
Root Cause
When hashOnPK=true, UniqueJoinKeys are populated via UnionBatch() which copies ALL rows including NULLs (hashmap.go:418). The hashOnPK path does not filter via zvals[k]==0 before appending to UniqueJoinKeys.
After InplaceSort(), null positions in the bitmap no longer correspond to actual null values. The corrupted vector is then MarshalBinary() and sent as a RuntimeFilter_IN filter to the probe side.
Impact
- JOIN queries with nullable join keys may produce incorrect results (missing rows)
- The corrupted runtime filter causes the probe side to incorrectly skip rows that should be included
- Affects any query where: (1) runtime filter is enabled, (2) build-side join key column has NULLs, (3) sort reorders NULL positions
Same Pattern As
This is the same class of bug as the InplaceSort null bitmap corruption fixed in PR #24054 (primaryKeysMayBeChanged).
Suggested Fix
Filter NULLs before sorting, or clone the vector:
// Before InplaceSort
if vec.HasNull() {
// Option 1: filter NULLs
filtered := filterNulls(vec, mp)
filtered.InplaceSort()
data, err = filtered.MarshalBinary()
// Option 2: just clear NULLs (they are irrelevant for IN-filter)
vec.GetNulls().Reset()
vec.InplaceSort()
}
Related
Description
InplaceSort()inpkg/sql/colexec/hashbuild/build.go(lines 366, 370) sortsUniqueJoinKeysvectors usingsort.Slice, which reorders data but does NOT reorder the null bitmap (nsp). This corrupts the null bitmap when the vector contains NULLs.Root Cause
When
hashOnPK=true,UniqueJoinKeysare populated viaUnionBatch()which copies ALL rows including NULLs (hashmap.go:418). ThehashOnPKpath does not filter viazvals[k]==0before appending toUniqueJoinKeys.After
InplaceSort(), null positions in the bitmap no longer correspond to actual null values. The corrupted vector is thenMarshalBinary()and sent as aRuntimeFilter_INfilter to the probe side.Impact
Same Pattern As
This is the same class of bug as the
InplaceSortnull bitmap corruption fixed in PR #24054 (primaryKeysMayBeChanged).Suggested Fix
Filter NULLs before sorting, or clone the vector:
Related
primaryKeysMayBeChangedpkg/container/vector/vector.go:4556—InplaceSortimplementation usessort.Slicewithout null bitmap handling