Skip to content

Bug: InplaceSort corrupts null bitmap in hashbuild runtime filter (JOIN with nullable keys) #24055

@XuPeng-SH

Description

@XuPeng-SH

Description

InplaceSort() in pkg/sql/colexec/hashbuild/build.go (lines 366, 370) sorts UniqueJoinKeys vectors using sort.Slice, which reorders data but does NOT reorder the null bitmap (nsp). This corrupts the null bitmap when the vector contains NULLs.

Root Cause

When hashOnPK=true, UniqueJoinKeys are populated via UnionBatch() which copies ALL rows including NULLs (hashmap.go:418). The hashOnPK path does not filter via zvals[k]==0 before appending to UniqueJoinKeys.

After InplaceSort(), null positions in the bitmap no longer correspond to actual null values. The corrupted vector is then MarshalBinary() and sent as a RuntimeFilter_IN filter to the probe side.

Impact

  • JOIN queries with nullable join keys may produce incorrect results (missing rows)
  • The corrupted runtime filter causes the probe side to incorrectly skip rows that should be included
  • Affects any query where: (1) runtime filter is enabled, (2) build-side join key column has NULLs, (3) sort reorders NULL positions

Same Pattern As

This is the same class of bug as the InplaceSort null bitmap corruption fixed in PR #24054 (primaryKeysMayBeChanged).

Suggested Fix

Filter NULLs before sorting, or clone the vector:

// Before InplaceSort
if vec.HasNull() {
    // Option 1: filter NULLs
    filtered := filterNulls(vec, mp)
    filtered.InplaceSort()
    data, err = filtered.MarshalBinary()
    // Option 2: just clear NULLs (they are irrelevant for IN-filter)
    vec.GetNulls().Reset()
    vec.InplaceSort()
}

Related

Metadata

Metadata

Assignees

Labels

kind/bugSomething isn't workingseverity/s0Extreme impact: Cause the application to break down and seriously affect the use

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions