Backport "Right Semi Join" (Hash Right Semi Join) from PostgreSQL 18

[kongfanshen-0801]

What

Backports two upstream PostgreSQL commits that add Hash Right Semi Join:

• aa86129e1 — Support "Right Semi Join" plan shapes
• 5668a857d — Fix right-semi-joins in HashJoin rescans

This lets the planner build the hash table on the smaller (LHS) side of an
IN/EXISTS semijoin instead of always hashing the inner relation.

Cloudberry/GPDB-specific adaptations

• nodes.h: JOIN_RIGHT_SEMI is appended at the end of the JoinType
  enum rather than in upstream's mid-list position. Inserting it mid-list
  shifts the integer values of the GPDB-only JOIN_DEDUP_SEMI/_REVERSE and
  JOIN_UNIQUE_* codes, which corrupts MPP motion planning and crashes during
  dispatch (SIGSEGV in setupCdbProcessList). Appending keeps every existing
  value stable.
• cdbpath.c (cdbpath_motion_for_join, serial + parallel switches):
  handle JOIN_RIGHT_SEMI like JOIN_RIGHT/JOIN_RIGHT_ANTI — the inner
  (build) side must not be replicated, since a right-semi join emits
  build-side rows.
• joinrels.c: add the JOIN_RIGHT_SEMI path alongside JOIN_SEMI while
  preserving the existing GPDB JOIN_DEDUP_SEMI handling.

Testing

• Functional: Hash Right Semi Join is chosen for small-build-side semijoins;
  results verified correct (dedup semantics, rescan correctness, MPP execution
  across segments).
• Regression: the join test expected output is still being reconciled
  against CI's canonical environment — hence this PR is opened as a draft.

Notes

• Only affects the PostgreSQL planner (optimizer=off); GPORCA does not
  generate JOIN_RIGHT_SEMI.

[my-ship-it]

Thanks for the back porting. Could we keep the original commit history, thanks

[kongfanshen-0801]

Thanks for the review! I've restructured the branch to preserve the original commit history:

• The two upstream PostgreSQL commits are now cherry-picked with their original author (Richard Guo), original commit messages, and (cherry picked from commit ...) provenance lines preserved:
  • Support "Right Semi Join" plan shapes (aa86129e1)
  • Fix right-semi-joins in HashJoin rescans (5668a857d)
• The Cloudberry/Greenplum-specific adaptations (moving JOIN_RIGHT_SEMI to the end of the JoinType enum to keep the GPDB enum values ABI-stable, and handling the new join type in cdbpath_motion_for_join) are isolated in a separate follow-up commit.

The code logic is unchanged from what I verified locally (Hash Right Semi Join chosen and correct via EXPLAIN ANALYZE + cross-validation). PTAL, thanks!

[my-ship-it]

Thanks for the review! I've restructured the branch to preserve the original commit history:

• The two upstream PostgreSQL commits are now cherry-picked with their original author (Richard Guo), original commit messages, and (cherry picked from commit ...) provenance lines preserved:

  • Support "Right Semi Join" plan shapes (aa86129e1)
  • Fix right-semi-joins in HashJoin rescans (5668a857d)

• The Cloudberry/Greenplum-specific adaptations (moving JOIN_RIGHT_SEMI to the end of the JoinType enum to keep the GPDB enum values ABI-stable, and handling the new join type in cdbpath_motion_for_join) are isolated in a separate follow-up commit.

The code logic is unchanged from what I verified locally (Hash Right Semi Join chosen and correct via EXPLAIN ANALYZE + cross-validation). PTAL, thanks!

Thanks, if have a chance, we could do some comparison of before and after for TPC-H Q 21

[kongfanshen-0801]

Thanks, if have a chance, we could do some comparison of before and after for TPC-H Q 21

Done — ran TPC-H Q21 before/after on the optimizer_enable_right_semi_join GUC.

Setup: SF=100 (lineitem 600,037,902 rows), GPORCA, 3-segment single-host demo cluster, 3 timed runs each.

optimizer_enable_right_semi_join	semijoin plan	runs (s)	best
on	Hash Right Semi Join	355.8 / 340.1 / 339.2	339.2
off	Hash Semi Join	353.0 / 349.3 / 352.2	349.3

Q21's EXISTS (l2 ...) semijoin is where the two plans diverge:

• on — Hash Right Semi Join: builds the hash on the small LHS (448,837 rows, 2 batches, ~17 MB) and streams lineitem l2 (200M rows).
• off — Hash Semi Join: builds the hash on lineitem l2 (200,042,924 rows, 512 batches → heavy spill), Work_mem wanted: 8.6 GB.

So the right-semi shape shrinks the build side from ~200M to ~449K rows, cuts the semijoin hash spill from 512 → 2 batches, and drops requested work_mem from ~8.6 GB → ~14 MB.

End-to-end wall-clock is ~3% better here because on this small cluster the total is dominated by the three lineitem scans (l1/l2/l3) and the l3 anti-join spill that are common to both plans; the build-side / memory win should grow on larger, properly-sized clusters and with tighter memory settings.

I also added an ORCA regression test (src/test/regress/sql/rightsemijoin.sql, with both planner and _optimizer answer files) asserting the GUC flips the plan between right-semi/anti and the regular semi/anti shapes with identical results, and wrote the Q21 numbers into the commit message. PTAL, thanks!

[kongfanshen-0801]

Follow-up on the Q21 numbers above — clarifying why the end-to-end gain is only ~3%, and showing the isolated win.

Where Q21's time actually goes (row executor)

Q21 scans lineitem three times and the cost is dominated by parts that are identical with the GUC on or off:

Hash Right Semi Join (l1 ⋈ l2)        <- the ONLY node right-semi changes
├─ Seq Scan lineitem l2 = 200,042,924
└─ Hash Anti Join (l1 ⋈ l3)
    ├─ Hash Join (l1 ⋈ supplier)
    │   └─ Seq Scan lineitem l1 = 126,464,414
    └─ Hash(build l3) = 126,464,414, Batches: 256 (spills)
        └─ Seq Scan lineitem l3 = 126,464,414

1. Three lineitem scans + per-row tuple deform ≈ 452M rows (~3 × 74 GB I/O plus deforming 16 columns/row in the row executor) — the single biggest chunk.
2. The l3 anti-join builds a 126M-row hash that spills to 256 batches — a separate large hash join that right-semi does not touch; identical on both sides.
3. The two big joins' probe work (hashing/comparing 200M + 126M rows + the l_suppkey <> l_suppkey join filter).

Right-semi only changes the build side of the outer semijoin (l1 ⋈ l2):

	semijoin build side	spill	work_mem wanted
on	small LHS, 448,837 rows, 2 batches, in-mem	none	14 MB
off	lineitem l2, 200,042,924 rows, 512 batches	~heavy	8.6 GB

The 200M l2 rows are scanned either way (as probe when on, as build when off), so the only net delta is the cost of building + spilling that 200M-row hash. Relative to the 452M-row scans and the l3 256-batch spill, that delta is small → ~3% wall-clock, even though total "Memory wanted" drops 60 GB → 38 GB.

Isolated semijoin: ~4× when it's the dominant cost

Removing the 3× full scans (small 10k-row LHS vs a unique 150M-row RHS, orders.o_orderkey, GPORCA), so the semijoin is the cost:

optimizer_enable_right_semi_join	plan	exec time (2 runs)
on	Hash Right Semi Join — build on 3,385-row LHS, no spill	19.7 / 19.7 s
off	fallback HashAggregate(dedup 50M) + Hash Join — 641 batches, 1.5 GB spill, 4.6 GB wanted	77.1 / 78.4 s

→ ~3.9× faster (≈20 s vs ≈77 s) once the build-side choice is the bottleneck.

Row vs vectorized

In the row executor, steps 1 + 3 (scanning/deforming ~452M rows and row-at-a-time hash/probe) dominate and dilute the win. In a vectorized/columnar engine those steps get much cheaper (column-pruned, batched), so the relative weight of avoiding the 200M-row build+spill grows and the Q21 wall-clock gain becomes more pronounced — consistent with the larger Q21 improvement seen on the vectorized build this was ported from.

Summary: the right-semi plan is genuinely chosen and the per-operator win is large (~4× isolated, 200M→449K build, 512→2 batches, 8.6 GB→14 MB); it just sits behind scan/deform-bound work in Q21 on the row executor, hence ~3% end-to-end here. optimizer_enable_right_semi_join defaults to on.