Skip to content

Three merged perf changes (#1782, #1783, #1784) have no test that fails if they are wrong #1810

Description

@lmeyerov

Found while auditing the six merges that landed without recorded approval. None of the three is incorrect — 707/707 differential cases are byte-identical between 84be35fb and 233b64c8. The problem is that nothing in the tree would have told us otherwise.

1. #1784 (84835988) — the regression suite is vacuous w.r.t. its own change

test_engine_polars_semi_key_dedup.py is the only test file for a polars-only change that removed .unique() from 7 semi-join key sites. Reverting the production hunks (3 of 4 in polars/chain.py, plus hop_eager.py and pattern_apply.py) leaves the suite at:

36 passed

The suite does catch the dangerous direction — removing the how="left" feeder's .unique() fails 14 — so it pins the boundary, not the change. If the removals had been wrong-headed, its own regression suite would have said nothing. (Independently reproduced; originally measured with all 7 hunks reverted, same result.)

The perf-regression theory the change invites was also tested and refuted rather than argued: removing .unique() makes the semi build side 2E instead of ≤N for undirected endpoint_ids, so at N=3000 / E=3,000,000 it should bite. Measured: undirected 235.0 → 235.1 ms, forward 102.7 → 95.7 ms, two-hop 454.7 → 446.3 ms. Unlike #1782 this PR shipped no cost guard, and the bad regime could not be made to bite.

2. #1783 (002bb914) — the polars EORD=EID hunk is invisible to every test

Reusing the synthetic edge id as the stable order column instead of building a second with_row_index. Reverting that hunk and running the entire polars chain suite:

1003 passed, 0 failures

It is a pure dedup so no value test is possible — but the sibling PR in the same stack shows the alternative: #1782 added test_typed_edge_predicate_only_reads_candidate_rows, an instrumentation test asserting the shape rather than a wall clock. The equivalent here is "exactly one with_row_index is built on the synthetic-EID path". Confirmed safe by argument plus 189 order-sensitive differential cases (explicit bind(edge=...) vs synthetic EID; float/str/int pandas indexes; empty intermediates) — 0 diffs — but nothing in the tree keeps it that way.

3. #1782 (4061e7ce) — the disclosed error-parity narrowing is untested

Its own CHANGELOG states the narrowing plainly:

A data-dependent comparison failure (e.g. a list-valued cell in an object column) is now observed only if it lands in the gathered candidates, so the indexed path can succeed where the scan raises.

I tried to weaponize it — an object column with a list cell, a datetime column compared against a string with a zero-candidate seed — and could not construct a divergence (the polars datetime case raised identically on both paths). So it is disclosed and unreached. That is the worst state for a known narrowing: nobody will notice if it widens. Either pin it with a test or establish that it is unreachable and close it out.

Suggested resolution

  1. Add an instrumentation/structural test for perf(gfql): empty-left merge shrink + polars EORD/EID row-index dedup #1783's EORD hunk (count with_row_index builds), mirroring perf(gfql): evaluate indexed edge_match on candidate rows, not the whole edge frame #1782's precedent.
  2. Add at least one perf(gfql/polars): stop deduplicating semi-join key sides #1784 case that fails with the .unique() calls restored — e.g. assert on the plan/instrumentation, since by construction the values cannot differ.
  3. Decide perf(gfql): evaluate indexed edge_match on candidate rows, not the whole edge frame #1782's narrowing: pin it, or demonstrate unreachability and delete the caveat from the CHANGELOG.

None of these is urgent — no known wrong answer rides on them. They are filed so the gap is a tracked decision rather than an accident of who was reading the diff.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions