Skip to content

Commit 5d01b91

Browse files
committed
perf(gfql/index): drop the redundant dedup on the polars select_by_ids semi key side
Same argument as the chain-side change: a semi-join emits a left row iff at least one match exists, so repeated ids can neither change the result nor multiply rows. The cudf/pandas branches of this same function already use `isin` with no dedup, so this also makes the three engines agree.
1 parent 4f692b9 commit 5d01b91

1 file changed

Lines changed: 5 additions & 1 deletion

File tree

graphistry/compute/gfql/index/engine_arrays.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,8 +79,12 @@ def select_by_ids(df: DataFrameT, col: str, ids: ArrayLike, engine: Engine) -> D
7979
# Semi-join (not Expr.is_in(Series), which polars 1.42 deprecates as ambiguous —
8080
# pola-rs/polars#22149) — vectorized AND preserves the left (df) row order, which
8181
# the node materialization relies on (table-order parity with the scan).
82+
# Not deduplicated: a semi-join emits a left row iff >=1 match exists, so repeated
83+
# ids neither change the result nor multiply rows — the dedup is a hash pass for
84+
# nothing. (cudf/pandas `isin` below has the same set semantics, also without a
85+
# dedup.) Anything needing a distinct id set must apply its own `.unique()`.
8286
ids_df = pl.DataFrame({col: np.asarray(ids)}).cast({col: df.schema[col]})
83-
return cast(DataFrameT, df.join(ids_df.unique(), on=col, how="semi"))
87+
return cast(DataFrameT, df.join(ids_df, on=col, how="semi"))
8488
if engine == Engine.CUDF:
8589
import cudf # type: ignore
8690

0 commit comments

Comments
 (0)