fix(gfql): resident-index gate — no float-collapsing id promotes, cudf device path, polars-gpu tags, serve-asserted tests

Review findings (plans/review-pr-1768/review.md):
B1 int64<->uint64 promotes to float64 and collapses ids >= 2^53 into
false matches (repro'd: seed 2^62+1 matched node 2^62) — integral pairs
whose promote is float now DECLINE to the exact-compare scan path.
M2 spies count SERVES (non-None), not calls, so a gate regression that
declines everything fails the suite. M1 native-chain hop indexed branch
+ reverse/EDGE_IN_ADJ now tested (serve-asserted, parity). M3 cudf ids
via .dropna().values (device array; to_numpy raised on nulls and
round-tripped host). M4 polars frames also accept polars-gpu-tagged
indexes (same numpy sidecars + polars gather).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL

Author: lmeyerov

CHANGELOG.md

@@ -10,7 +10,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 
 ### Performance
 - **GFQL seeded fast path covers single-alias property RETURNs (LDBC IS5 shape) (#1755)**: `MATCH (m {id})-[:T]->(p) RETURN p.a AS x, p.b` — which lowers to `rows(source=p)+select(items)` rather than a whole-row projection — now takes the seeded typed-hop fast path: the deduped destination rows are renamed/selected directly (value-identical to the rows-pivot + select pipeline; row order may differ). Dtype parity with the full pandas path is exact: non-id properties ride the rows-pivot there, which upcasts int/float to float64 and bool to object, so the lean tail applies the same casts (the id property keeps its dtype); dtype classes not verified against the pivot (datetimes, pandas extension dtypes, categoricals) decline. `res._edges` is the same empty edges frame the full path yields (never None). Conservative declines keep full-path semantics for everything else: cross-alias refs (`RETURN m.x, p.y`), mixed whole-row+property, DISTINCT/ORDER BY/LIMIT (extra lowered ops), expr items, properties absent from the node frame (the full path's null/error semantics must apply), and requested-vs-actual engine mismatches (the full path converts to the requested engine). Differential fast-vs-full sweep across all of the above on pandas + polars: exact parity incl. dtypes and edges, engagement pinned both ways. Measured on LDBC SNB SF1 (dgx, harness row-validated): `message-creator` (IS5) 116.3→38.3 ms on polars (the residue is the seed scan, targeted separately by resident-index coverage).
-- **GFQL seeded typed-hop fast paths use resident indexes (#1658 x #1755)**: with `gfql_index_all()` resident, the seeded fast-path helpers (`_seeded_typed_hop_pandas_cudf`, `_seeded_typed_return_dst_pandas_cudf`, `_seeded_typed_return_dst_polars`) replace their three full-frame scans — O(N) seed-row lookup, O(E) frontier `isin`, O(N) endpoint gather — with positional index lookups (node-id searchsorted + CSR adjacency gather), so a seeded Cypher lookup stops paying graph-size costs entirely. Decline-gated to fall back to the identical scan body: absent/stale index (fingerprint+identity via `get_valid`), non-numeric id families (object/str ids keep the scan path's null semantics), seed not id-filtered. Results identical either way (row order may differ; node-id index implies unique ids). Local 50k/200k covered-shape: pandas 3.11→1.85ms, polars 1.74→1.10ms; the win scales with graph size (the residue is parse/lowering, not scans). Pinned in `TestResidentIndexSeededFastPath` (parity+engagement pandas/polars, string-id decline, stale-index decline).
+- **GFQL seeded typed-hop fast paths use resident indexes (#1658 x #1755)**: with `gfql_index_all()` resident, the seeded fast-path helpers (`_seeded_typed_hop_pandas_cudf`, `_seeded_typed_return_dst_pandas_cudf`, `_seeded_typed_return_dst_polars`) replace their three full-frame scans — O(N) seed-row lookup, O(E) frontier `isin`, O(N) endpoint gather — with positional index lookups (node-id searchsorted + CSR adjacency gather), so a seeded Cypher lookup stops paying graph-size costs entirely. Decline-gated to fall back to the identical scan body: absent/stale index (fingerprint+identity via `get_valid`), non-numeric id families (object/str ids keep the scan path's null semantics), seed not id-filtered. Results identical either way (row order may differ; node-id index implies unique ids). dgx 50k/200k covered-shape: pandas 1.71→0.86ms, polars 2.02→0.69ms (vs Kuzu's same-box 1.06ms); SF1-scale 3.2M/17M polars 47.2→2.25ms — polars numbers measured with a polars-engine index resident, which today requires `gfql_index_all(engine='polars')` on polars frames (until #1767 lands, AUTO index builds swap polars frames to pandas; pandas/cuDF flows are unaffected and independent). Pinned in `TestResidentIndexSeededFastPath` (parity+engagement pandas/polars, string-id decline, stale-index decline, uint64/int64 promote decline, reverse-direction serving).
 - **GFQL seeded typed-hop fast path (#1755)**: a seeded typed 1-hop — native chain `[n({id}), e_forward(), n({type})]` or Cypher `MATCH (m {id})-[:T]->(p) RETURN p` — previously paid the full two-pass chain machinery (~20-40ms: whole-frame combines, full-column type filters, rows-pivot projection). A new seed-first fast path recognizes exactly this shape and reduces the graph to the seed's 1-hop neighborhood before any of that work: native chain 32.6→0.93ms (35×), Cypher RETURN 39.2→1.9ms (20×) on pandas, with cuDF covered via the shared DataFrame API (30.8→4.7ms). Value-identical to the full path by construction (same rows/columns/dtypes; row order and index may differ) — the helpers return `None` and fall through for anything outside the exact shape or carrying full-path side-channels (multi-hop, variable-length, undirected, predicate filters, reverse patterns, missing bindings, list-`labels` columns, policy hooks, same-path WHERE, OPTIONAL MATCH null rows, and WITH..MATCH carried seeds all decline). Null ids/endpoints never link (membership sets are null-dropped, matching the full pipeline's joins). Verified with an independent oracle (results checked against the creator set hand-computed from the raw frames, not merely fast-vs-slow agreement) plus a differential fast-vs-full sweep across shapes × engines in `test_seeded_typed_hop_fastpath.py`.
 - **GFQL seeded typed-hop fast path on polars/polars-gpu (#1755)**: extends the seeded typed-hop Cypher fast path to the polars engines via native polars filters (`_seeded_typed_return_dst_polars`), so a seeded typed 1-hop RETURN also lands in single-digit ms on polars (13.7→3.4ms, 4×) and polars-gpu (24.6→2.5ms, 10×). Dispatch is on the ACTUAL frame type (`is_polars_df`), not the requested engine, because WITH..MATCH reentry can request polars while handing pandas-materialized frames. Same decline contract as the pandas path (undirected/multi-hop/varlen/predicates fall through; value-identical for the covered shape), verified by the per-engine differential sweep and oracle tests in `test_seeded_typed_hop_fastpath.py`.
 - **GFQL two-star grouped-count OLAP queries collect once on polars (#1755)**: profiling showed the residual two-star path (graph-benchmark q5-q7 shapes) spent ~72% of its runtime on the fixed per-op cost of ~27 eager polars operations. When every residual translates natively (see the entry below), the whole plan — base filter_dicts, residual filters, typed-edge filters, semi-joins, group-prop lookup — now composes as ONE `pl.LazyFrame` plan collected once at the join (`filter_by_dict_polars` gains a `filter_expr_by_dict_polars` twin: same column/dtype resolution and typed-error/NIE contract, expression only). Value-identical to the eager lane it replaces (same filters/joins/aggregation), INCLUDING the empty-match boundary: the eager all-left-counts==1 shortcut's single `n=0` row (the openCypher count over no rows) is reproduced — the left-counts frame rides the same single collect. Edges are engine-converted before going lazy, so pandas frames under `engine='polars'` (the WITH..MATCH reentry shape) take the lane instead of crashing; LazyFrame inputs and untranslatable residuals decline to the eager path. Measured on dgx vs warm Kuzu, same session: q5 7.7→3.8ms (flips to a GFQL win vs 5.2), q6 9.8→6.1 (flips vs 7.3), q7 8.7→5.7 (flips vs 6.9) — moving the graph-benchmark scoreboard from 4W/2T/3L to 7W/1T/1L. Differential tests spy the extracted `_connected_join_two_star_fused_polars` helper and ASSERT lane engagement (a `count(*)` query declines the whole two-star path — pinned as a decline shape): fused vs forced-eager exact-row parity (ORDER BY pinned), the ungrouped empty-match n=0 row, pandas-frames-under-polars-engine parity, empty-result shape, pandas oracle.

graphistry/compute/chain_fast_paths.py

@@ -66,8 +66,17 @@ def _resident_seed_indexes(
     else:
         return None
     kind = EDGE_OUT_ADJ if direction == "forward" else EDGE_IN_ADJ
-    adj = registry.get_valid(kind, edges_df, (src, dst), engine)
-    nid = registry.get_valid(NODE_ID, nodes_df, (node,), engine)
+    engines = [engine]
+    if engine == Engine.POLARS:
+        # an index built with explicit engine='polars-gpu' serves the same eager
+        # polars frames (same numpy sidecars + polars row-gather)
+        engines.append(Engine.POLARS_GPU)
+    adj = nid = None
+    for eng_try in engines:
+        adj = registry.get_valid(kind, edges_df, (src, dst), eng_try)
+        nid = registry.get_valid(NODE_ID, nodes_df, (node,), eng_try)
+        if adj is not None and nid is not None:
+            break
     if adj is None or nid is None:
         return None
     xp, _ = array_namespace(engine)
@@ -80,13 +89,24 @@ def _ids_to_key_array(vals: Any, keys: Any, xp: Any) -> Optional[Any]:
     dropna semantics). None when the cast is not value-safe (mismatched families
     like str-vs-int decline to the scan path rather than risk false matches)."""
     try:
-        arr = xp.asarray(vals.to_numpy() if hasattr(vals, "to_numpy") else vals)
+        if 'cudf' in str(type(vals).__module__):
+            vals = vals.dropna()
+            raw = vals.values  # device array; to_numpy() raises on nulls + round-trips host
+        elif hasattr(vals, "to_numpy"):
+            raw = vals.to_numpy()
+        else:
+            raw = vals
+        arr = xp.asarray(raw)
         if arr.dtype.kind == "f":
             arr = arr[~xp.isnan(arr)]
         if arr.dtype.kind not in "iuf" or keys.dtype.kind not in "iuf":
             return None  # numeric id families only: object/str ids keep the scan path (null-object semantics)
         if arr.dtype != keys.dtype:
             common = xp.promote_types(arr.dtype, keys.dtype)
+            if arr.dtype.kind in "iu" and keys.dtype.kind in "iu" and common.kind == "f":
+                # int64<->uint64 promotes to float64, which collapses distinct ids
+                # >= 2^53 into false matches; the scan path compares exactly -> decline.
+                return None
             arr = arr.astype(common)
         return xp.unique(arr)
     except (TypeError, ValueError):

graphistry/tests/compute/gfql/test_seeded_typed_hop_fastpath.py

@@ -818,11 +818,17 @@ def _canon(self, res):
         return df[cols].sort_values(cols).reset_index(drop=True)
 
     def _spied(self, g, q, engine, monkeypatch):
+        # counts SERVES (non-None returns), not calls: a gate regression that
+        # declines everything must fail these tests, not silently pass them
         import graphistry.compute.chain_fast_paths as cfp
         counts = {"n": 0}
         on = cfp._index_node_rows
-        monkeypatch.setattr(cfp, "_index_node_rows",
-                            lambda *a, **k: (counts.__setitem__("n", counts["n"] + 1), on(*a, **k))[1])
+
+        def spy(*a, **k):
+            out = on(*a, **k)
+            counts["n"] += out is not None
+            return out
+        monkeypatch.setattr(cfp, "_index_node_rows", spy)
         res = g.gfql(q, engine=engine)
         monkeypatch.setattr(cfp, "_index_node_rows", on)
         return res, counts["n"]
@@ -840,7 +846,11 @@ def test_indexed_parity_and_engagement(self, engine, monkeypatch):
         else:
             mk = lambda: graphistry.nodes(ndf, "id").edges(edf, "src", "dst")  # noqa: E731
         plain = mk().gfql(self.Q, engine=engine)
-        indexed, hits = self._spied(mk().gfql_index_all(), self.Q, engine, monkeypatch)
+        # explicit engine: on master, gfql_index_all(AUTO) on polars frames still
+        # swaps them to pandas (resolve_engine legacy input-format policy), so the
+        # polars index must be requested explicitly; #1767 (separate PR) makes
+        # AUTO preserve polars frames, at which point bare gfql_index_all() also works.
+        indexed, hits = self._spied(mk().gfql_index_all(engine=engine), self.Q, engine, monkeypatch)
         assert hits > 0, "resident index did not serve the seeded fast path"
         pd.testing.assert_frame_equal(self._canon(indexed), self._canon(plain))
 
@@ -872,3 +882,54 @@ def test_stale_index_declines_to_scan(self, monkeypatch):
         plain = graphistry.nodes(ndf, "id").edges(edf2, "src", "dst").gfql(self.Q, engine="pandas")
         assert counts["n"] == 0, "stale index must not serve"
         pd.testing.assert_frame_equal(self._canon(got), self._canon(plain))
+
+
+    def test_native_chain_hop_indexed_parity_forward_and_reverse(self, monkeypatch):
+        """M1 pin: the native-chain hop helper's indexed branch (only reachable via
+        chain ops, never Cypher) — forward (EDGE_OUT_ADJ) and reverse (EDGE_IN_ADJ),
+        serve-asserted, parity vs the un-indexed graph."""
+        from graphistry.compute.ast import n, e_forward, e_reverse
+        import graphistry.compute.chain_fast_paths as cfp
+        ndf, edf = self._frames()
+        mk = lambda: graphistry.nodes(ndf, "id").edges(edf, "src", "dst")  # noqa: E731
+        for ops in (
+            [n({"id": 33}), e_forward(edge_match={"type": "KNOWS"}), n({"type": "Person"})],
+            [n({"id": 33}), e_reverse(edge_match={"type": "KNOWS"}), n({"type": "Person"})],
+        ):
+            serves = {"n": 0}
+            oe = cfp._index_edge_rows
+
+            def spy(*a, **k):
+                out = oe(*a, **k)
+                serves["n"] += out is not None
+                return out
+            monkeypatch.setattr(cfp, "_index_edge_rows", spy)
+            got = mk().gfql_index_all().gfql(ops, engine="pandas")
+            monkeypatch.setattr(cfp, "_index_edge_rows", oe)
+            assert serves["n"] > 0, f"indexed hop branch did not serve for {ops[1].direction}"
+            plain = mk().gfql(ops, engine="pandas")
+            pd.testing.assert_frame_equal(self._canon(got), self._canon(plain))
+
+    def test_uint64_int64_id_mix_declines_not_collapses(self, monkeypatch):
+        """B1 pin: int64<->uint64 promotes to float64, which collapses ids >= 2**53
+        into false matches; the gate must DECLINE (scan path compares exactly)."""
+        import graphistry.compute.chain_fast_paths as cfp
+        big = np.uint64(2**62)
+        ndf = pd.DataFrame({"id": np.array([big, big + np.uint64(1), np.uint64(7)], dtype=np.uint64),
+                            "type": ["Person"] * 3})
+        edf = pd.DataFrame({"src": np.array([int(big) + 1], dtype=np.int64),
+                            "dst": np.array([7], dtype=np.int64), "type": ["KNOWS"]})
+        g = graphistry.nodes(ndf, "id").edges(edf, "src", "dst").gfql_index_all()
+        q = f"MATCH (m {{id: {int(big) + 1}}})-[:KNOWS]->(p) RETURN p"
+        serves = {"n": 0}
+        on = cfp._index_node_rows
+
+        def spy(*a, **k):
+            out = on(*a, **k)
+            serves["n"] += out is not None
+            return out
+        monkeypatch.setattr(cfp, "_index_node_rows", spy)
+        got = g.gfql(q, engine="pandas")
+        monkeypatch.setattr(cfp, "_index_node_rows", on)
+        plain = graphistry.nodes(ndf, "id").edges(edf, "src", "dst").gfql(q, engine="pandas")
+        pd.testing.assert_frame_equal(self._canon(got), self._canon(plain))