Merge pull request #1766 from graphistry/perf/gfql-seeded-projection-slice

perf(gfql): seeded fast path covers single-alias property RETURNs — LDBC IS5 shape (#1755)

Author: lmeyerov

CHANGELOG.md

@@ -9,6 +9,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 <!-- Do Not Erase This Section - Used for tracking unreleased changes -->
 
 ### Performance
+- **GFQL seeded fast path covers single-alias property RETURNs (LDBC IS5 shape) (#1755)**: `MATCH (m {id})-[:T]->(p) RETURN p.a AS x, p.b` — which lowers to `rows(source=p)+select(items)` rather than a whole-row projection — now takes the seeded typed-hop fast path: the deduped destination rows are renamed/selected directly (value-identical to the rows-pivot + select pipeline; row order may differ). Dtype parity with the full pandas path is exact: non-id properties ride the rows-pivot there, which upcasts int/float to float64 and bool to object, so the lean tail applies the same casts (the id property keeps its dtype); dtype classes not verified against the pivot (datetimes, pandas extension dtypes, categoricals) decline. `res._edges` is the same empty edges frame the full path yields (never None). Conservative declines keep full-path semantics for everything else: cross-alias refs (`RETURN m.x, p.y`), mixed whole-row+property, DISTINCT/ORDER BY/LIMIT (extra lowered ops), expr items, properties absent from the node frame (the full path's null/error semantics must apply), and requested-vs-actual engine mismatches (the full path converts to the requested engine). Differential fast-vs-full sweep across all of the above on pandas + polars: exact parity incl. dtypes and edges, engagement pinned both ways. Measured on LDBC SNB SF1 (dgx, harness row-validated): `message-creator` (IS5) 116.3→38.3 ms on polars (the residue is the seed scan, targeted separately by resident-index coverage).
 - **GFQL seeded typed-hop fast path (#1755)**: a seeded typed 1-hop — native chain `[n({id}), e_forward(), n({type})]` or Cypher `MATCH (m {id})-[:T]->(p) RETURN p` — previously paid the full two-pass chain machinery (~20-40ms: whole-frame combines, full-column type filters, rows-pivot projection). A new seed-first fast path recognizes exactly this shape and reduces the graph to the seed's 1-hop neighborhood before any of that work: native chain 32.6→0.93ms (35×), Cypher RETURN 39.2→1.9ms (20×) on pandas, with cuDF covered via the shared DataFrame API (30.8→4.7ms). Value-identical to the full path by construction (same rows/columns/dtypes; row order and index may differ) — the helpers return `None` and fall through for anything outside the exact shape or carrying full-path side-channels (multi-hop, variable-length, undirected, predicate filters, reverse patterns, missing bindings, list-`labels` columns, policy hooks, same-path WHERE, OPTIONAL MATCH null rows, and WITH..MATCH carried seeds all decline). Null ids/endpoints never link (membership sets are null-dropped, matching the full pipeline's joins). Verified with an independent oracle (results checked against the creator set hand-computed from the raw frames, not merely fast-vs-slow agreement) plus a differential fast-vs-full sweep across shapes × engines in `test_seeded_typed_hop_fastpath.py`.
 - **GFQL seeded typed-hop fast path on polars/polars-gpu (#1755)**: extends the seeded typed-hop Cypher fast path to the polars engines via native polars filters (`_seeded_typed_return_dst_polars`), so a seeded typed 1-hop RETURN also lands in single-digit ms on polars (13.7→3.4ms, 4×) and polars-gpu (24.6→2.5ms, 10×). Dispatch is on the ACTUAL frame type (`is_polars_df`), not the requested engine, because WITH..MATCH reentry can request polars while handing pandas-materialized frames. Same decline contract as the pandas path (undirected/multi-hop/varlen/predicates fall through; value-identical for the covered shape), verified by the per-engine differential sweep and oracle tests in `test_seeded_typed_hop_fastpath.py`.
 - **GFQL two-star grouped-count OLAP queries collect once on polars (#1755)**: profiling showed the residual two-star path (graph-benchmark q5-q7 shapes) spent ~72% of its runtime on the fixed per-op cost of ~27 eager polars operations. When every residual translates natively (see the entry below), the whole plan — base filter_dicts, residual filters, typed-edge filters, semi-joins, group-prop lookup — now composes as ONE `pl.LazyFrame` plan collected once at the join (`filter_by_dict_polars` gains a `filter_expr_by_dict_polars` twin: same column/dtype resolution and typed-error/NIE contract, expression only). Value-identical to the eager lane it replaces (same filters/joins/aggregation), INCLUDING the empty-match boundary: the eager all-left-counts==1 shortcut's single `n=0` row (the openCypher count over no rows) is reproduced — the left-counts frame rides the same single collect. Edges are engine-converted before going lazy, so pandas frames under `engine='polars'` (the WITH..MATCH reentry shape) take the lane instead of crashing; LazyFrame inputs and untranslatable residuals decline to the eager path. Measured on dgx vs warm Kuzu, same session: q5 7.7→3.8ms (flips to a GFQL win vs 5.2), q6 9.8→6.1 (flips vs 7.3), q7 8.7→5.7 (flips vs 6.9) — moving the graph-benchmark scoreboard from 4W/2T/3L to 7W/1T/1L. Differential tests spy the extracted `_connected_join_two_star_fused_polars` helper and ASSERT lane engagement (a `count(*)` query declines the whole two-star path — pinned as a decline shape): fused vs forced-eager exact-row parity (ORDER BY pinned), the ungrouped empty-match n=0 row, pandas-frames-under-polars-engine parity, empty-result shape, pandas oracle.

graphistry/compute/gfql_fast_paths.py

@@ -2036,21 +2036,33 @@ def _execute_seeded_typed_hop_fast_path(
     if requested_engine not in (Engine.PANDAS, Engine.CUDF, Engine.POLARS, Engine.POLARS_GPU):
         return None
     projection = compiled_query.result_projection
-    if projection is None or projection.table != "nodes":
+    if projection is not None and projection.table != "nodes":
         return None
     # Only a single whole-row node alias (RETURN p). Multi-alias returns (RETURN
     # m, p) combine aliases into one row per match — different shape — so bail.
-    proj_cols = projection.columns
-    if len(proj_cols) != 1 or proj_cols[0].kind != "whole_row":
+    # (Single-alias PROPERTY returns lower with result_projection=None + a select
+    # op and are covered by the select-shape branch below.)
+    proj_cols = () if projection is None else projection.columns
+    if projection is not None and (len(proj_cols) != 1 or proj_cols[0].kind != "whole_row"):
         return None
     if compiled_query.execution_extras is not None and (
         compiled_query.execution_extras.connected_match_join is not None
         or compiled_query.execution_extras.connected_optional_match is not None
     ):
         return None
     ops = list(compiled_query.chain.chain)
-    if len(ops) != 4:
-        return None
+    select_op: Optional[ASTCall] = None
+    if projection is not None:
+        if len(ops) != 4:
+            return None
+    else:
+        # property-RETURN lowering: [n0, e1, n2, rows(source=alias), select(items)]
+        # (the LDBC IS5 shape: RETURN p.a AS x, p.b). Anything else (ORDER BY /
+        # LIMIT / DISTINCT add further ops; exprs lower differently) falls back.
+        if len(ops) != 5 or not isinstance(ops[4], ASTCall) or ops[4].function != "select":
+            return None
+        select_op = ops[4]
+        ops = ops[:4]
     n0, e1, n2, call = ops
     if not (isinstance(n0, ASTNode) and isinstance(e1, ASTEdge)
             and isinstance(n2, ASTNode) and isinstance(call, ASTCall)):
@@ -2071,8 +2083,31 @@ def _execute_seeded_typed_hop_fast_path(
     # source node (n0) — the forward seeded shape MATCH (m {id})-[:T]->(p) RETURN p.
     # Other alias/seed placements (e.g. reverse patterns where the seed is on the
     # RETURN node) fall back to the full path.
-    if n2._name != projection.alias:
+    return_alias = projection.alias if projection is not None else str((call.params or {}).get("source", ""))
+    if n2._name != return_alias:
         return None
+    select_items: Optional[list] = None
+    if select_op is not None:
+        raw_items = (select_op.params or {}).get("items")
+        if not raw_items or not isinstance(raw_items, (list, tuple)):
+            return None
+        nodes_frame_cols = None if base_graph._nodes is None else set(map(str, base_graph._nodes.columns))
+        if nodes_frame_cols is None:
+            return None
+        prefix = f"{return_alias}."
+        select_items = []
+        for it in raw_items:
+            if not (isinstance(it, (list, tuple)) and len(it) == 2):
+                return None
+            out_name, src_ref = str(it[0]), str(it[1])
+            # only same-alias property refs; the bare property must exist on the
+            # node frame (absent -> full path's null/error semantics must apply)
+            if not src_ref.startswith(prefix):
+                return None
+            prop = src_ref[len(prefix):]
+            if "." in prop or prop not in nodes_frame_cols:
+                return None
+            select_items.append((out_name, prop))
     if not (n0.filter_dict and any(not str(k).startswith("label__") for k in n0.filter_dict)):
         return None  # n0 must carry a selective (non-label) seed
     direction = e1.direction
@@ -2088,11 +2123,59 @@ def _execute_seeded_typed_hop_fast_path(
     is_polars = is_polars_df(nodes_frame)
     if is_polars != is_polars_df(base_graph._edges):
         return None  # mixed-engine node/edge frames: decline, full path decides
+    if select_items is not None and (requested_engine in (Engine.POLARS, Engine.POLARS_GPU)) != is_polars:
+        # Requested-vs-actual engine mismatch (e.g. polars frames + engine='pandas'):
+        # the full path CONVERTS the result to the requested engine, so the lean
+        # projection would leak actual-engine frames. Decline; the full path decides.
+        return None
     helper = _seeded_typed_return_dst_polars if is_polars else _seeded_typed_return_dst_pandas_cudf
     dst_res = helper(base_graph, n0, n2, e1, src, dst, node, direction)
     if dst_res is None:
         return None
     p_rows, _edges = dst_res
+    if select_items is not None:
+        # Lean property projection (IS5 shape): the deduped destination rows carry
+        # the raw property columns — rename/select directly, same values the
+        # rows-pivot + select pipeline emits (row order may differ; documented
+        # value-identical contract).
+        if is_polars:
+            import polars as pl
+            out_frame = p_rows.select([pl.col(prop).alias(out) for out, prop in select_items])
+        else:
+            # dtype parity with the full path: non-id properties ride the rows-pivot,
+            # which upcasts int/float -> float64 and bool -> object; the node-id
+            # property passes through the pivot key and keeps its dtype. Any dtype
+            # class not verified against the pivot (datetimes, pandas extension
+            # dtypes like nullable Int64/StringDtype, categoricals) declines to the
+            # full path rather than risk a silent dtype divergence.
+            import numpy as np
+            casts: Dict[str, str] = {}
+            for out_name, prop in select_items:
+                if prop == node:
+                    continue
+                d = p_rows[prop].dtype
+                if isinstance(d, pd.StringDtype):
+                    continue  # pandas>=3 default str dtype: pivot preserves it (verified parity)
+                if not isinstance(d, np.dtype):
+                    return None
+                if d == np.dtype(bool):
+                    casts[out_name] = "object"
+                elif d.kind in "iuf":
+                    casts[out_name] = "float64"
+                elif d.kind != "O":
+                    return None
+            out_frame = p_rows[[prop for _, prop in select_items]].copy()
+            out_frame.columns = [out for out, _ in select_items]
+            for out_name, target in casts.items():
+                out_frame[out_name] = out_frame[out_name].astype(target)
+        out = base_graph.bind()
+        out._nodes = out_frame
+        # full-path parity: a property RETURN yields an EMPTY edges frame with the
+        # edge schema (never None — res._edges must stay usable). The helper's
+        # edges are the matched hop edges, so take their zero-row head.
+        out._edges = _edges.head(0)
+        return out
+    assert projection is not None  # narrowed by the gate above
     # Lean projection: p_rows already IS the RETURN-alias (destination) node set.
     # Tag with the alias and reuse apply_result_projection for the exact
     # column-order/flatten semantics — all on a handful of rows, so seeded cypher