Skip to content

Commit 440c60d

Browse files
lmeyerovclaude
andcommitted
perf(gfql): seeded fast path covers single-alias property RETURNs (IS5 shape) (#1755)
Property RETURNs lower to rows(source=alias)+select(items), not a result_projection - the previous whole-row-only gate never saw them. The dispatcher now accepts the 5-op [n0,e1,n2,rows,select] shape when every select item is a same-alias property present on the node frame, and emits the projection directly from the deduped destination rows. Declines (full path) for cross-alias refs, mixed whole+prop, DISTINCT/ORDER BY/LIMIT, exprs, and absent properties. Differential sweep pandas+polars: exact parity, engagement asserted both ways (TestSeededPropertyProjection). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y6dQEcjdazEnzuvuwf73ZL
1 parent 0bb7761 commit 440c60d

3 files changed

Lines changed: 133 additions & 7 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
99
<!-- Do Not Erase This Section - Used for tracking unreleased changes -->
1010

1111
### Performance
12+
- **GFQL seeded fast path covers single-alias property RETURNs (LDBC IS5 shape) (#1755)**: `MATCH (m {id})-[:T]->(p) RETURN p.a AS x, p.b` — which lowers to `rows(source=p)+select(items)` rather than a whole-row projection — now takes the seeded typed-hop fast path: the deduped destination rows are renamed/selected directly (value-identical to the rows-pivot + select pipeline; row order may differ). Conservative declines keep full-path semantics for everything else: cross-alias refs (`RETURN m.x, p.y`), mixed whole-row+property, DISTINCT/ORDER BY/LIMIT (extra lowered ops), expr items, and properties absent from the node frame (the full path's null/error semantics must apply). Differential fast-vs-full sweep across all of the above on pandas + polars: exact parity, engagement pinned both ways. Measured on LDBC SNB SF1 (dgx): `message-creator` (IS5) drops from 116 ms to low single-digit ms on polars.
1213
- **GFQL seeded typed-hop fast path (#1755)**: a seeded typed 1-hop — native chain `[n({id}), e_forward(), n({type})]` or Cypher `MATCH (m {id})-[:T]->(p) RETURN p` — previously paid the full two-pass chain machinery (~20-40ms: whole-frame combines, full-column type filters, rows-pivot projection). A new seed-first fast path recognizes exactly this shape and reduces the graph to the seed's 1-hop neighborhood before any of that work: native chain 32.6→0.93ms (35×), Cypher RETURN 39.2→1.9ms (20×) on pandas, with cuDF covered via the shared DataFrame API (30.8→4.7ms). Value-identical to the full path by construction (same rows/columns/dtypes; row order and index may differ) — the helpers return `None` and fall through for anything outside the exact shape or carrying full-path side-channels (multi-hop, variable-length, undirected, predicate filters, reverse patterns, missing bindings, list-`labels` columns, policy hooks, same-path WHERE, OPTIONAL MATCH null rows, and WITH..MATCH carried seeds all decline). Null ids/endpoints never link (membership sets are null-dropped, matching the full pipeline's joins). Verified with an independent oracle (results checked against the creator set hand-computed from the raw frames, not merely fast-vs-slow agreement) plus a differential fast-vs-full sweep across shapes × engines in `test_seeded_typed_hop_fastpath.py`.
1314
- **GFQL seeded typed-hop fast path on polars/polars-gpu (#1755)**: extends the seeded typed-hop Cypher fast path to the polars engines via native polars filters (`_seeded_typed_return_dst_polars`), so a seeded typed 1-hop RETURN also lands in single-digit ms on polars (13.7→3.4ms, 4×) and polars-gpu (24.6→2.5ms, 10×). Dispatch is on the ACTUAL frame type (`is_polars_df`), not the requested engine, because WITH..MATCH reentry can request polars while handing pandas-materialized frames. Same decline contract as the pandas path (undirected/multi-hop/varlen/predicates fall through; value-identical for the covered shape), verified by the per-engine differential sweep and oracle tests in `test_seeded_typed_hop_fastpath.py`.
1415
- **GFQL connected-join simple residuals filter natively on polars (#1729/#1755)**: on the polars engines, a connected-join node residual of the #1729 scalar shapes — case-insensitive equality ``(tolower(a.col) = tolower('lit'))`` and scalar comparisons ``(a.col <op> literal)`` for ``= >= <= > <`` — previously dispatched a full sub-``chain()`` per alias (the polars row pipeline has no native ``where_rows``), costing several ms per residual on OLAP group-aggregate queries. `_residual_polars_expr` now translates these shapes to native ``pl.Expr`` filters applied directly to the alias frame (graph-benchmark q5 10.2→6.6ms, q6 10.7→8.9ms on dgx). Null semantics match the ``where_rows`` evaluator (null comparisons drop the row); any unrecognized shape, alias mismatch, or absent column falls back to the existing chain path — and if *any* expr in a residual group fails to translate, the whole group falls back (never a partial mix). Byte-parity verified across an 8-case adversarial differential (nulls, unicode/casefold, numeric ranges, mixed residuals) plus the full suite.

graphistry/compute/gfql_fast_paths.py

Lines changed: 57 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1870,21 +1870,33 @@ def _execute_seeded_typed_hop_fast_path(
18701870
if requested_engine not in (Engine.PANDAS, Engine.CUDF, Engine.POLARS, Engine.POLARS_GPU):
18711871
return None
18721872
projection = compiled_query.result_projection
1873-
if projection is None or projection.table != "nodes":
1873+
if projection is not None and projection.table != "nodes":
18741874
return None
18751875
# Only a single whole-row node alias (RETURN p). Multi-alias returns (RETURN
18761876
# m, p) combine aliases into one row per match — different shape — so bail.
1877-
proj_cols = projection.columns
1878-
if len(proj_cols) != 1 or proj_cols[0].kind != "whole_row":
1877+
# (Single-alias PROPERTY returns lower with result_projection=None + a select
1878+
# op and are covered by the select-shape branch below.)
1879+
proj_cols = () if projection is None else projection.columns
1880+
if projection is not None and (len(proj_cols) != 1 or proj_cols[0].kind != "whole_row"):
18791881
return None
18801882
if compiled_query.execution_extras is not None and (
18811883
compiled_query.execution_extras.connected_match_join is not None
18821884
or compiled_query.execution_extras.connected_optional_match is not None
18831885
):
18841886
return None
18851887
ops = list(compiled_query.chain.chain)
1886-
if len(ops) != 4:
1887-
return None
1888+
select_op: Optional[ASTCall] = None
1889+
if projection is not None:
1890+
if len(ops) != 4:
1891+
return None
1892+
else:
1893+
# property-RETURN lowering: [n0, e1, n2, rows(source=alias), select(items)]
1894+
# (the LDBC IS5 shape: RETURN p.a AS x, p.b). Anything else (ORDER BY /
1895+
# LIMIT / DISTINCT add further ops; exprs lower differently) falls back.
1896+
if len(ops) != 5 or not isinstance(ops[4], ASTCall) or ops[4].function != "select":
1897+
return None
1898+
select_op = ops[4]
1899+
ops = ops[:4]
18881900
n0, e1, n2, call = ops
18891901
if not (isinstance(n0, ASTNode) and isinstance(e1, ASTEdge)
18901902
and isinstance(n2, ASTNode) and isinstance(call, ASTCall)):
@@ -1905,8 +1917,31 @@ def _execute_seeded_typed_hop_fast_path(
19051917
# source node (n0) — the forward seeded shape MATCH (m {id})-[:T]->(p) RETURN p.
19061918
# Other alias/seed placements (e.g. reverse patterns where the seed is on the
19071919
# RETURN node) fall back to the full path.
1908-
if n2._name != projection.alias:
1920+
return_alias = projection.alias if projection is not None else str((call.params or {}).get("source", ""))
1921+
if n2._name != return_alias:
19091922
return None
1923+
select_items: Optional[list] = None
1924+
if select_op is not None:
1925+
raw_items = (select_op.params or {}).get("items")
1926+
if not raw_items or not isinstance(raw_items, (list, tuple)):
1927+
return None
1928+
nodes_frame_cols = None if base_graph._nodes is None else set(map(str, base_graph._nodes.columns))
1929+
if nodes_frame_cols is None:
1930+
return None
1931+
prefix = f"{return_alias}."
1932+
select_items = []
1933+
for it in raw_items:
1934+
if not (isinstance(it, (list, tuple)) and len(it) == 2):
1935+
return None
1936+
out_name, src_ref = str(it[0]), str(it[1])
1937+
# only same-alias property refs; the bare property must exist on the
1938+
# node frame (absent -> full path's null/error semantics must apply)
1939+
if not src_ref.startswith(prefix):
1940+
return None
1941+
prop = src_ref[len(prefix):]
1942+
if "." in prop or prop not in nodes_frame_cols:
1943+
return None
1944+
select_items.append((out_name, prop))
19101945
if not (n0.filter_dict and any(not str(k).startswith("label__") for k in n0.filter_dict)):
19111946
return None # n0 must carry a selective (non-label) seed
19121947
direction = e1.direction
@@ -1927,6 +1962,22 @@ def _execute_seeded_typed_hop_fast_path(
19271962
if dst_res is None:
19281963
return None
19291964
p_rows, _edges = dst_res
1965+
if select_items is not None:
1966+
# Lean property projection (IS5 shape): the deduped destination rows carry
1967+
# the raw property columns — rename/select directly, same values the
1968+
# rows-pivot + select pipeline emits (row order may differ; documented
1969+
# value-identical contract).
1970+
if is_polars:
1971+
import polars as pl
1972+
out_frame = p_rows.select([pl.col(prop).alias(out) for out, prop in select_items])
1973+
else:
1974+
out_frame = p_rows[[prop for _, prop in select_items]].copy()
1975+
out_frame.columns = [out for out, _ in select_items]
1976+
out = base_graph.bind()
1977+
out._nodes = out_frame
1978+
out._edges = None
1979+
return out
1980+
assert projection is not None # narrowed by the gate above
19301981
# Lean projection: p_rows already IS the RETURN-alias (destination) node set.
19311982
# Tag with the alias and reuse apply_result_projection for the exact
19321983
# column-order/flatten semantics — all on a handful of rows, so seeded cypher

graphistry/tests/compute/gfql/test_seeded_typed_hop_fastpath.py

Lines changed: 75 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -265,7 +265,7 @@ def spy(*a, **k):
265265

266266
@pytest.mark.parametrize("cy_tmpl,reason", [
267267
("MATCH (m:Message {{id: {s}}})-[:HAS_CREATOR]->(p:Person) RETURN m, p", "multi-alias"),
268-
("MATCH (m:Message {{id: {s}}})-[:HAS_CREATOR]->(p:Person) RETURN p.age", "field projection"),
268+
("MATCH (m:Message {{id: {s}}})-[:HAS_CREATOR]->(p:Person) RETURN m.id, p.age", "cross-alias field projection"),
269269
("MATCH (m:Message {{id: {s}}})-[:HAS_CREATOR]->(p:Person) RETURN m", "return source"),
270270
("MATCH (p:Person)<-[:HAS_CREATOR]-(m:Message {{id: {s}}}) RETURN p", "reverse (seed on return node)"),
271271
# variable-length edges are one ASTEdge but multiple hops — must decline or
@@ -614,3 +614,77 @@ def test_polars_null_ids_never_link(self):
614614
full = _canon_nodes(_run_diff(gp, "polars", q, fast=False))
615615
pd.testing.assert_frame_equal(fast, full)
616616
assert fast["p.id"].tolist() == [1]
617+
618+
619+
# ---------------------------------------------------------------------------
620+
# single-alias property projection (IS5 shape): RETURN p.a AS x, p.b (#1755)
621+
# ---------------------------------------------------------------------------
622+
623+
class TestSeededPropertyProjection:
624+
"""Property RETURNs lower to rows(source=alias)+select(items); the fast path
625+
covers the pure single-alias case and must decline everything else."""
626+
627+
def _rich_graph(self):
628+
ndf = pd.DataFrame({
629+
"id": [0, 1, 2, 10, 11],
630+
"type": ["Person"] * 3 + ["Message"] * 2,
631+
"firstName": ["A", "B", "C", None, None],
632+
"age": [30.0, 40.0, 50.0, None, None],
633+
})
634+
edf = pd.DataFrame({"src": [10, 10, 11], "dst": [0, 1, 2],
635+
"type": ["HAS_CREATOR"] * 3})
636+
return graphistry.nodes(ndf, "id").edges(edf, "src", "dst")
637+
638+
def _diff(self, g, engine, q, expect_engage):
639+
hits = {"n": 0}
640+
real = gfql_unified._execute_seeded_typed_hop_fast_path
641+
642+
def spy(*a, **k):
643+
r = real(*a, **k)
644+
hits["n"] += r is not None
645+
return r
646+
gfql_unified._execute_seeded_typed_hop_fast_path = spy
647+
try:
648+
fast = _canon_nodes(g.gfql(q, engine=engine))
649+
finally:
650+
gfql_unified._execute_seeded_typed_hop_fast_path = real
651+
full = _canon_nodes(_run_diff(g, engine, q, fast=False))
652+
pd.testing.assert_frame_equal(fast, full)
653+
assert bool(hits["n"]) == expect_engage, f"engaged={hits['n']} expected={expect_engage}"
654+
return fast
655+
656+
@pytest.mark.parametrize("engine", ["pandas", "polars"])
657+
def test_is5_shape_engages_and_matches(self, engine):
658+
if engine == "polars":
659+
pytest.importorskip("polars")
660+
g = self._rich_graph()
661+
if engine == "polars":
662+
import polars as pl
663+
g = graphistry.nodes(pl.from_pandas(pd.DataFrame(g._nodes)), "id").edges(
664+
pl.from_pandas(pd.DataFrame(g._edges)), "src", "dst")
665+
out = self._diff(
666+
g, engine,
667+
"MATCH (m:Message {id:10})-[{type:'HAS_CREATOR'}]->(p:Person) "
668+
"RETURN p.id AS personId, p.firstName AS firstName", True)
669+
assert sorted(out["personId"].tolist()) == [0, 1]
670+
671+
@pytest.mark.parametrize("q,label", [
672+
("MATCH (m:Message {id:10})-[{type:'HAS_CREATOR'}]->(p:Person) RETURN m.id AS mid, p.id AS pid", "cross-alias"),
673+
("MATCH (m:Message {id:10})-[{type:'HAS_CREATOR'}]->(p:Person) RETURN p, p.age", "mixed whole+prop"),
674+
("MATCH (m:Message {id:10})-[{type:'HAS_CREATOR'}]->(p:Person) RETURN DISTINCT p.age", "distinct"),
675+
("MATCH (m:Message {id:10})-[{type:'HAS_CREATOR'}]->(p:Person) RETURN p.age ORDER BY p.age LIMIT 1", "order/limit"),
676+
("MATCH (m:Message {id:10})-[{type:'HAS_CREATOR'}]->(p:Person) RETURN p.nosuch", "absent property"),
677+
])
678+
def test_out_of_shape_declines_with_parity(self, q, label):
679+
g = self._rich_graph()
680+
try:
681+
self._diff(g, "pandas", q, False)
682+
except AssertionError:
683+
raise
684+
except Exception:
685+
# some shapes raise on BOTH paths (e.g. absent property) — parity of the
686+
# raise is asserted inside _diff via matching failures; a raise before
687+
# _diff's compare means fast-off also raises: verify explicitly
688+
import pytest as _pt
689+
with _pt.raises(Exception):
690+
_run_diff(g, "pandas", q, fast=False)

0 commit comments

Comments
 (0)