Skip to content

Commit 233b64c

Browse files
lmeyerovclaude
andauthored
fix(gfql): stop WITH re-entry mutating the caller's Plottable (#1786) (#1793)
`g.gfql("MATCH (a {kind:'a'}) WITH a MATCH (a)-[*]->(b) RETURN count(*)")` left the re-entry seed on `g` itself, so the NEXT, entirely unrelated query on the same object was answered against it -- no error, just the previous query's count. Engine-independent (wrong on pandas too, so an engine A/B never surfaces it) and a direct violation of the repo's pure-functional contract: users reasonably reuse one Plottable for many queries. Leak sites (every `_gfql_*` field assigned during execution was audited): * gfql_unified `_execute_compiled_query_chain_non_union` -- the reported bug. `_seeded_dispatch_graph` returns `base_graph` ITSELF when the compiled query has no seed rows, so `dispatch_graph._gfql_start_nodes = ...` wrote straight onto the user's graph. * gfql_unified `gfql()` -- `dispatch_self = self` then `_gfql_shortest_path_backend = shortest_path_backend`: a per-CALL argument persisted on the caller's graph and silently became the default for its next query. * chain `_handle_boundary_calls` and its polars twin `_run_calls_polars` -- same in-place writes of `_gfql_start_nodes` / `_gfql_rows_base_graph`. Their target is an internal graph on today's paths rather than the caller's, but they are the same pattern one refactor away from the same bug, and without the clear below the polars one rides out on the RESULT. Approach: carry the state on an INTERNAL COPY, not a try/finally restore. The per-execution context idea already exists here -- `gfql/index/handoff.py` attaches its boundary decision with `g.bind()` -- so `gfql/exec_context.py` is the same shape for the row context, and every site now attaches on the way in and clears on the way out. Threading the seed through `ExecutionContext` instead would be the purest fix but it is read deep inside the row pipeline and both polars pattern appliers, so it is a large refactor; a `finally` restore was rejected because it still mutates a shared object for the duration of the call (anything holding `g` concurrently sees the seed) and it cannot fix the second half of the defect, where the state escapes on the RESULT graph and poisons queries run against that. Clearing on the way out matters independently: the result of a WITH query used to carry the seed, so a follow-up query on that result -- a different graph entirely -- got the same wrong answer one hop removed. Mutation-checked: reverting the gfql_unified seed write fails 8 of the new tests, reverting the shortest-path-backend write fails 2, removing the three clears fails 3, and a full revert fails 13. Tests run on both pandas and polars since the defect is engine-independent. Claude-Session: https://claude.ai/code/session_015YsqAZQLbqjSDrYSFz2GoB Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent c76f23a commit 233b64c

5 files changed

Lines changed: 231 additions & 11 deletions

File tree

graphistry/compute/chain.py

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -684,6 +684,7 @@ def _handle_boundary_calls(
684684
# Function-scope import: `gfql.index` transitively imports this module, so a
685685
# module-scope import would be a cycle.
686686
from .gfql.index.handoff import attach_handoff
687+
from .gfql.exec_context import attach_row_exec_context, clear_row_exec_context
687688

688689
g_temp = self
689690
suffix_base_graph = g_temp
@@ -732,9 +733,13 @@ def _handle_boundary_calls(
732733

733734
if suffix:
734735
logger.debug('Executing boundary suffix calls: %s', suffix)
735-
if start_nodes is not None:
736-
g_temp._gfql_start_nodes = start_nodes
737-
g_temp._gfql_rows_base_graph = suffix_base_graph
736+
# #1786: per-execution state rides on an INTERNAL COPY. Without prefix/middle
737+
# ops (or a handoff) nothing above rebuilt `g_temp`, so it is still the CALLER's
738+
# graph -- assigning here left the WITH re-entry seed on the user's object and
739+
# the next, unrelated query was answered against it (silent wrong count).
740+
g_temp = attach_row_exec_context(
741+
g_temp, start_nodes=start_nodes, rows_base_graph=suffix_base_graph
742+
)
738743
if (
739744
middle
740745
and any(getattr(op, "_name", None) is not None for op in middle)
@@ -779,7 +784,9 @@ def _handle_boundary_calls(
779784
start_nodes
780785
)
781786

782-
return g_temp
787+
# Each site that attaches the row context also detaches it: the suffix chain has run,
788+
# so the context is spent, and a caller who queries THIS result must not inherit it.
789+
return clear_row_exec_context(g_temp)
783790

784791

785792
def _chain_otel_attrs(
Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
"""Per-execution row context: the ``WITH`` re-entry seed and the boundary base graph.
2+
3+
The row pipeline reads its re-entry seed (``_gfql_start_nodes``) and the graph a
4+
boundary suffix must re-match against (``_gfql_rows_base_graph``) off the graph it is
5+
handed. Both are state of ONE execution, not properties of the user's graph, so they
6+
must ride on an INTERNAL COPY. Assigning them to the caller's ``Plottable`` leaves them
7+
behind after the query returns, and the next -- entirely unrelated -- query on the same
8+
object is then answered against the stale seed: a silent wrong count, no error, on every
9+
engine (#1786).
10+
11+
Mirrors ``gfql/index/handoff.attach_handoff``. The fields are DECLARED on ``Plottable``
12+
(defaults on ``PlotterBase``), so every access here is ordinary typed attribute access.
13+
"""
14+
from __future__ import annotations
15+
16+
from typing import TYPE_CHECKING, Optional
17+
18+
if TYPE_CHECKING:
19+
from graphistry.Plottable import Plottable
20+
from graphistry.compute.typing import DataFrameT
21+
22+
23+
def attach_row_exec_context(
24+
g: "Plottable",
25+
*,
26+
start_nodes: Optional["DataFrameT"] = None,
27+
rows_base_graph: Optional["Plottable"] = None,
28+
) -> "Plottable":
29+
"""Return an internal copy of ``g`` carrying this execution's row context.
30+
31+
``None`` means "keep whatever ``g`` already carries" for either field, so a nested
32+
execution inherits the enclosing one's seed exactly as it did before -- the change
33+
is only ever WHICH object the value lands on, never the value itself.
34+
"""
35+
out = g.bind()
36+
if start_nodes is not None:
37+
out._gfql_start_nodes = start_nodes
38+
if rows_base_graph is not None:
39+
out._gfql_rows_base_graph = rows_base_graph
40+
return out
41+
42+
43+
def clear_row_exec_context(g: "Plottable") -> "Plottable":
44+
"""Return ``g`` stripped of the row context, for a result handed back to the caller.
45+
46+
The mirror of ``attach_row_exec_context``: each site that attaches also detaches, so
47+
the plumbing never escapes on a user-visible result. Without this the RESULT of a
48+
``WITH`` query carries the seed, and a follow-up query on that result -- a different
49+
graph entirely -- is answered against it (the same #1786 defect, one hop removed).
50+
Returns ``g`` unchanged when there is nothing to strip, so the common path is free.
51+
"""
52+
if g._gfql_start_nodes is None and g._gfql_rows_base_graph is None:
53+
return g
54+
out = g.bind()
55+
out._gfql_start_nodes = None
56+
out._gfql_rows_base_graph = None
57+
return out

graphistry/compute/gfql/lazy/engine/polars/chain.py

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -503,14 +503,16 @@ def _run_calls_polars(g_cur, calls, start_nodes, base_graph, middle):
503503
"""
504504
from graphistry.compute.ast import ASTCall, ASTNode as _ASTNode, ASTEdge as _ASTEdge, rows as rows_fn
505505
from graphistry.compute.chain import serialize_binding_ops
506+
from graphistry.compute.gfql.exec_context import attach_row_exec_context, clear_row_exec_context
506507

507508
calls = list(calls)
508509
if not calls:
509510
return g_cur
510511

511-
if start_nodes is not None:
512-
g_cur._gfql_start_nodes = start_nodes
513-
g_cur._gfql_rows_base_graph = base_graph
512+
# #1786: twin of the generic chain -- per-execution state on an INTERNAL COPY.
513+
# `g_cur` is the CALLER's graph on the all-calls boundary run, so assigning here
514+
# left the WITH re-entry seed behind for the next, unrelated query.
515+
g_cur = attach_row_exec_context(g_cur, start_nodes=start_nodes, rows_base_graph=base_graph)
514516

515517
if (
516518
middle
@@ -585,7 +587,9 @@ def _run_calls_polars(g_cur, calls, start_nodes, base_graph, middle):
585587
f"{getattr(op, 'function', op)!r}; use engine='pandas' for this query "
586588
f"(no pandas fallback; parity-or-error by design)"
587589
)
588-
return g_cur
590+
# Attach/detach pair: the boundary run is done, so the context is spent and must not
591+
# ride out on the result the caller sees (see the twin in compute/chain.py).
592+
return clear_row_exec_context(g_cur)
589593

590594

591595

graphistry/compute/gfql_unified.py

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@
3030
)
3131
from graphistry.compute.exceptions import ErrorCode, GFQLValidationError
3232
from graphistry.compute.gfql.cypher.parser import parse_cypher
33+
from graphistry.compute.gfql.exec_context import attach_row_exec_context, clear_row_exec_context
3334
from graphistry.compute.gfql.cypher.lowering import (
3435
ConnectedMatchJoinPlan,
3536
CompiledCypherGraphQuery,
@@ -1120,9 +1121,14 @@ def _execute_compiled_query_chain_non_union(
11201121
and getattr(_first_op, "function", None) == "rows"
11211122
and getattr(_first_op, "params", {}).get("binding_ops") is not None
11221123
):
1123-
dispatch_graph._gfql_start_nodes = start_nodes
1124+
# #1786: on the no-seed-rows path `_seeded_dispatch_graph` hands back
1125+
# `base_graph` ITSELF (the caller's object), so this must land on a copy.
1126+
dispatch_graph = attach_row_exec_context(dispatch_graph, start_nodes=start_nodes)
11241127

11251128
result = _chain_dispatch(dispatch_graph, compiled_query.chain, engine, policy, context, start_nodes=start_nodes)
1129+
# Attach/detach pair (#1786): the chain has run, so the seed is spent and must not
1130+
# ride out on the result -- a follow-up query on it is about a DIFFERENT graph.
1131+
result = clear_row_exec_context(result)
11261132
if compiled_query.empty_result_row is not None:
11271133
result = _apply_empty_result_row(
11281134
result,
@@ -1847,8 +1853,14 @@ def gfql(self: Plottable,
18471853
e.query_type = policy_context.get('query_type')
18481854
raise
18491855

1856+
# #1786: `shortest_path_backend` is an argument to THIS call, not a property of
1857+
# the caller's graph, so it may not be written onto `self`. Copy only when the
1858+
# value actually differs: the default call then keeps `self`'s identity, and the
1859+
# compiled-query memo cache below is owned by `self` either way.
18501860
dispatch_self = self
1851-
dispatch_self._gfql_shortest_path_backend = shortest_path_backend
1861+
if self._gfql_shortest_path_backend != shortest_path_backend:
1862+
dispatch_self = self.bind()
1863+
dispatch_self._gfql_shortest_path_backend = shortest_path_backend
18521864
compiled_query = None
18531865

18541866
if where_param and isinstance(query, (dict, ASTLet)):
@@ -1904,7 +1916,9 @@ def gfql(self: Plottable,
19041916
query,
19051917
language=language,
19061918
params=params,
1907-
cache_owner=dispatch_self,
1919+
# `self`, not `dispatch_self`: the memo cache must outlive the call,
1920+
# and `dispatch_self` is a throwaway copy on the non-default backend.
1921+
cache_owner=self,
19081922
node_dtypes=_node_dtypes_for_pushdown(self, engine),
19091923
)
19101924
except GFQLValidationError as exc:
Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
"""A query must not mutate the ``Plottable`` it was run on (#1786).
2+
3+
``WITH`` re-entry seeds the follow-up ``MATCH`` from the carried nodes. That seed used to be
4+
assigned to the CALLER's graph and never cleared, so the next -- entirely unrelated -- query on
5+
the same object was answered against the stale seed: no error, just the previous query's count.
6+
Engine-independent (wrong on pandas too), so an engine A/B never surfaces it.
7+
8+
Everything here is written to FAIL if the leak comes back: the assertions compare a query run on
9+
a re-used graph against the SAME query run on a pristine one, and pin the absolute counts so a
10+
fix that merely makes both sides equally wrong is caught too.
11+
"""
12+
import pandas as pd
13+
import pytest
14+
15+
import graphistry
16+
17+
18+
def _engines():
19+
out = ["pandas"]
20+
try:
21+
import polars # noqa: F401
22+
out.append("polars")
23+
except Exception:
24+
pass
25+
return out
26+
27+
28+
ENGINES = _engines()
29+
30+
# The #1786 repro graph. `kind` alternates a/b/a/b/a/b/a, so `{kind:'a'}` selects 4 of 7 nodes.
31+
NODES = pd.DataFrame({"id": list(range(7)), "kind": ["a", "b"] * 3 + ["a"]})
32+
EDGES = pd.DataFrame(
33+
[(0, 3), (1, 2), (4, 6), (3, 5), (5, 6), (0, 6), (3, 4)], columns=["s", "d"]
34+
)
35+
36+
PLAIN = "MATCH (a)-[*]->(b) RETURN count(*) AS c"
37+
REENTRY = "MATCH (a {kind:'a'}) WITH a MATCH (a)-[*]->(b) RETURN count(*) AS c"
38+
NESTED = (
39+
"MATCH (a {kind:'a'}) WITH a MATCH (a)-[]->(b) WITH b MATCH (b)-[]->(c) "
40+
"RETURN count(*) AS c"
41+
)
42+
43+
# Independently established on this fixed graph (identical on both engines).
44+
PLAIN_COUNT = 13
45+
REENTRY_COUNT = 7
46+
NESTED_COUNT = 2
47+
48+
49+
def _g():
50+
"""A PRISTINE graph. Every oracle needs one: re-using a graph is the thing under test."""
51+
return graphistry.nodes(NODES, "id").edges(EDGES, "s", "d")
52+
53+
54+
def _count(g, query, engine):
55+
return int(g.gfql(query, engine=engine)._nodes["c"].to_list()[0])
56+
57+
58+
@pytest.mark.parametrize("engine", ENGINES)
59+
def test_reentry_then_unrelated_query_is_unaffected(engine):
60+
"""THE regression: two independent queries, one graph. The second must not see the first."""
61+
g = _g()
62+
assert _count(g, REENTRY, engine) == REENTRY_COUNT
63+
assert _count(g, PLAIN, engine) == PLAIN_COUNT
64+
# ... and identical to the same query on a graph that never ran the re-entry query.
65+
assert _count(g, PLAIN, engine) == _count(_g(), PLAIN, engine)
66+
67+
68+
@pytest.mark.parametrize("engine", ENGINES)
69+
def test_reentry_leaves_no_execution_state_on_caller_graph(engine):
70+
"""Directly: the per-execution fields on the caller's object are untouched."""
71+
g = _g()
72+
g.gfql(REENTRY, engine=engine)
73+
assert g._gfql_start_nodes is None
74+
assert g._gfql_rows_base_graph is None
75+
76+
77+
@pytest.mark.parametrize("engine", ENGINES)
78+
def test_reentry_still_works(engine):
79+
"""Do not "fix" the leak by breaking WITH: the re-entry answer itself must survive."""
80+
assert _count(_g(), REENTRY, engine) == REENTRY_COUNT
81+
assert REENTRY_COUNT != PLAIN_COUNT # else the test above proves nothing
82+
83+
84+
@pytest.mark.parametrize("engine", ENGINES)
85+
def test_repeated_reentry_is_stable(engine):
86+
"""Re-entry twice on one graph: the second run must not inherit the first's seed."""
87+
g = _g()
88+
assert [_count(g, REENTRY, engine) for _ in range(3)] == [REENTRY_COUNT] * 3
89+
90+
91+
@pytest.mark.parametrize("engine", ENGINES)
92+
def test_nested_reentry(engine):
93+
"""Two WITH boundaries in one query, then an unrelated query on the same graph."""
94+
g = _g()
95+
assert _count(g, NESTED, engine) == NESTED_COUNT
96+
assert g._gfql_start_nodes is None
97+
assert _count(g, PLAIN, engine) == PLAIN_COUNT
98+
99+
100+
@pytest.mark.parametrize("engine", ENGINES)
101+
def test_query_order_does_not_change_answers(engine):
102+
"""Whole point of immutability: the answers do not depend on execution order."""
103+
g1 = _g()
104+
plain_first = (_count(g1, PLAIN, engine), _count(g1, REENTRY, engine))
105+
g2 = _g()
106+
reentry_first = (_count(g2, REENTRY, engine), _count(g2, PLAIN, engine))
107+
assert plain_first == (PLAIN_COUNT, REENTRY_COUNT)
108+
assert reentry_first == (REENTRY_COUNT, PLAIN_COUNT)
109+
110+
111+
@pytest.mark.parametrize("engine", ENGINES)
112+
def test_reentry_result_does_not_carry_execution_state(engine):
113+
"""The same defect one hop out: a RESULT that carries the seed poisons queries on IT."""
114+
out = _g().gfql("MATCH (a {kind:'a'}) WITH a MATCH (a)-[]->(b) RETURN a", engine=engine)
115+
assert out._gfql_start_nodes is None
116+
assert out._gfql_rows_base_graph is None
117+
118+
119+
@pytest.mark.parametrize("engine", ENGINES)
120+
def test_pure_call_chain_result_does_not_carry_execution_state(engine):
121+
"""The all-calls boundary run (``let()`` bodies) hands its graph straight through.
122+
123+
On polars that graph is the one the run started from, so the boundary's base-graph
124+
write lands on the returned object unless it is attached to a copy and cleared.
125+
"""
126+
from graphistry.compute.ast import call
127+
128+
out = _g().gfql([call("rows", {"table": "nodes"})], engine=engine)
129+
assert out._gfql_rows_base_graph is None
130+
assert out._gfql_start_nodes is None
131+
132+
133+
@pytest.mark.parametrize("engine", ENGINES)
134+
def test_shortest_path_backend_is_not_written_onto_caller_graph(engine):
135+
"""A per-CALL argument is not a property of the graph; it must not persist on it."""
136+
g = _g()
137+
g.gfql(PLAIN, engine=engine, shortest_path_backend="bfs")
138+
assert g._gfql_shortest_path_backend == "auto"

0 commit comments

Comments
 (0)