Skip to content

Commit 8742dd2

Browse files
committed
test(gfql/index): GPU coverage for the indexed edge_match path, honestly scoped
The stack's own regression tests are all `parametrize("engine", _cpu_engines())` — pandas + polars — so the Engine.CUDF and POLARS_GPU branches had NO coverage from them. Switching those to `_engines()` is not the fix: that helper includes cudf whenever cudf is IMPORTABLE, so on any box with cudf installed but no CUDA runtime every such test fails (verified: 12 failures locally). So this is a separate, properly gated file. The gate is the operation itself — a tiny indexed typed hop on cudf — because every cheaper probe fails to discriminate on a half-installed box: `cudf.DataFrame(...)`, `.to_pandas()`, `.values` (a small cupy alloc), `==`, and even `groupby().sum()` all SUCCEED there and the suite then dies with `OSError: libnvrtc.so.12` in the first real kernel. Verified BOTH ways, which is the point of a gated file: local (no CUDA runtime): 12 skipped dgx-spark, graphistry/test-rapids-official:26.02-gfql-polars --gpus all: 12 passed (`--gpus all` is required; without it the 26.02 runtime reports cudaErrorInsufficientDriver and the file would skip everywhere — a silently always-skipping test file being strictly worse than none.) Covers, against the PANDAS oracle: null-bearing edge predicate columns across int64/Int64/float/string/boolean on cudf and polars-gpu, and the empty candidate batch on device. Discloses what it does NOT pin, mutation-checked: deleting the `fillna(False)` before `.values` in the cudf branch leaves all 12 GREEN on cudf 26.02, because `(sub == val)` there already yields a non-null boolean column. The fillna is defensive on this version, not load-bearing; the docstring says so rather than letting a green run imply otherwise.
1 parent 72ae3c3 commit 8742dd2

1 file changed

Lines changed: 127 additions & 0 deletions

File tree

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
"""GPU coverage for the indexed `edge_match` candidate-row path (#1782) and the semi-join
2+
key sides (#1784).
3+
4+
Why a separate file: the regression tests for those changes are parametrized over
5+
``_cpu_engines()`` — pandas + polars — so the ``Engine.CUDF`` and ``Engine.POLARS_GPU``
6+
branches had NO coverage from them. They cannot simply be switched to ``_engines()``:
7+
that helper includes cudf whenever cudf is *importable*, so on a dev box with cudf
8+
installed but no working GPU every such test fails. This file gates on a real runtime
9+
probe instead, so it runs where there is a GPU and skips cleanly where there is not.
10+
11+
The specific device risks these pin, all raised in review of #1782:
12+
* ``(sub == val).fillna(False).values`` on a GATHERED cudf column — ``.values`` RAISES on a
13+
null-bearing cudf column, so correctness rests entirely on ``fillna`` having cleared nulls
14+
for EVERY result dtype, not just the numeric ones.
15+
* the EMPTY candidate batch on device (a zero-length gather map).
16+
* the cost guard's whole-column fallback firing on device.
17+
Pandas is the oracle in every case — comparing a GPU engine only against another GPU engine
18+
would let a shared defect pass.
19+
20+
WHAT THESE DO **NOT** PIN, measured rather than assumed: deleting the ``fillna(False)`` from
21+
the cudf branch of ``_EdgeMatchRowFilter.mask_for`` leaves all of these GREEN on cudf 26.02.
22+
The review's concern was that ``.values`` raises on a null-bearing cudf column; on this
23+
version ``(sub == val)`` already yields a non-null boolean column, so the fillna is defensive
24+
rather than load-bearing *here*. It is kept because the pandas branch genuinely needs it and
25+
because a future cudf could restore null propagation — but do not read a green run of this
26+
file as evidence that the fillna is required. What the file DOES pin is device-vs-pandas
27+
parity across null-bearing dtypes and the empty-batch path, both of which had no coverage at
28+
all before: every new test the stack added is parametrized over ``_cpu_engines()``.
29+
"""
30+
import numpy as np
31+
import pandas as pd
32+
import pytest
33+
34+
import graphistry
35+
from graphistry.compute.gfql.index import create_index # noqa: F401 (registers plottable API)
36+
37+
cudf = pytest.importorskip("cudf")
38+
39+
40+
def _gpu_available() -> bool:
41+
"""Probe by running the smallest version of what these tests do.
42+
43+
Cheaper probes do not discriminate on a box with cudf installed but no CUDA runtime:
44+
`cudf.DataFrame(...)`, `.to_pandas()`, `.values` (a small cupy alloc), a comparison, and
45+
even `groupby().sum()` all SUCCEED there, and the suite then fails with
46+
`OSError: libnvrtc.so.12` inside the first real kernel. So the probe is an actual indexed
47+
typed hop on a two-edge graph — if that runs, everything below can.
48+
"""
49+
try:
50+
import numpy as _np
51+
import pandas as _pd
52+
from graphistry.Engine import Engine as _E, df_to_engine as _to
53+
_n = _to(_pd.DataFrame({"id": _np.arange(3, dtype=_np.int64)}), _E.CUDF)
54+
_e = _to(_pd.DataFrame({"src": [0, 1], "dst": [1, 2], "etype": [1, 2]}), _E.CUDF)
55+
_g = graphistry.nodes(_n, "id").edges(_e, "src", "dst").gfql_index_all(engine="cudf")
56+
_g.hop(nodes=_n[:1], hops=1, return_as_wave_front=True,
57+
edge_match={"etype": 1}, engine="cudf")
58+
return True
59+
except Exception:
60+
return False
61+
62+
63+
pytestmark = pytest.mark.skipif(not _gpu_available(), reason="no working cudf / GPU")
64+
65+
NULL_DTYPES = ["int64", "Int64", "float", "string", "boolean"]
66+
67+
68+
def _frames(null_col_dtype: str, n_nodes: int = 400, deg: int = 6):
69+
"""Typed graph whose edge predicate column carries NULLS — the case `.values` trips on."""
70+
rng = np.random.default_rng(3)
71+
m = n_nodes * deg
72+
etype = rng.integers(0, 3, m)
73+
edges = pd.DataFrame({"src": rng.integers(0, n_nodes, m),
74+
"dst": rng.integers(0, n_nodes, m),
75+
"etype": pd.Series(etype).astype(null_col_dtype)
76+
if null_col_dtype not in ("string", "boolean")
77+
else pd.Series([str(v) for v in etype] if null_col_dtype == "string"
78+
else (etype % 2).astype(bool)).astype(null_col_dtype)})
79+
edges.loc[edges.index[::7], "etype"] = None # ~14% nulls
80+
nodes = pd.DataFrame({"id": np.arange(n_nodes, dtype=np.int64)})
81+
return nodes, edges
82+
83+
84+
def _match_value(null_col_dtype: str):
85+
return {"string": "1", "boolean": True}.get(null_col_dtype, 1)
86+
87+
88+
@pytest.mark.parametrize("engine", ["cudf", "polars-gpu"])
89+
@pytest.mark.parametrize("null_col_dtype", NULL_DTYPES)
90+
def test_null_bearing_edge_predicate_matches_the_pandas_oracle_on_device(engine, null_col_dtype):
91+
"""`.values` on a gathered, null-bearing device column must neither raise nor diverge."""
92+
nodes, edges = _frames(null_col_dtype)
93+
g_pd = graphistry.nodes(nodes, "id").edges(edges, "src", "dst")
94+
kwargs = dict(hops=1, return_as_wave_front=True,
95+
edge_match={"etype": _match_value(null_col_dtype)})
96+
oracle = g_pd.hop(nodes=nodes[:1], engine="pandas", **kwargs)
97+
98+
from graphistry.Engine import Engine as _E, df_to_engine
99+
target = _E.CUDF if engine == "cudf" else _E.POLARS
100+
g = graphistry.nodes(df_to_engine(nodes, target), "id").edges(
101+
df_to_engine(edges, target), "src", "dst")
102+
got = g.gfql_index_all(engine=engine).hop(
103+
nodes=df_to_engine(nodes[:1], target), engine=engine, **kwargs)
104+
105+
def pairs(gg):
106+
df = gg._edges
107+
p = df.to_pandas() if hasattr(df, "to_pandas") else df
108+
return sorted(zip(p["src"].tolist(), p["dst"].tolist()))
109+
110+
assert pairs(got) == pairs(oracle), f"[{engine}/{null_col_dtype}] diverged from pandas"
111+
112+
113+
@pytest.mark.parametrize("engine", ["cudf", "polars-gpu"])
114+
def test_empty_candidate_batch_on_device(engine):
115+
"""A seed with no matching typed edges yields a zero-length gather map on device."""
116+
nodes, edges = _frames("int64")
117+
from graphistry.Engine import Engine as _E, df_to_engine
118+
target = _E.CUDF if engine == "cudf" else _E.POLARS
119+
g_pd = graphistry.nodes(nodes, "id").edges(edges, "src", "dst")
120+
kwargs = dict(hops=1, return_as_wave_front=True, edge_match={"etype": 99}) # matches nothing
121+
oracle = g_pd.hop(nodes=nodes[:1], engine="pandas", **kwargs)
122+
123+
g = graphistry.nodes(df_to_engine(nodes, target), "id").edges(
124+
df_to_engine(edges, target), "src", "dst")
125+
got = g.gfql_index_all(engine=engine).hop(
126+
nodes=df_to_engine(nodes[:1], target), engine=engine, **kwargs)
127+
assert int(got._edges.shape[0]) == int(oracle._edges.shape[0]) == 0

0 commit comments

Comments
 (0)