From b07e3305116354483016544708c85ab7954b29f3 Mon Sep 17 00:00:00 2001 From: Leo Meyerovich Date: Mon, 27 Jul 2026 18:08:59 -0700 Subject: [PATCH] test(gfql): run the #1788/#1790 suites on all four engines, in a lane that exists (CB3 a+b) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both suites tested the semantics of WHICH TABLE a query returns — engine-agnostic semantics — on pandas + polars only, and neither file was in `bin/test-polars.sh`. `test_rows_table_named_middle.py` also carried a module-level `pytest.importorskip("polars")`, and the `test-gfql-core` lane installs no polars, so that file ran in NO CI LANE AT ALL. `test_rewrite_param_discard.py` named cuDF but skipped polars-gpu outright. Now: fixed engine list (pandas / cuDF / polars / polars-gpu), classified availability check, both files in the polars lane, module-level importorskip dropped (5 pandas cases become live in test-gfql-core). TWO PROBE TRAPS, BOTH HIT BY TRYING RATHER THAN REASONED ABOUT: * A probe that runs THE SHAPE UNDER TEST disarms its own file. Reverting the #1788 `table` guard made the probe raise, and all 20 parameters reported SKIPPED instead of failing. The smoke query is now a plain traversal with no `rows()` call in it. * A probe that SWALLOWS every exception is worse than none. A transient `MemoryError: ... cudaErrorMemoryAllocation` in a fresh GPU container silently dropped cuDF from a run that otherwise passed — and a skipped GPU parameter reads as evidence of passing. `engine_skip_reason` now CLASSIFIES: a missing module skips, a recognisable GPU-stack error skips with its text quoted, and ANY other failure propagates. It is deliberately not `available_nonpandas_engines()`, which builds its list by importability, so a missing engine vanishes from the report rather than showing as SKIPPED. GPU RECEIPTS — dgx GB10, `graphistry/test-rapids-official:26.02-gfql-polars`, `docker run --gpus all`, cuDF 26.02.01 / polars 1.35.2: 45 passed, 3 xfailed, 0 skipped (repeated; all four engines running) MUTATION-CHECKED on the same box by reverting the #1788 and #1790 `table` guards: 24 failed / 0 skipped, spread EVENLY over the four engines (6 each) so the added parameters are not decorative. TWO REAL GAPS SURFACED BY THE NEW PARAMETERS, pinned strict-xfail, not papered over: * #1803 — the indexed bindings bypass gate admits only (PANDAS, CUDF, POLARS), while the native polars chain hands it `Engine.POLARS_GPU` whenever the GPU target is active. So polars-gpu ALWAYS reports `served: False, reason: 'unsupported_engine'` and silently runs the canonical scan. Values are unaffected; the optimization is lost with no signal. * #1804 — the native polars bindings builder never receives `alias_prefilters` (5 rows on pandas/cuDF vs 12 on polars). Already xfailed before this change, but with no issue number, so there was nothing to close and nothing to find. RUNTIME DELTA: zero. No production line changed — the diff is two test files, a shared test helper, `bin/test-polars.sh` and the CHANGELOG — so no pyg-bench lane run is required (CB5), and that claim is checkable from the diff. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_015YsqAZQLbqjSDrYSFz2GoB --- CHANGELOG.md | 1 + bin/test-polars.sh | 4 + .../tests/compute/gfql/polars_test_utils.py | 55 +++++++++++ .../gfql/test_rewrite_param_discard.py | 97 +++++++++++++++++-- .../gfql/test_rows_table_named_middle.py | 95 +++++++++++++++--- 5 files changed, 232 insertions(+), 20 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ac88700713..2b43f98e59 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,6 +24,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm - **Seeded typed-hop property projection keeps its fast path through DISTINCT / ORDER BY / SKIP / LIMIT**: those trailing row ops are plain frame operations, so the fast path now delegates them to the canonical chain instead of declining the whole query. ### Changed +- **`rows(table=...)` and rewrite-param regression suites now run on all four engines, and their polars params run in a lane at all** (tests only; zero runtime delta — no production line changed). The #1788 and #1790 suites were parametrized over pandas + polars only, and neither file was in `bin/test-polars.sh`, so `test_rows_table_named_middle.py` — which carried a module-level `pytest.importorskip("polars")` — ran in **no CI lane at all** (the `test-gfql-core` lane installs no polars, so the whole file skipped there). Both files are now parametrized over pandas / cuDF / polars / polars-gpu from a **fixed** engine list plus a classified availability check, and both are in the polars lane; dropping the module-level `importorskip` also makes 5 pandas cases live in `test-gfql-core`. The availability check is deliberately not `available_nonpandas_engines()`, which builds its list by importability so a missing engine vanishes from the report instead of showing as SKIPPED, and it is deliberately not a blanket `except Exception` either: a missing module skips, a recognisable GPU-stack error skips **with its text quoted**, and any other failure propagates — because a skipped GPU parameter reads as evidence of passing. Both traps were hit by trying rather than reasoned about: a probe that ran the shape under test turned a reverted production guard into 20 SKIPS instead of 20 failures, and a blanket probe silently dropped cuDF from an otherwise-green GPU run on a transient `cudaErrorMemoryAllocation`. GPU receipts on dgx GB10 / cuDF 26.02.01 / polars 1.35.2 / `docker run --gpus all`: **45 passed, 3 xfailed, 0 skipped**; mutation-checked by reverting the #1788 and #1790 `table` guards — **24 failures spread evenly over all four engines (6 each)**, so the added parameters are not decorative. Two gaps the new parameters surfaced are pinned as strict xfails rather than papered over: #1803 (the indexed bindings bypass excludes `Engine.POLARS_GPU`, so polars-gpu silently takes the scan path) and #1804 (the native polars bindings builder never receives `alias_prefilters` — previously an unnumbered xfail). - **`is_lazy` is a type predicate, so the polars engine's eager/lazy split is checked instead of asserted**: `dtypes.is_lazy` decides WHICH member of the two-member `PolarsFrame` union a frame is, but it was declared `-> bool`, so that answer was discarded at the call boundary — an eager-only attribute (`.height`, `.columns`, `.schema`) reached after a lazy guard was unprovable, and the native polars chain closed the gap with `cast("pl.DataFrame", frame)`. A cast is not a type: it re-asserts, unchecked, the exact fact the predicate had just established, once per call site, and a wrong one surfaces as a production `AttributeError` rather than a CI error. `is_lazy` is now declared `-> TypeIs["pl.LazyFrame"]` (PEP 742), which narrows the *negative* branch as well as the positive — the eager side is the one that needed it, so `TypeGuard` would not have done — and the cast is gone. Four previously untyped chain-combine helpers (`_semi`, `_combine_edges`, `_apply_node_names`, and the `_known_empty` guard) now carry real signatures: `_semi`'s two frames share the `PolarsT` TypeVar because polars joins do not mix eagerness, and the two combine helpers take the `_LazyShim` collect-once duck-type they are actually called with. Typing `_apply_node_names` also retired a `getattr(next_step, "edges_empty", None)` — the shim declares that tri-state in `__slots__`, so a typo is now a checker error instead of a silent `None` that would have re-armed the very cardinality gate the guard disarms. Internal only: no runtime behaviour, no public API, and the `TypeIs` import is `TYPE_CHECKING`-only, so no new runtime dependency floor. - **GFQL execution context is declared rather than attached at runtime**: the private `_gfql_*` per-execution fields (index policy and registry, row-pipeline base graph, carried seed nodes, edge aliases, shortest-path backend, and the indexed-bindings handoff) are now declared on `Plottable` with defaults on `PlotterBase`, instead of being set with `setattr` and read back with `getattr(..., default)`. No public API or behaviour change; internal call sites are typed, and hand-rolled `Plottable` stand-ins must now construct the full context. diff --git a/bin/test-polars.sh b/bin/test-polars.sh index 3bfc16a4dd..fa6e4e6eba 100755 --- a/bin/test-polars.sh +++ b/bin/test-polars.sh @@ -25,6 +25,10 @@ POLARS_TEST_FILES=( graphistry/tests/compute/gfql/test_engine_polars_row_pipeline.py graphistry/tests/compute/gfql/test_engine_polars_binding_rows.py graphistry/tests/compute/gfql/test_engine_polars_with_match_reentry.py + # #1788 / #1790 regression suites: engine-parametrized over pandas/cuDF/polars/polars-gpu, + # so their POLARS params only ever run here (test-gfql-core installs no polars) + graphistry/tests/compute/gfql/test_rows_table_named_middle.py + graphistry/tests/compute/gfql/test_rewrite_param_discard.py graphistry/tests/compute/gfql/test_engine_polars_cypher_conformance.py graphistry/tests/compute/gfql/test_engine_polars_conformance_matrix.py graphistry/tests/compute/gfql/test_polars_string_predicate_nonstring.py diff --git a/graphistry/tests/compute/gfql/polars_test_utils.py b/graphistry/tests/compute/gfql/polars_test_utils.py index a825d087f3..27927cebed 100644 --- a/graphistry/tests/compute/gfql/polars_test_utils.py +++ b/graphistry/tests/compute/gfql/polars_test_utils.py @@ -181,3 +181,58 @@ def assert_surfaces_agree(res_a, res_b, label): assert res_a[0] == res_b[0], f"{label}: surface divergence {res_a[0]} != {res_b[0]} (silent-bridge class)" if res_a[0] == "ok": assert res_a[1] == res_b[1], f"{label}: surfaces both ok but signatures differ" + + +# --- fixed-engine-list plumbing (#1795 / CB3). ONE definition, because the alternative is the +# trap this replaces: `available_nonpandas_engines()` above BUILDS its list by importability, so +# an engine that is missing simply vanishes from the report instead of showing as SKIPPED. --- + +#: Substrings that identify a broken/absent GPU STACK rather than a product defect. Matched +#: against `type(ex).__name__: str(ex)`, lowercased. Deliberately narrow: everything not listed +#: here is treated as a REAL failure, because the cost of guessing wrong in that direction is a +#: silently green GPU lane. +_GPU_ENVIRONMENT_MARKERS = ( + "libnvrtc", "libcuda", "libcudart", "nvrtc", "cuinit", + "no cuda-capable device", "cuda driver", "cuda runtime", + "cuda_error", "cudaerror", "driver version", + "gpu engine requested", # polars' own message when cudf_polars is absent +) + + +def gpu_environment_reason(ex: BaseException) -> "str | None": + """Return a skip reason iff ``ex`` is an ENVIRONMENT fact, else None (= a real failure). + + A probe that swallows every exception is worse than no probe: it converts a genuine + regression into a SKIP, and a skipped GPU parameter reads as evidence of passing. Measured + on the dgx box — a probe that ran the shape under test turned a reverted production guard + into 20 skips instead of 20 failures, and an intermittent cold-start error in a fresh + container silently dropped cuDF from a run that otherwise passed. + """ + if isinstance(ex, ImportError): + return f"{type(ex).__name__}: {ex}" + text = f"{type(ex).__name__}: {ex}".lower() + if any(marker in text for marker in _GPU_ENVIRONMENT_MARKERS): + return f"{type(ex).__name__}: {ex}" + return None + + +def engine_skip_reason(engine, smoke): + """``None`` => this engine MUST run here. A string => a stated, checkable skip reason. + + ``smoke`` is a zero-argument callable running a query that is NOT the shape under test — + running the asserted shape would let a regression disarm its own test. + """ + import importlib.util + + for module in {"polars": ("polars",), "polars-gpu": ("polars", "cudf_polars"), + "cudf": ("cudf",)}.get(engine, ()): + if importlib.util.find_spec(module) is None: + return f"{module} is not installed" + try: + smoke() + except BaseException as ex: # noqa: BLE001 — classified below, never blanket-swallowed + reason = gpu_environment_reason(ex) + if reason is not None: + return reason + raise + return None diff --git a/graphistry/tests/compute/gfql/test_rewrite_param_discard.py b/graphistry/tests/compute/gfql/test_rewrite_param_discard.py index 71de68d958..71d6f647fc 100644 --- a/graphistry/tests/compute/gfql/test_rewrite_param_discard.py +++ b/graphistry/tests/compute/gfql/test_rewrite_param_discard.py @@ -24,16 +24,36 @@ ``attach_prop_aliases`` — the #1711 projection pushdown — that is observable: naming the middle instead of spelling ``binding_ops`` by hand silently attaches EVERY alias's properties. Same query, same bindings, different output schema. + +ENGINE LIST IS FIXED, NOT PROBED-INTO-EXISTENCE. ``polars-gpu`` joins the three engines this +file already carried: it is the GPU collect target of the SAME native polars chain, so it +runs the same rewrite and the same indexed bypass, and it must therefore land on the same +side of every gate polars does — including the strict xfail at the bottom. An engine that +cannot run in the current environment reports SKIPPED with a reason; it never vanishes from +the report (the trap in ``polars_test_utils.available_nonpandas_engines()``, which silently +shrinks). The probe RUNS a query rather than importing a module, because a box where +``cudf``/``cudf_polars`` import against an incomplete CUDA runtime passes every import check +and then dies inside the first real kernel — which reads as a product failure rather than a +missing environment. + +COVERAGE BOUNDARY, stated rather than hidden behind a skip: no CI lane runs cuDF or +polars-gpu (``ci-gpu.yml`` is hard-disabled and does not install the ``polars`` extra), so +those two parameters report SKIPPED on CI. They are exercised out of band on the dgx GPU box +against ``graphistry/test-rapids-official:26.02-gfql-polars`` (``docker run --gpus all`` — +omitting ``--gpus all`` FABRICATES failures rather than skipping). Treat a green CI run as +evidence for pandas + polars only. """ from __future__ import annotations -from typing import Any, List +from functools import lru_cache +from typing import Any, List, Optional, Tuple import numpy as np import pandas as pd import pytest import graphistry +from graphistry.tests.compute.gfql.polars_test_utils import engine_skip_reason from graphistry.compute.ast import ( ASTObject, e_forward, e_undirected, n, rows, select, serialize_binding_ops, ) @@ -53,11 +73,13 @@ # comes back — so the tests below pin the SMALL number, not merely "not empty". ALL_EDGES = len(EDGES) -ENGINES = ["pandas", "polars", "cudf"] +PANDAS_API_ENGINES: Tuple[str, ...] = ("pandas", "cudf") +POLARS_API_ENGINES: Tuple[str, ...] = ("polars", "polars-gpu") +ENGINES: Tuple[str, ...] = PANDAS_API_ENGINES + POLARS_API_ENGINES def _frames(engine: str) -> Any: - if engine == "polars": + if engine in POLARS_API_ENGINES: pl = pytest.importorskip("polars") return pl.from_pandas(NODES), pl.from_pandas(EDGES) if engine == "cudf": @@ -72,6 +94,38 @@ def _graph(engine: str, *, indexed: bool) -> Any: return g.gfql_index_all(engine=engine) if indexed else g +@lru_cache(maxsize=None) +def _engine_skip_reason(engine: str) -> Optional[str]: + """``None`` => this engine MUST run here; a string => a stated, checkable skip reason. + + Runs BOTH halves this file needs, because the INDEXED path is the one that reaches cupy + kernels: a scan-only smoke test lets an incomplete CUDA install through and reports its + kernel error as a test failure (measured on a box with cudf 25.10 importable and + `libnvrtc.so.12` missing — 4 fabricated failures). + + Never the shape under test: a probe that runs it disarms the file — reverting the + indexed-bypass `table` guard turned every parameter into a SKIP instead of a failure. And + never a blanket `except Exception`: a missing module skips, a recognisable GPU-stack error + skips with its text quoted, and ANY other failure propagates, because a skipped GPU + parameter reads as evidence of passing. + """ + def smoke() -> None: + ops = [n({"id": 1}), e_forward(), n()] + _graph(engine, indexed=False).gfql(list(ops), engine=engine) + _graph(engine, indexed=True).gfql(list(ops), engine=engine, index_policy="force") + + return engine_skip_reason(engine, smoke) + + +def _require(engine: str) -> None: + reason = _engine_skip_reason(engine) + if reason is not None: + pytest.skip( + f"engine {engine!r} unavailable here ({reason}) — NOT evidence that it passes; " + "see the COVERAGE BOUNDARY note in this module's docstring" + ) + + def _run(engine: str, ops: List[ASTObject], *, indexed: bool) -> pd.DataFrame: kwargs: dict = {"engine": engine} if indexed: @@ -100,6 +154,7 @@ def test_indexed_bypass_honours_rows_table_edges(engine: str) -> None: Indexed and unindexed must agree, and both must be the narrowed set. The absolute count is pinned because the failure mode is exactly `ALL_EDGES`. """ + _require(engine) ops = [n({"id": 1}, name="a"), e_forward(name="r"), n(name="b"), rows(table="edges")] scan = _run(engine, ops, indexed=False) indexed = _run(engine, ops, indexed=True) @@ -114,6 +169,7 @@ def test_indexed_bypass_honours_rows_table_edges(engine: str) -> None: @pytest.mark.parametrize("engine", ENGINES) def test_indexed_bypass_honours_rows_table_edges_undirected(engine: str) -> None: """Undirected middle: node 1 has four incident edges, not twelve.""" + _require(engine) ops = [n({"id": 1}, name="a"), e_undirected(name="r"), n(name="b"), rows(table="edges")] scan = _run(engine, ops, indexed=False) indexed = _run(engine, ops, indexed=True) @@ -133,6 +189,7 @@ def test_indexed_bypass_table_edges_survives_a_projection(engine: str) -> None: values can distinguish them — and the indexed path returned every edge type in the graph rather than the three on node 1's outgoing edges. """ + _require(engine) ops = [n({"id": 1}, name="a"), e_forward(name="r"), n(name="b"), rows(table="edges"), select([("type", "type")])] scan = _run(engine, ops, indexed=False) @@ -143,14 +200,33 @@ def test_indexed_bypass_table_edges_survives_a_projection(engine: str) -> None: ) -@pytest.mark.parametrize("engine", ENGINES) +@pytest.mark.parametrize("engine", [ + "pandas", + "cudf", + "polars", + pytest.param("polars-gpu", marks=pytest.mark.xfail( + strict=True, + reason="#1803: the indexed bindings bypass excludes Engine.POLARS_GPU " + "(index/bindings.py gate), so polars-gpu always takes the scan path", + )), +]) def test_indexed_bypass_still_serves_a_bare_rows(engine: str) -> None: """THE NEGATIVE SIDE: declining on a non-default `table` must not decline everything. Without this, "never take the bypass" would pass every test above while silently retiring the indexed bindings path. Asserted on the trace, not on the answer — the answer is identical either way, which is precisely why the regression would be quiet. + + ``polars-gpu`` STRICT-xfails, and that is the point of adding the parameter (#1802). The + native polars chain hands `Engine.POLARS_GPU` to `try_indexed_connected_bindings_state` + whenever the GPU target is active, but that function's first gate admits only + `(PANDAS, CUDF, POLARS)` — so the bypass reports `served: False, + reason: 'unsupported_engine'` and polars-gpu silently runs the canonical scan. Values are + unaffected (every other case in this file passes on polars-gpu); what is lost is the + optimization, with no signal. A strict xfail rather than a skip so that whoever widens the + gate is told to delete the marker instead of leaving a stale "known gap" behind. """ + _require(engine) from graphistry.compute.gfql.index import index_trace ops = [n({"id": 1}, name="a"), e_forward(name="r"), n(name="b"), rows()] @@ -184,6 +260,7 @@ def test_named_middle_rewrite_keeps_attach_prop_aliases(engine: str) -> None: against the explicit spelling rather than a hardcoded column list, so the assertion tracks whatever the bindings builder emits. """ + _require(engine) binding_ops = serialize_binding_ops(MIDDLE) rewritten = _run(engine, list(MIDDLE) + [rows(attach_prop_aliases=["a"])], indexed=False) explicit = _run(engine, [rows(binding_ops=binding_ops, attach_prop_aliases=["a"])], @@ -206,6 +283,7 @@ def test_named_middle_rewrite_keeps_attach_prop_aliases(engine: str) -> None: @pytest.mark.parametrize("engine", ENGINES) def test_named_middle_rewrite_keeps_attach_prop_aliases_values(engine: str) -> None: """The pushdown must narrow the SCHEMA without changing the bindings themselves.""" + _require(engine) binding_ops = serialize_binding_ops(MIDDLE) rewritten = _run(engine, list(MIDDLE) + [rows(attach_prop_aliases=["a"])], indexed=False) explicit = _run(engine, [rows(binding_ops=binding_ops, attach_prop_aliases=["a"])], @@ -229,11 +307,15 @@ def test_named_middle_rewrite_keeps_attach_prop_aliases_values(engine: str) -> N @pytest.mark.parametrize("engine", [ "pandas", + "cudf", pytest.param("polars", marks=pytest.mark.xfail( strict=True, - reason="native polars bindings builder never receives alias_prefilters", + reason="#1804: native polars bindings builder never receives alias_prefilters", + )), + pytest.param("polars-gpu", marks=pytest.mark.xfail( + strict=True, + reason="#1804: polars-gpu is the same native builder as polars, same missing param", )), - "cudf", ]) def test_alias_prefilters_are_honoured_by_the_bindings_builder(engine: str) -> None: """`alias_prefilters` narrows the bindings on pandas/cuDF — and is IGNORED on polars. @@ -242,7 +324,7 @@ def test_alias_prefilters_are_honoured_by_the_bindings_builder(engine: str) -> N `attach_prop_aliases` only, so the param never reaches it; the generic path threads it into `_gfql_binding_ops_row_table`. Same query, 5 rows vs 12. - Not fixed here: the hint is documented as ADVISORY (the cypher lowering that emits it + Filed as #1804. Not fixed here: the hint is documented as ADVISORY (the cypher lowering that emits it always keeps the equivalent post-join filter, so Cypher answers agree across engines and only the pushdown is lost), and honouring it natively means porting the whole prefilter evaluator — a feature, not this fix. Hand-written GFQL that passes @@ -252,6 +334,7 @@ def test_alias_prefilters_are_honoured_by_the_bindings_builder(engine: str) -> N polars learns the param this turns into a failure, so whoever fixes it is told to delete the marker instead of leaving a stale "known gap" behind. """ + _require(engine) binding_ops = serialize_binding_ops(OPEN_MIDDLE) unfiltered = _run(engine, [rows(binding_ops=binding_ops)], indexed=False) filtered = _run(engine, [rows(binding_ops=binding_ops, alias_prefilters=ALIAS_PREFILTER)], diff --git a/graphistry/tests/compute/gfql/test_rows_table_named_middle.py b/graphistry/tests/compute/gfql/test_rows_table_named_middle.py index 75fbcfefa6..6e3cc1e622 100644 --- a/graphistry/tests/compute/gfql/test_rows_table_named_middle.py +++ b/graphistry/tests/compute/gfql/test_rows_table_named_middle.py @@ -12,16 +12,38 @@ `(person)-[r:KNOWS]-`) -> the rewritten op list is not an alternating node/edge path, so validation hard-errored with "require ... a single connected alternating node/edge path". -Pinned on both engines because the rewrite is duplicated in `compute/chain.py` (generic) and -`gfql/lazy/engine/polars/chain.py` (native polars); fixing one alone leaves the other wrong. +Pinned on all four engines because the rewrite is duplicated in `compute/chain.py` (generic) +and `gfql/lazy/engine/polars/chain.py` (native polars); fixing one alone leaves the other +wrong. cuDF runs the generic rewrite and polars-gpu the native one, so each surface is +covered by two engines rather than one — "which table comes back" is engine-agnostic +semantics, not a polars question. + +ENGINE LIST IS FIXED, NOT PROBED-INTO-EXISTENCE. An engine that cannot run in the current +environment reports SKIPPED with a reason; it never vanishes from the report (the trap in +`polars_test_utils.available_nonpandas_engines()`, which silently shrinks). There is also no +module-level `pytest.importorskip("polars")` any more: it caused the whole file — pandas +params included — to be skipped in the `test-gfql-core` lane, which does not install polars. + +COVERAGE BOUNDARY, stated rather than hidden behind a skip: no CI lane runs cuDF or +polars-gpu (`ci-gpu.yml` is hard-disabled and does not install the `polars` extra), so those +two parameters report SKIPPED on CI. They are exercised out of band on the dgx GPU box +against `graphistry/test-rapids-official:26.02-gfql-polars` (`docker run --gpus all` — +omitting `--gpus all` FABRICATES failures rather than skipping). Treat a green CI run as +evidence for pandas + polars only. """ +from functools import lru_cache +from typing import Any, Optional, Tuple + import pandas as pd import pytest import graphistry from graphistry.compute import ast +from graphistry.tests.compute.gfql.polars_test_utils import engine_skip_reason -pl = pytest.importorskip("polars") +PANDAS_API_ENGINES: Tuple[str, ...] = ("pandas", "cudf") +POLARS_API_ENGINES: Tuple[str, ...] = ("polars", "polars-gpu") +ENGINES: Tuple[str, ...] = PANDAS_API_ENGINES + POLARS_API_ENGINES NODES = pd.DataFrame({"id": [0, 1, 2, 3], "firstName": ["a", "b", "c", "d"]}) @@ -31,24 +53,67 @@ EDGE_COLS = {"s", "d", "type", "creationDate"} +def _frame(engine: str, df: pd.DataFrame) -> Any: + if engine in POLARS_API_ENGINES: + pl = pytest.importorskip("polars") + return pl.from_pandas(df) + if engine == "cudf": + cudf = pytest.importorskip("cudf") + return cudf.from_pandas(df) + return df + + def _graph(engine): - if engine == "polars": - return graphistry.nodes(pl.from_pandas(NODES), "id").edges( - pl.from_pandas(EDGES), "s", "d") - return graphistry.nodes(NODES, "id").edges(EDGES, "s", "d") + return graphistry.nodes(_frame(engine, NODES), "id").edges( + _frame(engine, EDGES), "s", "d") + + +@lru_cache(maxsize=None) +def _engine_skip_reason(engine: str) -> Optional[str]: + """``None`` => this engine MUST run here; a string => a stated, checkable skip reason. + + Two traps this steers between, both hit by trying: + + * An IMPORT-only probe does not discriminate on a box where cudf / cudf_polars import + against an incomplete CUDA runtime — construction and simple ops succeed and the suite + then dies inside the first real kernel, which reads as a product failure. + * A probe that SWALLOWS every exception is worse: reverting the production guard turned + all 20 parameters here into SKIPS instead of failures, and an intermittent cold-start + error in a fresh GPU container silently dropped cuDF from an otherwise-green run. A + skipped GPU parameter reads as evidence of passing, which is exactly the theatre this + file exists to avoid. + + So: a missing module skips, a recognisable GPU-stack error skips with its text quoted, and + ANY other failure propagates. The smoke query is a plain traversal — never the shape under + test, or a regression would disarm its own test. + """ + return engine_skip_reason( + engine, + lambda: _graph(engine).gfql([ast.n(), ast.e_undirected(), ast.n()], engine=engine), + ) + + +def _require(engine: str) -> None: + reason = _engine_skip_reason(engine) + if reason is not None: + pytest.skip( + f"engine {engine!r} unavailable here ({reason}) — NOT evidence that it passes; " + "see the COVERAGE BOUNDARY note in this module's docstring" + ) def _cols(frame): - return set(frame.columns) + return set(map(str, frame.columns)) -@pytest.mark.parametrize("engine", ["pandas", "polars"]) +@pytest.mark.parametrize("engine", ENGINES) def test_rows_table_edges_after_named_middle_ending_on_an_edge(engine): """LDBC IS3's edge lookup: `(n {id})-[r:KNOWS]-` then `rows(table='edges')`. The middle is [node, edge] — EVEN length, so the rewrite produced a non-alternating binding_ops list and the query raised instead of returning the edges. """ + _require(engine) g = _graph(engine) out = g.gfql([ ast.n({"id": 0}, name="n"), @@ -60,7 +125,7 @@ def test_rows_table_edges_after_named_middle_ending_on_an_edge(engine): assert len(out) == 2, f"[{engine}] expected node 0's two KNOWS edges, got {len(out)}" -@pytest.mark.parametrize("engine", ["pandas", "polars"]) +@pytest.mark.parametrize("engine", ENGINES) def test_rows_table_edges_after_named_middle_of_odd_length(engine): """The SILENT half: an odd-length named middle returned the bindings table instead. @@ -68,6 +133,7 @@ def test_rows_table_edges_after_named_middle_of_odd_length(engine): caller got alias columns (`n`, `r.type`, `f`, ...). Asserting the edge columns are present AND the alias columns are absent is what distinguishes the two tables. """ + _require(engine) g = _graph(engine) out = g.gfql([ ast.n({"id": 0}, name="n"), @@ -82,9 +148,10 @@ def test_rows_table_edges_after_named_middle_of_odd_length(engine): f"[{engine}] bindings columns leaked into a rows(table='edges') result: {sorted(cols)}" -@pytest.mark.parametrize("engine", ["pandas", "polars"]) +@pytest.mark.parametrize("engine", ENGINES) def test_rows_table_edges_is_unaffected_by_whether_the_middle_is_named(engine): """Naming an op is a projection concern; it must not change WHICH table comes back.""" + _require(engine) g = _graph(engine) named = g.gfql([ ast.n({"id": 0}, name="n"), @@ -102,13 +169,14 @@ def test_rows_table_edges_is_unaffected_by_whether_the_middle_is_named(engine): f"[{engine}] naming the middle changed the row count: {len(named)} vs {len(unnamed)}" -@pytest.mark.parametrize("engine", ["pandas", "polars"]) +@pytest.mark.parametrize("engine", ENGINES) def test_bare_rows_after_a_named_middle_still_gets_the_bindings_table(engine): """The NEGATIVE side of the boundary: the rewrite must still fire without `table=`. Without this, the fix could be "never rewrite", which would silently break every Cypher multi-alias RETURN — the thing the rewrite exists to serve. """ + _require(engine) g = _graph(engine) out = g.gfql([ ast.n({"id": 0}, name="n"), @@ -121,7 +189,7 @@ def test_bare_rows_after_a_named_middle_still_gets_the_bindings_table(engine): f"[{engine}] expected a bindings table with alias columns, got {sorted(cols)}" -@pytest.mark.parametrize("engine", ["pandas", "polars"]) +@pytest.mark.parametrize("engine", ENGINES) def test_explicit_table_nodes_still_rewrites_and_that_is_a_known_limitation(engine): """`rows(table='nodes')` DOES still rewrite, and that is deliberate — pin the limitation. @@ -136,6 +204,7 @@ def test_explicit_table_nodes_still_rewrites_and_that_is_a_known_limitation(engi wire-format change, out of scope here. This test exists so that limitation is pinned rather than discovered again. """ + _require(engine) g = _graph(engine) out = g.gfql([ ast.n({"id": 0}, name="n"),