fix(gfql): decline three silently-wrong bounded var-length polars shapes (#1787)

Pandas is the oracle and the contract is parity-or-NotImplementedError. Three
BOUNDED var-length shapes in the native polars `rows(binding_ops=...)` path
returned a DIFFERENT count with no error. Same root-cause family as the
unbounded case #1781 declined: pandas' step_pairs come from the var-length
`edge_op.execute` hop, whose hop-window pruning -- and, when seeded, its
per-seed BFS -- changes the edge multiplicity in a way a rebuild from the raw
matching edge table does not reproduce.

Declined (diverging graphs out of 60 random ones per shape, differential fuzz
against the pandas oracle):

  * directed `-[*k..m]->` with min_hops >= 3 (53/60; pandas 0 vs polars 35),
    and min_hops >= 2 off a FILTERED seed (27-30/60). Plain min_hops <= 2 is
    fuzz-clean, including as a non-first segment -- that is the graph-bench q3
    `-[*1..k]->` shape, still served.
  * undirected `-[*1..1]-` / `-[*1]-`, the degenerate window: 60/60, halves the
    count (pandas 36 vs polars 18). `-[*1..2]-` and wider agree, which is why
    the existing tests missed it.
  * undirected `-[*1..k]-` that does not start from the full node set: filtered
    seed 51/60, non-first segment 36/60. Every directed equivalent agrees, so
    this is specific to the undirected doubled-pair expansion.

The gate keys on an EXPLICIT var-length window rather than on
`EdgeSemantics.is_multihop`, because `-[*1..1]-` resolves to min == max == 1 and
so is NOT multihop -- yet pandas still routes it through the var-length hop
(`-[]-` gives 18 on a graph where `-[*1..1]-` gives 36). Plain `-[]-` is
untouched.

Tests: explicit decline + still-served neighbours for each of the three, plus a
seeded differential fuzz over random small graphs (cyclic, parallel edges,
self-loops) that asserts parity-or-raise AND that enough shapes are still served
for the check to mean anything. Bounded windows only: undirected unbounded
shapes through the pandas oracle can exhaust the box.

The new test file is REGISTERED in bin/test-polars.sh. That list is an explicit
allowlist, and the polars lane is the only CI lane with polars installed -- so
the only lane that can execute this file at all. Without the registration the
three `return None` decline statements were the only changed lines no lane ever
ran (10/13 = 76.92%, exactly the changed-line-coverage gate failure); with it
the block is 13/13.

Mutation-checked: dropping the min_hops clause fails 4 tests, the degenerate
undirected window 3, the undirected-seed clause 3, the whole gate 8. Full
graphistry/tests/compute is unchanged by the declines (9 pre-existing failures,
same set).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015YsqAZQLbqjSDrYSFz2GoB

Author: lmeyerov

bin/test-polars.sh

@@ -24,6 +24,7 @@ POLARS_TEST_FILES=(
     graphistry/tests/compute/gfql/test_engine_polars_chain.py
     graphistry/tests/compute/gfql/test_engine_polars_row_pipeline.py
     graphistry/tests/compute/gfql/test_engine_polars_binding_rows.py
+    graphistry/tests/compute/gfql/test_engine_polars_varlen_divergence_1787.py
     graphistry/tests/compute/gfql/test_engine_polars_with_match_reentry.py
     graphistry/tests/compute/gfql/test_engine_polars_cypher_conformance.py
     graphistry/tests/compute/gfql/test_engine_polars_conformance_matrix.py

graphistry/compute/gfql/lazy/engine/polars/row_pipeline.py

@@ -1312,6 +1312,52 @@ def _names(lf: pl.LazyFrame) -> List[str]:
                     )
                     if _resolved_min != 1:
                         return None
+            # #1787: three more BOUNDED var-length shapes this raw-edge reconstruction gets
+            # SILENTLY wrong, all found by differential fuzzing against the pandas oracle
+            # over random small graphs (the `n/60` counts below are diverging graphs out of
+            # 60 random ones per shape). Same root-cause family as the unbounded case #1781
+            # declined: pandas' step_pairs come from the var-length `edge_op.execute` hop,
+            # whose hop-window pruning -- and, when seeded, its per-seed BFS -- changes the
+            # edge multiplicity in a way a rebuild from the raw matching edge table does not
+            # reproduce. Pandas is the oracle and the contract is parity-or-NotImplementedError,
+            # so decline until the multiplicity is reconstructible.
+            #
+            # Gated on an EXPLICIT var-length window, not on `sem.is_multihop`: the degenerate
+            # `-[*1..1]-` resolves to min == max == 1 and so is NOT multihop, yet pandas still
+            # routes it through the var-length hop (`-[]-` gives 18 where `-[*1..1]-` gives 36
+            # on the same graph). That is exactly why the existing tests miss it.
+            if op.min_hops is not None or op.max_hops is not None:
+                _vl_max = op.max_hops if op.max_hops is not None else op.hops
+                _vl_min = op.min_hops if op.min_hops is not None else (
+                    op.hops if op.hops is not None else 1
+                )
+                # A seed is anything that starts the segment from less than the whole node
+                # set: a filtered start alias, or a re-entry / `WITH` seed frame.
+                _prev_op = ops[idx - 1] if idx >= 1 else None
+                _seeded_start = start_nodes is not None or (
+                    isinstance(_prev_op, ASTNode) and bool(_prev_op.filter_dict)
+                )
+                if _vl_max is not None:
+                    if op.direction == "undirected":
+                        #  - `-[*1..1]-` / `-[*1]-`: the DEGENERATE window halves the count
+                        #    (60/60; pandas 36 vs polars 18). `-[*1..2]-` and wider agree.
+                        if _vl_max == 1:
+                            return None
+                        #  - undirected `-[*1..k]-` OFF THE FULL NODE SET: the doubled-pair
+                        #    expansion over-counts when the segment does not start from every
+                        #    node -- filtered seed 51/60, non-first segment 36/60. Every
+                        #    DIRECTED equivalent agrees, so this is specific to the doubling.
+                        if _seeded_start or idx > 1:
+                            return None
+                    #  - directed `-[*k..m]->` with min_hops >= 3 (53/60; pandas 0 vs polars
+                    #    35), and min_hops >= 2 WITH a filtered seed (27-30/60).
+                    #    `max_reached_hop` in compute/hop.py is a dedup-by-node BFS
+                    #    eccentricity, not a longest-walk length, so pandas returns empty (or
+                    #    prunes) where this rebuild expands a different edge multiset.
+                    #    min_hops <= 2 off the full node set is fuzz-clean (0/60, including as
+                    #    a non-first segment) -- that is the graph-bench q3 `-[*1..k]->` shape.
+                    elif _vl_min >= 3 or (_vl_min >= 2 and _seeded_start):
+                        return None
             if op.direction not in ("forward", "reverse", "undirected"):
                 return None
             if any(

graphistry/tests/compute/gfql/test_engine_polars_varlen_divergence_1787.py

@@ -0,0 +1,145 @@
+"""Native polars var-length binding rows: parity with the pandas oracle, or decline (#1787).
+
+Three bounded shapes returned a DIFFERENT count from pandas with no error -- the contract is
+parity-or-``NotImplementedError``, so they are now declined, mirroring what #1781 did for the
+unbounded case:
+
+  1. directed ``-[*k..m]->`` with ``min_hops >= 3`` (and ``min_hops >= 2`` off a filtered seed);
+  2. undirected ``-[*1..1]-`` / ``-[*1]-``, the degenerate window, which HALVED the count;
+  3. undirected ``-[*1..k]-`` that does not start from the full node set (filtered seed, or a
+     non-first segment), which OVER-counted.
+
+The differential fuzz at the bottom is what found all three (and what proves the gate is not
+merely declining everything): it asserts served shapes still equal pandas AND that the
+neighbouring shapes are still served, so a lazily over-broad gate fails it.
+"""
+import random
+
+import pandas as pd
+import pytest
+
+import graphistry
+
+pl = pytest.importorskip("polars")
+
+
+def _pair(nodes, edges):
+    return (
+        graphistry.nodes(nodes, "id").edges(edges, "s", "d"),
+        graphistry.nodes(pl.from_pandas(nodes), "id").edges(pl.from_pandas(edges), "s", "d"),
+    )
+
+
+def _count(g, query, engine):
+    return int(g.gfql(query, engine=engine)._nodes["c"].to_list()[0])
+
+
+# The fixtures from the issue, one per divergence.
+BOUNDED_NODES = pd.DataFrame({"id": list(range(7))})
+BOUNDED_EDGES = pd.DataFrame(
+    [(4, 6), (0, 4), (1, 2), (4, 6), (5, 6), (4, 5), (4, 6)], columns=["s", "d"]
+)
+UNDIR_NODES = pd.DataFrame({"id": list(range(5))})
+UNDIR_EDGES = pd.DataFrame(
+    [(1, 2), (1, 2), (0, 2), (1, 0), (4, 2), (3, 2), (3, 4), (0, 3), (1, 2)], columns=["s", "d"]
+)
+SEEDED_NODES = pd.DataFrame({"id": list(range(7)), "kind": ["a", "b"] * 3 + ["a"]})
+SEEDED_EDGES = pd.DataFrame(
+    [(0, 3), (6, 3), (4, 2), (1, 3), (6, 3), (3, 1), (3, 5), (4, 1), (3, 3), (0, 3)],
+    columns=["s", "d"],
+)
+
+DECLINED = [
+    # (nodes, edges, query) -- each returned a wrong count before the gate.
+    (BOUNDED_NODES, BOUNDED_EDGES, "MATCH (a)-[*3..3]->(b) RETURN count(*) AS c"),
+    (BOUNDED_NODES, BOUNDED_EDGES, "MATCH (a)-[*3..4]->(b) RETURN count(*) AS c"),
+    (SEEDED_NODES, SEEDED_EDGES, "MATCH (a {kind:'a'})-[*2..3]->(b) RETURN count(*) AS c"),
+    (UNDIR_NODES, UNDIR_EDGES, "MATCH (a)-[*1..1]-(b) RETURN count(*) AS c"),
+    (UNDIR_NODES, UNDIR_EDGES, "MATCH (a)-[*1]-(b) RETURN count(*) AS c"),
+    (SEEDED_NODES, SEEDED_EDGES, "MATCH (a {kind:'a'})-[*1..2]-(b) RETURN count(*) AS c"),
+    (SEEDED_NODES, SEEDED_EDGES, "MATCH (a)-[]->(b)-[*1..2]-(c) RETURN count(*) AS c"),
+]
+
+STILL_SERVED = [
+    # Neighbours of each declined shape: the gate must not swallow these.
+    (BOUNDED_NODES, BOUNDED_EDGES, "MATCH (a)-[*1..2]->(b) RETURN count(*) AS c"),
+    (BOUNDED_NODES, BOUNDED_EDGES, "MATCH (a)-[*2..3]->(b) RETURN count(*) AS c"),
+    (UNDIR_NODES, UNDIR_EDGES, "MATCH (a)-[]-(b) RETURN count(*) AS c"),
+    (UNDIR_NODES, UNDIR_EDGES, "MATCH (a)-[*1..2]-(b) RETURN count(*) AS c"),
+    (UNDIR_NODES, UNDIR_EDGES, "MATCH (a)-[*1..3]-(b) RETURN count(*) AS c"),
+    (SEEDED_NODES, SEEDED_EDGES, "MATCH (a {kind:'a'})-[*1..2]->(b) RETURN count(*) AS c"),
+    (SEEDED_NODES, SEEDED_EDGES, "MATCH (a)-[]->(b)-[*2..3]->(c) RETURN count(*) AS c"),
+]
+
+
+@pytest.mark.parametrize("nodes,edges,query", DECLINED)
+def test_divergent_varlen_shapes_decline(nodes, edges, query):
+    """A shape that cannot be reproduced must RAISE, never answer differently."""
+    _, g_pl = _pair(nodes, edges)
+    with pytest.raises(NotImplementedError):
+        g_pl.gfql(query, engine="polars")
+
+
+@pytest.mark.parametrize("nodes,edges,query", STILL_SERVED)
+def test_neighbouring_varlen_shapes_still_served_and_match_pandas(nodes, edges, query):
+    """The gate is a scalpel: every neighbouring shape still runs natively AND matches pandas."""
+    g_pd, g_pl = _pair(nodes, edges)
+    assert _count(g_pl, query, "polars") == _count(g_pd, query, "pandas")
+
+
+@pytest.mark.parametrize("nodes,edges,query", DECLINED)
+def test_declined_shapes_still_answerable_on_pandas(nodes, edges, query):
+    """Declining is an ENGINE limit, not a query rejection: pandas still answers all of them."""
+    g_pd, _ = _pair(nodes, edges)
+    assert _count(g_pd, query, "pandas") >= 0
+
+
+FUZZ_SHAPES = [
+    "MATCH (a)-[*1..2]->(b) RETURN count(*) AS c",
+    "MATCH (a)-[*2..2]->(b) RETURN count(*) AS c",
+    "MATCH (a)-[*2..3]->(b) RETURN count(*) AS c",
+    "MATCH (a)-[*3..3]->(b) RETURN count(*) AS c",
+    "MATCH (a)-[*1..1]-(b) RETURN count(*) AS c",
+    "MATCH (a)-[*1..2]-(b) RETURN count(*) AS c",
+    "MATCH (a)-[*1..3]-(b) RETURN count(*) AS c",
+    "MATCH (a {kind:'a'})-[*1..2]-(b) RETURN count(*) AS c",
+    "MATCH (a {kind:'a'})-[*1..2]->(b) RETURN count(*) AS c",
+    "MATCH (a {kind:'a'})-[*2..3]->(b) RETURN count(*) AS c",
+    "MATCH (a)-[]->(b)-[*1..2]-(c) RETURN count(*) AS c",
+    "MATCH (a)-[]->(b)-[*2..3]->(c) RETURN count(*) AS c",
+]
+
+
+def test_bounded_varlen_differential_fuzz_against_pandas_oracle():
+    """Random small graphs (cyclic, parallel edges, self-loops): match pandas or raise.
+
+    Bounded windows only -- undirected UNBOUNDED shapes through the pandas oracle can blow the
+    box up (75 GiB RSS observed), and they are already declined elsewhere. Seeded so a failure
+    is reproducible; the shape list spans both sides of every gate boundary, so a gate that
+    declines too much shows up as the served-count assertion below going to zero.
+    """
+    rnd = random.Random(1787)
+    served = 0
+    for _ in range(25):
+        n_nodes = rnd.randint(3, 7)
+        nodes = pd.DataFrame({
+            "id": list(range(n_nodes)),
+            "kind": [rnd.choice("ab") for _ in range(n_nodes)],
+        })
+        edges = pd.DataFrame(
+            [(rnd.randrange(n_nodes), rnd.randrange(n_nodes)) for _ in range(rnd.randint(3, 10))],
+            columns=["s", "d"],
+        )
+        g_pd, g_pl = _pair(nodes, edges)
+        for query in FUZZ_SHAPES:
+            expected = _count(g_pd, query, "pandas")
+            try:
+                got = _count(g_pl, query, "polars")
+            except NotImplementedError:
+                continue  # an honest decline is allowed; a different answer is not
+            served += 1
+            assert got == expected, (
+                f"polars diverged from the pandas oracle on {query!r}: "
+                f"{got} != {expected}\nnodes={nodes.to_dict('list')}\nedges={edges.values.tolist()}"
+            )
+    assert served > 100, f"gate declines too much to be a useful parity check ({served})"