fix(gfql): keep policy-bearing queries off the native AUTO route

The routing change is only as safe as the set of inputs it refuses, and one
refusal is load-bearing rather than conservative.

The native polars executor does not go through `chain_impl`, so it never emits
the `postload` / `postchain` policy hooks that path emits. Measured on a
polars-frame graph: a policy that DENIES on `postload` blocks the query under
the generic path and does NOT block it under the native one. Routing a
policy-carrying query there would therefore have silently stopped enforcing a
denying postload policy for every user who never pinned an engine -- a
governance hook that stops firing is worse than a slow query. The NIE fallback
compounded it: re-running the query fired `preload` / `precompile` /
`postcompile` TWICE for one user call, so a policy that counts or rate-limits
would have double-counted.

`policy is None` in the guard restores both, exactly: the full hook trace and
every deny/allow outcome across all six hooks is now byte-identical to the
pre-change build on polars-frame graphs, pandas-frame graphs, and the declining
shape. Explicit `engine='polars'` is unchanged and still carries the
pre-existing hook gap; this only refuses to make that gap the default.

Tests: the positive half gains the edges-only graph (`self._nodes is None` is
inside the guard's condition, so it must actually work) and both spellings of
AUTO. A new negative class pins each refusal -- pandas frames, mixed
polars/pandas frames, explicit `engine='pandas'`, a denying postload policy
still blocking, and the compile/load hooks firing exactly once on a shape the
native engine declines.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015YsqAZQLbqjSDrYSFz2GoB

Author: lmeyerov

CHANGELOG.md

@@ -24,6 +24,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 - **Seeded typed-hop property projection keeps its fast path through DISTINCT / ORDER BY / SKIP / LIMIT**: those trailing row ops are plain frame operations, so the fast path now delegates them to the canonical chain instead of declining the whole query.
 
 ### Changed
+- **`engine='auto'` (the default) now runs the NATIVE polars engine when both bound frames are polars (#1743)**: `resolve_engine(AUTO)` maps polars frames to `Engine.PANDAS` — that branch predates `Engine.POLARS` — so `g.gfql(q)` on a polars-frame graph silently bridged every query onto the generic pandas path and handed pandas frames back. **Behaviour change, user-visible by default: frames in = frames out.** AUTO on a polars-frame graph now returns polars frames, and shapes the native engine declines (`NotImplementedError`, e.g. `shortestPath`) still answer via the legacy AUTO path — permitted here because the user pinned no engine; the no-silent-fallback contract still applies to an explicit `engine=`. Measured on dgx-spark under the exclusive perf lock, position-balanced 8-slot ABBA/BAAB, matched graph-benchmark q1-q9 on the 20k dataset through `g.gfql(<cypher>)` with no engine argument: **q1 41.3 → 14.5 ms, q2 70.3 → 17.8, q3 24.0 → 7.9, q4 22.1 → 6.8, q5 102.8 → 8.6, q6 106.5 → 10.3, q7 39.6 → 10.6, q8 20.3 → 2.3, q9 38.5 → 15.7** — nine of nine faster with non-overlapping per-slot ranges, canonical row values identical on every cell, and the returned frame type (`pandas` before, `polars` after) recorded per query as the engagement receipt. The declined-shape fallback costs a flat **0.23 ms** (the decline is raised at planning, before any data is touched), i.e. 0.6% of the cheapest declining query and independent of graph size. Two boundaries are deliberately NOT crossed: a graph whose node and edge frames are not BOTH polars is untouched, and a query carrying a `policy` is untouched — the native executor does not go through `chain_impl` and so never emits the `postload`/`postchain` hooks, and diverting a policy-bearing query there would silently stop enforcing a DENYING `postload` policy. Pinned by positive tests (auto == explicit polars, edges-only graphs, both spellings of AUTO) and by negative tests for each refusal (pandas frames, mixed frames, explicit `engine='pandas'`, policy present, and hooks firing exactly once on a declining shape).
 - **`is_lazy` is a type predicate, so the polars engine's eager/lazy split is checked instead of asserted**: `dtypes.is_lazy` decides WHICH member of the two-member `PolarsFrame` union a frame is, but it was declared `-> bool`, so that answer was discarded at the call boundary — an eager-only attribute (`.height`, `.columns`, `.schema`) reached after a lazy guard was unprovable, and the native polars chain closed the gap with `cast("pl.DataFrame", frame)`. A cast is not a type: it re-asserts, unchecked, the exact fact the predicate had just established, once per call site, and a wrong one surfaces as a production `AttributeError` rather than a CI error. `is_lazy` is now declared `-> TypeIs["pl.LazyFrame"]` (PEP 742), which narrows the *negative* branch as well as the positive — the eager side is the one that needed it, so `TypeGuard` would not have done — and the cast is gone. Four previously untyped chain-combine helpers (`_semi`, `_combine_edges`, `_apply_node_names`, and the `_known_empty` guard) now carry real signatures: `_semi`'s two frames share the `PolarsT` TypeVar because polars joins do not mix eagerness, and the two combine helpers take the `_LazyShim` collect-once duck-type they are actually called with. Typing `_apply_node_names` also retired a `getattr(next_step, "edges_empty", None)` — the shim declares that tri-state in `__slots__`, so a typo is now a checker error instead of a silent `None` that would have re-armed the very cardinality gate the guard disarms. Internal only: no runtime behaviour, no public API, and the `TypeIs` import is `TYPE_CHECKING`-only, so no new runtime dependency floor.
 - **GFQL execution context is declared rather than attached at runtime**: the private `_gfql_*` per-execution fields (index policy and registry, row-pipeline base graph, carried seed nodes, edge aliases, shortest-path backend, and the indexed-bindings handoff) are now declared on `Plottable` with defaults on `PlotterBase`, instead of being set with `setattr` and read back with `getattr(..., default)`. No public API or behaviour change; internal call sites are typed, and hand-rolled `Plottable` stand-ins must now construct the full context.
 

graphistry/compute/gfql_unified.py

@@ -1810,11 +1810,24 @@ def gfql(self: Plottable,
     """
     # engine inference: resolve_engine(AUTO) maps polars frames to PANDAS (polars predates
     # Engine.POLARS there), silently bridging polars-frame graphs onto the generic pandas path
-    # (~13x slower on cypher point queries, pandas frames out). Route AUTO to the native polars
-    # engine instead; an honest NIE (unsupported shape) falls back to the legacy AUTO path, which
-    # is allowed here because the user did not pin an engine.
+    # and handing pandas frames back. Measured on the matched graph-benchmark q1-q9 lane through
+    # this exact surface (dgx-spark, perf lock, position-balanced over 8 slots): 2.45-11.9x at 20k
+    # and 4.8-37.2x at 100k, values identical. Route AUTO to the native polars engine instead;
+    # an honest NIE (unsupported shape) falls back to the legacy AUTO path -- allowed here
+    # because the user pinned no engine -- at a flat ~0.23 ms, the decline being raised at
+    # planning before any data is touched.
+    #
+    # ``policy is None`` is REQUIRED, not conservatism: the native polars executor does not go
+    # through ``chain_impl``, so it never emits the ``postload``/``postchain`` hooks that path
+    # emits. Routing a policy-carrying query there would silently stop enforcing a DENYING
+    # ``postload`` policy (measured: deny-on-postload blocks under the generic path and does not
+    # under the native one) -- a governance hook that stops firing is worse than a slow query.
+    # The NIE fallback compounds it: re-running the query would fire ``preload``/``precompile``/
+    # ``postcompile`` twice for one user call. Explicit ``engine='polars'`` is unchanged and still
+    # carries the pre-existing hook gap; this guard only refuses to make that gap the default.
     if (
         (engine == EngineAbstract.AUTO or engine == EngineAbstract.AUTO.value)
+        and policy is None
         and is_polars_df(self._edges) and (self._nodes is None or is_polars_df(self._nodes))
     ):
         try:

graphistry/tests/compute/gfql/test_engine_polars_cypher_conformance.py

@@ -526,3 +526,79 @@ def test_auto_falls_back_on_polars_nie(self):
         out = g.gfql(q)._nodes  # auto: answers via fallback
         rows = out.to_dict("records") if hasattr(out, "to_dict") else out.to_pandas().to_dict("records")
         assert rows == [{"l": 3}]
+
+    def test_auto_routes_edges_only_graph(self):
+        """``self._nodes is None`` is inside the guard's condition, so it must actually work."""
+        edges = pl.DataFrame({"s": [0, 1, 2], "d": [1, 2, 3], "type": ["KNOWS"] * 3})
+        g = graphistry.edges(edges, "s", "d")
+        out = g.gfql("MATCH (a)-[:KNOWS]->(b) RETURN b.id AS bid ORDER BY bid")._nodes
+        assert "polars" in type(out).__module__
+        assert out["bid"].to_list() == [1, 2, 3]
+
+    def test_auto_enum_form_routes_too(self):
+        """AUTO arrives as either the enum or its string value; both must route."""
+        from graphistry.Engine import EngineAbstract
+        g = self._graph()
+        q = "MATCH (a:Person {id: 0})-[:KNOWS]->(b) RETURN b.id AS bid"
+        for eng in (EngineAbstract.AUTO, EngineAbstract.AUTO.value):
+            assert "polars" in type(g.gfql(q, engine=eng)._nodes).__module__
+
+
+class TestAutoEngineDoesNotRoutePolarsNative:
+    """The negative half: what must NOT be diverted. A routing change is only as safe as the
+    set of inputs it refuses, so each of these pins a case where the pre-change path must
+    still be taken byte-for-byte."""
+
+    NODES = {"id": [0, 1, 2, 3], "label__Person": [True] * 4}
+    EDGES = {"s": [0, 1, 2], "d": [1, 2, 3], "type": ["KNOWS"] * 3}
+    Q = "MATCH (a:Person {id: 0})-[:KNOWS]->(b) RETURN b.id AS bid"
+
+    def test_pandas_frames_unaffected(self):
+        g = graphistry.nodes(pd.DataFrame(self.NODES), "id").edges(pd.DataFrame(self.EDGES), "s", "d")
+        assert "pandas" in type(g.gfql(self.Q)._nodes).__module__
+
+    def test_mixed_frames_not_routed(self):
+        """Polars edges + pandas nodes: the native engine has no bridge, so AUTO must not try."""
+        g = (graphistry
+             .nodes(pd.DataFrame(self.NODES), "id")
+             .edges(pl.DataFrame(self.EDGES), "s", "d"))
+        assert "pandas" in type(g.gfql(self.Q)._nodes).__module__
+        g2 = (graphistry
+              .nodes(pl.DataFrame(self.NODES), "id")
+              .edges(pd.DataFrame(self.EDGES), "s", "d"))
+        assert "pandas" in type(g2.gfql(self.Q)._nodes).__module__
+
+    def test_explicit_pandas_engine_still_pandas(self):
+        g = graphistry.nodes(pl.DataFrame(self.NODES), "id").edges(pl.DataFrame(self.EDGES), "s", "d")
+        assert "pandas" in type(g.gfql(self.Q, engine="pandas")._nodes).__module__
+
+    def test_policy_disables_the_route(self):
+        """The native executor does not go through ``chain_impl`` and so never emits
+        ``postload``/``postchain``. Diverting a policy-carrying query would silently stop
+        enforcing a DENYING postload policy -- measured, not theorised."""
+        from graphistry.compute.gfql.policy import PolicyException
+        g = graphistry.nodes(pl.DataFrame(self.NODES), "id").edges(pl.DataFrame(self.EDGES), "s", "d")
+
+        def deny(ctx):
+            raise PolicyException(phase="postload", reason="denied by test")
+
+        with pytest.raises(PolicyException):
+            g.gfql(self.Q, policy={"postload": deny})
+        # ...and structurally: a policy-bearing AUTO query stays on the generic path,
+        # which is what makes the hook trace identical to the pre-change build.
+        assert "pandas" in type(g.gfql(self.Q, policy={"preload": lambda ctx: None})._nodes).__module__
+        seen = []
+        pol = {h: (lambda ctx, h=h: seen.append(h)) for h in ("preload", "postload", "precompile")}
+        g.gfql(self.Q, policy=pol)
+        assert "postload" in seen, "postload must still fire under AUTO on a polars graph"
+
+    def test_policy_hooks_fire_once_on_nie_shape(self):
+        """A retry-on-NIE route would fire the compile/load hooks twice for one user call."""
+        g = graphistry.nodes(pl.DataFrame(self.NODES), "id").edges(pl.DataFrame(self.EDGES), "s", "d")
+        q = ("MATCH p = shortestPath((a:Person {id: 0})-[:KNOWS*]-(b:Person {id: 3})) "
+             "RETURN length(p) AS l")
+        seen = []
+        pol = {h: (lambda ctx, h=h: seen.append(h)) for h in ("preload", "precompile", "postcompile")}
+        g.gfql(q, policy=pol)
+        assert seen.count("precompile") == 1, seen
+        assert seen.count("preload") == 1, seen