fix(gfql): show_indexes reports engine usability, not just fingerprint validity - #1841
Open
lmeyerov wants to merge 2 commits into
Open
fix(gfql): show_indexes reports engine usability, not just fingerprint validity#1841lmeyerov wants to merge 2 commits into
lmeyerov wants to merge 2 commits into
Conversation
…t validity show_indexes() claimed valid=True for an index the resolved query engine cannot use. Indexes are engine-specific, so a fingerprint-fresh index built for another engine (e.g. polars-built, inspected on a graph whose AUTO queries resolve to pandas) silently declines to a scan at query time while the inspection surface said everything was fine. This is the follow-up proposed in #1767's disposition comment ("stop show_indexes() reporting valid=True for an index the resolved query engine cannot use"). #1838 made the decline loud in gfql_explain via _engine_mismatch_reason; this completes the pair on show_indexes by reusing the same mechanism and wording rather than inventing a parallel one. - valid keeps its fingerprint-only meaning (backward compatible) - new columns: query_engine (what engine= resolves to for THIS graph, the same resolution a query makes; default 'auto'), usable (fresh AND engine-matched), reason (shared #1838 mismatch wording + stale reason) - show_indexes(engine=...) previews an explicit engine choice; SHOW GFQL INDEXES respects the gfql(engine=...) of the call; gfql_explain passes its engine through - reporting-only: no query routing changes (#1743 AUTO-routing stays held) Tests: 9 new (polars index vs AUTO/pandas query not-usable with exact reason, matching-engine usable, explicit-engine preview both directions, stale-but-engine-matched not usable, stale+mismatched combined reason, per-kind reasons across all four index kinds, DDL surface, cudf lane with import skip-guard). Index suite: 219 passed, 1 skipped locally (GPU lanes deselected, no GPU on this box). Lint + scoped mypy clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011AB4RZpph3uSFUpzKnZJcr
Resolve CHANGELOG.md conflict by keeping both Fixed entries: this PR's show_indexes engine-usability entry and master's gfql_clear_caches registry entry (#1836). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011AB4RZpph3uSFUpzKnZJcr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lineage: the one unlanded piece of #1767's disposition
#1767 was retracted after measurement showed it regressed the default path 125–534×. Its disposition comment proposed replacing it with a diagnostic instead:
#1838 landed the first half —
gfql_explain/index_tracenow name the mismatch via_engine_mismatch_reason. This PR lands the second half: the inspection surface itself. On master today,show_indexes()still saysvalid=Truefor every index in the disposition's measured cells, including the ones being declined at query time — the user inspecting their own index state is told it is fine while paying a silent 100–500× cliff (the disposition's words, not a new measurement; no perf claims are made here).What it does
validkeeps its fingerprint-only meaning, so nothing existing breaks. Three columns are added:query_engineengineargument (default'auto') resolves to for THIS graph — the sameresolve_enginecall a query makesusablereasonresident edge_out_adj index engine=polars, requested engine=pandas -> scan), plus a stale-fingerprint reasonSo the disposition's headline case now reads honestly: a polars-built index on a graph whose AUTO queries resolve to pandas shows
valid=True, usable=Falsewith the same reason stringgfql_explaingives for the decline.Surfaces wired through:
g.show_indexes(engine=...)previews an explicit engine choice;SHOW GFQL INDEXESrespects thegfql(engine=...)of the call;gfql_explainpasses its engine into the resident-index table it embeds.Mechanism reuse, not a parallel one
The per-row reason and the #1838 trace diagnostic share one formatter (
_engine_mismatch_text), so the two surfaces can never drift apart in wording; the existing exact-string pins on the explain side (test_explain_reports_bidirectional_engine_mismatch) keep guarding both.Explicitly NOT a routing change
No query planning or engine resolution is altered anywhere — reporting only. #1743 (AUTO → native routing) remains on hold, untouched.
Tests
9 new in
TestShowIndexesEngineUsability:engine='polars'preview usable; cudf lane (importorskip-guarded, mirrors the suite's existing cudf gating)SHOW GFQL INDEXEShonors AUTO vs explicit engineLocal (CPU, GPU lanes deselected — box has no GPU):
graphistry/tests/compute/gfql/index/219 passed, 1 skipped../bin/lint.sh(incl. type-hygiene guard) andMYPY_CMD="uvx mypy==2.3.0" ./bin/mypy.shon the touched files: clean.CHANGELOG: entry under Development/Fixed; docs:
indexing.rstliveness section now explainsusable/reason.🤖 Generated with Claude Code
https://claude.ai/code/session_011AB4RZpph3uSFUpzKnZJcr