docs(gfql): print the benchmark numbers pyg-bench publishes, and re-verify them

The owner's redirect on graphistry#1813: "should all that be there, and all we do is
pretty print their latest json? and if we have any boundary needs, ensure there
are data contract tests that run on the pyg-bench side, and we re-verify on our
side?" This is the consuming half. Measurement, provenance and publishability
live in pyg-bench, which owns the runs. This repository renders them.

`docs/source/_data/gfql_benchmarks.json` is a vendored copy of pyg-bench's
`published/docs-numbers.json`; `gfql_benchmarks.contract.json` is a vendored
copy of the contract it satisfies. The `gfql_bench` extension resolves `:bench:`
and `:bench-diag:` against the artifact and re-verifies the contract before
rendering anything, so the build fails on a key that is not published, a
measurement older than the policy allows, a diagnostic-only figure printed as a
result, or a page that drops a number's provenance or disclosures. The
re-verification is written against the contract DOCUMENT, independently of
pyg-bench's implementation - a shared library would fail identically on both
sides and prove nothing.

`benchmark_filter_pagerank.rst` is the first page to use it. Its numbers came
from a chart generator reading a results directory that has never existed in any
commit, so nobody could re-derive them; pyg-bench re-ran the benchmark on
dgx-spark against a named commit and published the result. Twitter: GFQL GPU
0.24s and CPU 1.58s against Neo4j + GDS 11.72s, 49.5x and 7.4x. GPlus: GFQL GPU
2.42s and CPU 32.10s, 13.3x. The two lifecycle SVGs are removed - they render
the withdrawn figures as glyph paths and their generator cannot be run.

GPlus publishes no Neo4j figure. That arm did not complete, and a ratio against
a run that did not finish is not a speedup.

`docs/test_bench_numbers.py` runs the contract checks in the ordinary test lane
rather than only in the docs build, so a number going stale fails CI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015YsqAZQLbqjSDrYSFz2GoB

Author: lmeyerov

CHANGELOG.md

@@ -37,6 +37,8 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 - **Seeded property-RETURN dtype divergence on cuDF**: the lean projection applied the pandas rows-pivot artifact (int → float64, bool → object) on every engine, but cuDF's canonical pivot preserves the source dtypes — so the fast path returned `float64`/`object` where cuDF's own canonical path returns `int64`/`bool`. The cast rule is now engine-aware; the dtype-class decline guard is unchanged.
 
 ### Documentation
+- **The filter -> PageRank -> filter benchmark is measured again, and the docs print it from data.** `gfql/benchmark_filter_pagerank.rst` published Twitter and GPlus figures whose charts were generated from a results directory that has never existed in any commit, so nobody could re-derive them. The benchmark now lives in `graphistry/pyg-bench`, which re-ran it on dgx-spark under the shared perf lock against a named PyGraphistry commit and committed both the raw artifacts and the published document. This page renders that document instead of restating it: Twitter GFQL GPU 0.24 s and CPU 1.58 s against Neo4j + GDS 11.72 s (49.5x and 7.4x), GPlus GFQL GPU 2.42 s and CPU 32.10 s (13.3x). Every arm reproduces the earlier run's final subgraphs exactly.
+- **Benchmark numbers are now referenced, not retyped.** `docs/source/_data/gfql_benchmarks.json` is a vendored copy of pyg-bench's published artifact and `gfql_benchmarks.contract.json` a vendored copy of the contract it satisfies. The new `gfql_bench` Sphinx extension resolves `:bench:` and `:bench-diag:` against it and re-verifies the contract independently before rendering anything, so the docs build fails on a key that is not published, a measurement older than the policy allows, a diagnostic-only figure printed as a result, or a page that drops a number's provenance or disclosures. `docs/test_bench_numbers.py` runs the same checks in the ordinary test lane, so a number going stale fails CI rather than a nightly.
 - **GFQL pay-as-you-go resident indexing user guide**: New :doc:`Pay-As-You-Go Resident Indexing <gfql/indexing>` page — the lifecycle guide to resident indexes (`gfql_index_all()` / `gfql_index_edges()` / `create_index()` / `show_indexes()` / `drop_index()`): what the node-id + CSR in/out adjacency sidecars are, what engages them on 0.58.0 (seeded typed-hop fast paths incl. property RETURNs and property-seeded lookups per #1768/#1770, direct `g.hop()`; the general polars chain traversal honestly noted as not yet covered), the staleness/validity contract (identity + fingerprint; rebind invalidates; declines are safe — identical results either way), engine notes (polars needs `gfql_index_all(engine='polars')` until #1767), 0.58.0-tag measured numbers, and a runnable end-to-end example. Wired into the GFQL toctree + recommended paths alongside :doc:`Seeded Traversal Indexes <gfql/index_adjacency>`.
 - **GFQL performance docs: 0.58.0 release-tag-verified numbers, siloed in one page**: `gfql/performance.rst` is now the canonical benchmark-numbers page (alongside `gfql/index_adjacency.rst` for the index benchmarks) — a benchmark rerun updates it alone. It carries the 0.58.0 tag sweep (DGX Spark GB10, warm medians N=30; four-engine numbers cross-engine parity-verified, competitor pairs validated against expected result rows): seeded typed-hop fast path across all four engines (e.g. pandas 29.9→2.46ms, 12.1×), native chain form, resident-index covered-shape lookups (with the `gfql_index_all(engine='polars')` caveat / PR #1767), flat seeded-hop scaling on pandas (0.159–0.164ms from 0.25M to 32M edges), the one-keyword `engine='polars'` LDBC SNB SF1 seed-lookup win (1,299.6→106.1ms, 12.3×), LDBC SNB interactive SF1 vs Neo4j 5.26 same-box pairs (GFQL 4 of 5; Neo4j wins recent-replies — reported as-is), OLAP multi-join vs embedded Kuzu (q8 200×, q9 14.2×) with the honest inverse (Kuzu wins single-table aggregates 2–4×, seeded property-projection lookups 2.4–64×), plus the prior Orkut/LiveJournal bulk sweep (moved from `engines.rst`, dated once) and its methodology. All other pages — `engines.rst`, `quick.rst`, `about.rst`, `overview.rst`, `index.rst` — now carry stable qualitative claims (e.g. "often an order of magnitude faster on query-heavy workloads") that link into `performance.rst` instead of inline figures, replacing the stale "up to ~38×" headlines and avoiding scattered per-claim version labels.
 

docs/.rstcheck.cfg

@@ -20,7 +20,9 @@ ignore_roles =
     py:attr,
     py:exc,
     py:obj,
-    py:data
+    py:data,
+    bench,
+    bench-diag
 
 # Ignore Sphinx-specific directives
 ignore_directives =
@@ -43,7 +45,9 @@ ignore_directives =
     glossary,
     productionlist,
     include,
-    graphviz
+    graphviz,
+    bench-provenance,
+    bench-disclosures
 
 # Ignore common informational messages
 ignore_messages = (Hyperlink target "[^"]*" is not referenced\.$)

docs/source/_data/gfql_benchmarks.contract.json

@@ -0,0 +1,102 @@
+{
+  "contract_version": 1,
+  "title": "pyg-bench published docs numbers",
+  "summary": [
+    "The boundary between this repository and the pygraphistry docs. pyg-bench MEASURES and",
+    "PUBLISHES; pygraphistry pretty-prints. Everything a consumer is allowed to assume about",
+    "published/docs-numbers.json is written down here, checked on the producer side before the",
+    "artifact is written, and re-verified independently on the consumer side before it renders.",
+    "Two independent implementations of these rules is the point: a shared library would fail",
+    "the same way on both sides of the boundary."
+  ],
+  "artifact_path": "published/docs-numbers.json",
+  "top_level_required": [
+    "contract_version",
+    "schema_version",
+    "generated_by",
+    "generated_at",
+    "policy",
+    "runs",
+    "cells"
+  ],
+  "policy_required": ["max_age_days", "managed_docs"],
+  "run_required": [
+    "measured_at",
+    "host",
+    "perf_lock_held",
+    "quiet_host",
+    "reps",
+    "pygraphistry_commit",
+    "pyg_bench_commit",
+    "runtime",
+    "dataset",
+    "artifact",
+    "row_validation"
+  ],
+  "cell_required": [
+    "run",
+    "workload",
+    "engine",
+    "value",
+    "unit",
+    "decimals",
+    "status",
+    "comparison_allowed",
+    "board_quotable",
+    "disclosures"
+  ],
+  "cell_status": ["ok", "partial", "adapter_workaround", "result_mismatch"],
+  "cell_unit": ["ms", "s", "x"],
+  "key_pattern": "^[a-z0-9][a-z0-9_]*(\\.[a-z0-9][a-z0-9_]*)+$",
+  "date_pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$",
+  "max_decimals": 4,
+  "invariants": [
+    {
+      "id": "cell_names_a_known_run",
+      "rule": "every cell.run must be a key of runs",
+      "why": "a number whose provenance record is absent is an unprovenanced number"
+    },
+    {
+      "id": "quotable_implies_comparable",
+      "rule": "board_quotable implies comparison_allowed",
+      "why": "a diagnostic-only figure must never reach a board"
+    },
+    {
+      "id": "quotable_implies_ok",
+      "rule": "board_quotable implies status == 'ok'",
+      "why": "partial / adapter_workaround / result_mismatch cells are not results"
+    },
+    {
+      "id": "caveat_travels_with_the_number",
+      "rule": "status != 'ok' or comparison_allowed is false implies disclosures is non-empty",
+      "why": "a bare ratio without its asterisk launders the caveat"
+    },
+    {
+      "id": "ratios_are_comparisons",
+      "rule": "unit 'x' implies comparison_allowed",
+      "why": "a ratio between two numbers that were never established as comparable is not a speedup"
+    },
+    {
+      "id": "value_is_rounded_as_published",
+      "rule": "value == round(value, decimals), value is finite and >= 0, 0 <= decimals <= max_decimals",
+      "why": "the consumer prints value verbatim; rounding must not happen at render time"
+    },
+    {
+      "id": "engine_is_named",
+      "rule": "cell.engine is a non-empty string",
+      "why": "pandas and polars are not interchangeable and the reader must be told which one ran"
+    },
+    {
+      "id": "measurement_is_dated",
+      "rule": "run.measured_at matches date_pattern",
+      "why": "staleness is checked against the MEASUREMENT date, not the export date"
+    }
+  ],
+  "consumer_obligations": [
+    "Refuse to render a key the artifact does not contain — a missing number is a build failure, never blank text.",
+    "Refuse to render a cell whose run.measured_at is older than policy.max_age_days.",
+    "Refuse to render a non-board_quotable cell as a bare number; it may appear only through a role that labels it diagnostic-only.",
+    "Render every referenced cell's disclosures and its run's provenance on the same page.",
+    "Re-verify every invariant above before rendering, with an implementation that does not import this repository."
+  ]
+}