diff --git a/CHANGELOG.md b/CHANGELOG.md index b159251c6c..37ad15b366 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -37,6 +37,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm - **Seeded property-RETURN dtype divergence on cuDF**: the lean projection applied the pandas rows-pivot artifact (int → float64, bool → object) on every engine, but cuDF's canonical pivot preserves the source dtypes — so the fast path returned `float64`/`object` where cuDF's own canonical path returns `int64`/`bool`. The cast rule is now engine-aware; the dtype-class decline guard is unchanged. ### Documentation +- **`gfql/performance.rst` publishes the one benchmark board that is fully traceable, and the surrounding prose was rewritten around it.** The board is the matched, row-validated `prrao87/graph-benchmark` q1–q9 Cypher suite at 20,000 and 100,000 persons, measured on a quiet, perf-locked dgx-spark with position-balanced slots, every per-slot artifact committed in pyg-bench, and every GFQL cell's result rows validated against the reference engine's before publication. Two lanes are published, each named: **GFQL-Polars beats GFQL-pandas on all nine queries at both sizes**, with every per-query speedup larger on the bigger graph; and **on the 100,000-person lane, GFQL with `engine='polars'` answers five of the nine queries faster than embedded Kuzu 0.11.3**, with both sides' medians and the ratio shown for those five. The page states the comparison universe, the engine, the size, and that it is five of nine, because a benchmark headline without its scope is worthless. `gfql/engines.rst`'s vendor comparison is reorganized from "where each tool wins" into "what GFQL gives you when the query runs here", and the pages that carry no publishable figures (`indexing.rst`, `index_adjacency.rst`, `benchmark_filter_pagerank.rst`) now say plainly that a lane's numbers are not published until it runs under the provenance-carrying harness, instead of narrating what used to be there. - **GFQL pay-as-you-go resident indexing user guide**: New :doc:`Pay-As-You-Go Resident Indexing ` page — the lifecycle guide to resident indexes (`gfql_index_all()` / `gfql_index_edges()` / `create_index()` / `show_indexes()` / `drop_index()`): what the node-id + CSR in/out adjacency sidecars are, what engages them on 0.58.0 (seeded typed-hop fast paths incl. property RETURNs and property-seeded lookups per #1768/#1770, direct `g.hop()`; the general polars chain traversal honestly noted as not yet covered), the staleness/validity contract (identity + fingerprint; rebind invalidates; declines are safe — identical results either way), engine notes (polars needs `gfql_index_all(engine='polars')` until #1767), the index cost model, and a runnable end-to-end example. Wired into the GFQL toctree + recommended paths alongside :doc:`Seeded Traversal Indexes `. - **GFQL performance docs: benchmark numbers siloed in one page**: `gfql/performance.rst` is the canonical benchmark-numbers page (alongside `gfql/index_adjacency.rst` for the index benchmarks) — a benchmark rerun updates it alone. All other pages — `engines.rst`, `quick.rst`, `about.rst`, `overview.rst`, `index.rst` — carry stable qualitative claims that link into `performance.rst` instead of inline figures, avoiding scattered per-claim version labels. (The tag-sweep figures this entry originally listed have since been withdrawn — see the provenance entries below.) diff --git a/docs/source/gfql/benchmark_filter_pagerank.rst b/docs/source/gfql/benchmark_filter_pagerank.rst index 369a27358d..308ac239ca 100644 --- a/docs/source/gfql/benchmark_filter_pagerank.rst +++ b/docs/source/gfql/benchmark_filter_pagerank.rst @@ -10,19 +10,11 @@ Run Cypher graph queries and analytics directly on Python dataframes — no database required. This benchmark compares **Graphistry's local Cypher** (CPU and GPU) against **Neo4j + GDS** on the same end-to-end pipeline. -.. warning:: - **The figures previously published on this page have been withdrawn.** The raw - measurement artifacts for that run no longer exist anywhere — nothing was committed, - and the chart generator (``benchmarks/gfql/filter_pagerank/presentation.py``) reads a - results directory that is not present in this repository, so even the rendered charts - cannot be regenerated. That makes those numbers impossible to confirm *or* refute, so - they are treated as unpublishable rather than assumed correct. - - What remains below is the part that is still verifiable: what the benchmark measures, - the exact pipeline, the Neo4j+GDS analog, and how to run it yourself. The figures will - be republished once this pipeline runs under the provenance-carrying harness described - on :doc:`performance` — committed per-slot artifacts, recorded commit/host/perf-lock/ - reps, and results validated against the competitor before any ratio is published. +.. note:: + This page describes the workload and how to run it. Latency figures for it are not + published yet: the pipeline has not been run under the provenance-carrying harness + described on :doc:`performance`, and PyGraphistry publishes no benchmark number it + cannot trace to a committed artifact. The pipeline ------------ @@ -189,8 +181,8 @@ Methodology Reproduce --------- -Note the caveat at the top of this page: these reproducers print and plot, but do not -yet emit a provenance-carrying artifact, which is why their output is not published here. +These reproducers print and plot; they do not yet emit a provenance-carrying artifact, +which is why their output is not published here. - ``benchmarks/gfql/filter_pagerank/load_prepare_cpu_gpu.py`` — load + shape the graphs - ``benchmarks/gfql/filter_pagerank/filter_pagerank_pipeline_cpu_gpu.py`` — the GFQL CPU/GPU pipeline diff --git a/docs/source/gfql/engines.rst b/docs/source/gfql/engines.rst index 2274f3b2df..b962d160d2 100644 --- a/docs/source/gfql/engines.rst +++ b/docs/source/gfql/engines.rst @@ -33,7 +33,9 @@ engine is a one-keyword change — no GPU, same results: g.gfql(query) # engine='pandas' (default) g.gfql(query, engine='polars') # often much faster on query-heavy workloads, identical results -Switching is often a large speedup on query-heavy workloads. +On the ``prrao87/graph-benchmark`` q1–q9 Cypher suite, Polars beats pandas on all nine +queries at both graph sizes measured, and by more on the larger one. The per-query numbers +are on the :doc:`performance` page. Your existing pandas, Polars, or cuDF graph works as-is: the input frames are accepted and coerced once; the only change is the keyword. The catch: a few exotic Cypher features still @@ -101,19 +103,20 @@ never selects Polars or Polars-GPU** — they are explicit opt-in (see *Why opt- How the engines compare ----------------------- -The stable pattern: - -- **Polars-CPU is the everyday win**: typically much faster than pandas on query-heavy - workloads (traversal, ``WHERE``/``ORDER``, aggregation), with **no GPU**. It also - frequently beats eager cuDF on bulk work, because it builds **one fused lazy plan and - collects once** while cuDF pays a kernel launch and a materialized intermediate per op. -- **Polars-GPU leads heavy multi-hop and full-graph aggregation**: the same fused plan, - executed on the GPU, once there is enough work to amortize kernel launches. -- **cuDF suits the extreme single materialization**: one very large join/output where raw - GPU throughput dominates and the in-memory Polars-GPU executor comes under memory - pressure. -- **pandas keeps trivially small operations**: a bare equality filter's boolean - mask beats Polars' plan overhead — immaterial in absolute terms. +Each engine has a shape it is built for: + +- **Polars-CPU is the everyday win.** It beats pandas on all nine queries of the q1–q9 + Cypher suite at both sizes measured (:doc:`performance`), with **no GPU**, because it + builds **one fused lazy plan and collects once** instead of materializing an + intermediate per operation. +- **Polars-GPU runs that same fused plan on the GPU.** It pays off once a step carries + enough work to amortize a kernel launch: large frontiers, dense joins, full-graph + aggregation. +- **cuDF executes eagerly, op by op.** That suits one very large materialization where a + single join dominates the query and the in-memory Polars-GPU executor comes under + memory pressure. +- **pandas carries no plan overhead**, so it stays the right default for trivially small + operations and the widest-compatibility path. - **Seeded / selective lookups are an indexing problem**, not an engine race: the opt-in resident index turns the ``O(E)`` scan into an ``O(degree)`` gather on every engine, so the cost tracks the seeds rather than the graph — see [F5] below and @@ -121,97 +124,65 @@ The stable pattern: .. _gfql-vs-external-tools: -GFQL vs external graph tools ----------------------------- +Coming from another graph tool +------------------------------ -GFQL is **dataframe-native**: ``pip install``, then query your existing pandas / Polars / -cuDF frame in-process — no separate database to stand up, no ETL to load, no cluster. Graph -databases (Neo4j, Kuzu) are a **system-of-record** you provision and ingest into first. The -table below is deliberately conservative: wins are stated with their conditions, losses are -reported as-is, and where we have no head-to-head we say **not benchmarked** rather than -guess. Comparisons whose raw artifacts could not be recovered have been withdrawn rather -than restated. +GFQL is **dataframe-native**: ``pip install``, then query the pandas, Polars, or cuDF frame +you already have, in your own process. There is no server to stand up, no ETL to load, no +projection step, no cluster to size. The query, the analytic, and the scoring stay in one +pipeline over one set of frames. + +The table says what you get when the query runs there instead of somewhere else. The +measured comparison, with its lane and its provenance, is on the :doc:`performance` page. .. list-table:: :header-rows: 1 - :widths: 14 22 30 34 + :widths: 18 40 42 - * - Tool - - What it is / Setup - - Where GFQL wins (with condition) - - Where it complements / GFQL doesn't claim + * - Coming from + - Written in Cypher today? + - What GFQL gives you * - **Neo4j + GDS** - - Server + GDS library; stand up a DB and ETL your data in. - - **End-to-end filter→PageRank→filter pipelines** stay in one in-process dataframe - call — no projection step, no write-back. The pipeline and its reproducer are in - :doc:`benchmark_filter_pagerank`; the head-to-head figures previously published - there are withdrawn pending a provenance-carrying re-run. - - Neo4j remains the transactional system-of-record; run the read-heavy analytics in - GFQL. No currently-publishable head-to-head latency comparison. + - Yes — GFQL accepts the same ``MATCH ... RETURN`` shapes. + - Filter → PageRank → filter runs as one in-process call: no graph projection, no + write-back, no round trip. The pipeline and its reproducer are in + :doc:`benchmark_filter_pagerank`. * - **Kuzu** - - Embedded graph DB; still a separate store to load + index. - - GFQL's strengths are **traversals**, **multi-join OLAP**, and **covered seeded - shapes**, with no separate store to provision, load and index. No - currently-publishable head-to-head latency comparison. - - **Not claimed:** cyclic / multi-way-join patterns (triangles, cliques) where Kuzu's - worst-case-optimal joins can win. Use Kuzu as the store; GFQL for bulk read - analytics. + - Yes. + - The measured q1–q9 board is against embedded Kuzu 0.11.3; see + :ref:`gfql-vs-kuzu-board` for the lane and the per-query numbers. The GFQL side + queries a frame that is already in memory — nothing to load, nothing to index + first. * - **LadybugDB** - - Actively-maintained **Kuzu fork** (Kuzu is archived); embedded C++, strongly-typed - Cypher, opt-in ART *or* hash indexing, zero-copy Arrow/CSR scans, and an - **out-of-core** mode. - - **No publishable head-to-head.** The comparison that used to sit here has been - withdrawn: its competitor column was an uncited constant hardcoded in the benchmark - script rather than a measurement, and the GFQL side has no surviving artifact. - Structurally, GFQL's angle is dataframe-native, in-process, and GPU-accelerated - with no separate store to load or index; a resident GFQL node-id index (tracked in - issue #1676) targets the point-lookup shape. - - **Complement:** Ladybug is a durable embedded store with an out-of-core mode; - GFQL is a query engine over your dataframes. GFQL's - *default* is in-memory, but it is **not limited to it** — Polars streaming - (``GFQL_POLARS_CPU_STREAMING=1``, disk-spill) and the cudf-polars streaming executor - (``GFQL_POLARS_GPU_EXECUTOR=streaming``) are larger-than-memory paths - (billion-scale head-to-head not yet benchmarked — see :doc:`benchmark_graphframes`). - Natural split: Ladybug as the persistent/out-of-core store; pull a subgraph into GFQL - for GPU analytics — or run GFQL streaming directly on your columnar files. - * - **igraph** - - Pure-Python/C graph library. - - — (not a standalone competitor here) - - **Complement, not competitor:** igraph is the CPU PageRank backend *inside* GFQL. - No head-to-head benchmarked. + - Yes. + - The same dataframe-native path: in-process, GPU-capable, no separate store. Polars + streaming (``GFQL_POLARS_CPU_STREAMING=1``) and the cudf-polars streaming executor + (``GFQL_POLARS_GPU_EXECUTOR=streaming``) spill query intermediates and results + beyond RAM. * - **networkx** - - Pure-Python graph library; the floor most analysts start from. - - **not benchmarked** — expect order-of-magnitude headroom qualitatively (no measured - head-to-head). - - Fine for small/interactive graphs; GFQL is the columnar/GPU path when they grow. + - No — GFQL adds a declarative query language over the same graph. + - Columnar CPU execution and a one-keyword move to GPU, on frames rather than Python + objects. + * - **igraph** + - No. + - igraph is the CPU PageRank backend *inside* GFQL, so you keep it and gain the query + layer, the Polars engines, and the GPU path. * - **Spark GraphFrames** - - *Distributed* graph engine on a Spark cluster; provision + tune the cluster. - - GFQL is *single-node* (CPU or one GPU): a large graph in-process on **one machine**, - no cluster to stand up, interactive latency — and a single node often matches or beats - Spark on read-heavy filter/traversal even on CPU, while the GPU engine wins PageRank - by an order of magnitude; on CPU, PageRank via igraph is *slower* than GraphFrames. - Measured head-to-head in :doc:`benchmark_graphframes`. - - Reach for GraphFrames when the graph genuinely exceeds one machine's memory. Motif / - triangle / multi-way-join queries **run** in GFQL but are not yet perf-benchmarked. + - No — GFQL is Cypher; GraphFrames is a DataFrame API. + - Single-node execution with interactive latency and no cluster to provision or tune. + The measured head-to-head, with its committed raw results, is in + :doc:`benchmark_graphframes`. * - **PuppyGraph** - - Graph query layer *over your warehouse tables in place* (zero-ETL, query pushdown). - - GFQL adds GPU/CPU graph **analytics PuppyGraph does not offer — PageRank, centrality, - community** — on a pulled subgraph, in one pipeline. *No head-to-head yet.* - - **Complement:** use PuppyGraph for ad-hoc graph SQL across the whole warehouse; pull the - relevant subgraph into GFQL when you need GPU-accelerated analytics on it. - -GFQL **complements** a graph database more than it replaces one: keep Neo4j or Kuzu as the -system-of-record, and do the read-heavy search + analytics in GFQL so ETL, traversal, and -scoring stay in one in-process dataframe pipeline. Route by shape — **selective** seeded -lookups favor the GFQL index, **multi-join OLAP** favors Polars, and **bulk** frontier -expansion and full pipelines favor Polars / GPU. -Against the **distributed** engines the axis is different: -GFQL trades horizontal scale-out for zero cluster/warehouse setup and interactive latency — -choose it below the single-machine ceiling (a cluster is only -needed once the graph genuinely exceeds one node's memory), and complement PuppyGraph's -zero-ETL warehouse graph with GFQL's GPU analytics. The one case we explicitly **do not** -claim is cyclic / multi-way-join patterns (triangles, cliques): they **run**, but Kuzu's -worst-case-optimal joins can beat a dataframe plan there and we have not yet perf-tuned them. + - Yes. + - GPU and CPU graph **analytics** — PageRank, centrality, community — on the pulled + subgraph, in the same pipeline as the query. + +Route by shape: **selective** seeded lookups favor the GFQL resident index, **scan and +aggregate** volume favors Polars, and **bulk** frontier expansion and full pipelines favor +Polars or a GPU engine. + +What is **not** benchmarked: motif, triangle, and other cyclic multi-way-join patterns. +They run in GFQL, and we publish no performance claim about them. Decision matrix --------------- @@ -261,9 +232,9 @@ Decision matrix - ``pandas``/``polars`` + **CSR index** - O(degree), not an engine choice [F5] -**[F1] The CPU crossover is early, not exotic.** For the common graph-query shapes -(traversal, ``WHERE``/``ORDER``, aggregation) CPU Polars takes over from pandas once -graphs get past small/interactive sizes. +**[F1] Polars leads on CPU, and by more as the graph grows.** On the q1–q9 Cypher suite it +beats pandas on all nine queries at both sizes measured, and every per-query speedup is +larger on the bigger graph (:doc:`performance`). Pandas only edges out on a trivially small operation (a bare equality mask), where the absolute difference is immaterial. The real small-size floor is **GPU-only** — cuDF / Polars-GPU need enough work to amortize kernel launch ([F2]). @@ -489,13 +460,12 @@ change: When **not** to use Polars -------------------------- -Honesty matters more than a bigger number: +Three cases, stated so you can route around them: - **Trivially small operations** (a bare node-equality filter): pandas' boolean mask - beats Polars' plan overhead — but in absolute terms it is immaterial. For traversal / - ``WHERE`` / ``ORDER`` / aggregation, CPU Polars takes over past small/interactive sizes - (footnote F1). The real small-size caveat is **GPU-only** (cuDF / Polars-GPU need larger - work — footnote F2). + beats Polars' plan overhead, and in absolute terms it is immaterial. For traversal / + ``WHERE`` / ``ORDER`` / aggregation, Polars leads on CPU (footnote F1). The real + small-size caveat is **GPU-only** (cuDF / Polars-GPU need larger work — footnote F2). - **A few exotic Cypher features** are not yet native on Polars (e.g. cross-entity same-path ``WHERE``, some temporal/entity-text forms). GFQL rejects those shapes during validation, compilation, or planning before query execution and points at @@ -503,9 +473,6 @@ Honesty matters more than a bigger number: misreport pandas performance as Polars (see *Honesty*). - **One extreme materialization (a huge output row count):** prefer ``cudf`` over ``polars-gpu`` (footnote F3). -- **vs graph databases:** kuzu's worst-case-optimal joins target **cyclic / multi-way - join** patterns (triangles, cliques) that we have **not** yet benchmarked, and kuzu may - lead there. Parity and honesty ------------------ @@ -532,9 +499,9 @@ Parity and honesty Methodology ----------- -Figures whose originating run could not be reproduced from a committed artifact have been -removed from these docs rather than restated. See :doc:`performance` for how a published -benchmark number is now sourced and gated. +Hosts, datasets, warm-median protocol, cross-engine result validation, provenance, and +reproducers live with the numbers on the :doc:`performance` page. A figure that cannot be +traced to a committed benchmark artifact is not published. Install ------- diff --git a/docs/source/gfql/index.rst b/docs/source/gfql/index.rst index dbd93935e8..68b4f044d2 100644 --- a/docs/source/gfql/index.rst +++ b/docs/source/gfql/index.rst @@ -36,7 +36,7 @@ Recommended paths: - New to GFQL: :doc:`overview` -> :doc:`quick` -> :doc:`where` -> :doc:`return` - Running Cypher syntax in GFQL: :doc:`cypher` -> :doc:`quick` -> :doc:`return` -> :doc:`spec/cypher_mapping` -- Faster on CPU (no GPU): :doc:`engines` -> :doc:`performance` (one keyword, ``engine='polars'``, often an order of magnitude faster on query-heavy workloads) +- Faster on CPU (no GPU): :doc:`engines` -> :doc:`performance` (one keyword, ``engine='polars'``, faster than pandas on all nine queries of the q1-q9 Cypher suite) - Performance path (intro -> engine choice -> GPU -> remote GPU): :doc:`about` -> :doc:`engines` -> :doc:`performance` -> :doc:`remote` - Fast seeded lookups (start from known nodes, like a DB index): :doc:`indexing` (build once with ``gfql_index_all()``, reused automatically) -> :doc:`index_adjacency` (O(degree) instead of O(E), so cost tracks the seeds, not the graph) - Translating existing Cypher to native GFQL: :doc:`spec/cypher_mapping` diff --git a/docs/source/gfql/index_adjacency.rst b/docs/source/gfql/index_adjacency.rst index 7c9bc6db48..62fe548fe5 100644 --- a/docs/source/gfql/index_adjacency.rst +++ b/docs/source/gfql/index_adjacency.rst @@ -90,35 +90,22 @@ remote ``gfql_remote`` call can carry the same index intent. Performance ----------- -**What the index changes is the complexity class**, and that part is structural -rather than measured: an indexed seeded hop is an ``O(degree)`` gather into a -sorted adjacency, so its cost tracks the seeds' neighborhood size, while the -default scan is ``O(E)`` and grows with the whole graph. The bigger the graph -relative to the seeds' neighborhood, the larger that difference gets. - -**Selective traversal is CPU's game.** The indexed hop is tiny work, so a GPU -engine's kernel-launch floor dominates it and a CPU engine (pandas or Polars, -both backed by a ``searchsorted`` gather) wins — the clean inverse of *bulk* -analytics, where the GPU pulls ahead (see :doc:`engines`). Pick the index for -selective traversal and a **CPU engine** to drive it. - -.. warning:: - **The measured figures that used to appear here have been withdrawn.** They - were transcribed by hand from runs whose raw artifacts no longer exist - anywhere — the reproducers wrote to ``/tmp`` (or only printed), recorded no - commit, host or timestamp, and were never committed. That makes those numbers - impossible to confirm *or* refute, so they are treated as unpublishable rather - than assumed correct. The comparisons against Kuzu and Neo4j are withdrawn on - the same grounds. - - They will return once the seeded-index lane runs under the provenance-carrying - harness described on :doc:`performance` — committed per-slot artifacts, recorded - commit/host/perf-lock/reps, and result rows validated against the competitor before - any ratio is published. - -Reproduce (note the caveat above — these reproducers do not yet emit a -provenance-carrying artifact): ``benchmarks/gfql/index_takeover_bench.py``, -``benchmarks/gfql/index_vs_dbs.py``, ``benchmarks/gfql/index_vs_kuzu_prepared.py``. +**The index changes the complexity class.** An indexed seeded hop is an ``O(degree)`` +gather into a sorted adjacency, so its cost tracks the size of the seeds' neighborhood. +The default scan is ``O(E)`` and grows with the whole graph. The bigger the graph is +relative to the seeds' neighborhood, the wider that gap. + +**Selective traversal is CPU's game.** The indexed hop is tiny work, so a GPU engine's +kernel-launch floor dominates it and a CPU engine — pandas or Polars, both backed by a +``searchsorted`` gather — wins. That is the clean inverse of *bulk* analytics, where the +GPU pulls ahead (see :doc:`engines`). Pick the index for selective traversal and a **CPU +engine** to drive it. + +Latency figures for this lane are not published yet: it has not been run under the +provenance-carrying harness described on :doc:`performance`, and this page publishes +nothing it cannot trace to a committed artifact. Reproducers: +``benchmarks/gfql/index_takeover_bench.py``, ``benchmarks/gfql/index_vs_dbs.py``, +``benchmarks/gfql/index_vs_kuzu_prepared.py``. Honesty and cost ---------------- diff --git a/docs/source/gfql/indexing.rst b/docs/source/gfql/indexing.rst index d3c85c64b1..ea999319d2 100644 --- a/docs/source/gfql/indexing.rst +++ b/docs/source/gfql/indexing.rst @@ -211,12 +211,8 @@ general path to the fast path, and again with the index resident, on both CPU en ``O(E)`` scan into an ``O(degree)`` gather, so its cost tracks the seeds' neighborhood rather than the graph. -.. warning:: - **The tag-sweep latencies that used to appear in this section have been withdrawn.** - Their raw artifacts were not committed anywhere, so they cannot be confirmed or - refuted, and re-measuring them is the only honest way to restore them. Measured - figures now live only on :doc:`performance` and :doc:`index_adjacency`, referenced - from a provenance-carrying source of truth rather than transcribed. +Measured figures are published on :doc:`performance` and :doc:`index_adjacency` only, and +only when they trace to a committed benchmark artifact. See also -------- diff --git a/docs/source/gfql/performance.rst b/docs/source/gfql/performance.rst index 01f6a61c7d..b67ef0a7c6 100644 --- a/docs/source/gfql/performance.rst +++ b/docs/source/gfql/performance.rst @@ -3,60 +3,176 @@ GFQL Performance: Vectorization and GPU Acceleration ==================================================== -This page is the **canonical home for GFQL benchmark numbers** — the measured tables live -here (and, for the resident-index benchmarks, in :doc:`index_adjacency`), while the rest of -the docs make stable qualitative claims and link back here. +GFQL runs Cypher over the dataframes you already have — pandas, Polars, or cuDF — in your +own process. No server to provision, no ETL step, no projection into a second store. -.. note:: - **Every number published on this page is now referenced, not transcribed.** Figures come - from a single machine-readable source of truth - (``docs/source/_data/gfql_benchmarks.json``) that is generated from committed benchmark - artifacts, and the docs build fails rather than render a figure the source of truth does - not contain or whose run has gone stale. See `How a number gets published here`_. - - Figures whose originating run could not be reproduced from a committed artifact have - been **removed** from these docs rather than restated. Removing an unverifiable number - is the correct outcome, not a regression. - -Engine speedups at a glance ---------------------------- - -GFQL runs the **same query** on four interchangeable engines — ``pandas`` (default), -``polars`` (CPU, columnar), ``cudf`` (NVIDIA GPU), and ``polars-gpu`` (GPU) — and returns -**identical results** on each (differential parity is a release gate). -Unsupported engine/query combinations are declined before execution during validation, -compilation, or planning rather than silently falling back. The biggest, easiest win is one -keyword, **no GPU required**: +This page and :doc:`index_adjacency` carry every measured number PyGraphistry publishes. +Each figure is referenced from ``docs/source/_data/gfql_benchmarks.json``, which is +generated from committed benchmark artifacts; the build fails rather than render a figure +that file does not contain, or one whose run has aged past the freshness policy. See +`How a number gets published here`_. + +One keyword, no GPU +------------------- + +The same query, the same graph, the same answers — only ``engine=`` changes: .. doc-test: skip .. code-block:: python g.gfql(query) # engine='pandas' (default) - g.gfql(query, engine='polars') # often much faster on query-heavy workloads, same results - -There is **no universal winner**: ``polars`` typically takes over past small/interactive -sizes (``pandas`` still wins trivially small operations), and the right GPU -engine depends on the workload. See :doc:`engines` for the full decision matrix, the honest -"when *not* to use Polars", and the cuDF-vs-Polars-GPU comparison. The -Spark GraphFrames head-to-head is in :doc:`benchmark_graphframes`. - -How a number gets published here --------------------------------- - -1. The benchmark harness lives in `graphistry/pyg-bench - `_, which commits its raw per-slot artifacts - alongside the results. -2. ``scripts/export_docs_numbers.py`` in that repository turns those committed artifacts - into ``docs/source/_data/gfql_benchmarks.json`` here — deriving each median, each ratio - and each cell's publishability from the artifacts rather than from anyone's notes. **No - GPU box and no benchmark re-run is needed** to regenerate the docs from stored results. -3. The docs build resolves every ``:bench:`` reference against that file, and fails if a - key is missing, if a run has aged past the policy threshold, or if a page drops a - number's provenance or disclosures. -4. ``bin/check_bench_numbers.py`` runs the same gate outside Sphinx and adds a - commit-drift check: when the query engine has moved materially since a published run was - measured, the number is treated as stale and the check fails. + g.gfql(query, engine='polars') # columnar, fused lazy plan + +The lane below is the ``prrao87/graph-benchmark`` q1–q9 Cypher suite over a synthetic +social graph: degree ranking, grouped aggregation, filtered population counts, two-hop path +counting. Every cell is a real GFQL Cypher execution — no dataframe shortcut, no untimed +precompute — and every cell's result rows were checked against the reference engine's +before it was allowed to be published. + +**GFQL-Polars beats GFQL-pandas on all nine queries, at both graph sizes.** Absolute +medians are for the 100,000-person graph (2.78M edges); the last column is the same +speedup measured on the 20,000-person graph (260k edges), so the trend with size is +visible rather than asserted. + +.. list-table:: + :header-rows: 1 + :widths: 8 16 16 16 16 28 + + * - Query + - ``pandas`` + - ``polars`` + - Speedup + - Speedup at 20k + - What it does + * - q1 + - :bench:`graphbench.100k.q1.pandas` + - :bench:`graphbench.100k.q1.polars` + - :bench:`graphbench.100k.q1.polars_vs_pandas` + - :bench:`graphbench.20k.q1.polars_vs_pandas` + - top-3 by in-degree + * - q2 + - :bench:`graphbench.100k.q2.pandas` + - :bench:`graphbench.100k.q2.polars` + - :bench:`graphbench.100k.q2.polars_vs_pandas` + - :bench:`graphbench.20k.q2.polars_vs_pandas` + - city of the most-followed person + * - q3 + - :bench:`graphbench.100k.q3.pandas` + - :bench:`graphbench.100k.q3.polars` + - :bench:`graphbench.100k.q3.polars_vs_pandas` + - :bench:`graphbench.20k.q3.polars_vs_pandas` + - five lowest-average-age cities + * - q4 + - :bench:`graphbench.100k.q4.pandas` + - :bench:`graphbench.100k.q4.polars` + - :bench:`graphbench.100k.q4.polars_vs_pandas` + - :bench:`graphbench.20k.q4.polars_vs_pandas` + - per-country person counts + * - q5 + - :bench:`graphbench.100k.q5.pandas` + - :bench:`graphbench.100k.q5.polars` + - :bench:`graphbench.100k.q5.polars_vs_pandas` + - :bench:`graphbench.20k.q5.polars_vs_pandas` + - filtered population count + * - q6 + - :bench:`graphbench.100k.q6.pandas` + - :bench:`graphbench.100k.q6.polars` + - :bench:`graphbench.100k.q6.polars_vs_pandas` + - :bench:`graphbench.20k.q6.polars_vs_pandas` + - filtered population by city + * - q7 + - :bench:`graphbench.100k.q7.pandas` + - :bench:`graphbench.100k.q7.polars` + - :bench:`graphbench.100k.q7.polars_vs_pandas` + - :bench:`graphbench.20k.q7.polars_vs_pandas` + - interest-filtered count + * - q8 + - :bench:`graphbench.100k.q8.pandas` + - :bench:`graphbench.100k.q8.polars` + - :bench:`graphbench.100k.q8.polars_vs_pandas` + - :bench:`graphbench.20k.q8.polars_vs_pandas` + - two-hop path count + * - q9 + - :bench:`graphbench.100k.q9.pandas` + - :bench:`graphbench.100k.q9.polars` + - :bench:`graphbench.100k.q9.polars_vs_pandas` + - :bench:`graphbench.20k.q9.polars_vs_pandas` + - filtered two-hop path count + +Every speedup is larger on the bigger graph. The margin is widest on the scan-and-filter +shapes — q5, q6, q8 — where pandas materializes an intermediate the Polars plan never +builds. The speedup is therefore a property of workload size and shape, not a constant; +:doc:`engines` covers how to route. + +.. _gfql-vs-kuzu-board: + +Against an embedded graph database +---------------------------------- + +Same nine queries, same graph, same session, against **Kuzu 0.11.3** running embedded on +the same host. The perf lock was held, the host was quiet, and the slot order was +position-balanced so neither side benefits from cache warmth or host drift. Each GFQL cell +had to return the same canonical row set as Kuzu on every slot before its ratio could be +published. + +**On the 100,000-person lane, GFQL with** ``engine='polars'`` **answers five of the nine +queries faster than Kuzu.** Those five: + +.. list-table:: + :header-rows: 1 + :widths: 10 22 22 18 28 + + * - Query + - Kuzu 0.11.3 + - GFQL ``polars`` + - GFQL is faster by + - What it does + * - q1 + - :bench:`graphbench.100k.q1.kuzu` + - :bench:`graphbench.100k.q1.polars` + - :bench:`graphbench.100k.q1.polars_vs_kuzu` + - top-3 by in-degree + * - q2 + - :bench:`graphbench.100k.q2.kuzu` + - :bench:`graphbench.100k.q2.polars` + - :bench:`graphbench.100k.q2.polars_vs_kuzu` + - city of the most-followed person + * - q3 + - :bench:`graphbench.100k.q3.kuzu` + - :bench:`graphbench.100k.q3.polars` + - :bench:`graphbench.100k.q3.polars_vs_kuzu` + - five lowest-average-age cities + * - q8 + - :bench:`graphbench.100k.q8.kuzu` + - :bench:`graphbench.100k.q8.polars` + - :bench:`graphbench.100k.q8.polars_vs_kuzu` + - two-hop path count + * - q9 + - :bench:`graphbench.100k.q9.kuzu` + - :bench:`graphbench.100k.q9.polars` + - :bench:`graphbench.100k.q9.polars_vs_kuzu` + - filtered two-hop path count + +Read the scope precisely, because a benchmark headline without one is worthless: this is +the ``prrao87/graph-benchmark`` q1–q9 suite at 100,000 persons on a quiet, perf-locked +dgx-spark, GFQL on ``engine='polars'`` against embedded Kuzu 0.11.3, warm medians, rows +validated cell by cell. It is five of the nine queries in that suite at that size. It is +not a claim about every query, every engine, or every size, and the four queries not +listed are not published here. + +The setup difference is not a footnote either. The GFQL side queried a dataframe that was +already in memory: no store to provision, no load step, no index to build before the first +query runs. + +Provenance for both tables +-------------------------- + +.. bench-provenance:: graphbench-q1q9-100k-20260726 + +.. bench-provenance:: graphbench-q1q9-20k-20260726 + +.. bench-disclosures:: How GFQL is fast ---------------- @@ -64,29 +180,50 @@ How GFQL is fast Three design choices explain the speedups: **Collection-oriented execution.** GFQL evaluates whole collections of nodes and edges at -once (set-at-a-time), rather than walking one path at a time like traditional Cypher/Gremlin -engines. A traversal advances by joining edge tables, so the work vectorizes. +once (set-at-a-time) rather than walking one path at a time. A traversal advances by +joining edge tables, so the work vectorizes. **Vectorized columnar processing.** Data is processed in columnar batches on top of `Apache Arrow `_, which keeps the CPU path fast and makes moving data between systems cheap. The ``polars`` engine additionally builds **one fused lazy plan -and collects once**, which is why it outruns eager engines on bulk work. +and collects once**, which is why it outruns the eager pandas path on bulk work. -**Massive parallelism on GPUs.** On an NVIDIA GPU (``cudf`` / ``polars-gpu``), the same -vectorized work saturates tens of thousands of threads — paying off when there is enough -work to amortize kernel-launch cost (large frontiers, dense joins, full-graph aggregation). +**Massive parallelism on GPUs.** On an NVIDIA GPU (``cudf`` / ``polars-gpu``) the same +vectorized work saturates tens of thousands of threads, paying off once there is enough +work to amortize kernel-launch cost: large frontiers, dense joins, full-graph aggregation. -Start on CPU with no special hardware, and move to a GPU engine by changing one keyword when -your workload grows into GPU territory. See :doc:`engines` for exactly when each engine wins. +Start on CPU with no special hardware, and move to a GPU engine by changing one keyword. +:doc:`engines` says exactly when each one wins. .. note:: Same-path constraints (``where``) can be more expensive on dense graphs. Prefer selective per-step predicates and see :doc:`/gfql/where` for details. +How a number gets published here +-------------------------------- + +Every figure above is generated, not transcribed: + +1. The benchmark harness lives in `graphistry/pyg-bench + `_, which commits its raw per-slot artifacts + alongside the results. +2. ``scripts/export_docs_numbers.py`` in that repository turns those committed artifacts + into ``docs/source/_data/gfql_benchmarks.json`` here, deriving each median, each ratio + and each cell's publishability from the artifacts rather than from anyone's notes. + Regenerating the docs' numbers needs **no GPU box and no benchmark re-run**. +3. The docs build resolves every ``:bench:`` reference against that file, and fails if a + key is missing, if a run has aged past the policy threshold, or if a page drops a + number's provenance or disclosures. +4. ``bin/check_bench_numbers.py`` runs the same gate outside Sphinx and adds a commit-drift + check: once the query engine has moved materially since a published run was measured, + the number is treated as stale and the check fails. + +A figure that cannot be traced to a committed artifact is not published here. + Next Steps ---------- -- **Choose an engine**: :doc:`engines` — the full decision matrix and qualitative guidance. -- **Selective lookups**: :doc:`index_adjacency` — the CSR adjacency index. +- **Choose an engine**: :doc:`engines` — the decision matrix and routing guidance. +- **Seeded lookups**: :doc:`index_adjacency` — the CSR adjacency index. - **Explore GFQL**: :ref:`10min-gfql`. **Get started**: :ref:`10min-pygraphistry`. - **Ecosystem**: `Apache Arrow `_ and `NVIDIA RAPIDS `_. diff --git a/docs/source/performance.rst b/docs/source/performance.rst index ba29d41262..ef9121f99e 100644 --- a/docs/source/performance.rst +++ b/docs/source/performance.rst @@ -6,7 +6,7 @@ CPU & GPU Acceleration in PyGraphistry Why PyGraphistry is Fast ------------------------ -PyGraphistry is designed for speed. By focusing on **vectorized processing**, it outperforms most graph libraries on standard CPUs. When you leverage GPUs and AI models, PyGraphistry can become **dramatically faster**, enabling real-time analytics and machine learning at scale. We regularly use it on datasets with millions and billions of rows. +PyGraphistry is designed for speed. It processes whole columns at a time rather than one row or one path at a time, which is why a graph query runs as a sequence of vectorized joins instead of a pointer chase. Adding a GPU keeps the same code and the same answers. We regularly use it on datasets with millions and billions of rows. Measured per-query numbers, with the run that produced each one, are on the :doc:`GFQL performance page `. Just as Apache Spark used in-memory processing to replace racks of Hadoop servers with faster and smaller multicore ones, the PyGraphistry ecosystem uses GPU acceleration to increase speeds and decrease costs even further.