Skip to content

Commit 7bc9b00

Browse files
lmeyerovclaude
andauthored
docs(gfql): withdraw the OLAP q8 magnitude claim (5.0 ms vs 1,004 ms, 200x) (#1817)
`gfql/performance.rst` published the graph-benchmark OLAP result as "q8 runs in 5.0 ms vs 1,004 ms for embedded Kuzu (200x)". The claim is withdrawn -- reconciled, not merely re-measured. Our own side held up: 5.0 ms published vs 5.06 ms re-measured. The competitor side did not. The old harness replaced q8/q9 path enumeration with an `OPTIONAL MATCH` degree-product rewrite and fed Kuzu pre-lowercased columns built OUTSIDE the timing loop, while GFQL paid `toLower()` in-query. Re-running that same handicapped configuration reproduces 964.60 ms on the same box -- a 98.9x handicap applied to the competitor, not GFQL speed. Canonical Kuzu 0.11.3, position-balanced and row-validated at 100k persons / 2.4M FOLLOWS, is 9.76 ms vs 5.06 ms = 1.93x. The directional claim (`engine='polars'` wins q8) is measured and is kept; only the magnitude goes. No replacement figure is published here. Scope is deliberately one sentence: the q9 14.2x figure, the LadybugDB comparison figures, and the `engines.rst` "one to two orders of magnitude" clause are all left at their master text -- those are being researched/re-measured separately, and a published number is not removed on a re-measurement alone. Claude-Session: https://claude.ai/code/session_01YYZRXegrALuXd3NHH5evqx Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent 1537e46 commit 7bc9b00

2 files changed

Lines changed: 3 additions & 3 deletions

File tree

CHANGELOG.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,8 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
4141

4242
### Documentation
4343
- **GFQL pay-as-you-go resident indexing user guide**: New :doc:`Pay-As-You-Go Resident Indexing <gfql/indexing>` page — the lifecycle guide to resident indexes (`gfql_index_all()` / `gfql_index_edges()` / `create_index()` / `show_indexes()` / `drop_index()`): what the node-id + CSR in/out adjacency sidecars are, what engages them on 0.58.0 (seeded typed-hop fast paths incl. property RETURNs and property-seeded lookups per #1768/#1770, direct `g.hop()`; the general polars chain traversal honestly noted as not yet covered), the staleness/validity contract (identity + fingerprint; rebind invalidates; declines are safe — identical results either way), engine notes (polars needs `gfql_index_all(engine='polars')` until #1767), 0.58.0-tag measured numbers, and a runnable end-to-end example. Wired into the GFQL toctree + recommended paths alongside :doc:`Seeded Traversal Indexes <gfql/index_adjacency>`.
44-
- **GFQL performance docs: 0.58.0 release-tag-verified numbers, siloed in one page**: `gfql/performance.rst` is now the canonical benchmark-numbers page (alongside `gfql/index_adjacency.rst` for the index benchmarks) — a benchmark rerun updates it alone. It carries the 0.58.0 tag sweep (DGX Spark GB10, warm medians N=30; four-engine numbers cross-engine parity-verified, competitor pairs validated against expected result rows): seeded typed-hop fast path across all four engines (e.g. pandas 29.9→2.46ms, 12.1×), native chain form, resident-index covered-shape lookups (with the `gfql_index_all(engine='polars')` caveat / PR #1767), flat seeded-hop scaling on pandas (0.159–0.164ms from 0.25M to 32M edges), the one-keyword `engine='polars'` LDBC SNB SF1 seed-lookup win (1,299.6→106.1ms, 12.3×), LDBC SNB interactive SF1 vs Neo4j 5.26 same-box pairs (GFQL 4 of 5; Neo4j wins recent-replies — reported as-is), OLAP multi-join vs embedded Kuzu (q8 200×, q9 14.2×) with the honest inverse (Kuzu wins single-table aggregates 2–4×, seeded property-projection lookups 2.4–64×), plus the prior Orkut/LiveJournal bulk sweep (moved from `engines.rst`, dated once) and its methodology. All other pages — `engines.rst`, `quick.rst`, `about.rst`, `overview.rst`, `index.rst` — now carry stable qualitative claims (e.g. "often an order of magnitude faster on query-heavy workloads") that link into `performance.rst` instead of inline figures, replacing the stale "up to ~38×" headlines and avoiding scattered per-claim version labels.
44+
- **GFQL performance docs: 0.58.0 release-tag-verified numbers, siloed in one page**: `gfql/performance.rst` is now the canonical benchmark-numbers page (alongside `gfql/index_adjacency.rst` for the index benchmarks) — a benchmark rerun updates it alone. It carries the 0.58.0 tag sweep (DGX Spark GB10, warm medians N=30; four-engine numbers cross-engine parity-verified, competitor pairs validated against expected result rows): seeded typed-hop fast path across all four engines (e.g. pandas 29.9→2.46ms, 12.1×), native chain form, resident-index covered-shape lookups (with the `gfql_index_all(engine='polars')` caveat / PR #1767), flat seeded-hop scaling on pandas (0.159–0.164ms from 0.25M to 32M edges), the one-keyword `engine='polars'` LDBC SNB SF1 seed-lookup win (1,299.6→106.1ms, 12.3×), LDBC SNB interactive SF1 vs Neo4j 5.26 same-box pairs (GFQL 4 of 5; Neo4j wins recent-replies — reported as-is), OLAP multi-join vs embedded Kuzu (q9 14.2×) with the honest inverse (Kuzu wins single-table aggregates 2–4×, seeded property-projection lookups 2.4–64×), plus the prior Orkut/LiveJournal bulk sweep (moved from `engines.rst`, dated once) and its methodology. All other pages — `engines.rst`, `quick.rst`, `about.rst`, `overview.rst`, `index.rst` — now carry stable qualitative claims (e.g. "often an order of magnitude faster on query-heavy workloads") that link into `performance.rst` instead of inline figures, replacing the stale "up to ~38×" headlines and avoiding scattered per-claim version labels.
45+
- **Withdrew the OLAP q8 magnitude claim from `gfql/performance.rst`** (no replacement figure published): the page read "**q8 runs in 5.0 ms vs 1,004 ms for embedded Kuzu (200×)**". Reconciled rather than merely re-measured — our own side held up (5.0 ms published vs 5.06 ms re-measured), but the 1,004 ms was not canonical Kuzu: the old harness replaced q8/q9 path enumeration with an `OPTIONAL MATCH` degree-product rewrite and fed Kuzu pre-lowercased columns built *outside* the timing loop, while GFQL paid `toLower()` in-query. Re-running that same handicapped configuration reproduces **964.60 ms on the same box**, i.e. a 98.9× handicap applied to the competitor; canonical Kuzu 0.11.3 on a position-balanced, row-validated run at 100k persons / 2.4M FOLLOWS is **9.76 ms vs 5.06 ms = 1.93×**. The directional claim (`engine='polars'` wins q8) survives and is kept; the magnitude is withdrawn. The companion q9 figure is unchanged here — it is being researched separately.
4546

4647
## [0.58.0 - 2026-07-22]
4748

docs/source/gfql/performance.rst

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -137,8 +137,7 @@ OLAP multi-join
137137
~~~~~~~~~~~~~~~
138138

139139
On the graph-benchmark OLAP multi-join queries at 100k-node scale with
140-
``engine='polars'``: **q8 runs in 5.0 ms vs 1,004 ms for embedded Kuzu (200×)**; q9 is
141-
**14.2×**.
140+
``engine='polars'``: GFQL wins q8 against embedded Kuzu; q9 is **14.2×**.
142141

143142
When not to use GFQL
144143
~~~~~~~~~~~~~~~~~~~~

0 commit comments

Comments
 (0)