Skip to content

Add experimental openCypher graph query frontend#17938

Open
xiangfu0 wants to merge 1 commit into
apache:masterfrom
xiangfu0:experimental-graph-query-frontend
Open

Add experimental openCypher graph query frontend#17938
xiangfu0 wants to merge 1 commit into
apache:masterfrom
xiangfu0:experimental-graph-query-frontend

Conversation

@xiangfu0

Copy link
Copy Markdown
Contributor

Summary

Adds an experimental graph query feature that translates a subset of openCypher into SQL JOINs and executes via Pinot's multi-stage engine. This enables graph-style traversal queries over existing Pinot tables without native graph runtime operators.

  • New pinot-graph module (SPI + planner) with hand-written Cypher parser → IR → SQL transpiler
  • POST /query/graph broker endpoint behind pinot.graph.enabled feature flag (default: false)
  • Supports 1-hop and 2-hop traversals, WHERE (comparisons, AND/OR/NOT, STARTS WITH, ENDS WITH, CONTAINS, IN), RETURN (DISTINCT, AS aliases, COUNT/SUM/AVG/MIN/MAX with GROUP BY), ORDER BY, SKIP, LIMIT
  • Graph schema passed inline in request JSON mapping vertex/edge labels to Pinot tables
  • Zero impact on existing SQL query path — purely additive changes

Design doc posted on #17935: #17935 (comment)

Closes #17935

Test plan

  • 211 unit tests (parser: 94, SQL generator: 49, translator: 52, golden: 16) — all pass
  • 12 broker handler tests — all pass
  • 10 E2E integration tests with 2 servers and distributed data (50 users, 200 edges, 4 segments/table) — all pass
  • spotless, checkstyle, license checks — all clean
  • Existing SQL path unaffected (no modifications to query execution)
  • Feature disabled by default; returns clear 400 error when disabled
  • Unsupported Cypher syntax rejected with clear error messages

🤖 Generated with Claude Code

@xiangfu0
xiangfu0 force-pushed the experimental-graph-query-frontend branch 3 times, most recently from c8d0f7a to 7902d0f Compare March 23, 2026 06:49
@codecov-commenter

codecov-commenter commented Mar 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.80228% with 213 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.50%. Comparing base (2892236) to head (99acfad).

Files with missing lines Patch % Lines
...a/org/apache/pinot/graph/planner/CypherParser.java 88.41% 30 Missing and 27 partials ⚠️
...che/pinot/graph/planner/CypherToSqlTranslator.java 72.32% 27 Missing and 17 partials ⚠️
...pinot/broker/api/resources/PinotClientRequest.java 0.00% 43 Missing ⚠️
...a/org/apache/pinot/graph/planner/SqlGenerator.java 85.50% 27 Missing and 12 partials ⚠️
...e/pinot/graph/planner/CypherSemanticValidator.java 79.77% 9 Missing and 9 partials ⚠️
.../java/org/apache/pinot/graph/planner/CypherIR.java 97.00% 3 Missing and 2 partials ⚠️
...not/broker/requesthandler/GraphRequestHandler.java 93.75% 0 Missing and 4 partials ⚠️
...ache/pinot/graph/planner/CypherParseException.java 50.00% 2 Missing ⚠️
.../org/apache/pinot/graph/spi/GraphSchemaConfig.java 96.42% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #17938      +/-   ##
============================================
+ Coverage     65.39%   65.50%   +0.11%     
+ Complexity     1405     1399       -6     
============================================
  Files          3423     3431       +8     
  Lines        215968   217283    +1315     
  Branches      34186    34382     +196     
============================================
+ Hits         141228   142333    +1105     
- Misses        63380    63520     +140     
- Partials      11360    11430      +70     
Flag Coverage Δ
custom-integration1 ?
integration ?
integration1 ?
integration2 ?
java-25 65.50% <83.80%> (+0.11%) ⬆️
temurin 65.50% <83.80%> (+0.11%) ⬆️
unittests 65.50% <83.80%> (+0.11%) ⬆️
unittests1 57.12% <ø> (+<0.01%) ⬆️
unittests2 37.86% <83.80%> (+0.28%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@xiangfu0
xiangfu0 force-pushed the experimental-graph-query-frontend branch from 7902d0f to 0263674 Compare March 30, 2026 10:43
@xiangfu0
xiangfu0 force-pushed the experimental-graph-query-frontend branch 2 times, most recently from bbdd050 to 2247441 Compare July 19, 2026 08:10
Implements an experimental graph query feature that translates a subset of openCypher into SQL JOINs and executes via Pinot's multi-stage engine (MSE). This enables graph-style traversal queries over existing Pinot tables without introducing native graph runtime operators.

Supported Cypher syntax:
- MATCH with 1-hop and 2-hop traversals (outgoing, incoming, undirected)
- WHERE with comparison operators, AND/OR/NOT, string predicates (STARTS WITH, ENDS WITH, CONTAINS), IN lists
- RETURN with DISTINCT, column aliases (AS), aggregations (COUNT, SUM, AVG, MIN, MAX) with auto GROUP BY
- ORDER BY (ASC/DESC), SKIP, LIMIT

Architecture:
- New pinot-graph module (pinot-graph-spi + pinot-graph-planner)
- Hand-written recursive descent Cypher parser to IR to SQL generator
- POST /query/graph broker endpoint (behind pinot.graph.enabled flag)
- Graph schema passed inline in request JSON (vertex/edge to table mapping)

Tests: 233 total (211 unit + 12 handler + 10 E2E integration with 2 servers)

Closes: apache#17935
@xiangfu0
xiangfu0 force-pushed the experimental-graph-query-frontend branch from 2247441 to 99acfad Compare July 20, 2026 08:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[discuss] Native Graph Query Language Support in Pinot

2 participants