Skip to content

GEOMESA-3582 Trino - Removes BBOX predicates from datastore SQL, adds as feature flag to the connector#3589

Merged
elahrvivaz merged 5 commits into
locationtech:mainfrom
cwdobbins:feature/sql-refactor
Jul 22, 2026
Merged

GEOMESA-3582 Trino - Removes BBOX predicates from datastore SQL, adds as feature flag to the connector#3589
elahrvivaz merged 5 commits into
locationtech:mainfrom
cwdobbins:feature/sql-refactor

Conversation

@cwdobbins

@cwdobbins cwdobbins commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

GEOMESA-3582: consolidate spatial pushdown in the connector

Splits spatial-filter handling cleanly between the two geomesa-trino modules and makes query plans deterministic regardless of how a query arrives.

Background

Previously the datastore rewrote CQL spatial predicates into composite SQL — __X_bbox__ overlap prefilters plus CASE WHEN row-level shortcuts — and the spatial_iceberg connector independently derived the same pruning from either ST_* calls or the recognized bbox-comparison pattern. The same logical query could therefore produce different plans depending on whether it came through the datastore or as hand-written Trino SQL, and near-identical logic was maintained in two places.

1. Datastore emits pure ST_* (commit 1)

TrinoFilterToSQL now emits only row-level ST_* function calls (plus temporal / attribute predicates). Swapped operand forms are normalized column-first via the OGC complement (within(a,b) ⇔ contains(b,a)) so the connector's ST_*(ST_GeomFromBinary(col), literal) recognizer always matches. The connector is now the single place bbox/Z2/XZ2 pushdown is derived, so datastore queries and hand-written Trino SQL get identical treatment and deterministic plans.

2. Connector-side bbox short-circuit for rectangle ST_Intersects (commit 2, feature-flagged)

Emitting pure ST_* removes the datastore's inline CASE WHEN row-level shortcut, which had let a row whose bbox was fully inside the query rectangle skip the WKB decode. This commit recovers a sound version of that saving entirely inside the connector, with no datastore coupling. pruning; those domains ride to the worker in the table handle's unenforcedPredicate. For the one case the bbox proves exactly — a rectangle ST_Intersects on a Z2/point geometry column — SpatialConnectorMetadata claims the predicate enforced (replacing only the st_intersects node with TRUE, never the is_visible row filter or any other conjunct) and wraps the handle so SpatialPageSourceProvider becomes the authoritative filter. Reading only the cheap bbox columns, the page source:

  • accepts rows whose bbox lies fully inside the query rectangle with no WKB decode,
  • rejects rows whose bbox cannot overlap it, and
  • decodes the geometry and runs the exact intersects test only on the thin envelope-boundary shell.

For point geometry columns the stored bbox is degenerate (xmin == xmax, ymin == ymax), so the classifier reads just the two coordinate columns and the box test collapses to a point-in-range check.

  • Sound / exact: directional float32 rounding (reject box rounded outward, accept box inward) guarantees any row that a nearest-rounded float32 bbox could misclassify falls through to the exact test, so the result is identical to the engine's exact predicate. Only the st_intersects call is claimed — is_visible and every other conjunct still execute.
  • Feature-flagged via geomesa.spatial.bbox-page-filter (on by default); see commit 3 for the flag's cost/benefit and how to disable it.

3. Connector-side bbox reject pre-filter for the general case (commit 3)

Commit 2's short-circuit engages only for the exactly-provable case (rectangle ST_Intersects on a Z2/point column). This commit extends the same page source with a reject-only mode covering every other spatial predicate the connector could not claim enforced — non-rectangular query geometries, non-point columns, ST_Contains, the outer envelope of a DWITHIN distance query, etc.

In reject-only mode the page source drops any row whose bbox cannot overlap the query envelope (via SourcePage.selectPositions, before the geometry column is materialized) and hands the survivors to the engine's exact ST_* residual unchanged. It reuses the same reject-bound machinery as the short-circuit's reject pass. This is a pure pre-filter:

  • Sound / non-destructive: only rows failing a necessary overlap condition are dropped, so no matching row is ever removed; the engine's exact predicate still runs on every survivor. Null bbox values are kept. Validated by cross-checking counts against the stock iceberg catalog (no pushdown) across rectangle, triangle, ST_Within, DWITHIN, and non-spatial filters.
  • Boosts coarsely-pruned, non-point / non-rectangular geometry: the WKB decode is the dominant per-row cost, and rejecting on the bbox first avoids it for the rows that file/row-group pruning missed.
  • Zero-overhead when inapplicable: non-spatial queries carry no bbox domains and delegate to Iceberg unchanged.

Both the short-circuit (commit 2) and this reject pre-filter are controlled by the single geomesa.spatial.bbox-page-filter catalog property, on by default. Its value is inversely proportional to spatial partition quality: on a well-truncate-partitioned Z2/XZ2 table, file/row-group pruning already removes most non-matching rows, so the extra bbox-column read can outweigh the saved decodes (most visibly for DWITHIN, which must materialize the geometry for the exact distance test regardless); on a coarsely-partitioned table the row-level rejection catches what file pruning misses. Set geomesa.spatial.bbox-page-filter=false to fall back to partition + per-file stat pruning plus the engine's exact ST_* on every surviving row.

@cwdobbins
cwdobbins marked this pull request as draft July 21, 2026 12:20
…ng datastore SQL as limited to ST_* predicates.
…uiting logic to the plugin connector, which shortcuts the more expensive WKB decode during row-filtering, provides additional performance improvement for coarsely-pruned datasets.
… bounding box coordinates of the envelope, which boosts non-point, non-rectangular geometry performance
@cwdobbins
cwdobbins requested a review from elahrvivaz July 22, 2026 03:22
@cwdobbins
cwdobbins marked this pull request as ready for review July 22, 2026 03:22
@cwdobbins
cwdobbins force-pushed the feature/sql-refactor branch from 972e4b6 to df8976a Compare July 22, 2026 03:35

@elahrvivaz elahrvivaz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just had a question about case-sensitivity of function names, but otherwise looks good! (although I don't know the internals of trino well enough to spot any issues)

@elahrvivaz elahrvivaz changed the title GEOMESA-3582 Removes BBOX predicates from datastore SQL, adds as feature flag to the connector GEOMESA-3582 Trino - Removes BBOX predicates from datastore SQL, adds as feature flag to the connector Jul 22, 2026
@cwdobbins
cwdobbins requested a review from elahrvivaz July 22, 2026 15:48
@elahrvivaz
elahrvivaz merged commit ddda0db into locationtech:main Jul 22, 2026
70 checks passed
@elahrvivaz

Copy link
Copy Markdown
Contributor

closes #3582

@cwdobbins
cwdobbins deleted the feature/sql-refactor branch July 22, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants