Spatial phase I: Layer (#370)

* grammatical accommodations

* Add Spatial geom type for choropleth/geographic visualization

Registers GeomType::Spatial across the parser, AST, and builder.
The spatial geom requires a `geometry` aesthetic and supports
fill, stroke, opacity, linewidth, and linetype.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add spatial feature: geometry auto-detection, geozero conversion, and SpatialRenderer

- Feature-gated `spatial` (default-on) with geozero + hex dependencies
- Auto-detect GEOMETRY columns via DESCRIBE for `DRAW spatial` layers
- SpatialRenderer converts WKB hex / GeoJSON strings to GeoJSON Features
- Vega-Lite geoshape mark with proper encoding channel handling

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add spatial tests, fix geometry encoding, remove auto-detection

- Skip geometry aesthetic in encoding builder (structural, not visual)
- Remove SpatialRenderer::modify_encoding stub (default suffices)
- Remove geometry column auto-detection (revisit after Arrow migration)
- Add end-to-end tests: GeoJSON features, WKB hex, mixed layers
- Add execute test for explicit geometry mapping

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Allow arbitrary SQL setup statements (INSTALL, LOAD, SET, etc.)

Relax the grammar's other_sql_statement rule to accept any non-delimiter
tokens, so statements like INSTALL/LOAD/SET/ATTACH parse without error.
Execute these setup statements before the main query in the pipeline.
Flip DDL detection in DuckDB and SQLite readers to a returns_rows
whitelist, so unknown statement types are handled gracefully.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(reader): replace ColumnBuilder with direct Arrow export via query_arrow

DuckDB's query_arrow API exports results directly as Arrow RecordBatches,
eliminating the manual row-by-row type mapping. Extension types like
GEOMETRY now flow through as native Binary columns instead of hitting a
lossy string fallback. Decimal128 columns are normalized to Float64 at
the boundary for downstream compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(execute): use DDL instead of Arrow round-trip for temp table materialisation

Replaces the execute_sql() → register() two-step with direct
CREATE TEMP TABLE AS DDL. This keeps data inside the database engine
and preserves native types (e.g. DuckDB GEOMETRY) that were lost
during the Arrow materialisation round-trip.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(spatial): conditional ST_AsWKB stat transform and native GEOMETRY tests

Only apply ST_AsWKB when the geometry column is Binary (native DuckDB
GEOMETRY via Arrow). String columns (GeoJSON, WKB hex) pass through
to the writer unchanged.

Adds tests using INSTALL spatial; LOAD spatial with ST_GeomFromText
to verify the full native GEOMETRY pipeline end-to-end.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert "refactor(execute): use DDL instead of Arrow round-trip for temp table materialisation"

This reverts commit 2694724.

* refactor(spatial): remove string geometry paths, require native GEOMETRY

Remove GeoJSON/WKB hex string handling from the writer and stat
transform. Spatial data should come from native GEOMETRY columns
(via ST_Read, ST_GeomFromText, etc.), not string columns.

Drops hex crate dependency from the spatial feature.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(spatial): use dialect for geometry-to-WKB conversion

Add sql_geometry_to_wkb() to SqlDialect trait with ST_AsBinary
default (OGC standard). DuckDB overrides with ST_AsWKB. The spatial
stat transform uses the dialect instead of hardcoding.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(data): add ggsql:world built-in dataset

Natural Earth 110m country boundaries as geoparquet. Columns: name,
iso_a3, continent, subregion, income_group, population, gdp, geom.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(spatial): auto-load spatial extension via dialect

The spatial stat transform now calls dialect.sql_spatial_setup() before
emitting ST_AsWKB, so users no longer need manual LOAD spatial statements.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* cargo fmt

* feat(spatial): auto-detect geometry column by name and type

When a geom declares a geometry aesthetic and the user hasn't mapped it
explicitly, scan the schema for a column with a conventional geometry
name (geom, geometry, wkb_geometry, the_geom, shape) and binary type.
Requires exactly one match to avoid ambiguity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(spatial): load spatial extension before registering ggsql:world

DuckDB reads geoparquet geometry columns as BLOB when the spatial
extension is not loaded, making ST_AsWKB fail later. Pre-load spatial
when the world dataset is referenced so the column is read as GEOMETRY.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* override defaults chosen by claude

* add docs

* candles in pentagram shape for clippy

* beg for clippy's forgiveness

* try installing the spatial module during the test

* Apply suggestions from code review

Co-authored-by: Thomas Lin Pedersen <thomasp85@gmail.com>

* apply suggestions from code review

* cargo fmt

* Detect geometry columns via Reader::geometry_columns()

Geometry columns lose their native type during Arrow materialisation
(DuckDB GEOMETRY becomes plain Binary). Add Reader::geometry_columns()
that queries the backend's type system (DESCRIBE for DuckDB) to find
actual geometry columns, with a name+type heuristic fallback.

The detection is implemented as GeomTrait::detect_aesthetics(), which
spatial overrides. This runs after global mapping merge so user-declared
geometry mappings take precedence. Multiple native geometry columns are
treated as ambiguous (user must declare explicitly).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert "Detect geometry columns via Reader::geometry_columns()"

This reverts commit 0f42b58.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Thomas Lin Pedersen <thomasp85@gmail.com>

Author: 3 people

Cargo.lock

@@ -57,6 +57,9 @@ thiserror = "1.0"
 # Color interpolation
 palette = { version = "0.7", default-features = false, features = ["std", "approx"] }
 
+# Spatial
+geozero = { version = "0.14", default-features = false }
+
 # Utilities
 regex = "1.10"
 chrono = "0.4"

Cargo.toml

@@ -142,6 +142,7 @@
       <item>arrow</item>
       <item>rule</item>
       <item>range</item>
+      <item>spatial</item>
     </list>
 
     <!-- Aesthetics -->
@@ -188,6 +189,7 @@
       <!-- Specialty aesthetics -->
       <item>slope</item>
       <item>intercept</item>
+      <item>geometry</item>
       <!-- Facet aesthetics -->
       <item>panel</item>
       <item>row</item>

doc/ggsql.xml

@@ -33,6 +33,7 @@ There are many different layers to choose from when visualising your data. Some
 - [`boxplot`](layer/type/boxplot.qmd) displays continuous variables as 5-number summaries.
 - [`range`](layer/type/range.qmd) a line segment between two values along an axis, with optional hinges at the endpoints.
 - [`smooth`](layer/type/smooth.qmd) a trendline that follows the data shape.
+- [`spatial`](layer/type/spatial.qmd) simple features from geometry.
 
 ### Position adjustments
 - [`stack`](layer/position/stack.qmd) places objects with a shared baseline on top of each other.

doc/syntax/index.qmd

@@ -0,0 +1,86 @@
+---
+title: "Spatial"
+---
+
+> Layers are declared with the [`DRAW` clause](../../clause/draw.qmd). Read the documentation for this clause for a thorough description of how to use it.
+
+The spatial layer is used to render geographic geometries consisting of polygons, lines and points used to make maps like choropleths.
+It differs from other layers in that it uses a special [simple features](https://en.wikipedia.org/wiki/Simple_Features) geometry column that defines the shapes.
+
+## Aesthetics
+The following aesthetics are recognised by the spatial layer.
+
+### Required
+* `geometry`: a column of simple features.
+
+Note that the `geometry` column is required, but an attempt is made to detect such a column automatically.
+In practise, this mapping does not often need to be declared.
+
+### Optional
+* `stroke` The colour of the lines.
+* `fill` The colour of the inner area.
+* `colour` Shorthand for setting `stroke` and `fill` simultaneously.
+* `opacity` The opacity of colours.
+* `linewidth` The width of the lines.
+* `linetype` The dash pattern of the line.
+
+## Settings
+The spatial layer has no additional settings.
+
+## Data transformation
+The spatial layer transforms the `geometry` column to [Well-Known Binary](https://libgeos.org/specifications/wkb/).
+
+## Orientation
+The spatial layer has no orientations.
+
+## Examples
+
+Note that depending on your reader, you may need to activate modules for spatial analysis.
+
+```{ggsql}
+-- For example, for DuckDB, one could use:
+INSTALL spatial;
+LOAD spatial;
+```
+
+A basic map of the world using built-in data.
+Note that the geometry column is automatically detected.
+
+```{ggsql}
+VISUALISE FROM ggsql:world
+DRAW spatial
+```
+
+If the geometry column isn't automatically detected —for example because it has a non-standard name— you may need to declare the mapping explicitly.
+
+```{ggsql}
+SELECT geom AS foo FROM ggsql:world
+VISUALISE
+DRAW spatial 
+  MAPPING foo AS geometry
+```
+
+Filtering on other columns.
+
+```{ggsql}
+VISUALISE FROM ggsql:world
+DRAW spatial 
+  FILTER continent == 'Asia'
+```
+
+Filtering based on spatial operations.
+
+```{ggsql}
+VISUALISE FROM ggsql:world
+DRAW spatial 
+  FILTER ST_Intersects(geom, ST_MakeEnvelope(-20.0, -35.0, 55.0, 38.0))
+```
+
+Make a choropleth map by mapping a variable to a fill aesthetic.
+
+```{ggsql}
+VISUALISE FROM ggsql:world
+DRAW spatial 
+  MAPPING population AS fill
+  SETTING opacity => 1
+```

doc/syntax/layer/type/spatial.qmd

@@ -269,7 +269,7 @@
         {
           "comment": "Specialty and computed aesthetics",
           "name": "support.type.aesthetic.ggsql",
-          "match": "\\b(weight|coef|intercept|offset|density|count|intensity)\\b"
+          "match": "\\b(weight|coef|intercept|offset|density|count|intensity|geometry)\\b"
         },
         {
           "comment": "Facet aesthetics",
@@ -320,7 +320,7 @@
         {
           "comment": "Geom types from grammar.js",
           "name": "support.type.geom.ggsql",
-          "match": "\\b(point|line|path|bar|col|area|tile|polygon|ribbon|histogram|density|smooth|boxplot|violin|text|label|segment|arrow|rule|range)\\b"
+          "match": "\\b(point|line|path|bar|col|area|tile|polygon|ribbon|histogram|density|smooth|boxplot|violin|text|label|segment|arrow|rule|range|spatial)\\b"
         },
         { "include": "#common-clause-patterns" }
       ]
@@ -334,7 +334,7 @@
       "patterns": [
         {
           "name": "support.type.geom.ggsql",
-          "match": "\\b(point|line|path|bar|col|area|tile|polygon|ribbon|histogram|density|smooth|boxplot|violin|text|label|segment|arrow|rule|range)\\b"
+          "match": "\\b(point|line|path|bar|col|area|tile|polygon|ribbon|histogram|density|smooth|boxplot|violin|text|label|segment|arrow|rule|range|spatial)\\b"
         },
         { "include": "#common-clause-patterns" }
       ]

ggsql-vscode/syntaxes/ggsql.tmLanguage.json

@@ -32,6 +32,9 @@ libloading = { workspace = true, optional = true }
 parquet = { workspace = true, optional = true }
 bytes = { workspace = true }
 
+# Spatial
+geozero = { workspace = true, optional = true, features = ["with-wkb", "with-geojson"] }
+
 # Serialization
 serde.workspace = true
 serde_json.workspace = true
@@ -53,11 +56,12 @@ tempfile = "3.8"
 ureq = "3"
 
 [features]
-default = ["duckdb", "sqlite", "vegalite", "parquet", "builtin-data", "odbc"]
+default = ["duckdb", "sqlite", "vegalite", "parquet", "builtin-data", "odbc", "spatial"]
 duckdb = ["dep:duckdb"]
 parquet = ["dep:parquet"]
 sqlite = ["dep:rusqlite"]
 odbc = ["dep:toml_edit", "dep:libloading"]
+spatial = ["dep:geozero"]
 vegalite = []
 builtin-data = []
 all-readers = ["duckdb", "sqlite", "odbc"]