Add Overture data source support (#541)

Add preliminary support for Overture as a data source.

* Recoded Claude-provided plan for Overture addition

* Added script to download Overture data as Parquet

* Updated Overture plan with some manual edits

* Implemented --overture CLI flag and got it working with just land & water at first

* Expanded Overture theme=base to landuse layer

* Added Overture theme=buildings to buildings layer

* Added Overture theme=transportation to roads layer

* Added Overture theme=places to pois layer

* Added first failing Overture POI tests

* Added feature finder script to assist with Overture test cases

* Sped up feature finder with RTree

* Added more failing Overture POI tests

* Replaced failing OGR SQL queries that were too long

* Added failing Overture POI test

* Sped up feature finder with more parallel ops

* Added numerous failing RoadsTest cases

* Added failing trunk and motorway tests

* Added test for all classes of link roads

* Switched from raw tags to assigned kind in POI zooms to prepare for Overture

* Passed all Overture POI tests by applying new rules

* Added QRank for Overture

* Added basic kind assignments for Overture roads

* Added basic zoom assignments for Overture roads

* Add comprehensive tests for Overture roads oneway/is_link extraction and line splitting

- Updated 6 existing tests to include access_restrictions and road_flags data
- Added 6 new splitting tests for partial bridge/tunnel/oneway/level application
- Tests use simple geometries (0,0)-(1,0) for easy verification
- All 12 tests failing as expected (property extraction and splitting not yet implemented)
- References real Overture feature IDs and OSM way IDs in comments

* Implement Overture roads line splitting for partial bridge/tunnel/oneway/level properties

Major Changes:
- Created com.protomaps.basemap.geometry.Linear utility class for line splitting operations
- Rewrote Roads.processOverture() to handle fractional 'between' ranges from Overture data
- Implemented collectSplitPoints() to gather all split positions from road_flags, access_restrictions, and level_rules
- Implemented extractSegmentProperties() to determine which properties apply to each split segment
- Added emitRoadFeature() to create features with custom split geometries

Results:
- 15/21 tests now passing (6 original property extraction tests now pass)
- 6 splitting tests create correct features with correct attributes and geometries
- Only remaining issue: cosmetic Norm{} wrapper in test assertions (geometries are actually correct)

Implementation handles:
- Partial bridges via road_flags with 'is_bridge' flag
- Partial tunnels via road_flags with 'is_tunnel' flag
- Partial oneway restrictions via access_restrictions with heading='backward'
- Partial level changes via level_rules
- Overlapping property ranges (e.g., bridge + oneway on same segment)
- Multiple split points creating 2-5 output features per input feature

* Fix Linear.splitAtFractions() to preserve intermediate vertices for curved roads

- Add comprehensive unit tests for line splitting with curves
- Rewrite Linear.splitAtFractions() to preserve all vertices between split points
- Add coordinate transformation from lat/lon to world coordinates before emitting
- All 9 new Linear tests pass, roads render correctly with curves preserved

* Reimplemented Linear.java using existing JTS linear referencing support

* Fixed split geometry tests to correctly handle world coordinates

* Corrected oneway=yes to get arrows showing up

* Shortened some superlong function bodies

* Migrated OSM minZoom to highwayZoomsIndex

* Separated remaining zoom-related rules for OSM roads

* Claude's version of Places.java

* Organized Places rules to separate kinds and zooms indexes

* kindRank can come from osmKindsIndex (for now!) and zoomsIndex, but not overtureKindsIndex

* Building parts look nice

* Added rendering and tests for theme=transportation/type=segment rail and water

* Added basic Overture landcover

* Replaced Python scripts with https://gist.github.com/migurski/8765492ef831457d7c21c20f4e454e1b

* Bumped version to 4.14

* Improved Landcover test coverage

* Added Overture water tests

* Read bounds from GeoParquet metadata and pass to tile archive

When using --overture with a Parquet file, the basemap now reads the
bounding box from the GeoParquet metadata and uses it to set the bounds
in the output PMTiles archive. This enables "fit bounds" functionality
on maps.protomaps.com to zoom to the precise area covered by the data.

The bounds are only applied when no --bounds argument is provided,
allowing users to still override with manual bounds if needed.

* Add test coverage for GeoParquet bounds extraction

Refactors bounds reading logic into a separate testable method and adds comprehensive test coverage including valid bounds extraction, missing file handling, and invalid file handling. Includes test GeoParquet file with Alcatraz Island buildings.

Author: migurski

CHANGELOG.md

@@ -1,3 +1,7 @@
+Tiles 4.14.0
+------
+- Add experimental Overture data source support [#541]
+
 Styles 5.7.1
 ------
 - Fix appearance of highway and major road tunnel dashed strokes [#558]

tiles/src/main/java/com/protomaps/basemap/Basemap.java

@@ -1,8 +1,10 @@
 package com.protomaps.basemap;
 
+import blue.strategic.parquet.ParquetReader;
 import com.onthegomap.planetiler.ForwardingProfile;
 import com.onthegomap.planetiler.Planetiler;
 import com.onthegomap.planetiler.config.Arguments;
+import com.onthegomap.planetiler.reader.parquet.GeoParquetMetadata;
 import com.onthegomap.planetiler.util.Downloader;
 import com.protomaps.basemap.feature.CountryCoder;
 import com.protomaps.basemap.feature.QrankDb;
@@ -24,6 +26,9 @@
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
+import org.apache.parquet.ParquetReadOptions;
+import org.apache.parquet.hadoop.ParquetFileReader;
+import org.locationtech.jts.geom.Envelope;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -46,37 +51,43 @@ public Basemap(QrankDb qrankDb, CountryCoder countryCoder, Clip clip,
       var buildings = new Buildings();
       registerHandler(buildings);
       registerSourceHandler("osm", buildings::processOsm);
+      registerSourceHandler("pm:overture", buildings::processOverture);
     }
 
     if (layer.isEmpty() || layer.equals(Landuse.LAYER_NAME)) {
       var landuse = new Landuse();
       registerHandler(landuse);
       registerSourceHandler("osm", landuse::processOsm);
+      registerSourceHandler("pm:overture", landuse::processOverture);
     }
 
     if (layer.isEmpty() || layer.equals(Landcover.LAYER_NAME)) {
       var landcover = new Landcover();
       registerHandler(landcover);
       registerSourceHandler("landcover", landcover::processLandcover);
       registerSourceHandler("ne", landcover::processNe);
+      registerSourceHandler("pm:overture", landcover::processOverture);
     }
 
     if (layer.isEmpty() || layer.equals(Places.LAYER_NAME)) {
       var place = new Places(countryCoder);
       registerHandler(place);
       registerSourceHandler("osm", place::processOsm);
+      registerSourceHandler("pm:overture", place::processOverture);
     }
 
     if (layer.isEmpty() || layer.equals(Pois.LAYER_NAME)) {
       var poi = new Pois(qrankDb);
       registerHandler(poi);
       registerSourceHandler("osm", poi::processOsm);
+      registerSourceHandler("pm:overture", poi::processOverture);
     }
 
     if (layer.isEmpty() || layer.equals(Roads.LAYER_NAME)) {
       var roads = new Roads(countryCoder);
       registerHandler(roads);
       registerSourceHandler("osm", roads::processOsm);
+      registerSourceHandler("pm:overture", roads::processOverture);
     }
 
     if (layer.isEmpty() || layer.equals(Transit.LAYER_NAME)) {
@@ -91,6 +102,7 @@ public Basemap(QrankDb qrankDb, CountryCoder countryCoder, Clip clip,
       registerSourceHandler("osm", water::processOsm);
       registerSourceHandler("osm_water", water::processPreparedOsm);
       registerSourceHandler("ne", water::processNe);
+      registerSourceHandler("pm:overture", water::processOverture);
     }
 
     if (layer.isEmpty() || layer.equals(Earth.LAYER_NAME)) {
@@ -100,6 +112,7 @@ public Basemap(QrankDb qrankDb, CountryCoder countryCoder, Clip clip,
       registerSourceHandler("osm", earth::processOsm);
       registerSourceHandler("osm_land", earth::processPreparedOsm);
       registerSourceHandler("ne", earth::processNe);
+      registerSourceHandler("pm:overture", earth::processOverture);
     }
 
     if (clip != null) {
@@ -119,7 +132,7 @@ public String description() {
 
   @Override
   public String version() {
-    return "4.13.7";
+    return "4.14.0";
   }
 
   @Override
@@ -149,6 +162,32 @@ public Map<String, String> extraArchiveMetadata() {
     return result;
   }
 
+  /**
+   * Extracts bounds from GeoParquet metadata and returns them as a formatted string.
+   *
+   * @param parquetPath Path to the GeoParquet file
+   * @return Bounds string in format "minX,minY,maxX,maxY" if successful, empty otherwise
+   */
+  static java.util.Optional<String> extractBoundsFromGeoParquet(Path parquetPath) {
+    try {
+      var inputFile = ParquetReader.makeInputFile(parquetPath.toFile());
+      try (var reader = ParquetFileReader.open(inputFile, ParquetReadOptions.builder().build())) {
+        var fileMetadata = reader.getFooter().getFileMetaData();
+        var geoparquet = GeoParquetMetadata.parse(fileMetadata);
+        Envelope bounds = geoparquet.primaryColumnMetadata().envelope();
+
+        if (bounds != null && !bounds.isNull() && bounds.getArea() > 0) {
+          String boundsStr = String.format("%f,%f,%f,%f",
+            bounds.getMinX(), bounds.getMinY(), bounds.getMaxX(), bounds.getMaxY());
+          return java.util.Optional.of(boundsStr);
+        }
+      }
+    } catch (Exception e) {
+      LOGGER.warn("Failed to read bounds from GeoParquet file: {}", e.getMessage());
+    }
+    return java.util.Optional.empty();
+  }
+
   public static void main(String[] args) throws IOException {
     // Check for help flag
     for (String arg : args) {
@@ -183,6 +222,7 @@ private static void printHelp() {
         --help, -h              Show this help message and exit
         --area=<name>           Geofabrik area name to download, or filename in data/sources/
                                 (default: monaco, e.g., us/california, washington)
+        --overture=<path>       Path to Overture Maps Parquet file (mutually exclusive with --area)
         --maxzoom=<n>           Maximum zoom level (default: 15)
         --layer=<name>          Process only a single layer (optional)
                                 Valid values: boundaries, buildings, landuse, landcover,
@@ -210,6 +250,7 @@ private static void printHelp() {
       """, basemap.name(), basemap.version(), basemap.description()));
   }
 
+  @java.lang.SuppressWarnings("java:S5738")
   static void run(Arguments args) throws IOException {
     args = args.orElse(Arguments.of("maxzoom", 15));
 
@@ -221,17 +262,50 @@ static void run(Arguments args) throws IOException {
 
     var countryCoder = CountryCoder.fromJarResource();
 
-    String area = args.getString("area", "Geofabrik area name to download, or filename in data/sources/", "monaco");
+    String area = args.getString("area", "Geofabrik area name to download, or filename in data/sources/", "");
+    String overtureFile = args.getString("overture", "Path to Overture Maps Parquet file", "");
+
+    if (!area.isEmpty() && !overtureFile.isEmpty()) {
+      LOGGER.error("Error: Cannot specify both --area and --overture");
+      System.exit(1);
+    }
+    if (area.isEmpty() && overtureFile.isEmpty()) {
+      area = "monaco"; // default
+    }
+
+    // Extract bounds from GeoParquet metadata if using Overture source
+    if (!overtureFile.isEmpty() && args.getString("bounds", "", "").isEmpty()) {
+      Path parquetPath = Path.of(overtureFile);
+      var boundsOpt = extractBoundsFromGeoParquet(parquetPath);
+      if (boundsOpt.isPresent()) {
+        String boundsStr = boundsOpt.get();
+        LOGGER.info("Setting bounds from GeoParquet metadata: {}", boundsStr);
+        args = args.orElse(Arguments.of("bounds", boundsStr));
+      }
+    }
 
     var planetiler = Planetiler.create(args)
-      .addNaturalEarthSource("ne", nePath, neUrl)
-      .addOsmSource("osm", Path.of("data", "sources", area + ".osm.pbf"), "geofabrik:" + area)
-      .addShapefileSource("osm_water", sourcesDir.resolve("water-polygons-split-3857.zip"),
-        "https://osmdata.openstreetmap.de/download/water-polygons-split-3857.zip")
-      .addShapefileSource("osm_land", sourcesDir.resolve("land-polygons-split-3857.zip"),
-        "https://osmdata.openstreetmap.de/download/land-polygons-split-3857.zip")
-      .addGeoPackageSource("landcover", sourcesDir.resolve("daylight-landcover.gpkg"),
-        "https://r2-public.protomaps.com/datasets/daylight-landcover.gpkg");
+      .addNaturalEarthSource("ne", nePath, neUrl);
+
+    if (!overtureFile.isEmpty()) {
+      // Add Overture Parquet source
+      planetiler.addParquetSource("pm:overture",
+        List.of(Path.of(overtureFile)),
+        false, // not Hive partitioned dirname, just a single file
+        fields -> fields.get("id"),
+        fields -> fields.get("type") // source layer
+      );
+    } else {
+      // Add OSM and GeoPackage sources
+      planetiler
+        .addOsmSource("osm", Path.of("data", "sources", area + ".osm.pbf"), "geofabrik:" + area)
+        .addShapefileSource("osm_water", sourcesDir.resolve("water-polygons-split-3857.zip"),
+          "https://osmdata.openstreetmap.de/download/water-polygons-split-3857.zip")
+        .addShapefileSource("osm_land", sourcesDir.resolve("land-polygons-split-3857.zip"),
+          "https://osmdata.openstreetmap.de/download/land-polygons-split-3857.zip")
+        .addGeoPackageSource("landcover", sourcesDir.resolve("daylight-landcover.gpkg"),
+          "https://r2-public.protomaps.com/datasets/daylight-landcover.gpkg");
+    }
 
     Path pgfEncodingZip = sourcesDir.resolve("pgf-encoding.zip");
     Path qrankCsv = sourcesDir.resolve("qrank.csv.gz");
@@ -281,8 +355,20 @@ static void run(Arguments args) throws IOException {
 
     fontRegistry.loadFontBundle("NotoSansDevanagari-Regular", "1", "Devanagari");
 
+    String outputName;
+    if (!overtureFile.isEmpty()) {
+      String filename = Path.of(overtureFile).getFileName().toString();
+      if (filename.endsWith(".parquet")) {
+        outputName = filename.substring(0, filename.length() - ".parquet".length());
+      } else {
+        outputName = filename;
+      }
+    } else {
+      outputName = area;
+    }
+
     planetiler.setProfile(new Basemap(qrankDb, countryCoder, clip, layer))
-      .setOutput(Path.of(area + ".pmtiles"))
+      .setOutput(Path.of(outputName + ".pmtiles"))
       .run();
   }
 }

tiles/src/main/java/com/protomaps/basemap/feature/Matcher.java

@@ -155,7 +155,7 @@ public static Expression without(String... arguments) {
    * Creates an {@link Expression} that matches when a numeric tag value is within a specified range.
    *
    * <p>
-   * The lower bound is inclusive. The upper bound, if provided, is exclusive.
+   * The lower bound is inclusive. The upper bound is exclusive.
    * </p>
    *
    * <p>

tiles/src/main/java/com/protomaps/basemap/geometry/Linear.java

@@ -0,0 +1,113 @@
+package com.protomaps.basemap.geometry;
+
+import java.util.*;
+import org.locationtech.jts.geom.Geometry;
+import org.locationtech.jts.geom.LineString;
+import org.locationtech.jts.linearref.LengthIndexedLine;
+
+/**
+ * Utility class for linear geometry operations, particularly splitting LineStrings at fractional positions.
+ */
+public class Linear {
+
+  private Linear() {
+    // Utility class, prevent instantiation
+  }
+
+  /**
+   * Represents a segment of a line with fractional start/end positions.
+   */
+  public static class Segment {
+    public final double start; // Fractional position 0.0-1.0
+    public final double end; // Fractional position 0.0-1.0
+
+    public Segment(double start, double end) {
+      this.start = start;
+      this.end = end;
+    }
+  }
+
+  /**
+   * Split a LineString at fractional positions and return list of split LineStrings. Preserves all intermediate
+   * vertices between split points to maintain curve geometry.
+   *
+   * @param line        The LineString to split
+   * @param splitPoints List of fractional positions (0.0-1.0) where to split
+   * @return List of LineString segments
+   */
+  public static List<LineString> splitAtFractions(LineString line, List<Double> splitPoints) {
+    if (splitPoints.isEmpty()) {
+      return List.of(line);
+    }
+
+    // Sort split points and remove duplicates, ensure 0.0 and 1.0 are included
+    Set<Double> pointSet = new TreeSet<>();
+    pointSet.add(0.0);
+    pointSet.addAll(splitPoints);
+    pointSet.add(1.0);
+
+    List<Double> points = new ArrayList<>(pointSet);
+    List<LineString> segments = new ArrayList<>();
+
+    // Use JTS LengthIndexedLine for efficient extraction
+    double totalLength = line.getLength();
+    LengthIndexedLine indexedLine = new LengthIndexedLine(line);
+
+    // For each pair of split points, create a segment preserving intermediate vertices
+    for (int i = 0; i < points.size() - 1; i++) {
+      double startFrac = points.get(i);
+      double endFrac = points.get(i + 1);
+
+      double startLength = startFrac * totalLength;
+      double endLength = endFrac * totalLength;
+
+      Geometry segment = indexedLine.extractLine(startLength, endLength);
+      if (segment instanceof LineString ls && ls.getNumPoints() >= 2) {
+        segments.add(ls);
+      }
+    }
+
+    return segments;
+  }
+
+  /**
+   * Create list of Segments representing the split ranges between all split points.
+   *
+   * @param splitPoints List of fractional positions (0.0-1.0) where to split
+   * @return List of Segment objects with start/end fractions
+   */
+  public static List<Segment> createSegments(List<Double> splitPoints) {
+    if (splitPoints.isEmpty()) {
+      return List.of(new Segment(0.0, 1.0));
+    }
+
+    // Sort split points and remove duplicates, ensure 0.0 and 1.0 are included
+    Set<Double> pointSet = new TreeSet<>();
+    pointSet.add(0.0);
+    pointSet.addAll(splitPoints);
+    pointSet.add(1.0);
+
+    List<Double> points = new ArrayList<>(pointSet);
+    List<Segment> segments = new ArrayList<>();
+
+    for (int i = 0; i < points.size() - 1; i++) {
+      segments.add(new Segment(points.get(i), points.get(i + 1)));
+    }
+
+    return segments;
+  }
+
+  /**
+   * Check if a segment defined by [segStart, segEnd] overlaps with a range [rangeStart, rangeEnd].
+   *
+   * @param segStart   Start of segment (0.0-1.0)
+   * @param segEnd     End of segment (0.0-1.0)
+   * @param rangeStart Start of range (0.0-1.0)
+   * @param rangeEnd   End of range (0.0-1.0)
+   * @return true if the segment overlaps with the range
+   */
+  public static boolean overlaps(double segStart, double segEnd, double rangeStart, double rangeEnd) {
+    // Segments overlap if they share any fractional position
+    return segEnd > rangeStart && segStart < rangeEnd;
+  }
+}

tiles/src/main/java/com/protomaps/basemap/layers/Buildings.java

@@ -137,6 +137,29 @@ public void processOsm(SourceFeature sf, FeatureCollector features) {
     }
   }
 
+  public void processOverture(SourceFeature sf, FeatureCollector features) {
+    // Filter by type field - Overture theme
+    if (!"buildings".equals(sf.getString("theme"))) {
+      return;
+    }
+
+    // Ignore non-building types
+    if (!"building".equals(sf.getString("type")) && !"building_part".equals(sf.getString("type"))) {
+      return;
+    }
+
+    features.polygon(this.name())
+      //.setId(FeatureId.create(sf))
+      // Core Tilezen schema properties
+      .setAttr("kind", "building")
+      // Core OSM tags for different kinds of places
+      //.setAttrWithMinzoom("layer", Parse.parseIntOrNull(sf.getString("layer")), 13)
+      // NOTE: Height is quantized by zoom in a post-process step
+      //.setAttr(HEIGHT_KEY, height.height())
+      .setAttr("sort_rank", 400)
+      .setZoomRange(11, 15);
+  }
+
   @Override
   public List<VectorTile.Feature> postProcess(int zoom, List<VectorTile.Feature> items) throws GeometryException {
     if (zoom == 15) {