GDAL/OGR + QGIS read-only support

[bdon]

Motivation

Read PMTiles, particularly vector archives, from the cloud or local storage using QGIS.

Currently MBTiles works by "drag and drop" into QGIS, and may work through adding a Vector Tile layer type. We want the same while also supporting vsicurl for accessing tiles via HTTP Ranges.

The priority should be on vector tiles, since raster tiles are well covered by existing formats. We only care about read support for now, because writing vector tilesets is best left to specialized tools like tippecanoe/planetiler.

pmtiles.hpp is the existing C++ implementation of pmtiles v3 utility functions that is copied into the tippecanoe source tree. This is intentionally written to be unopinionated about I/O and compression, as to re-use those already in GDAL.

Links

• GDAL frmts/mbtilesdataset.cpp
• See GDAL Wiki on Reading COG with GDAL
• Current PMTiles specification See More detailed Version 3 Specification #172 for some more details

Tagging @rouault (via @cholmes)

[rouault]

OGR PMTiles driver proposed per OSGeo/gdal#7938

[bdon]

the GDAL PR addresses opening PMTiles to retrieve OGR features, however given the above requirements maybe what we want is a QGIS native driver analogous to qgsmbtiles.cpp for PMTiles. This would enable proper dynamic zoom and tile display in QGIS.

However, the QGIS native driver for MBTiles opens a SQLite database in-process, and can't benefit from the existing abstraction over /vsicurl/ etc that GDAL provides. The infrastructure for this already exists at qgsvectortileconnection.cpp

[four43]

I believe my issue (as commented qgis/QGIS#57041) was related to QGIS just picking the highest zoom level and trying to render everything at once...

[bdon]

@four43 that is correct, for multi-resolution tiled data that chooses a zoom level with the viewport, the driver needs to be implemented in QGIS itself and not in GDAL (MBTiles for example)

[jo-chemla]

Hi there, just wanted to know if anyone has made progress on implementing the qgis driver to read remote pmtiles - also asked in the closed discussion thread here. Would love to load this VIDA osm + google + microsoft open-buildings dataset from 2024-09 without downloading the whole 250GB :)

Edit It's actually not loading the whole file, just seems that the initial load takes quite some time when copy-pasting the url, maybe to load tile indexes or the dataset/file hierarchcal structure.

[wipfli]

I would do the following:

put the following in ./config.json:

{
    "options": {
        "paths": {
            "root": "",
            "pmtiles": "data"
        }
    },
    "data": {
        "buildings": {
            "pmtiles": "https://data.source.coop/vida/google-microsoft-osm-open-buildings/pmtiles/goog_msft_osm.pmtiles"
        }
    }
}

Then create an empty data directory with mkdir data.

Then run tileserver-gl with:

docker run --rm -it -v $(pwd):/data -p 8080:8080 maptiler/tileserver-gl:latest

You now have a normal zxy endpoint on your machine for the building tiles at

http://localhost:8080/data/buildings/{z}/{x}/{y}.pbf

[wipfli]

Or maybe even simpler just look at the pmtiles file on pmtiles.io: https://pmtiles.io/#url=https%3A%2F%2Fdata.source.coop%2Fvida%2Fgoogle-microsoft-osm-open-buildings%2Fpmtiles%2Fgoog_msft_osm.pmtiles&map=12.93/47.36621/8.53346

[jo-chemla]

Thanks for getting back, I was indeed exploring the data with protomaps viewer pmtiles.io. But I wanted to find a way to import into qgis from the http url, and noticed the lag when pasting the url to the import vector layer dialog, hence the comment.

[bdon]

You can also use the pmtiles serve with a remote archive and read ZXY from localhost in QGIS:

https://docs.protomaps.com/pmtiles/cli#serve

[jo-chemla]

Thanks that pmtiles serve CLI looks like a good idea. I've tried multiple variations and finally stumbled on the below working one - putting here in case it's useful to anyone, could not find an example with HTTP, only s3 or file:/// within the docs and repo.

Adding the below mvt vector tile layer does indeed loads way faster than the pmtiles url as a vector layer file/service, so thank a lot!

pmtiles serve . --bucket=https://data.source.coop/vida/google-microsoft-osm-open-buildings/pmtiles --port 8081
# http://127.0.0.1:8081/goog_msft_osm/{z}/{x}/{y}.mvt

Quick note: I think the bucket CLI flag does not remove trailing slash, plus such an example for http could be useful in the docs. Thanks again for the great format and utility!

[bdon]

Welcome to add an example for HTTPS sources to the docs!