Add QuackOSM support for high-performance OSM data download#1286
Conversation
- Add quackosm_gdf_from_place() for downloading OSM data by place name - Add quackosm_gdf_from_bbox() for downloading OSM data by bounding box - Add quackosm_gdf_from_geometry() for downloading OSM data by geometry - Add quackosm_gdf_from_pbf() for loading local PBF files - Add quackosm_to_parquet() for exporting to GeoParquet format - Add helper function _convert_tags_to_quackosm_filter() - Create notebook example (107_quackosm.ipynb) demonstrating usage Closes #1144
There was a problem hiding this comment.
Pull request overview
This PR adds support for QuackOSM, a high-performance library for downloading OpenStreetMap data using DuckDB, providing significant speed improvements over traditional methods. The implementation creates a parallel API to the existing osmnx-based functions with support for multiple data sources.
Changes:
- Added five new QuackOSM functions for downloading OSM data from different sources (place name, bounding box, geometry, PBF file) and exporting to Parquet
- Added helper function to convert tags to QuackOSM filter format
- Created comprehensive Jupyter notebook demonstrating usage with multiple examples
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 13 comments.
| File | Description |
|---|---|
| leafmap/osm.py | Adds 6 new functions for QuackOSM integration: place-based, bbox-based, geometry-based, PBF file loading, Parquet export, and tag conversion helper |
| docs/notebooks/107_quackosm.ipynb | Comprehensive tutorial notebook demonstrating QuackOSM functionality with filtering examples, multiple layers, and usage patterns |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Try convert_osm_extract_to_geodataframe first, fall back to geometry-based approach | ||
| try: | ||
| gdf = qosm.convert_osm_extract_to_geodataframe( | ||
| query, tags_filter=tags_filter, **kwargs |
There was a problem hiding this comment.
Using a bare except Exception: is too broad and could hide unexpected errors. Consider catching specific exceptions that QuackOSM might raise when the extract is not found (e.g., a specific error type from QuackOSM). This would make debugging easier and avoid catching unintended errors like programming mistakes or system errors.
| """Convert osmnx-style tags dict to QuackOSM filter format. | ||
|
|
||
| QuackOSM uses a similar but slightly different format for tag filtering. | ||
| This function converts the osmnx-style dict to QuackOSM format. | ||
|
|
||
| Args: | ||
| tags (dict): Dict with OSM tags as keys and True, string, or list of strings as values. | ||
|
|
||
| Returns: | ||
| dict: Tags filter in QuackOSM format. | ||
| """ | ||
| if tags is None: | ||
| return None | ||
|
|
||
| # QuackOSM expects the same format as osmnx: | ||
| # {key: True} - match any value for the key | ||
| # {key: value} - match specific value | ||
| # {key: [value1, value2]} - match any of the values | ||
| quackosm_filter = {} | ||
|
|
||
| for key, value in tags.items(): | ||
| if value is True: | ||
| # Match any value for this key | ||
| quackosm_filter[key] = True | ||
| elif isinstance(value, str): | ||
| # Match specific value | ||
| quackosm_filter[key] = value | ||
| elif isinstance(value, list): | ||
| # Match any of these values | ||
| quackosm_filter[key] = value | ||
| else: | ||
| quackosm_filter[key] = value | ||
|
|
||
| return quackosm_filter | ||
|
|
||
|
|
There was a problem hiding this comment.
The helper function _convert_tags_to_quackosm_filter is redundant. According to the docstring comment at lines 752-755, QuackOSM expects the same format as osmnx. The function essentially copies the tags dictionary without any transformation - lines 759-769 just copy each value as-is. This helper function can be removed, and the tags can be passed directly to QuackOSM functions.
| """Convert osmnx-style tags dict to QuackOSM filter format. | |
| QuackOSM uses a similar but slightly different format for tag filtering. | |
| This function converts the osmnx-style dict to QuackOSM format. | |
| Args: | |
| tags (dict): Dict with OSM tags as keys and True, string, or list of strings as values. | |
| Returns: | |
| dict: Tags filter in QuackOSM format. | |
| """ | |
| if tags is None: | |
| return None | |
| # QuackOSM expects the same format as osmnx: | |
| # {key: True} - match any value for the key | |
| # {key: value} - match specific value | |
| # {key: [value1, value2]} - match any of the values | |
| quackosm_filter = {} | |
| for key, value in tags.items(): | |
| if value is True: | |
| # Match any value for this key | |
| quackosm_filter[key] = True | |
| elif isinstance(value, str): | |
| # Match specific value | |
| quackosm_filter[key] = value | |
| elif isinstance(value, list): | |
| # Match any of these values | |
| quackosm_filter[key] = value | |
| else: | |
| quackosm_filter[key] = value | |
| return quackosm_filter | |
| """Pass through osmnx-style tags dict for use with QuackOSM. | |
| QuackOSM expects the same tag filter format as osmnx: | |
| {key: True} - match any value for the key | |
| {key: value} - match specific value | |
| {key: [value1, value2]} - match any of the values | |
| This helper is kept for backwards compatibility but no longer | |
| performs any transformation. | |
| Args: | |
| tags (dict): Dict with OSM tags as keys and True, string, or list of strings as values. | |
| Returns: | |
| dict: Tags filter in QuackOSM format (identical to input). | |
| """ | |
| if tags is None: | |
| return None | |
| # QuackOSM uses the same format as osmnx, so we can return tags directly. | |
| return tags |
| check_package("quackosm", "https://github.com/kraina-ai/quackosm") | ||
|
|
||
| try: | ||
| import quackosm as qosm | ||
| from quackosm.osm_extracts import OsmExtractSource | ||
| except ImportError: | ||
| raise ImportError( | ||
| "quackosm package is required. Please install it using " | ||
| "'pip install quackosm' or 'conda install -c conda-forge quackosm'" | ||
| ) |
There was a problem hiding this comment.
The check_package function call is followed by a try-except block that attempts to import the same package. This is redundant because check_package already raises an ImportError if the package is not available. You can remove the try-except block and rely on the check_package function to handle missing dependencies.
| >>> gdf = osm.quackosm_gdf_from_geometry(geometry, tags={"natural": "water"}) | ||
| """ | ||
| check_package("quackosm", "https://github.com/kraina-ai/quackosm") | ||
|
|
||
| try: | ||
| import quackosm as qosm | ||
| from shapely import wkt | ||
| from shapely.geometry import shape | ||
| except ImportError: | ||
| raise ImportError( | ||
| "quackosm package is required. Please install it using " |
There was a problem hiding this comment.
The check_package function call is followed by a try-except block that attempts to import the same package. This is redundant because check_package already raises an ImportError if the package is not available. You can remove the try-except block and rely on the check_package function to handle missing dependencies.
| >>> gdf = osm.quackosm_gdf_from_pbf("monaco.osm.pbf", tags={"building": True}) | ||
| """ | ||
| check_package("quackosm", "https://github.com/kraina-ai/quackosm") | ||
|
|
||
| try: | ||
| import quackosm as qosm | ||
| except ImportError: | ||
| raise ImportError( | ||
| "quackosm package is required. Please install it using " |
There was a problem hiding this comment.
The check_package function call is followed by a try-except block that attempts to import the same package. This is redundant because check_package already raises an ImportError if the package is not available. You can remove the try-except block and rely on the check_package function to handle missing dependencies.
| >>> gdf = osm.quackosm_gdf_from_bbox(bbox, tags={"highway": True}) | ||
| """ | ||
| check_package("quackosm", "https://github.com/kraina-ai/quackosm") | ||
|
|
||
| try: | ||
| import quackosm as qosm | ||
| from shapely.geometry import box | ||
| except ImportError: | ||
| raise ImportError( | ||
| "quackosm package is required. Please install it using " |
There was a problem hiding this comment.
The check_package function call is followed by a try-except block that attempts to import the same package. This is redundant because check_package already raises an ImportError if the package is not available. You can remove the try-except block and rely on the check_package function to handle missing dependencies.
|
|
||
| def quackosm_gdf_from_bbox( | ||
| bbox: Union[Tuple[float, float, float, float], List[float]], | ||
| tags: Optional[Dict] = None, | ||
| verbosity_mode: Optional[str] = "transient", | ||
| **kwargs, | ||
| ) -> gpd.GeoDataFrame: | ||
| """Download OSM data for a bounding box using QuackOSM. | ||
|
|
||
| QuackOSM is a high-performance library for reading OpenStreetMap data using DuckDB. | ||
| It automatically downloads the required PBF files and converts them to GeoDataFrame. | ||
|
|
||
| Args: |
There was a problem hiding this comment.
The bbox parameter format differs from the existing osm_gdf_from_bbox function in this module, which uses individual parameters (north, south, east, west) at line 280-281. For API consistency within the module, consider whether the bbox should follow the same pattern. However, using a tuple (west, south, east, north) is also a common convention in geospatial libraries, so this may be an intentional design choice. The docstring clearly documents the format, so this is acceptable if the difference is intentional.
| get a single tag-value combination, or a list of strings to get multiple values | ||
| for the given tag. For example, tags = {'building': True} would return all | ||
| building footprints in the area. tags = {'amenity': True, 'highway': 'bus_stop'} | ||
| would return all amenities and highway=bus_stop features. Defaults to None (all features). |
There was a problem hiding this comment.
The tags parameter is optional in the QuackOSM functions (defaults to None), whereas in the existing osmnx-based functions like osm_gdf_from_place (line 82), tags is required. This is a difference in API design. While this may be intentional to provide more flexibility (returning all features when tags is None), it should be clearly communicated that this is a deliberate difference from the osmnx functions. The current documentation does mention "Defaults to None (all features)" which is good.
| would return all amenities and highway=bus_stop features. Defaults to None (all features). | |
| would return all amenities and highway=bus_stop features. Defaults to None (all features). | |
| Note: Unlike the osmnx-based functions in this module, the tags parameter is optional | |
| here; passing None will return all available OSM features. This difference is intentional. |
| # Set verbosity mode | ||
| kwargs["verbosity_mode"] = verbosity_mode |
There was a problem hiding this comment.
Setting kwargs["verbosity_mode"] will overwrite any user-provided verbosity_mode value in kwargs. If a user passes verbosity_mode both as a named parameter and in kwargs, this will silently override the kwargs value. Consider checking if "verbosity_mode" already exists in kwargs before setting it, or document that the named parameter takes precedence.
| # Set verbosity mode | |
| kwargs["verbosity_mode"] = verbosity_mode | |
| # Set verbosity mode only if not already provided in kwargs | |
| if "verbosity_mode" not in kwargs: | |
| kwargs["verbosity_mode"] = verbosity_mode |
|
|
||
| return gdf | ||
|
|
||
|
|
There was a problem hiding this comment.
The geometry parameter is missing a type hint. Based on the docstring, it accepts multiple types: Shapely geometry, WKT string, GeoJSON dict, or GeoDataFrame. Consider adding a type hint like geometry: Union[Any, str, Dict, gpd.GeoDataFrame] or geometry: Any for flexibility.
|
🚀 Deployed on https://69781a5b5ecd3aa6681455d8--opengeos.netlify.app |
* Add QuackOSM support for high-performance OSM data download - Add quackosm_gdf_from_place() for downloading OSM data by place name - Add quackosm_gdf_from_bbox() for downloading OSM data by bounding box - Add quackosm_gdf_from_geometry() for downloading OSM data by geometry - Add quackosm_gdf_from_pbf() for loading local PBF files - Add quackosm_to_parquet() for exporting to GeoParquet format - Add helper function _convert_tags_to_quackosm_filter() - Create notebook example (107_quackosm.ipynb) demonstrating usage Closes #1144 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Closes #1144