Skip to content

feat[pmtiles]: support multiple shapes per routes#1365

Merged
davidgamez merged 9 commits into
mainfrom
feat/support_multiple_route_shapes
Sep 23, 2025
Merged

feat[pmtiles]: support multiple shapes per routes#1365
davidgamez merged 9 commits into
mainfrom
feat/support_multiple_route_shapes

Conversation

@davidgamez

@davidgamez davidgamez commented Sep 19, 2025

Copy link
Copy Markdown
Member

Summary:

This PR adds support for multiple shapes per route. No additional information has been added to the routes.json regarding shapes.

Expected behavior:

All shapes are represented in the pmtiles.

Left image, one shape per route. Right image, multiple shapes per route.
image (4)

From our AI friend

This pull request introduces improvements to the GTFS CSV caching and GeoJSON conversion logic, focusing on safer data extraction, more robust handling of missing or malformed data, and enhanced support for routes with multiple shapes and trips. The most important changes are grouped below.

Data Extraction and Safety Improvements

  • Added the get_safe_int helper for safely extracting integer values, and updated get_safe_value and get_safe_float to use explicit type hints for better type safety. [1] [2] [3] [4]
  • Refactored code in gtfs_stops_to_geojson.py and main.py to consistently use get_safe_value and get_safe_float for extracting values from CSV rows, reducing the risk of exceptions and ensuring missing or malformed data is handled gracefully. [1] [2] [3] [4] [5] [6]

Route and Shape Handling Enhancements

  • Redesigned the shape and trip mapping in CsvCache: get_shape_from_route now returns all shape IDs and their associated trips for a route, and added tracking of trips without shapes. This enables more accurate representation of GTFS data where multiple shapes or trips may exist per route. [1] [2] [3]
  • Updated the route GeoJSON generation logic in main.py to support multiple shapes and trips per route. Each route feature now includes shape and trip information, and falls back to using stops when shape data is missing. [1] [2] [3]

Type Hints and Structure

  • Introduced new TypedDicts (ShapeTrips, RouteCoordinates) and added type hints throughout the codebase for improved readability and static analysis. [1] [2] [3]

Dependency Updates

  • Added new development dependencies (psutil, gcp_storage_emulator) to requirements_dev.txt for enhanced testing and local development.
    Testing tips:

Provide tips, procedures and sample files on how to test the feature.
Testers are invited to follow the tips AND to try anything they deem relevant outside the bounds of the testing tips.

Please make sure these boxes are checked before submitting your pull request - thanks!

  • Run the unit tests with ./scripts/api-tests.sh to make sure you didn't break anything
  • Add or update any needed documentation to the repo
  • Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
  • Linked all relevant issues
  • Include screenshot(s) showing how this pull request works and fixes the issue(s)

@davidgamez
davidgamez requested a review from Copilot September 22, 2025 17:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the PMTiles builder to support multiple route shapes by allowing routes to have multiple shape_ids with associated trip_ids. The main purpose is to handle cases where a single route can have different shapes for different trips.

  • Transforms route-to-shape relationship from single shape_id to multiple shapes with trip associations
  • Adds support for trips without shapes by falling back to stop coordinates
  • Improves error handling and data validation using safe value getters

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
functions-python/pmtiles_builder/src/main.py Core refactoring to support multiple route shapes with new RouteCoordinates type and get_route_coordinates method
functions-python/pmtiles_builder/src/csv_cache.py Updates CsvCache to track multiple shapes per route and trips without shapes
functions-python/pmtiles_builder/src/gtfs_stops_to_geojson.py Replaces direct row access with safe value getters for better data validation
functions-python/helpers/transform.py Adds get_safe_int function and improves type hints for existing safe value functions
functions-python/helpers/tests/test_transform.py Adds unit tests for the new get_safe_int function
functions-python/pmtiles_builder/src/scripts/pmtiles_builder_verifier.py New verification script for testing PMTiles builder functionality
functions-python/pmtiles_builder/requirements_dev.txt Adds development dependencies for testing and emulation

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread functions-python/pmtiles_builder/src/csv_cache.py Outdated
Comment thread functions-python/pmtiles_builder/src/csv_cache.py Outdated
Comment thread functions-python/pmtiles_builder/src/csv_cache.py Outdated
Comment thread functions-python/pmtiles_builder/src/main.py Outdated
@davidgamez davidgamez changed the title feat: support multiple route shapes feat[pmtiles]: support multiple shapes per routes Sep 22, 2025
@davidgamez
davidgamez marked this pull request as ready for review September 22, 2025 18:32
@davidgamez davidgamez self-assigned this Sep 22, 2025
@davidgamez
davidgamez requested review from cka-y and jcpitre September 22, 2025 21:16
Comment thread functions-python/pmtiles_builder/src/main.py
Comment thread functions-python/pmtiles_builder/src/csv_cache.py Outdated
Comment thread functions-python/pmtiles_builder/src/csv_cache.py Outdated
Comment thread functions-python/pmtiles_builder/src/csv_cache.py Outdated

@cka-y cka-y left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment thread functions-python/pmtiles_builder/src/main.py Outdated
Comment thread functions-python/pmtiles_builder/src/main.py Outdated

@jcpitre jcpitre left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!

@davidgamez
davidgamez merged commit a552071 into main Sep 23, 2025
3 checks passed
@davidgamez
davidgamez deleted the feat/support_multiple_route_shapes branch September 23, 2025 17:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants