feat: reverse geolocation per polygon strategy

[davidgamez]

Summary:

Closes #1301, #1289, #965
Significant changes in this PR:

• Introduced per_polygon strategy
  • This strategy, described here, looks for clustering points within a polygon to reduce the amount of DB calls. This works for cases where the points are close together and the geolocation query has "high" admin levels in their respective country. In cases where the admin levels are below the municipality, the algorithm performs as a "per_point" strategy.
• Fixed duplicate key value violates unique constraint. This was happening because the per-point strategy was attempting to create the same location multiple times within the same database session.
• Added a guard against multiple executions of the function within the same day.
• Stop using stop caching for GBFS, as vehicle positions change constantly.
• Fixed duplicate admin levels. This issue had two causes(Investigate location extraction failures caused by duplicate admin levels #965):
  • The difference between the two representations (plane 2D and geospatial 3D) was causing duplicate intersections between polygons on large areas and edges. To fix this, the code now is casting the geometry as Geography.
  • OSM doesn’t enforce uniqueness at each level; different contributors can model administrative boundaries differently. To fix this, a tie-break mechanism was implemented.

Minor fixes

• Fixed refresh materialized view function name in cloud task
• Missing GCP function region and project ID parameters

Out of scope:

• Address timeout for large feeds. This should be part of Large feeds fails to extract location information #1324
• Create a configuration service. This is part of Add global and per feed configuration(integrate feeds download) #1325

Expected behavior:

• ✅ A reduced running time for feeds with a high number of stops per polygon.
• ✅ Reverse geolocation function runs a maximum of once per feed day and feed ID.

Performance results

The performance results were collected for a few feeds without using the geometry cache.

Feed	Stops	Locations	Strategy	Time	Memory(peak)	Notes
GTFS	---	---	---	---	---	---
mdb-2014	308611	225	Per-Point	33130s(9.2h)	129.57 MB
			Per-Polygon	25870s(7.2h)	133.11 MB
---	---	---	---	---	---	---
mdb-1139	663	324	Per-Point	327s(5.45m)	19.48 MB
			Per-Polygon	100(1.6m)	129.57 MB
---	---	---	---	---	---	---
mdb-2825	10985	1541	Per-Point	1052.04(17.5)	25.55 MB
			Per-Polygon	348.9s(5.8m)	34.48 MB
GBFS	---	---	---	---	---	---
gbfs-flamingo_auckland	1539	1	Per-Point	37.2s	1.34 MB
			Per-Polygon	37.9	1.24 MB	All points reverse geolocation queries returns adminlevel 4
gbfs-hellocycling	11777	241	Per-Point	337.76s(5.6m)	14.16 MB
			Per-Polygon	31.59	9.81 MB

Testing tips:
Local testing

• Use reverse_geolocation_process_verifier.py for local testing

From our AI friend:

This pull request introduces new geospatial utilities and refactors location handling logic to improve reverse geolocation and location grouping features. It adds helper functions for working with geopolygons, enhances enum conversion error reporting, and expands test coverage for location utilities. Several dependencies and configuration files are updated to support these changes.

Geospatial utilities and location logic enhancements

• Added new helper functions in locations.py for converting geometry types (to_shapely), selecting geopolygons by admin level (select_highest_level_polygon, select_lowest_level_polygon), extracting country codes from polygons, and querying geopolygons that cover a point with BigQuery-compatible semantics.
• Introduced new reverse geolocation utilities in location_group_utils.py, including functions for extracting location aggregates, creating/updating stop groups, and managing location group entities. These functions leverage the new geopolygon helpers for more robust location assignment and grouping. [1] [2]

Test coverage improvements

• Expanded test_locations.py with comprehensive tests for the new geospatial helper functions, including edge cases for admin level selection and country code extraction from geopolygons.

Dependency and configuration updates

• Added shapely to requirements.txt to support geometry conversions and spatial operations.
• Updated .coveragerc to omit coverage for files in */shared/common/*, reflecting the new shared helpers.

Error handling and logging improvements

• Improved error reporting in enum conversion by logging failed conversions in transform.py.
• Updated logger initialization to import google.cloud.logging only when needed, improving startup reliability.

Minor fixes

• Fixed a typo in the Cloud Function URL construction in gcp_utils.py (tasks_executor instead of tasks-executor).
• Added missing import for logging in transform.py.
• Cleaned up unused imports in logger.py.
• Added new enum values and documentation for reverse geocoding strategies in locations.py.

Please make sure these boxes are checked before submitting your pull request - thanks!

• Run the unit tests with ./scripts/api-tests.sh to make sure you didn't break anything
• Add or update any needed documentation to the repo
• Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
• Linked all relevant issues
• Include screenshot(s) showing how this pull request works and fixes the issue(s)

[davidgamez]

Moving this import inside the function, as the unit tests can fail when running locally, and the console is not authenticated with Google services

[qcdyx]

Excellent work!

[cka-y]

LGTM 🏆

[cka-y]

[question] what does this have to do w big query?

[davidgamez]

The fact that we have to cast(Geopolygon.geometry, Geography(srid=4326)) due to the original values from BigQuery tables being Geography instead of geometry

[cka-y]

great addition 🤩