Skip to content

Add IBTrACS USA wind buffers pipeline#14

Closed
t-downing wants to merge 5 commits into
mainfrom
usa-wind-buffers
Closed

Add IBTrACS USA wind buffers pipeline#14
t-downing wants to merge 5 commits into
mainfrom
usa-wind-buffers

Conversation

@t-downing
Copy link
Copy Markdown
Contributor

Summary

  • Adds storms.ibtracs_wind_buffers table (one row per storm per wind speed threshold: 34/50/64 kt)
  • New pipeline src/pipelines/wind_buffers.py reads USA quadrant wind radii from PROD ibtracs_tracks_geo, calls ocha-lens buffer calculation functions, and writes to DEV
  • Registered as wind-buffers subcommand in run_pipeline.py with optional --basin and --start-year filters

Depends on the usa-wind-buffers PR in ocha-lens being merged first (for calculate_wind_buffers_gdf and expand_quad_col).

Usage

# Full historical run
python run_pipeline.py wind-buffers --mode dev

# Subset (e.g. for testing)
python run_pipeline.py wind-buffers --mode dev --basin NA --start-year 2020

Test plan

  • Tested NA basin since 2020 against DEV DB (135 storms, 405 buffer polygons written successfully)
  • Run SQL schema against target DB before first run
  • Run full historical backfill once ocha-lens PR is merged and version bumped in requirements.txt

Reads USA quadrant wind radii from PROD ibtracs_tracks_geo, computes
per-storm wind buffer polygons using ocha-lens, and writes to DEV
storms.ibtracs_wind_buffers. Supports --basin and --start-year filters
for partial runs.
- New storms.nhc_wind_buffers table: one row per (atcf_id, issued_time, wind_speed_kt)
- nhc-wind-buffers subcommand reads NHC forecast tracks from PROD and writes buffers to DEV
- Refactored both pipelines to write in batches of 50 as calculation proceeds, keeping the DB connection alive and making runs crash-safe
- Fixed expand_quad_col in ocha-lens to handle JSON [] format (NHC) alongside PostgreSQL {} format (IBTrACS)
@t-downing t-downing changed the base branch from main to add-wsp-data May 4, 2026 18:03
Base automatically changed from add-wsp-data to main May 11, 2026 18:37
@t-downing
Copy link
Copy Markdown
Contributor Author

Superseded by #15.

All commits on this branch (72687de, 7a7e9e7, 5b5bdcf, README updates) were merged into wsp-adm0-exp via merge commit e9239f9, and the standalone src/pipelines/wind_buffers.py was later folded into src/pipelines/ibtracs.py and src/pipelines/nhc.py (commit 49dd298). The IBTrACS + NHC wind-buffer pipelines now ship as part of #15 alongside the full exposure cascade and Databricks bundle.

Closing — please review there.

@t-downing t-downing closed this May 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant