Fix rscontext issues with current national datasets#1260
Conversation
LANDFIRE changed filename abbreviations between LF 2020/2022 and LF 2023: - HDst → HDist (Historic Disturbance) - FDst → FDist (Fuel Disturbance) - SCla → SClass (Succession Classes) Changes: - Add LANDFIRE_INPUTS config dict in rs_context.py for easier version updates - Update layer_descriptions.json in rscontext and blm_context
Successful TNM API responses don't include an 'error' key, but _get_urls()
assumed it always exists. This caused a KeyError when downloading NTD data.
Changed items['error'] to items.get('error') for safe dictionary access.
The function was producing empty shapefiles for ecoregions and geology when source data CRS differs from output EPSG. Root cause: SetSpatialFilter() requires geometry in the input layer's CRS, but the clip/intersect geometry was in output EPSG (WGS84). National ecoregions and geology datasets use Albers Equal Area Conic projection, so the spatial filter found zero matching features. For the clip=True path, the same geometry was also used for Intersection() after being transformed, causing CRS mismatch there too. Fix: Create separate geometry objects: - clip_geom_filter/intersect_geom_filter: transformed to input CRS for filter - clip_geom: kept in output EPSG for Intersection() operation
Re-running rscontext failed with "UNIQUE constraint failed: gpkg_contents.identifier" when the output directory already existed. Root cause: create_spatial_view() uses DROP VIEW IF EXISTS but doesn't clean up the corresponding metadata in gpkg_contents and gpkg_geometry_columns tables. Fix: Delete existing metadata rows before inserting new ones.
Comma-separated reach codes with spaces (e.g., "43100, 40300") were parsed without stripping whitespace, resulting in SQL filters like: FCode = ' 40300' which fail to match database values. Fix: Strip whitespace when parsing flowline, flowarea, and waterbody reach codes using list comprehension with .strip().
|
Thanks @ecopony We will review these changes and get them integrated. How are you using RSContext? Is this a professional or hobby project? Over what geographic extent are you interested in RSContext? How have you been developing it, with CodeSpaces or on your local machine? Thanks again |
Hi @philipbaileynar — thanks! I used BRAT to write my master's thesis in GIS back then, and I'm just now finally getting around to contributing that documentation! To answer your questions: this is a hobby/personal interest project. I work professionally in software development (ed tech), but my GIS master's got me hooked on riverscapes analysis. Geographically, I'm most interested in the Pacific Northwest and the Southwest. I've been developing locally on my machine. I'd be interested in contributing more to the project if you'd like to point me in a direction. Otherwise I could start looking at bugs in the issues repository. |
3 participants
Summary
Fixes several issues encountered when running rscontext with current national datasets (LANDFIRE LF 2023, USGS ecoregions/geology via TNM API).
These were discovered while setting up rscontext to run locally with freshly downloaded national datasets so I could document the process.
I'm opening a separate docs PR. This code PR just addresses issues that blocked local runs.
Changes
1. LANDFIRE filename updates for LF 2023
LANDFIRE changed filename abbreviations between LF 2020/2022 and LF 2023:
HDst→HDist(Historic Disturbance)FDst→FDist(Fuel Disturbance)SCla→SClass(Succession Classes)Also adds
LANDFIRE_INPUTSconfig dict to centralize these mappings for easier future updates.2. TNM API response handling
_get_urls()assumed the API response always contains an'error'key, but successful responses omit it. Changeditems['error']toitems.get('error').3. CRS mismatch in vector clipping
copy_feature_class_shapefile()was producing empty shapefiles for ecoregions and geology.SetSpatialFilter()requires the filter geometry in the input layer's CRS, but the clip geometry was in output EPSG (WGS84). National datasets use Albers Equal Area Conic, so the spatial filter found zero matches.4. GeoPackage view recreation on re-runs
Re-running rscontext failed with
UNIQUE constraint failed: gpkg_contents.identifier.DROP VIEW IF EXISTSdoesn't remove metadata from GeoPackage system tables.5. Reach code parsing whitespace
Comma-separated reach codes with spaces (e.g.,
"43100, 40300") weren't stripped, causing SQL filters to fail silently.Testing
Verified with rscontext run on HUC 17060304: