Merge pull request #461 from SimmerV/feature-add

feat: Coverage: per-pixel canopy + building DSM, multi-origin merge

Author: SimmerV

Dockerfile.buildings

@@ -0,0 +1,21 @@
+# One-shot bake image. Run via `docker compose --profile bake run --rm building-bake`.
+# rasterio's manylinux wheels link against libexpat which python:slim variants
+# don't bundle — install it explicitly.
+FROM python:3.14-slim
+
+RUN apt-get update \
+    && apt-get install -y --no-install-recommends libexpat1 \
+    && rm -rf /var/lib/apt/lists/*
+
+# Cap GDAL's per-process cache. Default 5% of RAM × N workers blows past
+# Docker Desktop's 16 GB ceiling and OOMs mid-bake.
+ENV GDAL_CACHEMAX=64
+
+WORKDIR /app
+
+COPY scripts/requirements-buildings.txt scripts/
+RUN pip install --no-cache-dir -r scripts/requirements-buildings.txt
+
+COPY scripts/building_tiles.py scripts/
+
+CMD ["python", "scripts/building_tiles.py"]

Dockerfile.canopy

@@ -0,0 +1,21 @@
+# One-shot bake image. Run via `docker compose --profile bake run --rm canopy-bake`.
+# rasterio's manylinux wheels link against libexpat which python:slim variants
+# don't bundle — install it explicitly.
+FROM python:3.14-slim
+
+RUN apt-get update \
+    && apt-get install -y --no-install-recommends libexpat1 \
+    && rm -rf /var/lib/apt/lists/*
+
+# Cap GDAL's per-process cache. Default 5% of RAM × N workers blows past
+# Docker Desktop's 16 GB ceiling and OOMs mid-bake.
+ENV GDAL_CACHEMAX=64
+
+WORKDIR /app
+
+COPY scripts/requirements-canopy.txt scripts/
+RUN pip install --no-cache-dir -r scripts/requirements-canopy.txt
+
+COPY scripts/canopy_tiles.py scripts/
+
+CMD ["python", "scripts/canopy_tiles.py"]

Dockerfile.landcover

@@ -1,8 +1,12 @@
 # One-shot bake image. Run via `docker compose --profile bake run --rm landcover-bake`.
-# Kept separate from the prod image to keep that image lean. rasterio ships GDAL
-# wheels on PyPI, so no system GDAL install is needed.
+# rasterio's manylinux wheels link against libexpat which python:slim variants
+# don't bundle — install it explicitly.
 FROM python:3.14-slim
 
+RUN apt-get update \
+    && apt-get install -y --no-install-recommends libexpat1 \
+    && rm -rf /var/lib/apt/lists/*
+
 WORKDIR /app
 
 COPY scripts/requirements-landcover.txt scripts/

RF-MODEL.md

@@ -103,6 +103,38 @@ The full per-class P.452 (hₐ, dₖ) and P.833 (γ, A) parameters are visible i
 
 Outside the United States (or anywhere the NLCD bake doesn't cover), the model falls back to **Mixed Forest** as a conservative default for every pixel. Future work tracks adding ESA WorldCover as a global fallback.
 
+## Per-pixel building heights — JRC GHS-BUILT-H 100 m (optional)
+
+NLCD's developed-intensity classes (21/22/23/24) ship with class-nominal heights of 4 / 9 / 15 / 25 m from P.452 Table 4. These are continental averages — wrong by ±15 m for any specific neighbourhood. They also can't put real diffractors (apartment blocks, industrial silos, isolated office towers) into the ITM propagation profile.
+
+When building tiles are baked from **JRC GHS-BUILT-H R2023A ANBH** (Average Net Building Height, 100 m global, CC BY 4.0), two integration sites in the model use the measurements:
+
+1. **DSM into the ITM profile.** Mid-path samples become `terrain + measured_building_height` so ITM's diffraction algorithm sees rooftops as actual obstacles. Endpoints (TX and RX) stay bare-earth — your `txHeightM` / `rxHeightM` are AGL above local ground, not on top of a presumed building. Operators with rooftop-mounted antennas should add the building's height to their antenna AGL field manually.
+2. **Measured `h_a` in P.452 endpoint clutter.** For developed-class pixels (21/22/23/24) at TX and RX, the measured ANBH replaces `cls.nominalHeightM` in the height-gain formula. A 2 m handheld in a 100 m cell that ANBH says is 4 m gets the same ~19 dB endpoint loss; in a cell that's actually 14 m, the cell-specific value drives the calculation. Non-developed classes (vegetation, water, etc.) keep their class-nominal heights so trees don't get erased when a forest pixel happens to read ANBH = 0.
+
+**100 m averaging caveats.** ANBH averages over the *built portion* of each 100 m cell, so a neighbourhood of 30% 9 m houses + 5% 25 m apartment block reads as ~11 m — the apartment block is captured as elevated DSM but its sharp edges get smeared. ITM's above-rooftop diffraction (the dominant path at typical link distances) is well-served by this; ground-level street-canyon waveguide effects need ray-tracing, not raster, regardless of resolution.
+
+The "Building heights" toggle in the Coverage and Scan settings panels lets operators turn the refinement off (compare measured-vs-nominal, or use bare-earth ITM for a baseline). Default is on.
+
+**Operator runbook for the building bake:** [scripts/README-buildings.md](scripts/README-buildings.md).
+
+## Per-pixel canopy heights — ETH 10 m (optional)
+
+NLCD tells the model *what kind* of canopy is at a pixel; class-nominal heights from P.452 Table 4 (15 m deciduous, 20 m evergreen, 15 m mixed, 10 m woody-wetland, 2 m emergent-wetland) tell it *how tall* that canopy is. Real canopy varies — a 50 m redwood next to a 5 m madrone in the same NLCD "Evergreen Forest" pixel both got coded as 20 m without measurement.
+
+When canopy tiles are baked from **ETH Global Canopy Height 2020** (Lang et al. 2023, 10 m global, CC BY 4.0), the P.833 MED loop uses the measured height per sample instead of the class-nominal:
+
+- The path-integral gate `zPath < terrain + canopyHeight` evaluates against measured canopy at every profile sample.
+- Where the measured value is 0 (clearings, fire scars, recent logging), MED contributes nothing for that sample — the path is above any canopy.
+- Where the bake doesn't cover a pixel (out-of-bbox / ocean adjacency), the loop falls back to the class-nominal value silently.
+- Endpoint clutter (P.452) still uses class-nominal heights — Tier 3 only refines the path-integrated MED. Endpoint heights are a Tier-2 (per-pixel building heights) refinement.
+
+If the bake includes the optional standard-deviation file (`scripts/canopy_tiles.py --include-stddev`), pixels with σ ≥ height are blended 50/50 with the class-nominal so a single noisy ETH pixel can't dominate the integral.
+
+The "Canopy heights" toggle in the Coverage and Scan settings panels lets operators turn the refinement off (e.g. to compare measured-vs-nominal predictions, or to validate the older behavior). Default is on.
+
+**Operator runbook for the canopy bake:** [scripts/README-canopy.md](scripts/README-canopy.md).
+
 ## Setup — running the bake
 
 NLCD data is not bundled with MeshInfo (it's ~2 GB). Operators run a one-shot bake script after deploying:
@@ -119,6 +151,22 @@ CONUS at full resolution takes 15–90 min depending on CPU. Tiles are bind-moun
 
 Once tiles exist, the API mounts them at `/tiles/landcover/{z}/{x}/{y}.png` and the frontend fetches them on every coverage compute. The "Land cover: USGS NLCD" status chip in the Coverage panel shows whether tiles are healthy or the model is using the fallback class.
 
+For the optional canopy-height refinement, run the canopy bake separately:
+
+```bash
+docker compose --profile bake run --rm canopy-bake     # CONUS default; --scope global also supported
+```
+
+CONUS canopy bake is ~30 GB of source COGs + ~1–2 GB of output tiles. Time: 1–4 hours depending on the ETH file server and CPU. Without this bake, the model uses class-nominal canopy heights — the coverage tool still works.
+
+For the optional building-height refinement, run the building bake:
+
+```bash
+docker compose --profile bake run --rm building-bake   # global default; --scope conus also supported
+```
+
+Global building bake is ~1.8 GB of source GeoTIFF + ~1–2 GB of output tiles. Time: 1–3 hours. Without this bake, the model uses class-nominal heights for developed classes and bare-earth ITM — the coverage tool still works.
+
 ## The aggression scaler
 
 Inside the Coverage and Scan panels, the **Clutter** control is a 3-stop slider:
@@ -142,7 +190,7 @@ The default (1.0×) reflects ITU-R P.452 / P.833 published values. Don't sit at
 These are scope limits that affect prediction accuracy in specific scenarios:
 
 - **Indoor receivers (ITU-R P.2109)** — RX inside a building gets +10–20 dB additional loss not modeled. Outdoor-to-outdoor only.
-- **Per-tree / per-building height** — uses class-nominal canopy heights (15 m deciduous, 20 m evergreen, etc.), not measured heights from LIDAR. A single tall redwood next to your antenna isn't picked up.
+- **Per-building height** — when the building bake is in place, JRC GHS-BUILT-H 100 m measurements replace class-nominal heights for developed classes and feed buildings into the ITM DSM. The 100 m averaging captures first-order diffraction physics but smooths individual rooftops; per-building precision (a single tall office tower across the street from your antenna) needs vector pipelines beyond this raster model.
 - **Seasonal foliage** — deciduous classes assume leaf-on (summer) at 0.5 dB/m. Out-of-leaf is ~0.15 dB/m; not modeled. Future work.
 - **Frequencies other than 915 MHz** — the P.833 specific-attenuation values are calibrated at 915 MHz. Adding 868 MHz (EU) or 433 MHz would need a per-band lookup.
 - **Polarization-dependent vegetation loss** — uses unpolarized averages (the slight per-polarization difference is in the noise floor here).
@@ -156,6 +204,8 @@ These are scope limits that affect prediction accuracy in specific scenarios:
 - ITU-R Rec. **P.2108** — newer clutter-loss recommendation (future migration target)
 - ITU-R Rec. **P.2109** — building entry loss (indoor RX, deferred)
 - USGS NLCD: <https://www.mrlc.gov/data>
+- ETH Global Canopy Height 2020 (Lang et al. 2023): <https://langnico.github.io/globalcanopyheight/>
+- JRC GHS-BUILT-H R2023A: <https://human-settlement.emergency.copernicus.eu/ghs_buH2023.php>
 - ITM v1.4: <https://github.com/NTIA/itm>
 
 ## How to validate

api/api.py

@@ -449,6 +449,46 @@ async def server_config(request: Request) -> JSONResponse:
                     tile_dir,
                 )
 
+        # Canopy-height tiles for the P.833-9 vegetation loss loop. Pre-baked by
+        # scripts/canopy_tiles.py; missing tiles 404 and the frontend falls back
+        # to class-nominal heights. See RF-MODEL.md.
+        canopy_cfg = self.config.get("canopy", {}) or {}
+        if canopy_cfg.get("enabled", False):
+            tile_dir = Path(canopy_cfg.get("tile_dir", "output/canopy"))
+            if tile_dir.is_dir():
+                app.mount(
+                    "/tiles/canopy",
+                    TileFiles(directory=str(tile_dir)),
+                    name="canopy_tiles",
+                )
+                logger.info("Mounted canopy-height tiles at /tiles/canopy from %s", tile_dir.resolve())
+            else:
+                logger.info(
+                    "Canopy-height tiles enabled but %s does not exist — frontend will use class-nominal heights. "
+                    "Run scripts/canopy_tiles.py to populate.",
+                    tile_dir,
+                )
+
+        # Building-height tiles for the P.452 endpoint formula and ITM DSM.
+        # Pre-baked by scripts/building_tiles.py; missing tiles 404 and the
+        # frontend falls back to class-nominal. See RF-MODEL.md.
+        buildings_cfg = self.config.get("buildings", {}) or {}
+        if buildings_cfg.get("enabled", False):
+            tile_dir = Path(buildings_cfg.get("tile_dir", "output/buildings"))
+            if tile_dir.is_dir():
+                app.mount(
+                    "/tiles/buildings",
+                    TileFiles(directory=str(tile_dir)),
+                    name="building_tiles",
+                )
+                logger.info("Mounted building-height tiles at /tiles/buildings from %s", tile_dir.resolve())
+            else:
+                logger.info(
+                    "Building-height tiles enabled but %s does not exist — frontend will use class-nominal heights. "
+                    "Run scripts/building_tiles.py to populate.",
+                    tile_dir,
+                )
+
         allow_origins = os.getenv("ALLOW_ORIGINS", "").split(",")
         logger.info("Allowed origins: %s (%d)", allow_origins, len(allow_origins))
 

config.py

@@ -138,6 +138,22 @@
         "tile_dir": "output/landcover",
         "source": "USGS NLCD 2024",
     },
+    # Per-pixel measured canopy heights for the P.833 vegetation loss loop.
+    # Tiles are pre-baked via scripts/canopy_tiles.py; absent tile_dir → frontend
+    # falls back to class-nominal heights.
+    "canopy": {
+        "enabled": True,
+        "tile_dir": "output/canopy",
+        "source": "ETH Global Canopy Height 2020",
+    },
+    # Per-pixel measured building heights for P.452 endpoint clutter and the
+    # ITM DSM. Tiles are pre-baked via scripts/building_tiles.py; absent
+    # tile_dir → frontend falls back to class-nominal heights.
+    "buildings": {
+        "enabled": True,
+        "tile_dir": "output/buildings",
+        "source": "JRC GHS-BUILT-H R2023A",
+    },
     "debug": False,
 }
 

config.toml.sample

@@ -306,16 +306,56 @@ style = "mapbox/dark-v11"
 # mounts it at /tiles/landcover. Without a bake, the frontend falls back to a
 # default class everywhere — coverage still works, just less accurate.
 #
-# To populate tiles, run scripts/landcover_tiles.py once. See
-# scripts/README-landcover.md for the operator guide and RF-MODEL.md for
-# the RF model.
+# To populate tiles, run the one-time bake (idempotent, safe to re-run):
+#   docker compose --profile bake run --rm landcover-bake
+# See scripts/README-landcover.md for sub-region bakes, disk/time estimates,
+# and the host-Python alternative. RF model details in RF-MODEL.md.
 # ─────────────────────────────────────────────────────────────────────────────
 
 [landcover]
 enabled = true
 tile_dir = "output/landcover"   # bind-mounted into the meshinfo container
 source = "USGS NLCD 2024"       # surfaced in the map UI attribution chip
 
+# ─────────────────────────────────────────────────────────────────────────────
+# Canopy heights — Optional
+# Per-pixel measured canopy heights for ITU-R P.833-9 path-integrated
+# vegetation loss. Replaces class-nominal heights (15-20 m by class) with
+# measurements from ETH Global Canopy Height 2020 (Lang et al. 2023, 10 m).
+# When tile_dir exists and enabled = true, the API mounts it at /tiles/canopy.
+# Without a bake, the frontend falls back to class-nominal heights.
+#
+# To populate tiles, run the one-time bake (idempotent, safe to re-run):
+#   docker compose --profile bake run --rm canopy-bake
+# See scripts/README-canopy.md for sub-region bakes, disk/time estimates,
+# and the host-Python alternative.
+# ─────────────────────────────────────────────────────────────────────────────
+
+[canopy]
+enabled = true
+tile_dir = "output/canopy"      # bind-mounted into the meshinfo container
+source = "ETH Global Canopy Height 2020"
+
+# ─────────────────────────────────────────────────────────────────────────────
+# Building heights — Optional
+# Per-pixel measured building heights for ITU-R P.452-17 endpoint clutter and
+# the ITM DSM along the propagation path. Replaces class-nominal heights
+# (4/9/15/25 m for the four NLCD developed-intensity classes) with
+# measurements from JRC GHS-BUILT-H ANBH (100 m global, CC-BY-4.0).
+# When tile_dir exists and enabled = true, the API mounts it at /tiles/buildings.
+# Without a bake, the frontend falls back to class-nominal heights.
+#
+# To populate tiles, run the one-time bake (idempotent, safe to re-run):
+#   docker compose --profile bake run --rm building-bake
+# See scripts/README-buildings.md for sub-region bakes, disk/time estimates,
+# and the host-Python alternative.
+# ─────────────────────────────────────────────────────────────────────────────
+
+[buildings]
+enabled = true
+tile_dir = "output/buildings"   # bind-mounted into the meshinfo container
+source = "JRC GHS-BUILT-H R2023A"
+
 # ─────────────────────────────────────────────────────────────────────────────
 # UI Customization — Optional
 # External tools and node detail links shown in the web UI.

docker-compose-dev.yml

@@ -87,5 +87,23 @@ services:
     volumes:
       - ./output:/app/output
 
+  # `docker compose -f docker-compose-dev.yml --profile bake run --rm canopy-bake` — see scripts/README-canopy.md.
+  canopy-bake:
+    profiles: ["bake"]
+    build:
+      context: .
+      dockerfile: Dockerfile.canopy
+    volumes:
+      - ./output:/app/output
+
+  # `docker compose -f docker-compose-dev.yml --profile bake run --rm building-bake` — see scripts/README-buildings.md.
+  building-bake:
+    profiles: ["bake"]
+    build:
+      context: .
+      dockerfile: Dockerfile.buildings
+    volumes:
+      - ./output:/app/output
+
 volumes:
   meshinfo_pgdata:

docker-compose.yml

@@ -85,5 +85,23 @@ services:
     volumes:
       - ./output:/app/output
 
+  # `docker compose --profile bake run --rm canopy-bake` — see scripts/README-canopy.md.
+  canopy-bake:
+    profiles: ["bake"]
+    build:
+      context: .
+      dockerfile: Dockerfile.canopy
+    volumes:
+      - ./output:/app/output
+
+  # `docker compose --profile bake run --rm building-bake` — see scripts/README-buildings.md.
+  building-bake:
+    profiles: ["bake"]
+    build:
+      context: .
+      dockerfile: Dockerfile.buildings
+    volumes:
+      - ./output:/app/output
+
 volumes:
   meshinfo_pgdata: