Cover write_geotiff_gpu compression modes (zstd default + jpeg) (#1646)

.claude/sweep-test-coverage-state.csv

@@ -1,3 +1,3 @@
 module,last_inspected,issue,severity_max,categories_found,notes
-geotiff,2026-05-11,,HIGH,2;3;4,"Pass 6 (2026-05-11): added test_overview_resampling_min_max_median_2026_05_11.py covering Cat 4 HIGH parameter-coverage gap on overview_resampling=min/max/median. CPU end-to-end paths were already covered by test_cog_overview_nodata_1613::test_cpu_cog_overview_aggregations_ignore_sentinel; the GPU end-to-end paths and the direct CPU+GPU block-reducer branches had no targeted tests, so a regression on those code paths would ship undetected. 26 tests, all passing on GPU host: block-reducer unit tests (finite + partial-NaN), end-to-end COG writes for both to_geotiff and write_geotiff_gpu, CPU/GPU parity for to_geotiff(gpu=True), CPU nodata-sentinel regression check, and ValueError error-path tests for unknown method names on both backends. Pass 5 (2026-05-11): added test_degenerate_shapes_backends_2026_05_11.py covering Cat 3 HIGH geometric gaps (1x1 / 1xN / Nx1 reads on dask+numpy, GPU, dask+cupy backends; 1x1 / 1xN / Nx1 writes through write_geotiff_gpu) and Cat 2 MEDIUM NaN/Inf gaps (all-NaN read on GPU + dask+cupy, Inf / -Inf reads on all non-eager backends, NaN sentinel mask on dask read path including sentinel block split across chunk boundary). 23 tests, all passing on GPU host. Prior passes still hold: pass 4 (r4) closed read_geotiff_gpu/dask name= + max_pixels= kwargs (Cat 4), pass 3 (r3) closed read_vrt GPU/dask+GPU backend dispatch (Cat 1) and dtype/name kwargs (Cat 4)."
+geotiff,2026-05-11,,HIGH,2;3;4,"Pass 7 (2026-05-11): added test_gpu_writer_compression_modes_2026_05_11.py closing Cat 4 HIGH gap on write_geotiff_gpu compression= modes. The writer documents zstd (default, fastest GPU), deflate, jpeg, and none, but only deflate + none had round-trip tests; the default zstd and the jpeg (nvJPEG/Pillow) paths shipped without targeted coverage. 11 new tests, all passing on GPU host: zstd round-trip + default-codec pinning, jpeg round-trip on 3-band RGB uint8 + 1-band greyscale, TIFF compression-tag header check across none/deflate/zstd/jpeg, plain deflate + none round-trips outside the COG/sentinel paths, and a cross-codec lossless parity check (zstd/deflate/none agree pixel-exact). nvJPEG path was exercised live, not just the Pillow fallback. Pass 6 (2026-05-11): added test_overview_resampling_min_max_median_2026_05_11.py covering Cat 4 HIGH parameter-coverage gap on overview_resampling=min/max/median. CPU end-to-end paths were already covered by test_cog_overview_nodata_1613::test_cpu_cog_overview_aggregations_ignore_sentinel; the GPU end-to-end paths and the direct CPU+GPU block-reducer branches had no targeted tests, so a regression on those code paths would ship undetected. 26 tests, all passing on GPU host: block-reducer unit tests (finite + partial-NaN), end-to-end COG writes for both to_geotiff and write_geotiff_gpu, CPU/GPU parity for to_geotiff(gpu=True), CPU nodata-sentinel regression check, and ValueError error-path tests for unknown method names on both backends. Pass 5 (2026-05-11): added test_degenerate_shapes_backends_2026_05_11.py covering Cat 3 HIGH geometric gaps (1x1 / 1xN / Nx1 reads on dask+numpy, GPU, dask+cupy backends; 1x1 / 1xN / Nx1 writes through write_geotiff_gpu) and Cat 2 MEDIUM NaN/Inf gaps (all-NaN read on GPU + dask+cupy, Inf / -Inf reads on all non-eager backends, NaN sentinel mask on dask read path including sentinel block split across chunk boundary). 23 tests, all passing on GPU host. Prior passes still hold: pass 4 (r4) closed read_geotiff_gpu/dask name= + max_pixels= kwargs (Cat 4), pass 3 (r3) closed read_vrt GPU/dask+GPU backend dispatch (Cat 1) and dtype/name kwargs (Cat 4)."
 reproject,2026-05-10,,HIGH,1;4;5,"Added 39 tests: LiteCRS direct coverage, itrf_transform behaviour/roundtrip/array, itrf_frames, geoid_height numerical correctness + raster happy-path, vertical helpers (ellipsoidal<->orthometric/depth), reproject() lat/lon and latitude/longitude dim propagation. Note: _merge_arrays_cupy is imported but unused (no cupy merge dispatch in merge()); flagged as feature gap not test gap."

xrspatial/geotiff/tests/test_gpu_writer_compression_modes_2026_05_11.py

@@ -0,0 +1,308 @@
+"""Coverage for ``write_geotiff_gpu`` compression modes.
+
+The GPU writer documents four ``compression=`` modes: ``'zstd'``
+(default, "fastest on GPU"), ``'deflate'``, ``'jpeg'`` (nvJPEG with
+Pillow fallback), and ``'none'``. The existing test suite exercises
+only ``'none'`` and ``'deflate'`` with direct round-trip assertions.
+
+* ``'zstd'`` is the default and is hit implicitly by tests that omit
+  ``compression=``, but no test asserts pixel fidelity for the zstd
+  path. A regression in the nvCOMP zstd encoder (or in the writer's
+  zstd codec-tag wiring) would not surface against the implicit
+  callers because they only assert metadata-level properties.
+
+* ``'jpeg'`` routes to ``_nvjpeg_batch_encode`` with a CPU Pillow
+  fallback. Neither code path is exercised through ``write_geotiff_gpu``
+  anywhere else in the suite. ``to_geotiff(compression='jpeg')``
+  rejects the CPU path with the JPEGTables interop error, so the only
+  way to reach the GPU JPEG encoder via the public API is through
+  ``write_geotiff_gpu``.
+
+This module closes the Cat 4 HIGH parameter-coverage gap by pinning a
+round-trip test for each documented mode (zstd, deflate, jpeg, none)
+plus a parametrised TIFF compression-tag check that the file header
+advertises the right codec.
+"""
+from __future__ import annotations
+
+import importlib.util
+
+import numpy as np
+import pytest
+import xarray as xr
+
+from xrspatial.geotiff import (
+    open_geotiff,
+    write_geotiff_gpu,
+)
+from xrspatial.geotiff._header import parse_header, parse_ifd
+
+
+def _gpu_available() -> bool:
+    if importlib.util.find_spec("cupy") is None:
+        return False
+    try:
+        import cupy
+        return bool(cupy.cuda.is_available())
+    except Exception:
+        return False
+
+
+_HAS_GPU = _gpu_available()
+_gpu_only = pytest.mark.skipif(not _HAS_GPU, reason="cupy + CUDA required")
+
+
+# Compression-tag IDs from the TIFF specification, mirroring the table
+# in ``_writer._compression_tag``. Pinned here so an accidental change
+# to the codec-tag wiring is caught.
+_TIFF_COMPRESSION_TAG = 259
+_COMPRESSION_TAGS = {
+    'none': 1,
+    'deflate': 8,
+    'jpeg': 7,
+    'zstd': 50000,
+}
+
+
+def _read_compression_tag(path: str) -> int:
+    """Return the TIFF Compression (tag 259) value from *path*."""
+    with open(path, 'rb') as f:
+        data = f.read()
+    hdr = parse_header(data)
+    ifd = parse_ifd(data, hdr.first_ifd_offset, hdr)
+    entry = ifd.entries[_TIFF_COMPRESSION_TAG]
+    val = entry.value
+    # value is either an int scalar or a 1-tuple depending on count;
+    # the TIFF spec allows count=1 to be inlined.
+    if isinstance(val, (tuple, list)):
+        return int(val[0])
+    return int(val)
+
+
+def _make_int_da(h=64, w=64, dtype=np.int32):
+    """Build a deterministic CuPy-backed DataArray for lossless codecs."""
+    import cupy
+    arr = (np.arange(h * w, dtype=np.int64) % 1000).astype(dtype).reshape(h, w)
+    return xr.DataArray(
+        cupy.asarray(arr),
+        dims=('y', 'x'),
+        coords={'y': np.arange(h), 'x': np.arange(w)},
+    ), arr
+
+
+def _make_rgb_uint8_da(h=64, w=64, seed=0):
+    """Build a CuPy-backed uint8 3-band DataArray for JPEG."""
+    import cupy
+    rng = np.random.default_rng(seed)
+    arr = rng.integers(0, 256, size=(h, w, 3), dtype=np.uint8)
+    return xr.DataArray(
+        cupy.asarray(arr),
+        dims=('y', 'x', 'band'),
+        coords={'y': np.arange(h), 'x': np.arange(w), 'band': [1, 2, 3]},
+    ), arr
+
+
+# ---------------------------------------------------------------------------
+# Cat 4 HIGH: zstd is the documented default, never round-tripped explicitly
+# ---------------------------------------------------------------------------
+
+@_gpu_only
+def test_write_geotiff_gpu_zstd_roundtrip(tmp_path):
+    """Default ``compression='zstd'`` round-trips pixel-exact.
+
+    The GPU writer advertises zstd as the fastest GPU codec and uses it
+    as the default. nvCOMP zstd is lossless, so the read-back must
+    equal the input bit-for-bit.
+    """
+    da, arr = _make_int_da()
+    path = str(tmp_path / "zstd_roundtrip.tif")
+
+    write_geotiff_gpu(da, path, compression='zstd')
+
+    out = open_geotiff(path)
+    np.testing.assert_array_equal(out.values, arr)
+    assert out.dtype == arr.dtype
+
+
+@_gpu_only
+def test_write_geotiff_gpu_zstd_default_matches_explicit(tmp_path):
+    """Omitting ``compression=`` defaults to zstd; bytes must match an
+    explicit ``compression='zstd'`` call.
+
+    Pins the default so a silent change to the default codec (eg. to
+    'deflate') would fail this test.
+    """
+    da, _ = _make_int_da()
+    default_path = str(tmp_path / "default.tif")
+    explicit_path = str(tmp_path / "explicit_zstd.tif")
+
+    write_geotiff_gpu(da, default_path)
+    write_geotiff_gpu(da, explicit_path, compression='zstd')
+
+    # Both files must advertise zstd in the TIFF header.
+    assert _read_compression_tag(default_path) == _COMPRESSION_TAGS['zstd']
+    assert _read_compression_tag(explicit_path) == _COMPRESSION_TAGS['zstd']
+
+
+# ---------------------------------------------------------------------------
+# Cat 4 HIGH: jpeg is documented but never round-tripped
+# ---------------------------------------------------------------------------
+
+@_gpu_only
+def test_write_geotiff_gpu_jpeg_rgb_roundtrip(tmp_path):
+    """``compression='jpeg'`` round-trips a 3-band uint8 RGB raster.
+
+    JPEG is lossy so we tolerate a moderate per-pixel error budget but
+    require the mean error to stay within typical JPEG quality bounds
+    (well under 50 for default-quality 8-bit).
+    """
+    da, arr = _make_rgb_uint8_da()
+    path = str(tmp_path / "jpeg_rgb.tif")
+
+    write_geotiff_gpu(da, path, compression='jpeg')
+
+    out = open_geotiff(path)
+    assert out.shape == arr.shape
+    assert out.dtype == arr.dtype
+    diff = np.abs(out.values.astype(np.int32) - arr.astype(np.int32))
+    # Random uint8 is the worst case for JPEG; we just want to catch a
+    # codec that emits all-zero or all-255 output rather than measure
+    # quality. Mean-abs-diff below 50 is comfortable for default quality.
+    assert diff.mean() < 50, (
+        f"JPEG round-trip mean diff {diff.mean()} suggests encoder/decoder break"
+    )
+
+
+@_gpu_only
+def test_write_geotiff_gpu_jpeg_uint8_single_band_roundtrip(tmp_path):
+    """``compression='jpeg'`` round-trips a 1-band uint8 (greyscale)
+    raster.
+
+    Single-band JPEG exercises a different nvJPEG path (luminance-only
+    vs. RGB) and the Pillow fallback's monochrome branch.
+    """
+    import cupy
+    rng = np.random.default_rng(0)
+    arr = rng.integers(0, 256, size=(64, 64), dtype=np.uint8)
+    da = xr.DataArray(
+        cupy.asarray(arr),
+        dims=('y', 'x'),
+        coords={'y': np.arange(64), 'x': np.arange(64)},
+    )
+    path = str(tmp_path / "jpeg_mono.tif")
+
+    write_geotiff_gpu(da, path, compression='jpeg')
+
+    out = open_geotiff(path)
+    assert out.shape == arr.shape
+    assert out.dtype == arr.dtype
+    diff = np.abs(out.values.astype(np.int32) - arr.astype(np.int32))
+    assert diff.mean() < 50
+
+
+# ---------------------------------------------------------------------------
+# Cat 4 MEDIUM: compression-tag header check across all documented modes
+# ---------------------------------------------------------------------------
+
+@_gpu_only
+@pytest.mark.parametrize("compression", ['none', 'deflate', 'zstd'])
+def test_write_geotiff_gpu_compression_tag(tmp_path, compression):
+    """The TIFF Compression tag in the output matches the requested
+    codec.
+
+    A regression that wired the writer to a different codec tag would
+    produce files that decode correctly through the internal reader
+    (it inspects the same wired tag) but break interop with GDAL /
+    rasterio / libtiff.
+    """
+    da, _ = _make_int_da()
+    path = str(tmp_path / f"compression_tag_{compression}.tif")
+
+    write_geotiff_gpu(da, path, compression=compression)
+
+    assert _read_compression_tag(path) == _COMPRESSION_TAGS[compression]
+
+
+@_gpu_only
+def test_write_geotiff_gpu_jpeg_compression_tag(tmp_path):
+    """The JPEG compression tag (7) is written for uint8 RGB input."""
+    da, _ = _make_rgb_uint8_da()
+    path = str(tmp_path / "jpeg_tag.tif")
+
+    write_geotiff_gpu(da, path, compression='jpeg')
+
+    assert _read_compression_tag(path) == _COMPRESSION_TAGS['jpeg']
+
+
+# ---------------------------------------------------------------------------
+# Cat 4 MEDIUM: explicit deflate round-trip (already covered indirectly
+# but no test in the suite asserts pixel equality on the GPU writer
+# deflate path with a non-COG/non-overview layout).
+# ---------------------------------------------------------------------------
+
+@_gpu_only
+def test_write_geotiff_gpu_deflate_roundtrip(tmp_path):
+    """``compression='deflate'`` round-trips pixel-exact for the plain
+    (non-COG) GPU writer path.
+
+    The existing deflate coverage on the GPU writer runs through the
+    COG path or through NaN-sentinel scenarios. This test pins the
+    plain tiled-deflate layout against a deterministic integer raster.
+    """
+    da, arr = _make_int_da()
+    path = str(tmp_path / "deflate_plain.tif")
+
+    write_geotiff_gpu(da, path, compression='deflate')
+
+    out = open_geotiff(path)
+    np.testing.assert_array_equal(out.values, arr)
+    assert _read_compression_tag(path) == _COMPRESSION_TAGS['deflate']
+
+
+# ---------------------------------------------------------------------------
+# Cat 4 MEDIUM: none / uncompressed round-trip
+# ---------------------------------------------------------------------------
+
+@_gpu_only
+def test_write_geotiff_gpu_none_roundtrip(tmp_path):
+    """``compression='none'`` round-trips pixel-exact.
+
+    The GPU writer still chunks the image into tile buffers even when
+    no codec is applied; this test pins that the no-codec assembly
+    path emits a valid, readable file.
+    """
+    da, arr = _make_int_da()
+    path = str(tmp_path / "none_plain.tif")
+
+    write_geotiff_gpu(da, path, compression='none')
+
+    out = open_geotiff(path)
+    np.testing.assert_array_equal(out.values, arr)
+    assert _read_compression_tag(path) == _COMPRESSION_TAGS['none']
+
+
+# ---------------------------------------------------------------------------
+# Cross-codec parity: pixel-exact for lossless codecs
+# ---------------------------------------------------------------------------
+
+@_gpu_only
+def test_write_geotiff_gpu_lossless_codecs_agree(tmp_path):
+    """zstd / deflate / none must produce pixel-identical read-backs.
+
+    The codecs are lossless, so for the same input the decoded
+    pixel arrays must match exactly. Catches regressions where a codec
+    path silently corrupts data (eg. wrong predictor wiring).
+    """
+    da, arr = _make_int_da()
+    paths = {
+        codec: str(tmp_path / f"parity_{codec}.tif")
+        for codec in ('none', 'deflate', 'zstd')
+    }
+    for codec, path in paths.items():
+        write_geotiff_gpu(da, path, compression=codec)
+
+    reads = {codec: open_geotiff(path).values for codec, path in paths.items()}
+
+    np.testing.assert_array_equal(reads['none'], arr)
+    np.testing.assert_array_equal(reads['deflate'], reads['none'])
+    np.testing.assert_array_equal(reads['zstd'], reads['none'])