refactor(errors)!: a lean, idiomatic DataRetrievalError taxonomy

Every request failure raises a subclass of DataRetrievalError, so a caller can
handle any of them with a single `except dataretrieval.DataRetrievalError`. The
taxonomy stays small -- it adds only what the underlying httpx exceptions can't
express:

  DataRetrievalError(Exception)
  |- HTTPError                   # .status_code -- the server returned an error status
  |   '- TransientError          # .retry_after -- retryable (429 / 5xx)
  |       |- RateLimited         #   429
  |       '- ServiceUnavailable  #   5xx
  |- RequestTooLarge             # the request can't fit
  |   |- URLTooLong              #   414 / client-side over-long URL
  |   '- Unchunkable             #   the Water Data chunker can't split the call
  '- NoDataError                 # a 200 response with no data

One factory -- error_for_status(status, message, *, retry_after) -- maps a
status to its type, and every request path routes through it (the legacy
`query` path, the Water Data chunker, nldi, nadp, streamstats), so a given
status surfaces as the same type everywhere. A fatal 4xx is a generic HTTPError
carrying .status_code (inspect the code rather than a class per code). The
chunker keys retry/resume on TransientError; connection-level failures
(timeouts, DNS) surface as httpx exceptions on the single-shot paths. The typed
errors are picklable, so they propagate cleanly back from multiprocessing /
lithops workers.

BREAKING CHANGES
- Request failures raise typed DataRetrievalError subclasses instead of bare
  ValueError / RuntimeError / httpx.HTTPStatusError. The exceptions root only at
  DataRetrievalError(Exception) and no longer also inherit ValueError /
  RuntimeError -- catch DataRetrievalError (or a subclass), not the builtins.
- A fatal 4xx raises HTTPError (read .status_code); there are no per-code types.
- The empty-result error is renamed NoSitesError -> NoDataError (it is raised
  from the shared query path for any module, not just NWIS "sites"). NoSitesError
  stays as a deprecated alias and will be removed in a future release.

Also adds a dataretrieval.exceptions API docs page and a NEWS.md changelog entry.

mypy --strict clean; ruff clean; full suite green (485 passed, 2 skipped); the
Water Data chunker's resume tests pass unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: thodson-usgs

NEWS.md

@@ -1,3 +1,5 @@
+**06/03/2026:** The request-error hierarchy is now unified. Every module (`nwis`, `wqp`, `nldi`, `waterdata`, `nadp`, `streamstats`) raises a subclass of `dataretrieval.DataRetrievalError` on a failed request, so a single `except dataretrieval.DataRetrievalError` spans them all. An HTTP error status surfaces as an `HTTPError` carrying `.status_code` (inspect it to branch on a specific code); the retryable 429/5xx subset is `TransientError` (`RateLimited` / `ServiceUnavailable`, carrying `.retry_after`); and a request too large to satisfy is a `RequestTooLarge` (`URLTooLong` for an over-long single request, `Unchunkable` when the Water Data chunker cannot split a call small enough). Connection-level failures (timeouts, DNS) still surface as `httpx` exceptions on the single-shot paths. **Breaking change:** these exceptions no longer multiply-inherit a built-in — code that caught request failures with `except ValueError` or `except RuntimeError` should switch to `except dataretrieval.DataRetrievalError` (or a specific subclass). The error raised on a 200-but-empty result, formerly `NoSitesError`, is renamed `NoDataError` (the old name leaked NWIS-era "sites" terminology and the condition is general); `NoSitesError` remains as a deprecated alias and will be removed in a future release.
+
 **05/17/2026:** The OGC `waterdata` getters (`get_daily`, `get_continuous`, `get_field_measurements`, and the rest of the multi-value-capable functions) now transparently chunk requests whose URLs would otherwise exceed the server's ~8 KB byte limit.
 
 **05/16/2026:** Fixed silent truncation in the paginated `waterdata` request loops (`_walk_pages` and `get_stats_data`). Mid-pagination failures (HTTP 429, 5xx, network error) were previously swallowed — pagination would quietly stop and the function would return whatever rows it had collected, leaving callers with truncated DataFrames they had no way to detect. The loops now status-check every page like the initial request and raise `RuntimeError` on any failure, with the upstream exception chained as `__cause__` and a short menu of recovery actions (wait and retry, reduce the request, or obtain an API token) in the message. **Behavior change**: callers that previously consumed partial DataFrames on transient upstream blips will now see an exception; retry the call (possibly with a smaller `limit` or narrower query).

dataretrieval/__init__.py

@@ -17,8 +17,10 @@
 ``nldi`` requires geopandas (``pip install dataretrieval[nldi]``) and is
 imported on demand: ``from dataretrieval import nldi``.
 
-Every request failure raises a subclass of :class:`dataretrieval.DataRetrievalError`;
-the taxonomy lives in ``dataretrieval.exceptions``.
+When a request gets an HTTP error response it raises a subclass of
+:class:`dataretrieval.DataRetrievalError` (the taxonomy lives in
+``dataretrieval.exceptions``). Connection-level failures (timeouts, DNS) still
+surface as ``httpx`` exceptions on the single-shot service paths.
 """
 
 from importlib.metadata import PackageNotFoundError, version
@@ -29,10 +31,10 @@
     __version__ = "version-unknown"
 
 from dataretrieval.exceptions import (
-    BadRequestError,
     DataRetrievalError,
+    HTTPError,
+    NoDataError,
     NoSitesError,
-    NotFoundError,
     RateLimited,
     RequestTooLarge,
     ServiceUnavailable,
@@ -64,10 +66,10 @@
     # error taxonomy (canonical home: ``dataretrieval.exceptions``), re-exported
     # so callers can ``except dataretrieval.DataRetrievalError``
     "exceptions",
-    "BadRequestError",
     "DataRetrievalError",
-    "NoSitesError",
-    "NotFoundError",
+    "HTTPError",
+    "NoDataError",
+    "NoSitesError",  # deprecated alias for NoDataError
     "RateLimited",
     "RequestTooLarge",
     "ServiceUnavailable",

dataretrieval/exceptions.py

@@ -1,74 +1,176 @@
 """Exception taxonomy for ``dataretrieval``.
 
-A failed request from any service module (``nwis``, ``wqp``, ``waterdata``,
-``nldi``, ...) raises a subclass of :class:`DataRetrievalError`, so a caller can
-handle any request failure with a single ``except dataretrieval.DataRetrievalError``.
-
-The tree has two intermediate bases a caller can catch to span a whole family:
-:class:`RequestTooLarge` (the request can't fit, however it was issued) and
-:class:`TransientError` (a temporary failure worth retrying).
-
-This module deliberately has no third-party dependencies, so any module can
-import it without pulling in pandas/httpx.
+When a request gets an HTTP error response, the service modules (``nwis``,
+``wqp``, ``nldi``, ``waterdata``, ``nadp``, ``streamstats``) raise a subclass of
+:class:`DataRetrievalError`, so a caller can handle any of them with one
+``except dataretrieval.DataRetrievalError``. Connection-level failures (timeouts,
+DNS, refused connections) surface as ``httpx`` exceptions on the single-shot
+request paths.
+
+A status error is an :class:`HTTPError` carrying ``.status_code`` (inspect it to
+branch on the specific code); :class:`TransientError` is the retryable subset
+(429 / 5xx). A few failures are not a plain status -- :class:`RequestTooLarge`
+(:class:`URLTooLong` / :class:`Unchunkable`) and :class:`NoDataError`.
+
+This module imports only ``httpx`` (the package's core HTTP dependency, always
+installed) -- not pandas/geopandas -- so it stays cheap to import and free of
+import cycles.
 """
 
 from __future__ import annotations
 
-from typing import TYPE_CHECKING
+from typing import Any, ClassVar
 
-if TYPE_CHECKING:
-    import httpx
+import httpx
 
 __all__ = [
     "DataRetrievalError",
-    "BadRequestError",
-    "NotFoundError",
-    "RequestTooLarge",
-    "URLTooLong",
-    "Unchunkable",
-    "NoSitesError",
+    "HTTPError",
     "TransientError",
     "RateLimited",
     "ServiceUnavailable",
+    "RequestTooLarge",
+    "URLTooLong",
+    "Unchunkable",
+    "NoDataError",
+    "NoSitesError",  # deprecated alias for NoDataError
+    "error_for_status",
 ]
 
 
 class DataRetrievalError(Exception):
-    """Base class for errors raised when a request to a USGS or EPA web
+    """Base class for every error raised when a request to a USGS or EPA web
     service fails.
 
-    Every service module (``nwis``, ``wqp``, ``waterdata``, ``nldi``, ...)
-    raises a subclass of this when a request fails, so a caller can handle any
-    request failure uniformly::
+    Service modules raise a subclass of this on a failed request, so a caller
+    can handle them uniformly::
 
         try:
             df, md = dataretrieval.wqp.get_results(...)
         except dataretrieval.DataRetrievalError:
             ...
 
-    Subclasses also inherit from the built-in exception this package has
-    historically raised for the condition's *kind* -- :class:`ValueError` for a
-    request that can't succeed as written (bad params, too large), and
-    :class:`RuntimeError` for a transient transport failure -- so existing
-    ``except ValueError`` / ``except RuntimeError`` handlers keep working.
+    Connection-level failures (timeouts, DNS) still surface as ``httpx``
+    exceptions on the single-shot request paths.
+    """
+
+    def __reduce__(self) -> tuple[Any, ...]:
+        # The status subclasses declare keyword-only fields (status_code /
+        # retry_after); the default ``BaseException.__reduce__`` rebuilds via
+        # ``cls(*self.args)``, which drops them and raises TypeError on unpickle
+        # / deepcopy. Reconstruct from args + ``__dict__`` instead so every
+        # subclass round-trips -- these get pickled back from multiprocessing /
+        # lithops workers.
+        return (_rebuild_error, (self.__class__, self.args, self.__dict__))
+
+
+def _rebuild_error(
+    cls: type[DataRetrievalError],
+    args: tuple[Any, ...],
+    state: dict[str, Any],
+) -> DataRetrievalError:
+    """Rebuild a :class:`DataRetrievalError` without calling ``__init__``.
+
+    See :meth:`DataRetrievalError.__reduce__`.
+    """
+    err = cls.__new__(cls)
+    err.args = args
+    err.__dict__.update(state)
+    return err
+
+
+# --- HTTP status errors --------------------------------------------------
+
+
+class HTTPError(DataRetrievalError):
+    """The service returned an error HTTP status.
+
+    The numeric status is on :attr:`status_code`; inspect it to branch on the
+    specific code, e.g. ``except HTTPError as e: ... e.status_code == 404``.
+    :class:`TransientError` (429 / 5xx) is the retryable subset and is itself an
+    ``HTTPError``. The one carve-out: a 413/414 surfaces as :class:`URLTooLong`
+    (a :class:`RequestTooLarge`), *not* an ``HTTPError`` -- catch
+    :class:`DataRetrievalError` to span every failure.
+
+    Parameters
+    ----------
+    message : str
+        Human-readable error message.
+    status_code : int
+        The HTTP status the service returned.
+    """
+
+    def __init__(self, message: str, *, status_code: int) -> None:
+        super().__init__(message)
+        self.status_code = status_code
+
+
+class TransientError(HTTPError):
+    """A 429 or 5xx the server may serve on a later try (:class:`RateLimited`
+    for 429, :class:`ServiceUnavailable` for 5xx).
+
+    This classifies the HTTP condition; it does not by itself retry the request.
+    Whether a transient is retried is up to the calling path -- a single-shot
+    request raises it for the caller to handle (e.g. wait :attr:`retry_after`
+    and re-issue).
+
+    Parameters
+    ----------
+    message : str
+        Human-readable error message.
+    status_code : int
+        The HTTP status the service returned.
+    status_code : int, optional
+        The HTTP status the service returned. Defaults to the concrete leaf's
+        canonical code (:attr:`_DEFAULT_STATUS`) when omitted;
+        :func:`error_for_status` always passes the real status.
+    retry_after : float, optional
+        Seconds to wait before retrying, parsed from the ``Retry-After``
+        response header; ``None`` when the header is absent or unparseable.
     """
 
+    #: Canonical status a concrete transient stamps when built without an
+    #: explicit ``status_code`` (:class:`RateLimited` = 429,
+    #: :class:`ServiceUnavailable` = 503). ``TransientError`` itself is abstract
+    #: and sets none, so constructing it bare requires ``status_code``.
+    _DEFAULT_STATUS: ClassVar[int]
+
+    def __init__(
+        self,
+        message: str,
+        *,
+        status_code: int | None = None,
+        retry_after: float | None = None,
+    ) -> None:
+        super().__init__(
+            message,
+            status_code=self._DEFAULT_STATUS if status_code is None else status_code,
+        )
+        self.retry_after = retry_after
+
 
-# --- Fatal client errors -------------------------------------------------
-# The request can't succeed as written; retrying it unchanged won't help. Each
-# is also a ``ValueError`` -- the built-in the legacy ``query`` path has always
-# raised -- so existing ``except ValueError`` handlers keep working.
+class RateLimited(TransientError):
+    """A request was rejected with HTTP 429 (too many requests)."""
 
+    _DEFAULT_STATUS = 429
 
-class BadRequestError(DataRetrievalError, ValueError):
-    """The service rejected the request parameters (HTTP 400)."""
 
+class ServiceUnavailable(TransientError):
+    """A request was rejected with a server error (HTTP 5xx).
 
-class NotFoundError(DataRetrievalError, ValueError):
-    """The requested resource was not found; often an empty query (HTTP 404)."""
+    Raised by both the legacy ``query`` path and the Water Data path, so a 5xx
+    surfaces as one type regardless of which subsystem issued the request.
+    ``status_code`` defaults to 503 (its namesake) only when built by hand
+    without one; the factory always supplies the real 5xx.
+    """
 
+    _DEFAULT_STATUS = 503
 
-class RequestTooLarge(DataRetrievalError, ValueError):
+
+# --- Request can't fit (not necessarily an HTTP status) ------------------
+
+
+class RequestTooLarge(DataRetrievalError):
     """The request is too large for the service to satisfy.
 
     A base for the two ways a request can exceed what the service accepts;
@@ -99,56 +201,45 @@ class Unchunkable(RequestTooLarge):
     """
 
 
-class NoSitesError(DataRetrievalError):
-    """The selection criteria matched no sites/data."""
+# --- Empty result --------------------------------------------------------
+
+
+class NoDataError(DataRetrievalError):
+    """The request succeeded (HTTP 200) but the selection criteria matched
+    no data."""
 
     def __init__(self, url: httpx.URL) -> None:
         self.url = url
 
     def __str__(self) -> str:
         return (
-            "No sites/data found using the selection criteria specified in "
-            f"url: {self.url}"
+            f"No data found using the selection criteria specified in url: {self.url}"
         )
 
 
-# --- Transient transport errors ------------------------------------------
-# The service was reachable but temporarily refused the request; the same call
-# may succeed if retried. Each is also a ``RuntimeError`` (the built-in the
-# waterdata path has always raised). The Water Data chunker recognizes them via
-# ``isinstance(exc, TransientError)`` and wraps them as resumable
-# ``ChunkInterrupted`` subclasses.
+#: Deprecated alias for :class:`NoDataError`. The original name leaked NWIS-era
+#: "sites" terminology; it is retained so existing ``except NoSitesError``
+#: handlers keep working, and will be removed in a future release.
+NoSitesError = NoDataError
 
 
-class TransientError(DataRetrievalError, RuntimeError):
-    """Base for transient HTTP failures that are worth an automatic retry.
+def error_for_status(
+    status: int, message: str, *, retry_after: float | None = None
+) -> DataRetrievalError:
+    """Return the typed :class:`DataRetrievalError` for an HTTP error *status*.
 
-    One subclass per recoverable HTTP status family (429 -> :class:`RateLimited`,
-    5xx -> :class:`ServiceUnavailable`); the Water Data chunker recognizes them
-    by this shared base and wraps them as resumable interruptions.
-
-    Parameters
-    ----------
-    message : str
-        Human-readable error message.
-    retry_after : float, optional
-        Seconds to wait before retrying, parsed from the ``Retry-After``
-        response header; stored on the :attr:`retry_after` attribute (``None``
-        when the header is absent or unparseable).
-    """
-
-    def __init__(self, message: str, *, retry_after: float | None = None) -> None:
-        super().__init__(message)
-        self.retry_after = retry_after
-
-
-class RateLimited(TransientError):
-    """A request was rejected with HTTP 429 (too many requests)."""
-
-
-class ServiceUnavailable(TransientError):
-    """A request was rejected with a server error (HTTP 5xx).
-
-    Raised by both the legacy ``query`` path and the Water Data path, so a 5xx
-    surfaces as one type regardless of which subsystem issued the request.
+    The single status-to-type mapping shared by every request path (the legacy
+    ``query`` path, ``waterdata``, ``nadp`` / ``streamstats``), so a given status
+    surfaces as the same type everywhere. ``message`` is used verbatim;
+    ``retry_after`` is attached only to the transient (:class:`TransientError`)
+    types. A 413/414 surfaces as :class:`URLTooLong` (a :class:`RequestTooLarge`)
+    rather than a generic :class:`HTTPError`, matching the client-side
+    over-long-URL case.
     """
+    if status in (413, 414):
+        return URLTooLong(message)
+    if status == 429:
+        return RateLimited(message, status_code=status, retry_after=retry_after)
+    if 500 <= status < 600:
+        return ServiceUnavailable(message, status_code=status, retry_after=retry_after)
+    return HTTPError(message, status_code=status)

dataretrieval/nadp.py

@@ -45,7 +45,7 @@
 
 import httpx
 
-from dataretrieval.utils import HTTPX_DEFAULTS
+from dataretrieval.utils import HTTPX_DEFAULTS, _raise_for_status
 
 _DEPRECATION_MESSAGE = (
     "The `nadp` module is deprecated and will be removed from `dataretrieval` "
@@ -230,7 +230,7 @@ def get_zip(url: str, filename: str) -> NADP_ZipFile:
     _warn_deprecated()
 
     req = httpx.get(url + filename, **HTTPX_DEFAULTS)
-    req.raise_for_status()
+    _raise_for_status(req)
 
     # z = zipfile.ZipFile(io.BytesIO(req.content))
     z = NADP_ZipFile(io.BytesIO(req.content))