refactor(errors)!: unify request failures under a DataRetrievalError taxonomy

Before, an HTTP failure surfaced as a different exception type depending on
which module made the request -- a ValueError (or bare Exception) on the legacy
query() path, RuntimeError-based types on the waterdata path, a bare
httpx.HTTPStatusError elsewhere -- so there was no single `except` for "any
dataretrieval request failure".

Introduce dataretrieval/exceptions.py (dependency-free, re-exported at top level
as dataretrieval.<Name>), rooted at DataRetrievalError, with two intermediate
bases that name the axes a caller reasons about:

  DataRetrievalError(Exception)
  |- BadRequestError(.., ValueError)     # 400
  |- NotFoundError(.., ValueError)       # 404
  |- RequestTooLarge(.., ValueError)     # base: request too large to satisfy
  |   |- URLTooLong                      #   414 / client-side URL reject
  |   '- Unchunkable                     #   chunker planner floor
  |- NoSitesError                        # empty result
  '- TransientError(.., RuntimeError)    # base: retryable; carries retry_after
      |- RateLimited                     #   429
      '- ServiceUnavailable              #   5xx (both paths)

- One type per condition, raised by both the legacy query() path and the Water
  Data chunker. Callers can catch a whole family (`except RequestTooLarge` /
  `except TransientError`); the chunker's retry check is a single
  isinstance(exc, TransientError).
- query()'s inline status ladder is extracted into a reusable _raise_for_status().
- NoSitesError now subclasses DataRetrievalError (was Exception).
- Built-in compatibility by kind: fatal client errors are also ValueError,
  transient transport errors also RuntimeError, so existing `except ValueError`
  / `except RuntimeError` handlers keep working.

BREAKING CHANGES
- The legacy query() path raises typed errors instead of ad-hoc ValueErrors
  (400 -> BadRequestError, 404 -> NotFoundError, 414/over-long URL -> URLTooLong).
- A 5xx on the legacy query() path now raises ServiceUnavailable, a RuntimeError
  (was a ValueError): a transient server failure is a runtime condition, not a
  bad value.
- The Water Data chunker's planner-floor error is Unchunkable (a RequestTooLarge
  subclass).
- Import the transport types/bases from dataretrieval / dataretrieval.exceptions,
  not from dataretrieval.waterdata.chunking.

Verified: 477 passed / 2 skipped, ruff clean; live API spot checks (404/400/
over-long URL raise the typed errors, 200 unaffected); all 21 example notebooks
execute end-to-end against the live API (227/227 cells, 0 errors).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: thodson-usgs

dataretrieval/__init__.py

@@ -5,6 +5,7 @@
 except PackageNotFoundError:
     __version__ = "version-unknown"
 
+from dataretrieval.exceptions import *
 from dataretrieval.nadp import *
 from dataretrieval.nwis import *
 from dataretrieval.samples import *

dataretrieval/exceptions.py

@@ -0,0 +1,149 @@
+"""Exception taxonomy for ``dataretrieval``.
+
+A failed request from any service module (``nwis``, ``wqp``, ``waterdata``,
+``nldi``, ...) raises a subclass of :class:`DataRetrievalError`, so a caller can
+handle any request failure with a single ``except dataretrieval.DataRetrievalError``.
+
+The tree has two intermediate bases a caller can catch to span a whole family:
+:class:`RequestTooLarge` (the request can't fit, however it was issued) and
+:class:`TransientError` (a temporary failure worth retrying).
+
+This module deliberately has no third-party dependencies, so any module can
+import it without pulling in pandas/httpx.
+"""
+
+from __future__ import annotations
+
+__all__ = [
+    "DataRetrievalError",
+    "BadRequestError",
+    "NotFoundError",
+    "RequestTooLarge",
+    "URLTooLong",
+    "Unchunkable",
+    "NoSitesError",
+    "TransientError",
+    "RateLimited",
+    "ServiceUnavailable",
+]
+
+
+class DataRetrievalError(Exception):
+    """Base class for errors raised when a request to a USGS or EPA web
+    service fails.
+
+    Every service module (``nwis``, ``wqp``, ``waterdata``, ``nldi``, ...)
+    raises a subclass of this when a request fails, so a caller can handle any
+    request failure uniformly::
+
+        try:
+            df, md = dataretrieval.wqp.get_results(...)
+        except dataretrieval.DataRetrievalError:
+            ...
+
+    Subclasses also inherit from the built-in exception this package has
+    historically raised for the condition's *kind* -- :class:`ValueError` for a
+    request that can't succeed as written (bad params, too large), and
+    :class:`RuntimeError` for a transient transport failure -- so existing
+    ``except ValueError`` / ``except RuntimeError`` handlers keep working.
+    """
+
+
+# --- Fatal client errors -------------------------------------------------
+# The request can't succeed as written; retrying it unchanged won't help. Each
+# is also a ``ValueError`` -- the built-in the legacy ``query`` path has always
+# raised -- so existing ``except ValueError`` handlers keep working.
+
+
+class BadRequestError(DataRetrievalError, ValueError):
+    """The service rejected the request parameters (HTTP 400)."""
+
+
+class NotFoundError(DataRetrievalError, ValueError):
+    """The requested resource was not found; often an empty query (HTTP 404)."""
+
+
+class RequestTooLarge(DataRetrievalError, ValueError):
+    """The request is too large for the service to satisfy.
+
+    A base for the two ways a request can exceed what the service accepts;
+    catch it to handle either. The concrete subclasses are :class:`URLTooLong`
+    (a single request the server rejected) and :class:`Unchunkable` (the Water
+    Data chunker could not split the call small enough to fit).
+    """
+
+
+class URLTooLong(RequestTooLarge):
+    """A single request URL exceeded the service's limit (HTTP 414, or rejected
+    client-side before it was sent).
+
+    Raised by the legacy ``query`` path, which issues one request without
+    chunking. Remediation: query fewer sites, or split the call manually.
+    """
+
+
+class Unchunkable(RequestTooLarge):
+    """No chunking plan fits the URL byte limit.
+
+    Raised by the Water Data chunker when even the smallest reducible plan
+    (every list axis at one atom per sub-request, the filter at one clause per
+    sub-request) still exceeds the server's byte limit -- so unlike
+    :class:`URLTooLong`, automatic splitting has already been tried and
+    exhausted. Shrink the input lists, simplify the filter, or split the call
+    manually.
+    """
+
+
+class NoSitesError(DataRetrievalError):
+    """The selection criteria matched no sites/data."""
+
+    def __init__(self, url):
+        self.url = url
+
+    def __str__(self):
+        return (
+            "No sites/data found using the selection criteria specified in "
+            f"url: {self.url}"
+        )
+
+
+# --- Transient transport errors ------------------------------------------
+# The service was reachable but temporarily refused the request; the same call
+# may succeed if retried. Each is also a ``RuntimeError`` (the built-in the
+# waterdata path has always raised). The Water Data chunker recognizes them via
+# ``isinstance(exc, TransientError)`` and wraps them as resumable
+# ``ChunkInterrupted`` subclasses.
+
+
+class TransientError(DataRetrievalError, RuntimeError):
+    """Base for transient HTTP failures that are worth an automatic retry.
+
+    One subclass per recoverable HTTP status family (429 -> :class:`RateLimited`,
+    5xx -> :class:`ServiceUnavailable`); the Water Data chunker recognizes them
+    by this shared base and wraps them as resumable interruptions.
+
+    Parameters
+    ----------
+    message : str
+        Human-readable error message.
+    retry_after : float, optional
+        Seconds to wait before retrying, parsed from the ``Retry-After``
+        response header; stored on the :attr:`retry_after` attribute (``None``
+        when the header is absent or unparseable).
+    """
+
+    def __init__(self, message: str, *, retry_after: float | None = None) -> None:
+        super().__init__(message)
+        self.retry_after = retry_after
+
+
+class RateLimited(TransientError):
+    """A request was rejected with HTTP 429 (too many requests)."""
+
+
+class ServiceUnavailable(TransientError):
+    """A request was rejected with a server error (HTTP 5xx).
+
+    Raised by both the legacy ``query`` path and the Water Data path, so a 5xx
+    surfaces as one type regardless of which subsystem issued the request.
+    """

dataretrieval/utils.py

@@ -10,6 +10,13 @@
 
 import dataretrieval
 from dataretrieval.codes import tz
+from dataretrieval.exceptions import (
+    BadRequestError,
+    NoSitesError,
+    NotFoundError,
+    ServiceUnavailable,
+    URLTooLong,
+)
 
 HTTPX_DEFAULTS = {
     "follow_redirects": True,
@@ -270,14 +277,42 @@ def __repr__(self) -> str:
                         data_list.append(data)  # append results to list"""
 
 
-def _url_too_long_error(detail: str) -> ValueError:
-    return ValueError(
+def _url_too_long_error(detail: str) -> URLTooLong:
+    return URLTooLong(
         "Request URL too long. Modify your query to use fewer sites. "
         f"{detail}. Pseudo-code example of how to split your query: "
         f"\n {_URL_TOO_LONG_EXAMPLE}"
     )
 
 
+def _raise_for_status(response: httpx.Response) -> None:
+    """Map an unsuccessful HTTP status to a typed :class:`DataRetrievalError`;
+    return ``None`` on success.
+
+    Shared by the legacy :func:`query` path. The 4xx types stay
+    :class:`ValueError`-compatible (this path's historical contract), but a 5xx
+    raises the transient :class:`ServiceUnavailable` (a :class:`RuntimeError`),
+    since a server failure is retryable rather than a bad request.
+    """
+    status = response.status_code
+    if status == 400:
+        raise BadRequestError(
+            f"Bad Request, check that your parameters are correct. URL: {response.url}"
+        )
+    elif status == 404:
+        raise NotFoundError(
+            "Page Not Found Error. May be the result of an empty query. "
+            f"URL: {response.url}"
+        )
+    elif status == 414:
+        raise _url_too_long_error(f"API response reason: {response.reason_phrase}")
+    elif 500 <= status < 600:
+        raise ServiceUnavailable(
+            f"Service Unavailable: {status} {response.reason_phrase}. "
+            f"The service at {response.url} may be down or experiencing issues."
+        )
+
+
 def query(url, payload, delimiter=",", ssl_check=True):
     """Send a query.
 
@@ -321,37 +356,9 @@ def query(url, payload, delimiter=",", ssl_check=True):
     except httpx.InvalidURL as exc:
         raise _url_too_long_error(f"httpx rejected the URL client-side: {exc}") from exc
 
-    if response.status_code == 400:
-        raise ValueError(
-            f"Bad Request, check that your parameters are correct. URL: {response.url}"
-        )
-    elif response.status_code == 404:
-        raise ValueError(
-            "Page Not Found Error. May be the result of an empty query. "
-            + f"URL: {response.url}"
-        )
-    elif response.status_code == 414:
-        raise _url_too_long_error(f"API response reason: {response.reason_phrase}")
-    elif 500 <= response.status_code < 600:
-        raise ValueError(
-            f"Service Unavailable: {response.status_code} {response.reason_phrase}. "
-            + f"The service at {response.url} may be down or experiencing issues."
-        )
+    _raise_for_status(response)
 
     if response.text.startswith("No sites/data"):
         raise NoSitesError(response.url)
 
     return response
-
-
-class NoSitesError(Exception):
-    """Custom error class used when selection criteria returns no sites/data."""
-
-    def __init__(self, url):
-        self.url = url
-
-    def __str__(self):
-        return (
-            "No sites/data found using the selection criteria specified in "
-            f"url: {self.url}"
-        )

dataretrieval/waterdata/chunking.py

@@ -66,6 +66,13 @@
 import pandas as pd
 from anyio.from_thread import start_blocking_portal
 
+from dataretrieval.exceptions import (
+    DataRetrievalError,
+    RateLimited,
+    ServiceUnavailable,
+    TransientError,
+    Unchunkable,
+)
 from dataretrieval.utils import HTTPX_DEFAULTS
 
 from . import _progress
@@ -383,70 +390,7 @@ def _passthrough_result(
     return frame, response
 
 
-class _RetryableTransportError(RuntimeError):
-    """
-    Base for typed HTTP transport failures the chunker recognizes as
-    transient.
-
-    Raised by :func:`dataretrieval.waterdata.utils._raise_for_non_200`
-    and walked by :func:`_classify_chunk_error`. One subclass per
-    recoverable HTTP status family (429 → :class:`RateLimited`,
-    5xx → :class:`ServiceUnavailable`); ``ChunkedCall`` wraps them as
-    resumable :class:`ChunkInterrupted` subclasses.
-
-    Parameters
-    ----------
-    message : str
-        Human-readable error message.
-    retry_after : float, optional
-        Seconds to wait before retrying, parsed from the
-        ``Retry-After`` response header.
-
-    Attributes
-    ----------
-    retry_after : float or None
-        Seconds to wait before retrying, parsed from the
-        ``Retry-After`` response header. ``None`` when the header was
-        absent or unparseable.
-    """
-
-    def __init__(self, message: str, *, retry_after: float | None = None) -> None:
-        super().__init__(message)
-        self.retry_after = retry_after
-
-
-class RateLimited(_RetryableTransportError):
-    """
-    A USGS Water Data API request was rejected with HTTP 429.
-
-    Exposed as a typed exception so callers (notably the multi-value
-    chunker) can detect rate-limit failures via ``isinstance`` instead
-    of string-matching error messages.
-    """
-
-
-class ServiceUnavailable(_RetryableTransportError):
-    """
-    A USGS Water Data API request was rejected with HTTP 5xx.
-
-    Surfaced as a typed exception (parallel to :class:`RateLimited`)
-    so ``ChunkedCall`` can treat transient server failures as
-    resumable interruptions rather than fatal programmer errors.
-    """
-
-
-class RequestTooLarge(ValueError):
-    """
-    No chunking plan fits the URL byte limit.
-
-    Raised when even the smallest reducible plan (every list axis at
-    singleton chunks and the filter at one clause per sub-request)
-    still exceeds the server's byte limit. Shrink the input lists,
-    simplify the filter, or split the call manually.
-    """
-
-
-class ChunkInterrupted(RuntimeError):
+class ChunkInterrupted(DataRetrievalError, RuntimeError):
     """
     Base class for mid-stream chunk failures whose completed work is
     preserved and resumable.
@@ -854,7 +798,7 @@ class ChunkPlan:
 
     Raises
     ------
-    RequestTooLarge
+    Unchunkable
         If the request needs chunking but even the singleton plan
         doesn't fit ``url_limit``.
     """
@@ -923,7 +867,7 @@ def _plan(
 
         Raises
         ------
-        RequestTooLarge
+        Unchunkable
             If even the singleton plan (every axis at one atom per
             chunk) still exceeds ``url_limit``.
         """
@@ -944,7 +888,7 @@ def _plan(
                         biggest_axis, biggest_idx, biggest_size = axis, idx, size
 
             if biggest_axis is None:
-                raise RequestTooLarge(
+                raise Unchunkable(
                     f"Request exceeds {url_limit} bytes (URL + body) at the "
                     f"smallest reducible plan (every axis at one atom per "
                     f"sub-request). Reduce input sizes, shorten or simplify "
@@ -1119,7 +1063,7 @@ def _retryable(exc: BaseException) -> tuple[bool, float | None]:
         ``(retryable, retry_after)`` — the server ``Retry-After`` hint
         (seconds) when the transient carried one, else ``None``.
     """
-    if isinstance(exc, (RateLimited, ServiceUnavailable)):
+    if isinstance(exc, TransientError):
         return True, exc.retry_after
     if isinstance(exc, httpx.TransportError):
         return True, None
@@ -1708,7 +1652,7 @@ def multi_value_chunked(
 
     Raises
     ------
-    RequestTooLarge
+    Unchunkable
         If no plan can fit ``url_limit``.
     ChunkInterrupted
         On a mid-execution transient — 429, 5xx, or a bare transport