Commit 4587964
refactor(errors)!: a lean, idiomatic DataRetrievalError taxonomy
Every request failure raises a subclass of DataRetrievalError, so a caller can
handle any of them with a single `except dataretrieval.DataRetrievalError`. The
taxonomy stays small -- it adds only what the underlying httpx exceptions can't
express:
DataRetrievalError(Exception)
|- HTTPError # .status_code -- the server returned an error status
| '- TransientError # .retry_after -- retryable (429 / 5xx)
| |- RateLimited # 429
| '- ServiceUnavailable # 5xx
|- RequestTooLarge # the request can't fit
| |- URLTooLong # 414 / client-side over-long URL
| '- Unchunkable # the Water Data chunker can't split the call
|- NetworkError # no response: timeout / DNS / refused connection
'- NoDataError # a 200 response with no data
One factory -- error_for_status(status, message, *, retry_after) -- maps a
status to its type, and every request path routes through it (the legacy
`query` path, the Water Data chunker, nldi, nadp, streamstats), so a given
status surfaces as the same type everywhere. A fatal 4xx is a generic HTTPError
carrying .status_code (inspect the code rather than a class per code). The
chunker keys retry/resume on TransientError.
Connection-level failures (no HTTP response: timeout, DNS, refused connection)
are wrapped as NetworkError, with the underlying httpx exception on __cause__,
so one `except DataRetrievalError` truly spans every failure. The single-shot
paths route their GETs through a thin `utils._get` wrapper that does the
translation; the chunker keeps its own client and wraps transport failures as
resumable interruptions instead. NetworkError carries no .status_code (no
response arrived); with TransientError it forms the retryable set.
The typed errors are picklable via the standard __getstate__/__setstate__
protocol, so they survive a pickle / deepcopy back from a multiprocessing /
lithops worker. A chunk-interruption error sheds its live resume handle (.call)
on that trip -- keeping the diagnostic counts and partial frame/response --
while in-process callers still get full `exc.call.resume()`.
A too-long-URL status (413 / 414) on the legacy `query` path keeps the
actionable "split your query" remediation message (the same one the client-side
over-long-URL case raises), rather than degrading to a bare HTTP-status line.
BREAKING CHANGES
- Request failures raise typed DataRetrievalError subclasses instead of bare
ValueError / RuntimeError / httpx.HTTPStatusError. The exceptions root only at
DataRetrievalError(Exception) and no longer also inherit ValueError /
RuntimeError -- catch DataRetrievalError (or a subclass), not the builtins.
This now also covers ChunkInterrupted (previously a RuntimeError) and
mid-pagination failures (previously a bare RuntimeError).
- Connection-level failures are wrapped as NetworkError instead of surfacing as
raw httpx exceptions on the single-shot paths -- catch NetworkError (or
DataRetrievalError); the httpx exception is preserved on __cause__.
- A fatal 4xx raises HTTPError (read .status_code); there are no per-code types.
- The empty-result error is renamed NoSitesError -> NoDataError (it is raised
from the shared query path for any module, not just NWIS "sites"). NoSitesError
stays as a deprecated alias -- referencing it now emits a DeprecationWarning --
and will be removed in a future release.
Also adds a dataretrieval.exceptions API docs page and a NEWS.md changelog entry.
mypy --strict clean; ruff clean; full suite green (489 passed, 2 skipped); the
Water Data chunker's resume tests pass unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>1 parent ecf2833 commit 4587964
18 files changed
Lines changed: 632 additions & 318 deletions
File tree
- dataretrieval
- waterdata
- docs/source/reference
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
1 | 3 | | |
2 | 4 | | |
3 | 5 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
21 | | - | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
22 | 23 | | |
23 | 24 | | |
24 | 25 | | |
| 26 | + | |
25 | 27 | | |
26 | 28 | | |
27 | 29 | | |
28 | 30 | | |
29 | 31 | | |
30 | 32 | | |
31 | 33 | | |
32 | | - | |
33 | 34 | | |
34 | | - | |
35 | | - | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
36 | 38 | | |
37 | 39 | | |
38 | 40 | | |
| |||
64 | 66 | | |
65 | 67 | | |
66 | 68 | | |
67 | | - | |
68 | 69 | | |
69 | | - | |
70 | | - | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
71 | 73 | | |
72 | 74 | | |
73 | 75 | | |
| |||
76 | 78 | | |
77 | 79 | | |
78 | 80 | | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
0 commit comments