Commit ff8f535
refactor(errors)!: a lean, idiomatic DataRetrievalError taxonomy (#319)
Every request failure raises a subclass of DataRetrievalError, so a caller can
handle any of them with a single `except dataretrieval.DataRetrievalError`. The
taxonomy stays small -- it adds only what the underlying httpx exceptions can't
express:
DataRetrievalError(Exception) # .status_code / .retry_after / .retryable
|- HTTPError # .status_code -- the server returned an error status
| '- TransientError # .retry_after -- retryable (429 / 5xx)
| |- RateLimited # 429
| '- ServiceUnavailable # 5xx
|- RequestTooLarge # the request can't fit
| |- URLTooLong # 414 / client-side over-long URL
| '- Unchunkable # the Water Data chunker can't split the call
|- NetworkError # no response: timeout / DNS / refused connection
'- NoSitesError # no-data on the legacy nwis path (see below)
One factory -- error_for_status(status, message, *, retry_after) -- maps a
status to its type, and every request path routes through it (the legacy
`query` path, the Water Data chunker, nldi, nadp, streamstats), so a given
status surfaces as the same type everywhere. A fatal 4xx is a generic HTTPError
carrying .status_code (inspect the code rather than a class per code). The
chunker keys retry/resume on TransientError.
Every DataRetrievalError exposes three read-anywhere fields -- .status_code
(None when there is no HTTP status), .retry_after, and .retryable -- so a single
`except DataRetrievalError as e` clause can branch on the status or drive a
backoff loop without importing or isinstance-checking the concrete subclass.
Connection-level failures (no HTTP response: timeout, DNS, refused connection)
are wrapped as NetworkError, with the underlying httpx exception on __cause__,
so one `except DataRetrievalError` truly spans every failure. The single-shot
paths route their GETs through a thin `utils._get` wrapper that does the
translation; the chunker keeps its own client and wraps transport failures as
resumable interruptions instead. NetworkError carries no .status_code but is
.retryable; with TransientError it forms the retryable set.
A no-data result is not an error: the modern getters (waterdata, wqp, nldi)
return an empty DataFrame when nothing matches. Only the deprecated nwis
(waterservices) path still raises NoSitesError on no data.
The typed errors are picklable via the standard __getstate__/__setstate__
protocol, so they survive a pickle / deepcopy back from a multiprocessing /
lithops worker. A chunk-interruption error sheds its live resume handle (.call)
on that trip -- keeping the diagnostic counts and partial frame/response --
while in-process callers still get full `exc.call.resume()`.
A too-long-URL status (413 / 414) on the legacy `query` path keeps the
actionable "split your query" remediation message (the same one the client-side
over-long-URL case raises), rather than degrading to a bare HTTP-status line.
Also adds a dataretrieval.exceptions API docs page, a "Handling errors" user
guide, and a NEWS.md changelog entry.
ruff clean (pre-commit hooks); mypy --strict and the full pytest suite are
re-verified by CI on push.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>1 parent ecf2833 commit ff8f535
21 files changed
Lines changed: 726 additions & 324 deletions
File tree
- dataretrieval
- waterdata
- docs/source
- reference
- userguide
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
1 | 3 | | |
2 | 4 | | |
3 | 5 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
21 | | - | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
22 | 23 | | |
23 | 24 | | |
24 | 25 | | |
| |||
29 | 30 | | |
30 | 31 | | |
31 | 32 | | |
32 | | - | |
33 | 33 | | |
| 34 | + | |
| 35 | + | |
34 | 36 | | |
35 | | - | |
36 | 37 | | |
37 | 38 | | |
38 | 39 | | |
| |||
64 | 65 | | |
65 | 66 | | |
66 | 67 | | |
67 | | - | |
68 | 68 | | |
| 69 | + | |
| 70 | + | |
69 | 71 | | |
70 | | - | |
71 | 72 | | |
72 | 73 | | |
73 | 74 | | |
| |||
0 commit comments