Skip to content

Commit af9b238

Browse files
thodson-usgsclaude
andauthored
chore!: require Python >=3.10; remove the deprecated variable_info metadata property (#332)
Ahead of the breaking 1.2.0 release. Require Python >=3.10 - 3.9 support was already effectively broken: `waterdata`'s `anyio` dependency and the test stack require 3.10+, and the `waterdata` test modules already skipped on <3.10. `requires-python = ">=3.10"`; ruff `target-version = "py310"`; mypy `python_version = "3.10"` (drops the `mypy<2` pin that only existed to keep targeting 3.9); declare `anyio>=4.0` as a direct dependency (imported directly by `waterdata`); CI matrix `["3.10", "3.13", "3.14"]`; fill in per-version Trove classifiers. - The py310 ruff target newly enables `B905` (zip `strict=`) and `UP036` (now-dead `sys.version_info < (3, 10)` skip guards): add explicit `strict=False` to the remaining `zip()` calls (`nwis`, `ogc/planning`, `waterdata/nearest`) and drop the dead version-skip guards in the four `waterdata` test modules. Behavior-identical. Remove the deprecated `variable_info` metadata property entirely - The `NWIS_Metadata` override only warned and returned `None` (it relied on the defunct `get_pmcodes`). Nothing else implemented it, so the `BaseMetadata` abstract is removed too — accessing `.variable_info` now raises `AttributeError`. - `site_info` is kept: `NWIS_Metadata`/`WQP_Metadata` implement it, and the modern `waterdata` metadata intentionally leaves it abstract (use `waterdata.get_monitoring_locations()` for site descriptions). Fix the live-API flaky-rerun marker - A live test that hits an SSL/connection timeout surfaces as a typed `NetworkError`, but the `flaky_api` rerun patterns matched only the raw httpx exception names. `pytest-rerunfailures` matches the crash line (the `NetworkError`), not the chained `ConnectTimeout`, so the transient was never retried and failed CI. Add `NetworkError` to `_TRANSIENT_RERUN_PATTERNS`. BREAKING CHANGE: Python 3.9 is no longer supported; `pip install dataretrieval>=1.2.0` requires Python >=3.10. The `variable_info` metadata property is removed (it always returned `None`); use `site_info` or `waterdata.get_monitoring_locations()`. Claude-Session: https://claude.ai/code/session_01Sjb14HkwuCydKSKMsaXsgd Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 566ef86 commit af9b238

14 files changed

Lines changed: 40 additions & 64 deletions

.github/workflows/python-package.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ jobs:
4949
fail-fast: false
5050
matrix:
5151
os: [ubuntu-latest, windows-latest]
52-
python-version: ["3.9", "3.13", "3.14"]
52+
python-version: ["3.10", "3.13", "3.14"]
5353

5454
steps:
5555
- uses: actions/checkout@v6

NEWS.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
**06/23/2026:** **Breaking change (1.2.0):** the minimum supported Python is now **3.10** (`requires-python = ">=3.10"`). 3.9 support was already effectively broken — the `waterdata` module's dependencies (`anyio`, the test stack) require 3.10+, and the `waterdata` test modules already skipped on <3.10. `anyio` is now declared as a direct dependency (it is imported directly by `waterdata`), and the CI/ruff/mypy targets move to 3.10. Also fully removed the deprecated `variable_info` metadata property: the `NWIS_Metadata` override only warned and returned `None` (it relied on the defunct `get_pmcodes`), and the `BaseMetadata` abstract is gone too since nothing implemented it — accessing `.variable_info` now raises `AttributeError`. `site_info` is unaffected.
2+
13
**06/23/2026:** **Breaking change (1.2.0):** removed the `nadp` module and the deprecated `samples` module ahead of the 1.2.0 release. `nadp` was deprecated on 05/01/2026 — NADP is not a USGS data source, so retrieve NADP data directly from https://nadp.slh.wisc.edu/. The `samples.get_usgs_samples` shim (a deprecated forward to the modern getter) is gone; use `waterdata.get_samples()` instead. `import dataretrieval.nadp` / `import dataretrieval.samples` now raise `ModuleNotFoundError`.
24

35
**06/03/2026:** The request-error hierarchy is now unified. Every module (`nwis`, `wqp`, `nldi`, `waterdata`, `nadp`, `streamstats`) raises a subclass of `dataretrieval.DataRetrievalError` on a failed request, so a single `except dataretrieval.DataRetrievalError` spans them all. An HTTP error status surfaces as an `HTTPError` carrying `.status_code` (inspect it to branch on a specific code); the retryable 429/5xx subset is `TransientError` (`RateLimited` / `ServiceUnavailable`, carrying `.retry_after`); and a request too large to satisfy is a `RequestTooLarge` (`URLTooLong` for an over-long single request, `Unchunkable` when the Water Data chunker cannot split a call small enough). Connection-level failures (timeouts, DNS, refused connections) are wrapped as a `NetworkError`, with the underlying `httpx` exception on `__cause__`. Every `DataRetrievalError` also exposes `.status_code` (`None` when there is no HTTP status), `.retry_after`, and `.retryable`, so a single `except dataretrieval.DataRetrievalError as e` clause can branch on the status or retry transient failures without knowing the concrete subclass. **Breaking change:** these exceptions no longer multiply-inherit a built-in — code that caught request failures with `except ValueError` or `except RuntimeError` should switch to `except dataretrieval.DataRetrievalError` (or a specific subclass). A no-data result is **not** an error: the modern getters (`waterdata`, `wqp`, `nldi`) return an empty DataFrame when nothing matches. Only the deprecated `nwis` (waterservices) path still raises `NoSitesError` on no data.

dataretrieval/nwis.py

Lines changed: 8 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1033,11 +1033,15 @@ def _read_json(json: dict[str, Any]) -> pd.DataFrame:
10331033
# for example, [0, 21, 22] would be the first and last indices
10341034
index_list = [0]
10351035
index_list.extend(
1036-
[i + 1 for i, (a, b) in enumerate(zip(site_list[:-1], site_list[1:])) if a != b]
1036+
[
1037+
i + 1
1038+
for i, (a, b) in enumerate(zip(site_list[:-1], site_list[1:], strict=False))
1039+
if a != b
1040+
]
10371041
)
10381042
index_list.append(len(site_list))
10391043

1040-
for start, end in zip(index_list[:-1], index_list[1:]):
1044+
for start, end in zip(index_list[:-1], index_list[1:], strict=False):
10411045
# grab a block containing timeseries 0:21,
10421046
# which are all from the same site
10431047
site_block = json["value"]["timeSeries"][start:end]
@@ -1133,8 +1137,8 @@ class NWIS_Metadata(BaseMetadata):
11331137
11341138
Notes
11351139
-----
1136-
``site_info`` and ``variable_info`` are exposed as properties (documented
1137-
below) rather than plain attributes.
1140+
``site_info`` is exposed as a property (documented below) rather than a
1141+
plain attribute.
11381142
11391143
"""
11401144

@@ -1196,17 +1200,3 @@ def site_info(self) -> tuple[pd.DataFrame, BaseMetadata] | None:
11961200

11971201
else:
11981202
return None # don't set metadata site_info attribute
1199-
1200-
@property
1201-
def variable_info(self) -> None:
1202-
"""
1203-
Deprecated. Accessing variable_info via NWIS_Metadata is deprecated.
1204-
Returns None.
1205-
"""
1206-
warnings.warn(
1207-
"Accessing variable_info via NWIS_Metadata is deprecated as "
1208-
"it relies on the defunct get_pmcodes function.",
1209-
DeprecationWarning,
1210-
stacklevel=2,
1211-
)
1212-
return None

dataretrieval/ogc/planning.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -497,7 +497,7 @@ def iter_sub_args(self) -> Iterator[dict[str, Any]]:
497497
chunk_lists = [self.chunks[ax.arg_key] for ax in self.axes]
498498
for combo in itertools.product(*chunk_lists):
499499
sub_args = dict(self.args)
500-
for axis, chunk in zip(self.axes, combo):
500+
for axis, chunk in zip(self.axes, combo, strict=False):
501501
sub_args[axis.arg_key] = axis.render(chunk)
502502
yield sub_args
503503

dataretrieval/utils.py

Lines changed: 3 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -252,19 +252,15 @@ def __init__(self, response: httpx.Response) -> None:
252252
# # disclaimer seems to be only part of importWaterML1
253253
# self.disclaimer = None
254254

255-
# These properties are to be set by `nwis` or `wqp`-specific metadata classes.
255+
# ``site_info`` is set by ``nwis`` / ``wqp``-specific metadata classes; the
256+
# modern ``waterdata`` metadata leaves it unimplemented (use
257+
# ``waterdata.get_monitoring_locations`` to retrieve site descriptions).
256258
@property
257259
def site_info(self) -> Any:
258260
raise NotImplementedError(
259261
"site_info must be implemented by utils.BaseMetadata children"
260262
)
261263

262-
@property
263-
def variable_info(self) -> Any:
264-
raise NotImplementedError(
265-
"variable_info must be implemented by utils.BaseMetadata children"
266-
)
267-
268264
def __repr__(self) -> str:
269265
return f"{type(self).__name__}(url={self.url})"
270266

dataretrieval/waterdata/nearest.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -197,7 +197,8 @@ def _build_window_or_filter(targets: pd.DatetimeIndex, window_td: pd.Timedelta)
197197
lowers = (targets - window_td).strftime(fmt)
198198
uppers = (targets + window_td).strftime(fmt)
199199
return " OR ".join(
200-
f"(time >= '{lo}' AND time <= '{up}')" for lo, up in zip(lowers, uppers)
200+
f"(time >= '{lo}' AND time <= '{up}')"
201+
for lo, up in zip(lowers, uppers, strict=False)
201202
)
202203

203204

pyproject.toml

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
66
name = "dataretrieval"
77
description = "Discover and retrieve water data from U.S. federal hydrologic web services."
88
readme = "README.md"
9-
requires-python = ">=3.9"
9+
requires-python = ">=3.10"
1010
keywords = ["USGS", "water data"]
1111
license = "CC0-1.0"
1212
license-files = ["LICENSE.md"]
@@ -17,11 +17,22 @@ maintainers = [
1717
{name = "Elise Hinman", email = "ehinman@usgs.gov"},
1818
]
1919
classifiers = [
20+
"Development Status :: 5 - Production/Stable",
21+
"Intended Audience :: Science/Research",
22+
"Topic :: Scientific/Engineering :: Hydrology",
2023
"Programming Language :: Python :: 3",
24+
"Programming Language :: Python :: 3.10",
25+
"Programming Language :: Python :: 3.11",
26+
"Programming Language :: Python :: 3.12",
27+
"Programming Language :: Python :: 3.13",
28+
"Programming Language :: Python :: 3.14",
2129
]
2230
dependencies = [
2331
"httpx",
2432
"pandas>=2.0.0,<4.0.0",
33+
# Directly imported by ``waterdata`` (``anyio.from_thread.start_blocking_portal``),
34+
# so declared here rather than relied on transitively via httpx.
35+
"anyio>=4.0",
2536
]
2637
dynamic = ["version"]
2738

@@ -35,7 +46,7 @@ dataretrieval = ["py.typed"]
3546
# Minimal set the CI ``type-check`` job installs — just mypy + the package,
3647
# not the whole test stack.
3748
type-check = [
38-
"mypy<2", # <2 so it can still target Python 3.9 (the project's floor)
49+
"mypy",
3950
]
4051
test = [
4152
"pytest > 5.0.0",
@@ -77,7 +88,7 @@ repository = "https://github.com/DOI-USGS/dataretrieval-python.git"
7788
write_to = "dataretrieval/_version.py"
7889

7990
[tool.ruff]
80-
target-version = "py39"
91+
target-version = "py310"
8192
extend-exclude = ["demos"]
8293

8394
[tool.ruff.lint]
@@ -115,7 +126,7 @@ docstring-code-line-length = 72
115126
# libraries (pandas, geopandas) are treated as ``Any`` instead of
116127
# requiring stub packages. Dropping that — via pandas-stubs/types-requests and
117128
# per-module overrides — can follow.
118-
python_version = "3.9" # the project's minimum supported version
129+
python_version = "3.10" # the project's minimum supported version
119130
files = ["dataretrieval"]
120131
strict = true
121132
ignore_missing_imports = true

tests/conftest.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,11 @@
2626
r"(?:QuotaExhausted|ServiceInterrupted):",
2727
r"Connect(ion)?Error", # requests' ConnectionError + httpx' ConnectError
2828
r"ReadTimeout|ConnectTimeout|Timeout",
29+
# ``dataretrieval`` wraps connection-level failures (timeout / DNS / refused)
30+
# in a typed ``NetworkError``; rerunfailures matches the crash line (the
31+
# ``NetworkError``), not the chained raw httpx exception, so match the
32+
# wrapper too -- otherwise a transient SSL/handshake timeout fails CI.
33+
r"NetworkError",
2934
]
3035

3136
#: Apply to a test module (``pytestmark = flaky_api``) or class (``@flaky_api``)

tests/ngwmn_test.py

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -6,14 +6,8 @@
66
behavior change still fails on the first run.
77
"""
88

9-
import sys
10-
11-
import pytest
129
from pandas import DataFrame
1310

14-
if sys.version_info < (3, 10):
15-
pytest.skip("Skip entire module on Python < 3.10", allow_module_level=True)
16-
1711
from dataretrieval import ngwmn
1812
from dataretrieval.utils import BaseMetadata
1913
from tests.conftest import flaky_api

tests/nwis_test.py

Lines changed: 1 addition & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -359,7 +359,7 @@ class TestMetaData:
359359
-----
360360
361361
- Originally based on GitHub Issue #73.
362-
- Modified to site_info and variable_info as properties, not callables.
362+
- Modified to expose site_info as a property, not a callable.
363363
"""
364364

365365
def test_set_metadata_info_site(self):
@@ -416,17 +416,6 @@ def test_set_metadata_info_countyCd(self):
416416
# assert that site_info is implemented
417417
assert md.site_info
418418

419-
def test_variable_info_deprecated(self):
420-
"""Test that variable_info raises a DeprecationWarning and returns None."""
421-
response = mock.MagicMock()
422-
md = NWIS_Metadata(response)
423-
with pytest.warns(
424-
DeprecationWarning,
425-
match="Accessing variable_info via NWIS_Metadata is deprecated",
426-
):
427-
result = md.variable_info
428-
assert result is None
429-
430419

431420
class TestReadRdb:
432421
"""Tests for the NWIS-specific _read_rdb wrapper.

0 commit comments

Comments
 (0)