Skip to content

Commit 86ac286

Browse files
thodson-usgsclaude
andcommitted
Limit live-test retry scope to waterdata module
PR DOI-USGS#273 fixed pagination loops in dataretrieval.waterdata.utils specifically; other modules (wqp, samples, nadp, streamstats, nldi, nwis) reach the network through different paths and don't share that regression. Narrow _PUBLIC_GETTERS to waterdata's 16 user-facing getters and add a tests/waterdata file-path scope check, which also prevents name-collisions with legacy nwis getters (get_ratings, get_daily) from spuriously marking nwis tests. After this change, the retry marker is applied to 55 tests across tests/waterdata_test.py, tests/waterdata_nearest_test.py, tests/waterdata_filters_test.py, and tests/waterdata_ratings_test.py, with zero matches outside that directory. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 6f41cb3 commit 86ac286

1 file changed

Lines changed: 26 additions & 45 deletions

File tree

tests/conftest.py

Lines changed: 26 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,25 @@
66
Water Data API susceptible to flaking on a transient upstream blip
77
(e.g. the HTTP 502 Bad Gateway that broke CI on PR #273's merge).
88
9-
Selection: a test is "live" iff (a) it does not request the
10-
``requests_mock`` fixture, AND (b) its function body references one of
11-
the package's public user-facing getters (the ``_PUBLIC_GETTERS`` set).
12-
This skips pure unit tests of internal helpers (``_get_args``,
13-
``_check_*``, ``_normalize_*``, ``_construct_api_requests``) that
14-
share the test file with live tests.
9+
Scope: ``waterdata`` only. PR #273 specifically fixed the pagination
10+
loops in ``dataretrieval.waterdata.utils`` (``_walk_pages`` /
11+
``get_stats_data``); other modules (``wqp``, ``samples``, ``nadp``,
12+
``streamstats``, ``nldi``, ``nwis``) reach the network through
13+
different paths with their own error semantics and don't share PR
14+
#273's regression.
15+
16+
Selection: a test is "live" iff (a) its file path is under
17+
``tests/waterdata`` (other modules' tests are out of scope), AND (b)
18+
it does not request the ``requests_mock`` fixture, AND (c) its
19+
function body references one of the waterdata public user-facing
20+
getters (the ``_PUBLIC_GETTERS`` set). The file-path scope is
21+
necessary because some getter names (``get_ratings``, ``get_daily``)
22+
collide with legacy ``nwis`` function names; without the scope, a
23+
test of the legacy ``nwis.get_ratings`` would be retried under
24+
waterdata's transient-error patterns. The function-body check skips
25+
pure unit tests of internal helpers (``_get_args``, ``_check_*``,
26+
``_normalize_*``, ``_construct_api_requests``) that share the test
27+
file with live tests.
1528
1629
Retry: live tests are retried up to twice on a 5-second backoff,
1730
but only when the failure trace matches a narrow transient-upstream
@@ -24,12 +37,11 @@
2437

2538
import pytest
2639

27-
# Public, network-doing entry points. The set is intentionally exhaustive
28-
# so the source-inspection heuristic catches every test that exercises
29-
# the library's user-facing API surface. Add new public getters here.
40+
# Public, network-doing entry points of the waterdata module.
41+
# Other modules' getters are intentionally out of scope (see module
42+
# docstring). Add new waterdata getters here.
3043
_PUBLIC_GETTERS = frozenset(
3144
{
32-
# waterdata
3345
"get_channel",
3446
"get_combined_metadata",
3547
"get_continuous",
@@ -46,41 +58,6 @@
4658
"get_time_series_metadata",
4759
"get_stats_por",
4860
"get_stats_date_range",
49-
# wqp / samples
50-
"get_results",
51-
"get_samples",
52-
"get_samples_summary",
53-
"what_activities",
54-
"what_activity_metrics",
55-
"what_detection_limits",
56-
"what_habitat_metrics",
57-
"what_organizations",
58-
"what_project_weights",
59-
"what_projects",
60-
"what_sites",
61-
# nadp
62-
"get_annual_MDN_map",
63-
"get_annual_NTN_map",
64-
"get_zip",
65-
# streamstats / nldi
66-
"get_basin",
67-
"get_features",
68-
"get_features_by_data_source",
69-
"get_flowlines",
70-
"get_sample_watershed",
71-
"get_watershed",
72-
# nwis (legacy)
73-
"get_discharge_measurements",
74-
"get_discharge_peaks",
75-
"get_dv",
76-
"get_gwlevels",
77-
"get_info",
78-
"get_iv",
79-
"get_pmcodes",
80-
"get_qwdata",
81-
"get_record",
82-
"get_stats",
83-
"get_water_use",
8461
}
8562
)
8663

@@ -102,6 +79,10 @@
10279

10380
def pytest_collection_modifyitems(config, items):
10481
for item in items:
82+
# Scope: waterdata test files only. See module docstring for why
83+
# this file-path filter is needed in addition to the name check.
84+
if "tests/waterdata" not in item.nodeid.replace("\\", "/"):
85+
continue
10586
if "requests_mock" in item.fixturenames:
10687
continue
10788
try:

0 commit comments

Comments
 (0)