Auto-retry live-API tests on transient upstream errors (DOI-USGS#277)

DOI-USGS#273 surfaced a transient HTTP 502 from upstream on its merge-to-main
CI run that would have been silently swallowed by
_walk_pages and turned into an empty DataFrame. The status-code-aware
behavior is correct for users, but it makes every test that hits the
live USGS Water Data API susceptible to flaking on a transient blip.

This PR adds:

- pytest-rerunfailures to the `test` optional dependency set.
- tests/conftest.py that (a) registers a `live` pytest marker; (b)
  auto-applies it to every test that does not take `requests_mock` as
  a fixture (the existing mock-driven convention in this repo); and
  (c) configures live-marked tests to retry up to twice on a 5-second
  backoff — but ONLY when the failure trace matches one of a narrow
  set of transient-upstream patterns: `429:` / `5xx:` prefixes from
  `_raise_for_non_200`, `ConnectionError` shapes, and timeout strings
  from the requests/urllib3 stack.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: thodson-usgs

.github/workflows/python-package.yml

@@ -13,9 +13,9 @@ jobs:
   lint:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@v6
       - name: Set up Python 3.14
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@v6
         with:
           python-version: "3.14"
           cache: "pip"
@@ -36,9 +36,9 @@ jobs:
         python-version: ["3.9", "3.13", "3.14"]
 
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@v6
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@v6
         with:
           python-version: ${{ matrix.python-version }}
           cache: "pip"

.github/workflows/python-publish.yml

@@ -21,9 +21,9 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-    - uses: actions/checkout@v4
+    - uses: actions/checkout@v6
     - name: Set up Python
-      uses: actions/setup-python@v5
+      uses: actions/setup-python@v6
       with:
         python-version: '3.x'
         cache: 'pip'

.github/workflows/sphinx-docs.yml

@@ -11,11 +11,11 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
-        uses: actions/checkout@v4
+        uses: actions/checkout@v6
         with:
           persist-credentials: false
       - name: Set up Python
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@v6
         with:
           python-version: "3.13"
           cache: "pip"

pyproject.toml

@@ -35,6 +35,7 @@ dataretrieval = ["py.typed"]
 test = [
   "pytest > 5.0.0",
   "pytest-cov[all]",
+  "pytest-rerunfailures",
   "coverage",
   "requests-mock",
   "ruff",

tests/waterdata_test.py

@@ -33,6 +33,26 @@
     _normalize_str_iterable,
 )
 
+# Most tests in this module call the live USGS Water Data API. After
+# PR #273, transient upstream errors (5xx / 429 / connection drops)
+# propagate instead of silently truncating, which makes CI susceptible
+# to flaking on a brief upstream blip. Auto-retry such failures, but
+# only for the narrow set of transient-error trace patterns below —
+# library bugs raising other exception types still fail on the first
+# try. The marker is attached to every test in the module, but the
+# patterns match only traces produced by real network round-trips
+# (``_raise_for_non_200`` output, ``requests`` exceptions), so tests
+# using ``requests_mock`` or ``mock.patch`` are no-ops for the rerun.
+pytestmark = pytest.mark.flaky(
+    reruns=2,
+    reruns_delay=5,
+    only_rerun=[
+        r"RuntimeError:\s*(?:429|5\d\d):",  # _raise_for_non_200 output
+        r"ConnectionError",
+        r"ReadTimeout|ConnectTimeout|Timeout",
+    ],
+)
+
 
 def mock_request(requests_mock, request_url, file_path):
     """Mock request code"""