Return latest paginated/chunked response so QuotaExhausted floor sees current quota

The ``multi_value_chunked`` decorator reads ``x-ratelimit-remaining``
from the response returned by ``fetch_once(sub_args)`` to honor its
documented ``QuotaExhausted`` safety floor. That response was two
layers stale:

1. ``_walk_pages`` captured ``initial_response = resp`` before
   pagination and returned it, so any sub-request with N > 1 pages
   bubbled up only the first page's headers — the loop already kept
   overwriting ``resp`` each iteration; we just weren't returning the
   latest.
2. ``_combine_chunk_responses`` returned ``responses[0]`` with summed
   ``elapsed``, so when ``filters.chunked`` fanned out a long
   OR-filter into N sub-chunks the outer wrapper only saw the first
   sub-chunk's headers.

Composed, the staleness gap per outer chunk was
``inner_chunks × pages_per_inner_chunk − 1`` HTTP requests of quota
consumption the chunker was blind to. For the canonical workload
(chained query, long site list, paginated filter) that gap easily
exceeds the default floor of 50, so the guard never tripped — users
hit ``RuntimeError("429: Too many requests...")`` from
``_raise_for_non_200`` instead of the structured ``QuotaExhausted``
with ``partial_frame``/``completed_chunks`` they were promised.

Fix both layers: ``_walk_pages`` returns the latest ``resp`` (which
the loop was already maintaining), and ``_combine_chunk_responses``
returns ``responses[-1]`` (with ``elapsed`` summed onto it instead
of onto ``responses[0]``). Both changes match
``QuotaExhausted.partial_response``'s docstring ("metadata for the
last successful sub-request"). Same fix applied to the parallel
pagination loop in the stats helper for consistency.

No behavior change for single-page mocked tests (initial == latest).
209 waterdata unit tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: thodson-usgs

dataretrieval/waterdata/filters.py

@@ -268,15 +268,20 @@ def _combine_chunk_frames(frames: list[pd.DataFrame]) -> pd.DataFrame:
 def _combine_chunk_responses(
     responses: list[requests.Response],
 ) -> requests.Response:
-    """Return one response: first chunk's URL/headers + summed ``elapsed``.
+    """Return one response: last chunk's URL/headers + summed ``elapsed``.
 
-    Mutates the first response in place (only ``elapsed``); downstream only
+    Returning the latest sub-response (rather than the first) preserves
+    current rate-limit headers (e.g. ``x-ratelimit-remaining``), which the
+    outer ``multi_value_chunked`` decorator inspects to honor its
+    ``QuotaExhausted`` safety floor between sub-requests.
+
+    Mutates the last response in place (only ``elapsed``); downstream only
     reads ``elapsed`` (in ``BaseMetadata.query_time``), URL, and headers.
     """
-    head = responses[0]
+    tail = responses[-1]
     if len(responses) > 1:
-        head.elapsed = sum((r.elapsed for r in responses[1:]), start=head.elapsed)
-    return head
+        tail.elapsed = sum((r.elapsed for r in responses[:-1]), start=tail.elapsed)
+    return tail
 
 
 _FetchOnce = TypeVar(

dataretrieval/waterdata/utils.py

@@ -643,7 +643,10 @@ def _walk_pages(
     pd.DataFrame
         A DataFrame containing the aggregated results from all pages.
     requests.Response
-        The initial response object containing metadata about the first request.
+        The latest response from the pagination walk. Returning the most
+        recent response (not the first) lets downstream callers observe
+        current rate-limit headers (e.g. ``x-ratelimit-remaining``) on
+        which the multi-value chunker's ``QuotaExhausted`` guard relies.
 
     Raises
     ------
@@ -666,9 +669,6 @@ def _walk_pages(
         resp = client.send(req)
         _raise_for_non_200(resp)
 
-        # Store the initial response for metadata
-        initial_response = resp
-
         # Grab some aspects of the original request: headers and the
         # request type (GET or POST)
         method = req.method.upper()
@@ -697,7 +697,7 @@ def _walk_pages(
                 curr_url = None
 
         # Concatenate all pages at once for efficiency
-        return pd.concat(dfs, ignore_index=True), initial_response
+        return pd.concat(dfs, ignore_index=True), resp
     finally:
         if close_client:
             client.close()
@@ -1128,9 +1128,6 @@ def get_stats_data(
         resp = client.send(req)
         _raise_for_non_200(resp)
 
-        # Store the initial response for metadata
-        initial_response = resp
-
         # Grab some aspects of the original request: headers and the
         # request type (GET or POST)
         method = req.method.upper()
@@ -1173,7 +1170,7 @@ def get_stats_data(
         if expand_percentiles:
             dfs = _expand_percentiles(dfs)
 
-        return dfs, BaseMetadata(initial_response)
+        return dfs, BaseMetadata(resp)
     finally:
         if close_client:
             client.close()