docs: omit parallel_chunks NEWS entry; refresh README benchmark with verified numbers

thodson-usgs · claude · thodson-usgs · commit 8ae1b9a4249a · 2026-07-17T13:15:48.000-05:00
- Remove the parallel_chunks NEWS.md entry (per maintainer request).
- Re-ran the README benchmark against the live API with a hardened,
  cache-cold methodology: a fixed 271-site subset of Ohio stream gages,
  fixed page size (limit=250), each n on its own distinct 1-year window so no
  run is served from the server's data-window cache (off on 2003/2005, the
  fanned runs on 2007/2009/2011 — no shared cache keys).

  The multi-fold speedup is real but the earlier figures were optimistic:
  off 9.3s -&gt; n=8 ~2.0s (~4.5x) -&gt; n=32 1.2s (~8x), vs the previously stated
  ~6x/~12x. Page counts (~30/32/54) match. Noted that the multiplier scales
  with page count. The old note said "271 Ohio discharge sites", but
  state="Ohio",site_type_code="ST" now returns 2889, so the benchmark is
  pinned to a reproducible 271-site subset.

Signed-off-by: thodson-usgs &lt;thodson@usgs.gov&gt;

Co-Authored-By: Claude Opus 4.8 (1M context) &lt;noreply@anthropic.com&gt;
diff --git a/NEWS.md b/NEWS.md
@@ -1,5 +1,3 @@
-**07/01/2026:** Added `waterdata.parallel_chunks(n)` — a context manager to control how finely the OGC `waterdata` getters split multi-value requests. By default the getters chunk only as much as the server's ~8 KB URL limit forces (the fewest sub-requests). But because every sub-request paginates, splitting a large result further costs little or no extra quota when each sub-request still spans many pages, so a caller who *knows* their pull is large can opt into a finer split for smoother progress, more even concurrency, and a smaller unit of retry/resume: `with waterdata.parallel_chunks(32): df, md = waterdata.get_daily(monitoring_location_id=many_sites)`. `n` is a positive integer (e.g. `2`, `8`, `32`) — the number of sub-requests to fan the call out into; a non-integer or non-positive value raises `ValueError` at the `with`. It caps the *total* sub-request count across every multi-value argument combined (not per argument), bounded below by what the byte limit already forces and above by how many values there are to split, so several multi-value arguments can't multiply past it and `n=1` asks for no extra fan-out. Each sub-request costs a request against your hourly rate limit, and because how many run *at once* is capped separately by `API_USGS_CONCURRENT` (default 32) an `n` beyond that adds quota without adding parallelism, so the useful range is roughly `2` up to `API_USGS_CONCURRENT`. It is deliberately a scoped `with` block rather than an automatic behavior or an environment variable, since the library can't tell in advance whether a query is large — a short-window query might fit in a single page, where extra chunks would only burn quota. Also exported at the top level as `dataretrieval.parallel_chunks`.
-
 **06/23/2026:** **Breaking change (1.2.0):** the minimum supported Python is now **3.10** (`requires-python = ">=3.10"`). 3.9 support was already effectively broken — the `waterdata` module's dependencies (`anyio`, the test stack) require 3.10+, and the `waterdata` test modules already skipped on <3.10. `anyio` is now declared as a direct dependency (it is imported directly by `waterdata`), and the CI/ruff/mypy targets move to 3.10. Also fully removed the deprecated `variable_info` metadata property: the `NWIS_Metadata` override only warned and returned `None` (it relied on the defunct `get_pmcodes`), and the `BaseMetadata` abstract is gone too since nothing implemented it — accessing `.variable_info` now raises `AttributeError`. `site_info` is unaffected.
 
 **06/23/2026:** **Breaking change (1.2.0):** removed the `nadp` module and the deprecated `samples` module ahead of the 1.2.0 release. `nadp` was deprecated on 05/01/2026 — NADP is not a USGS data source, so retrieve NADP data directly from https://nadp.slh.wisc.edu/. The `samples.get_usgs_samples` shim (a deprecated forward to the modern getter) is gone; use `waterdata.get_samples()` instead. `import dataretrieval.nadp` / `import dataretrieval.samples` now raise `ModuleNotFoundError`.
diff --git a/README.md b/README.md
@@ -139,21 +139,23 @@ your hourly [rate limit](https://api.waterdata.usgs.gov/signup/); since how many
 run *at once* is capped separately by `API_USGS_CONCURRENT` (default 32), the
 useful range is roughly `2` up to that value.
 
-Benchmark — 271 Ohio discharge sites (`get_daily`, `parameter_code="00060"`),
-cold cache, each `n` run against its own time window so results are not
-cache-served, with the page size fixed so every run fetches roughly the same
-number of pages (isolating the effect of parallelism):
-
-| `n`  | parallelism | pages | wall-clock                | speedup |
-| ---- | ----------- | ----- | ------------------------- | ------- |
-| off  | 1           | 32    | 23.9 s / 27.4 s (2 runs)  | 1×      |
-| `8`  | 8           | 37    | 4.1 s / 4.3 s             | ~6×     |
-| `32` | 32          | 44    | 2.0 s                     | ~12×    |
+Benchmark — a fixed 271-site subset of Ohio stream gages
+(`get_daily`, `parameter_code="00060"`), with a small fixed page size
+(`limit=250`) so every run fetches roughly the same number of pages (isolating
+the effect of parallelism). Each `n` was run against its own cold 1-year time
+window so no run is served from the server's data-window cache:
+
+| `n`  | parallelism | pages | wall-clock              | speedup |
+| ---- | ----------- | ----- | ----------------------- | ------- |
+| off  | 1           | ~30   | 9.5 s / 9.1 s (2 runs)  | 1×      |
+| `8`  | 8           | ~32   | 2.2 s / 1.9 s           | ~4.5×   |
+| `32` | 32          | 54    | 1.2 s                   | ~8×     |
 
 The gain comes from overlapping each sub-request's per-page latency and
-server-side work — a genuinely large pull at the default page size shows a
-similar multi-fold speedup. The extra sub-requests each cost quota, so reserve a
-large `n` for pulls you know are large.
+server-side work, so the exact multiplier scales with how many pages the pull
+spans — a larger pull (more pages) has more parallelism to exploit. The extra
+sub-requests each cost quota, so reserve a large `n` for pulls you know are
+large.
 
 Visit the
 [API Reference](https://doi-usgs.github.io/dataretrieval-python/reference/waterdata.html)

Original file line number	Diff line number	Diff line change
`@@ -1,5 +1,3 @@`
`1`		-07/01/2026: Added `waterdata.parallel_chunks(n)` — a context manager to control how finely the OGC `waterdata` getters split multi-value requests. By default the getters chunk only as much as the server's ~8 KB URL limit forces (the fewest sub-requests). But because every sub-request paginates, splitting a large result further costs little or no extra quota when each sub-request still spans many pages, so a caller who knows their pull is large can opt into a finer split for smoother progress, more even concurrency, and a smaller unit of retry/resume: `with waterdata.parallel_chunks(32): df, md = waterdata.get_daily(monitoring_location_id=many_sites)`. `n` is a positive integer (e.g. `2`, `8`, `32`) — the number of sub-requests to fan the call out into; a non-integer or non-positive value raises `ValueError` at the `with`. It caps the total sub-request count across every multi-value argument combined (not per argument), bounded below by what the byte limit already forces and above by how many values there are to split, so several multi-value arguments can't multiply past it and `n=1` asks for no extra fan-out. Each sub-request costs a request against your hourly rate limit, and because how many run at once is capped separately by `API_USGS_CONCURRENT` (default 32) an `n` beyond that adds quota without adding parallelism, so the useful range is roughly `2` up to `API_USGS_CONCURRENT`. It is deliberately a scoped `with` block rather than an automatic behavior or an environment variable, since the library can't tell in advance whether a query is large — a short-window query might fit in a single page, where extra chunks would only burn quota. Also exported at the top level as `dataretrieval.parallel_chunks`.
`2`		`-`
`3`	`1`	06/23/2026: Breaking change (1.2.0): the minimum supported Python is now 3.10 (`requires-python = ">=3.10"`). 3.9 support was already effectively broken — the `waterdata` module's dependencies (`anyio`, the test stack) require 3.10+, and the `waterdata` test modules already skipped on <3.10. `anyio` is now declared as a direct dependency (it is imported directly by `waterdata`), and the CI/ruff/mypy targets move to 3.10. Also fully removed the deprecated `variable_info` metadata property: the `NWIS_Metadata` override only warned and returned `None` (it relied on the defunct `get_pmcodes`), and the `BaseMetadata` abstract is gone too since nothing implemented it — accessing `.variable_info` now raises `AttributeError`. `site_info` is unaffected.
`4`	`2`
`5`	`3`	06/23/2026: Breaking change (1.2.0): removed the `nadp` module and the deprecated `samples` module ahead of the 1.2.0 release. `nadp` was deprecated on 05/01/2026 — NADP is not a USGS data source, so retrieve NADP data directly from https://nadp.slh.wisc.edu/. The `samples.get_usgs_samples` shim (a deprecated forward to the modern getter) is gone; use `waterdata.get_samples()` instead. `import dataretrieval.nadp` / `import dataretrieval.samples` now raise `ModuleNotFoundError`.