Commit 551aa5e
Isolate filter/chunking code in a dedicated filters.py module
Move everything ``filter`` / ``filter_lang`` / CQL-chunking-related out
of ``utils.py`` into a new ``dataretrieval/waterdata/filters.py``:
- ``FILTER_LANG`` type alias (previously in ``types.py``)
- Constants: ``_CQL_FILTER_CHUNK_LEN``, ``_WATERDATA_URL_BYTE_LIMIT``,
``_NON_FILTER_URL_HEADROOM``
- Regexes: ``_NUM`` / ``_IDENT`` / ``_OP`` subpatterns,
``_NUMERIC_COMPARE_RE``, ``_IN_NUMERIC_RE``, ``_BETWEEN_NUMERIC_RE``
- Splitter: ``_iter_or_boundaries``, ``_split_top_level_or``
- Chunker: ``_chunk_cql_or``
- Per-request URL budget: ``_effective_filter_budget``
- Lexicographic-pitfall guard: ``_check_numeric_filter_pitfall``,
``_raise_pitfall``
- Fan-out orchestration: ``_plan_filter_chunks``, ``_fetch_chunks``,
``_combine_chunk_frames``, ``_aggregate_response_metadata``
``utils.get_ogc_data`` now calls one entry point —
``filters.fetch_combined(args, build_request=..., walk_pages=...)`` —
which returns ``(combined_df, metadata)``. The ``build_request`` /
``walk_pages`` callables are injected so ``filters.py`` doesn't
depend on ``utils.py`` (breaks the potential circular import).
Tests that exercise filter helpers move to a new
``tests/waterdata_filters_test.py``; ``tests/waterdata_utils_test.py``
keeps only the handful of non-filter tests (``_get_args``, the
``_walk_pages`` mock suite).
The motivation is isolation: if the filter feature is rolled back
later, the rollback is:
1. Delete ``dataretrieval/waterdata/filters.py`` and
``tests/waterdata_filters_test.py``.
2. Replace the single ``filters.fetch_combined(...)`` call in
``utils.get_ogc_data`` with an inline
``_construct_api_requests(**args)`` + ``_walk_pages(...)``
(~3 lines).
3. Remove ``FILTER_LANG`` from ``waterdata/__init__.py`` (one
line) and ``api.py`` (one import, plus the eight ``filter`` /
``filter_lang`` kwargs on the OGC getters).
No behaviour change. All 162 non-live tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent 9314d34 commit 551aa5e
7 files changed
Lines changed: 1005 additions & 905 deletions
File tree
- dataretrieval/waterdata
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| 29 | + | |
29 | 30 | | |
30 | 31 | | |
31 | | - | |
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
| 19 | + | |
19 | 20 | | |
20 | 21 | | |
21 | | - | |
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| |||
0 commit comments