Skip to content

Commit 995bb40

Browse files
derek73claude
andcommitted
Sweep docs for the relocated script table
Closing sweep for the _SCRIPT_RANGES move to _policy.py. Two stale spots, both corrected: - nameparser/_policy.py: _script_matcher's docstring claimed the closure keeps derived matchers out of test_regex_sync.py's hand-copy completeness sweep as if that sweep scanned everywhere; it scans a fixed set of pipeline modules (_assign, _post_rules, _render, _tokenize, _vocab), which covers _vocab but never _policy or the packs. Reworded to claim only what holds: the closure spares _vocab's derived matchers a declaration row in that sweep. - docs/locales.rst: the contributing contract told authors to declare DEVIATES but never said how a range-declaring pack builds one, while test_locales.py now reads any module-level re.Pattern as a marker regex needing branch coverage and fails a pack importing re without one. Added the mechanism: build the predicate with _policy's _script_matcher, the way zh and ja do, never by compiling private character ranges. Verified rather than edited: AGENTS.md's test_regex_sync.py line ("wherever the copy lives -- including copies outside the package") stays true because the differential toml copy in tools/ remains and is still pinned; its layering line already lists _policy among allowed pack imports. Acceptance greps: 0x4E00 appears in the package only in _policy.py's table; the dead symbols (_HAN_RANGES, _JA_RANGES, _in_repertoire, _SCRIPT_PATTERNS, _JA_PATTERN) are gone from nameparser/, tests/, and tools/; no pack references _vocab. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent c78adde commit 995bb40

2 files changed

Lines changed: 12 additions & 4 deletions

File tree

docs/locales.rst

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -280,7 +280,14 @@ by ``tests/v2/test_locales.py``:
280280
#. Declare a module-level ``DEVIATES(name)`` predicate: given a name
281281
string, return whether *this pack alone* might parse it differently
282282
from the default parser. Over-declaring is safe; under-declaring is
283-
not — when in doubt, ``DEVIATES`` should say yes.
283+
not — when in doubt, ``DEVIATES`` should say yes. A pack whose
284+
scope is a *script* builds the predicate with ``_script_matcher``
285+
from ``nameparser/_policy.py``, the way ``zh`` and ``ja`` do —
286+
never by compiling its own character ranges: the table in
287+
``_policy.py`` is the single source for script codepoint spans, and
288+
the contract test reads any module-level ``re.Pattern`` in a pack
289+
as a marker regex needing branch coverage (a pack that imports
290+
``re`` without exposing one fails outright).
284291
#. Add a rotator list to ``tests/v2/test_locales.py``. Every pack needs
285292
one, but what it has to contain follows from how the pack declares
286293
its scope. A pack declaring by *marker regex* (``ru``, ``tr_az``)

nameparser/_policy.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -155,9 +155,10 @@ def _script_matcher(*scripts: Script,
155155
so much as IMPORTS re without exposing such a pattern fails
156156
"imports re but exposes no module-level pattern" -- so a pack must
157157
not import re at all; predicates built here keep the packs
158-
invisible to that sweep by construction (and keep derived
159-
matchers out of tests/v2/test_regex_sync.py's hand-copy
160-
completeness sweep, which scans for module-level patterns)."""
158+
invisible to that sweep by construction (and spare _vocab's
159+
derived matchers a declaration row in tests/v2/test_regex_sync.py's
160+
completeness sweep, which scans the pipeline modules for
161+
module-level patterns)."""
161162
if not scripts:
162163
raise ValueError("_script_matcher needs at least one Script")
163164
cls = "".join(f"\\U{lo:08x}-\\U{hi:08x}"

0 commit comments

Comments
 (0)