Skip to content

Commit d8feb3e

Browse files
derek73claude
andcommitted
Reject given_name_titles entries that are not titles
Lexicon guarded two of its three marker-over-base relationships: particles_ambiguous <= particles and suffix_acronyms_ambiguous <= suffix_acronyms both raised ValueError, while given_name_titles <= titles was unchecked. An entry present only in given_name_titles is never consulted by any rule, so the misconfiguration did nothing and said nothing -- the silent-degradation failure mode #256 was deprecated for. Found while writing the locales.rst example: the doctest failed because the honorific was added to given_name_titles alone. Fold all three checks into one _SUBSET_FIELDS loop; the messages are unchanged. v1 compatibility is preserved by intersecting at the shim boundary. v1 treats titles and first_name_titles as independent sets and only consults first_name_titles for a word already recognized as a title, so an entry missing from titles was unreachable in v1 too -- dropping it changes no v1 behavior while satisfying the invariant. Verified against a live 1.4.0 with a word absent from the defaults: first_name_titles alone yields no title on both 1.4 and the 2.0 facade; both fields yield the title on both. Without this the guard broke six compat tests, since Constants(titles=[...]) replaces titles wholesale while first_name_titles keeps its defaults. (An earlier check of the same question used "sheikh", which is already in the default TITLES, so it proved nothing -- the retest with a novel word reversed the conclusion.) Docs: customize.rst states the subset rule for all three pairs, and its remove() example moves from "sir" to "professor" -- "sir" is itself a given_name_title, so removing it now correctly raises. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 3edfe4e commit d8feb3e

5 files changed

Lines changed: 56 additions & 19 deletions

File tree

docs/customize.rst

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,10 +29,17 @@ Removing works the same way, and drops the word from recognition:
2929

3030
.. doctest::
3131

32-
>>> lean = Lexicon.default().remove(titles={"sir"})
33-
>>> Parser(lexicon=lean).parse("Sir Robert Johns").title
32+
>>> lean = Lexicon.default().remove(titles={"professor"})
33+
>>> Parser(lexicon=lean).parse("Professor Robert Johns").title
3434
''
3535

36+
A few fields mark a subset of another — ``given_name_titles`` over
37+
``titles``, ``particles_ambiguous`` over ``particles``,
38+
``suffix_acronyms_ambiguous`` over ``suffix_acronyms``. Entries must
39+
appear in the base field too, so add to both and remove from the marker
40+
first; anything else raises ``ValueError`` naming the orphans rather
41+
than leaving a marker entry that no rule will ever consult.
42+
3643
Whole lexicons compose with ``|``, which unions field by field — handy
3744
for keeping a shared house vocabulary separate from a per-source one
3845
and combining them at parser construction:

docs/locales.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,8 +49,8 @@ Both fields, because ``given_name_titles`` is a marker over ``titles``
4949
rather than a separate vocabulary: ``titles`` makes the word a title at
5050
all, and listing it in ``given_name_titles`` says the honorific
5151
precedes the *given* name — as Arabic ones do — so the word after it
52-
isn't read as a family name. Adding it to ``given_name_titles`` alone
53-
has no effect.
52+
isn't read as a family name. Listing it in ``given_name_titles`` alone
53+
raises ``ValueError`` rather than quietly doing nothing.
5454

5555
A pack is for something different: a *structural* rule, like reordering
5656
a patronymic, that vocabulary alone can't express.

nameparser/_config_shim.py

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -933,11 +933,19 @@ def _build_snapshot(self) -> tuple[Lexicon, Policy, _RenderDefaults]:
933933
"""
934934
from nameparser.config.maiden_markers import MAIDEN_MARKERS
935935
acronyms = frozenset(self.suffix_acronyms)
936+
titles = frozenset(self.titles)
936937
# keep in sync with _lexicon._default_lexicon() (pinned by the
937938
# default-Constants equality test in tests/v2/test_config_shim.py)
938939
lexicon = Lexicon(
939-
titles=frozenset(self.titles),
940-
given_name_titles=frozenset(self.first_name_titles),
940+
titles=titles,
941+
# intersect, same reasoning as suffix_acronyms_ambiguous
942+
# below: v1 treats titles and first_name_titles as
943+
# independent sets, and only consults first_name_titles for
944+
# a word already recognized as a title -- so an entry
945+
# missing from titles was never reachable in v1 either, and
946+
# dropping it preserves v1 behavior exactly while
947+
# satisfying Lexicon's subset invariant
948+
given_name_titles=frozenset(self.first_name_titles) & titles,
941949
suffix_acronyms=acronyms,
942950
suffix_words=frozenset(self.suffix_not_acronyms),
943951
# intersect: Lexicon enforces ambiguous <= acronyms; v1

nameparser/_lexicon.py

Lines changed: 19 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,18 @@
2323
"conjunctions", "bound_given_names", "maiden_markers",
2424
)
2525

26+
#: (marker, base) pairs: each marker field narrows how entries of its
27+
#: base vocabulary are read, and carries no vocabulary of its own. An
28+
#: entry present in the marker but absent from the base is never
29+
#: consulted by any rule, so it is rejected rather than silently
30+
#: ignored -- a no-op configuration is the failure mode these fields
31+
#: are most likely to be misused into.
32+
_SUBSET_FIELDS = (
33+
("particles_ambiguous", "particles"),
34+
("suffix_acronyms_ambiguous", "suffix_acronyms"),
35+
("given_name_titles", "titles"),
36+
)
37+
2638

2739
def _normalize(word: str) -> str:
2840
"""Lowercase, strip whitespace and EDGE periods -- v1's lc()
@@ -215,19 +227,13 @@ def __post_init__(self) -> None:
215227
canonical = _normpairs(self.capitalization_exceptions)
216228
object.__setattr__(self, "capitalization_exceptions", canonical)
217229
object.__setattr__(self, "_cap_map", MappingProxyType(dict(canonical)))
218-
if not self.particles_ambiguous <= self.particles:
219-
extra = ", ".join(sorted(self.particles_ambiguous - self.particles))
220-
raise ValueError(
221-
f"particles_ambiguous must be a subset of particles; "
222-
f"not in particles: {extra}"
223-
)
224-
if not self.suffix_acronyms_ambiguous <= self.suffix_acronyms:
225-
extra = ", ".join(sorted(
226-
self.suffix_acronyms_ambiguous - self.suffix_acronyms))
227-
raise ValueError(
228-
f"suffix_acronyms_ambiguous must be a subset of "
229-
f"suffix_acronyms; not in suffix_acronyms: {extra}"
230-
)
230+
for marker, base in _SUBSET_FIELDS:
231+
orphans = getattr(self, marker) - getattr(self, base)
232+
if orphans:
233+
raise ValueError(
234+
f"{marker} must be a subset of {base}; "
235+
f"not in {base}: {', '.join(sorted(orphans))}"
236+
)
231237

232238
# -- constructors ----------------------------------------------------
233239

tests/v2/test_lexicon.py

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -227,3 +227,19 @@ def test_suffix_ambiguous_must_be_subset_of_acronyms() -> None:
227227
# depends on the membership tests agreeing about it
228228
with pytest.raises(ValueError, match="subset"):
229229
Lexicon(suffix_acronyms_ambiguous=frozenset({"ma"}))
230+
231+
232+
def test_given_name_titles_must_be_subset_of_titles() -> None:
233+
# third marker-over-base pair, same rule: given_name_titles marks
234+
# which TITLES precede the given name rather than the family name,
235+
# so an entry missing from titles is never consulted -- a silent
236+
# no-op is exactly the failure mode the other two guards prevent
237+
with pytest.raises(ValueError, match="subset"):
238+
Lexicon(given_name_titles=frozenset({"sheikh"}))
239+
240+
241+
def test_remove_orphaning_given_name_titles_raises() -> None:
242+
lex = Lexicon(titles=frozenset({"sheikh"}),
243+
given_name_titles=frozenset({"sheikh"}))
244+
with pytest.raises(ValueError, match="subset"):
245+
lex.remove(titles={"sheikh"})

0 commit comments

Comments
 (0)