Skip to content

Commit 08b979d

Browse files
derek73claude
andcommitted
Record what the in-place merge requires, and a second empty-tokens case
Two documentation defects from the aliasing review, which otherwise cleared the merge rewrite: ~1.1M instrumented parses with no assertion firing, and byte-identical output against the pre-fix spelling over 8,036 names x 16 configs (all three name orders, middle_as_family, both patronymic rules, custom delimiters, both locale packs, the HumanName facade and the Constants shim) plus 120,375 names x 8 policies of targeted chain fuzz. merge() now REQUIRES lo < hi, where the old spelling did not. Every call site passes it, but with lo >= hi the slice assignment would insert rather than replace, putting a second reference to pieces[lo] into the array so the next merge extends the same list twice. Stated at the call boundary rather than guarded: a silent no-op return would hide a caller bug, and the constraint only matters to someone adding a call site. AmbiguityKind.UNBALANCED_DELIMITER called an empty token tuple a "rare exception" for a character inside a masked region. Since content-free input started keeping its diagnostics there is a second source, and it covers the canonical example -- parse("(") reports with tokens=() because no token survived. The docstring now admits both. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 0ae0b70 commit 08b979d

2 files changed

Lines changed: 14 additions & 3 deletions

File tree

nameparser/_pipeline/_group.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,13 @@ def merge(lo: int, hi: int, add: Set[str] = frozenset(),
129129
# pieces[k] before a merge), so mutating is safe; verified
130130
# identical token/role/tag/span/ambiguity output over 54,877
131131
# names. tests/v2/test_benchmark.py's "and " shape is the guard.
132+
#
133+
# Every call site passes lo < hi, and this REQUIRES it: with
134+
# lo >= hi the slice assignment would insert rather than
135+
# replace, putting a second reference to pieces[lo] into the
136+
# array, and the next merge to touch either index would extend
137+
# the same list twice. The old rebuild-a-fresh-list spelling
138+
# was harmless there. Keep the bound if you add a caller.
132139
combined = pieces[lo]
133140
for piece in pieces[lo + 1:hi]:
134141
combined.extend(piece)

nameparser/_types.py

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -259,9 +259,13 @@ class AmbiguityKind(StrEnum):
259259
PARTICLE_OR_GIVEN = "particle-or-given"
260260
#: A nickname/maiden delimiter opened without closing (or closed
261261
#: without opening); the text was kept as literal name content, so
262-
#: the tokens are the one the stray character ended up inside. Rare
263-
#: exception: a character that lands in no token at all (inside a
264-
#: masked region) leaves the tuple empty.
262+
#: the tokens are the one the stray character ended up inside.
263+
#: Two cases leave that tuple empty: a character that lands in no
264+
#: token at all (inside a masked region), and an input with no
265+
#: alphanumeric content anywhere, which parses to an empty name --
266+
#: the report survives because "was this malformed?" is the only
267+
#: question left, but there is no token for it to point at.
268+
#: ``parse("(")`` is the second case, not an exotic one.
265269
UNBALANCED_DELIMITER = "unbalanced-delimiter"
266270
#: More comma-separated segments than any recognized name shape;
267271
#: the parse is best-effort over the extra segments.

0 commit comments

Comments
 (0)