Skip to content

Commit de264af

Browse files
derek73claude
andcommitted
Only report the particle fork when the chain actually claims something
Third instance of the same mistake in this emitter, and the same root cause both previous times: keyed on a SHAPE rather than the DECISION. When the piece after an ambiguous particle is a suffix, the chain's inner scan never advances, so merge(k, k+1) folds a piece into itself. Nothing is chained and the particle stays a lone leading piece -- but the record was appended before the merge, keyed only on "is a prefix" plus the ambiguous tag. Result: 'Dr. Van Jr.' -> given='Van' AMB "'Van' was chained into the family name" The detail asserted the opposite of what happened, for all 39 particles_ambiguous members. For do/freiherr/st the token lands in the TITLE instead. Worse, the no-op leaves exactly the shape _assign triggers on, so 36 of 39 double-reported the same token from two stages. `j > k + 1` is the discriminator -- it distinguishes a chain that claimed a following piece from one that claimed nothing -- and it fixes both defects, since the no-op was the only overlap between the two emitters' domains. They now partition cleanly. Also fixed the detail's wording. _group runs before assignment, so it cannot know which field the chained piece lands in: "Dr. Van Johnson de la Cruz" puts it in GIVEN, not family. This is the same defect the role-readback cured in _assign one commit earlier -- the lesson did not carry to the new emitter, and _group cannot read a role back, so the text now describes the decision rather than guessing an outcome. A "Dr. Van Jr." case row would have caught both criticals; 1539 tests passed with them present because the only titled-particle test used an UNAMBIGUOUS particle, pinning vocabulary instead of the decision. Verified: 0 spurious reports across all 39 ambiguous particles; corpus 486 names, 2 intentional diffs, 0 unexplained. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 2ff9189 commit de264af

3 files changed

Lines changed: 45 additions & 3 deletions

File tree

nameparser/_pipeline/_group.py

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -174,7 +174,14 @@ def merge(lo: int, hi: int, add: Set[str] = frozenset(),
174174
j += 1
175175
while j < len(pieces) and not prefix(j) and not suffix(j):
176176
j += 1
177-
if (all(title(x) for x in range(k))
177+
# j > k + 1 is what makes this a DECISION rather than a
178+
# shape: when the next piece is a suffix the inner scan
179+
# never advances, merge(k, k+1) folds a piece into itself,
180+
# and the particle stays a lone leading piece -- nothing
181+
# was chained, and _assign reports that case instead.
182+
# Without this the two emitters both fire on the same token.
183+
if (j > k + 1
184+
and all(title(x) for x in range(k))
178185
and "vocab:particle-ambiguous"
179186
in tokens[pieces[k][0]].tags):
180187
particle_forks.append(pieces[k][0])
@@ -295,8 +302,9 @@ def group(state: ParseState) -> ParseState:
295302
for i in particle_forks:
296303
ambiguities.append(PendingAmbiguity(
297304
AmbiguityKind.PARTICLE_OR_GIVEN,
298-
f"{tokens[i].text!r} was chained into the family "
299-
f"name; it is also a given name in other names",
305+
f"{tokens[i].text!r} was chained onto the following "
306+
f"name piece; it is also a given name in other "
307+
f"names",
300308
(i,)))
301309
return dataclasses.replace(
302310
state, tokens=tuple(tokens), pieces=tuple(all_pieces),

tests/v2/cases.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,13 @@ def __post_init__(self) -> None:
115115
"shifts Van off the given position, the prefix chain "
116116
"fires, and the fork is reported from group rather than "
117117
"assign (v1 parity on the fields)"),
118+
Case("titled_ambiguous_particle_no_op_chain", "Dr. Van Jr.",
119+
{"title": "Dr.", "given": "Van", "suffix": "Jr."},
120+
notes="the piece after the particle is a suffix, so the chain "
121+
"scan never advances and the merge is a no-op -- nothing "
122+
"was chained, so there is no fork to report (the emitter "
123+
"fired here for all 39 ambiguous particles, and _assign "
124+
"double-reported the same token)"),
118125
Case("initial_shaped_not_conjunction", "john e. smith",
119126
{"given": "john", "middle": "e.", "family": "smith"},
120127
notes="v1 is_conjunction excludes initials at classify too"),

tests/v2/test_parser.py

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -271,3 +271,30 @@ def test_ambiguous_particle_reports_both_branches_of_its_fork() -> None:
271271
def test_unambiguous_particle_chain_reports_nothing() -> None:
272272
# 'de' is never a given name, so merging it is not a fork
273273
assert parse("Dr. de la Vega").ambiguities == ()
274+
275+
276+
@pytest.mark.parametrize("text", [
277+
"Dr. Van Jr.", # the piece after the particle is a suffix, so
278+
"Dr. Van MD", # the chain scan never advances and merge() is
279+
"Dr. Do Jr.", # a no-op -- nothing was chained, no fork taken
280+
])
281+
def test_no_op_prefix_chain_is_not_a_fork(text: str) -> None:
282+
assert parse(text).ambiguities == ()
283+
284+
285+
def test_a_fork_is_reported_by_exactly_one_stage() -> None:
286+
# the no-op merge left the particle a lone leading piece, which is
287+
# _assign's trigger, so both stages reported the same token
288+
n = parse("Dr. Van Jr Smith")
289+
assert n.given == "Van"
290+
assert len(n.ambiguities) == 1
291+
292+
293+
def test_chained_particle_detail_does_not_claim_a_role() -> None:
294+
# _group runs before assignment, so it cannot know which field the
295+
# chained piece lands in -- "Dr. Van Johnson de la Cruz" puts it in
296+
# GIVEN. The detail must describe the decision, not guess a role.
297+
n = parse("Dr. Van Johnson de la Cruz")
298+
assert n.given == "Van Johnson"
299+
(amb,) = n.ambiguities
300+
assert "family name" not in amb.detail

0 commit comments

Comments
 (0)