Skip to content

Commit 3f0b754

Browse files
derek73claude
andcommitted
A word-final apostrophe is not an unbalanced delimiter
The close sweep added in d993db0 flagged "Mari' Aube'". It should not: an ambiguity means the part could REASONABLY read two ways where it sits, and a "'" after a word character cannot -- it is an apostrophe. "'" is the only delimiter character that occurs inside and at the end of real name parts, and the least likely of them to mark a nickname. The same position with a quote IS ambiguous, because quotes do not appear inside names, so the carve-out is deliberately this one character: "Mari' Aube'" -> 0 ambiguities 'Mari" Aube"' -> 2 ambiguities #273 excluded the curly apostrophe from the delimiter set outright for this reason. The straight one has to stay a delimiter (v1's quoted_word), so it gets the narrower treatment here. The carve-out is about what PRECEDES the character, so an apostrophe opening a quote is untouched: "John 'Jack Smith" still reports an unmatched open. Also allows a preceding period, for initials ("O'B. Smith'"). Corpus back to 0 of 486 reporting, where the naive close sweep had 1. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent d993db0 commit 3f0b754

2 files changed

Lines changed: 42 additions & 0 deletions

File tree

nameparser/_pipeline/_extract.py

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,24 @@ def _close_ok(text: str, j: int, width: int) -> bool:
6161
return k >= len(text) or text[k].isspace() or text[k] in COMMA_CHARS
6262

6363

64+
def _apostrophe_after_a_word(text: str, j: int, close: str) -> bool:
65+
"""A "'" directly after a word character is an apostrophe, not a
66+
dangling close quote -- "Mari' Aube'", "Ali Baba'".
67+
68+
An ambiguity means the part could REASONABLY read two ways where it
69+
sits, and this one cannot: "'" is the only delimiter character that
70+
occurs inside and at the end of real name parts, and it is the
71+
least likely of them to mark a nickname. The same position with a
72+
quote IS ambiguous, since quotes do not appear inside names, so the
73+
carve-out is deliberately this one character. #273 excluded the
74+
curly apostrophe from the delimiter set outright for the same
75+
reason; the straight one has to stay a delimiter (v1's
76+
quoted_word), so it is handled here instead.
77+
"""
78+
return close == "'" and j > 0 and (text[j - 1].isalnum()
79+
or text[j - 1] == ".")
80+
81+
6482
def _overlaps(span: Span, taken: list[Span]) -> bool:
6583
return any(span.start < t.end and t.start < span.end for t in taken)
6684

@@ -191,6 +209,7 @@ def extract_delimited(state: ParseState) -> ParseState:
191209
start = j + 1
192210
if (j in reported # already an open
193211
or not _close_ok(text, j, len(close))
212+
or _apostrophe_after_a_word(text, j, close)
194213
or _overlaps(Span(j, j + len(close)), masked)):
195214
continue
196215
reported.add(j)

tests/v2/pipeline/test_extract.py

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,6 +199,29 @@ def test_unmatched_close_is_reported(text: str, char: str) -> None:
199199
assert char in parse(text).ambiguities[0].detail
200200

201201

202+
def test_word_final_apostrophe_is_not_an_unbalanced_delimiter() -> None:
203+
# ambiguity means the part could REASONABLY read two ways where it
204+
# sits. A "'" after a word character is an apostrophe -- it is the
205+
# one delimiter character that occurs inside and at the end of real
206+
# name parts, and the least likely to mark a nickname. The same
207+
# position with a quote IS ambiguous, because quotes do not appear
208+
# inside names. (#273 excluded the curly apostrophe from the
209+
# delimiters entirely for this reason; the straight one is a
210+
# delimiter, so it needs the carve-out here instead.)
211+
assert parse("Mari' Aube'").ambiguities == ()
212+
assert parse("Ali Baba'").ambiguities == ()
213+
quoted = parse('Mari" Aube"')
214+
assert [a.kind for a in quoted.ambiguities] == \
215+
[AmbiguityKind.UNBALANCED_DELIMITER] * 2
216+
217+
218+
def test_apostrophe_after_a_space_is_still_reported() -> None:
219+
# the carve-out is about what PRECEDES the character: an apostrophe
220+
# opening a quote is still an unmatched open when nothing closes it
221+
assert [a.kind for a in parse("John 'Jack Smith").ambiguities] == \
222+
[AmbiguityKind.UNBALANCED_DELIMITER]
223+
224+
202225
@pytest.mark.parametrize("text", [
203226
"Brian O'connor", # apostrophe mid-word, not a delimiter
204227
"O'B. John Smith",

0 commit comments

Comments
 (0)