Commit 211f8c3
committed
Address PR review: pin down suffix_delimiter edge cases, clarify comments
- Lock in hn.middle for the "Doe, Mary - Kate, RN" case so the stray
hyphen artifact (a pre-existing tokenization quirk, reproducible
without suffix_delimiter set) can't silently drift.
- Add coverage for the parts[1:] loop expanding more than one comma
segment, for a multi-word token on one side of the delimiter in the
detection check, and for delimiter no-op parity with the no-delimiter
baseline when the format isn't detected as suffix-comma.
- Note in expand_suffix_delimiter's docstring that it's a no-op without
a configured delimiter, and comment why detection needs the
delimiter-expanded view of parts[1].1 parent 021823e commit 211f8c3
2 files changed
Lines changed: 39 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
678 | 678 | | |
679 | 679 | | |
680 | 680 | | |
681 | | - | |
| 681 | + | |
| 682 | + | |
682 | 683 | | |
683 | 684 | | |
684 | 685 | | |
| |||
1067 | 1068 | | |
1068 | 1069 | | |
1069 | 1070 | | |
| 1071 | + | |
| 1072 | + | |
| 1073 | + | |
1070 | 1074 | | |
1071 | 1075 | | |
1072 | 1076 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
273 | 273 | | |
274 | 274 | | |
275 | 275 | | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
276 | 310 | | |
277 | 311 | | |
278 | 312 | | |
| |||
0 commit comments