Skip to content

Glued CJK honorifics are absorbed into the name (山田太郎様 → family 山田太郎様) #308

Description

@derek73

The common way to write CJK honorifics is glued to the name: 田中さん, 王先生, and the standard mail-addressing form 山田太郎様. Whole-token vocabulary can never match these, so today the honorific is absorbed whole — and a kana honorific even changes classification (田中さん is Han+hiragana → kana-licensed → family=田中さん), and would mislead a configured ja segmenter.

The fix is a mechanism, not constants: a list-driven tail-peel on the last name token, running before classification/order/segmentation — the honorific-stripping analogue of script_segment. Design questions this issue exists to settle:

  • Safety: 山田さん must peel while 山田さくら must not — tail-matching against a closed honorific list is probably sufficient (no Japanese given name is さん/様/先生), but the boundary needs the same treatment the segmenter contract got (what may peel, what must decline).
  • Default-on or opt-in: glued 様/さん at token tail looks unambiguous (the doctrine's "orthography settles it" test), but that claim needs adversarial review against real given names.
  • Interaction order: the peel must precede effective_script, the family-first license, and the segmentation stage, and the peeled honorific needs a field (suffix) and span bookkeeping consistent with the anti-Strange parsing of name w lastname prefix and title before and after #100 invariant (spans index the original — a peel is a token split, not a rewrite).

Depends on the spaced-honorific vocabulary issue for the list itself.

Metadata

Metadata

Assignees

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions