Skip to content

Commit 53611b1

Browse files
derek73claude
andcommitted
Name the escape hatch that actually works (#308)
The peel costs a couple of microseconds per non-ASCII parse and is not gated on segment_scripts -- deliberate, and the prose already says so, but a performance-sensitive caller reaches for that field first. Emptying honorific_tails hits the peel's first-line guard and recovers the cost; customize.rst already documents that removal as the BEHAVIORAL off-switch, so one clause there covers the other reason to reach for it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 46fdc3e commit 53611b1

1 file changed

Lines changed: 5 additions & 1 deletion

File tree

docs/customize.rst

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -369,7 +369,11 @@ vocabulary is also the peel's off-switch —
369369
``Lexicon.default().remove(honorific_tails={"さん"})`` leaves
370370
``田中さん`` unsplit while the spaced ``田中 さん`` still reads ``さん``
371371
as a suffix. Dropping a word from a marker field alone orphans
372-
nothing, so that one needs no matching ``suffix_words`` edit.
372+
nothing, so that one needs no matching ``suffix_words`` edit. Emptying
373+
the field is also the way to opt out of what the peel *costs* a
374+
non-ASCII parse — an empty ``honorific_tails`` stops it at its first
375+
guard, whereas ``segment_scripts`` never gated it and so cannot turn it
376+
off.
373377

374378
.. note::
375379

0 commit comments

Comments
 (0)