Correct the migration doc's CJK baseline claims (#271)

derek73 · claude · derek73 · commit 5ff113123455 · 2026-07-28T12:24:11.000-07:00
Two claims about 1.4 in migrate.rst were wrong, both verified against the pinned 1.4 worker rather than from memory: - 1.4 routes a LONE token to first, universally, so HumanName("김민준") gave first="김민준" and not last. The doc said last. - "Unspaced Chinese is unaffected either way" was false. Segmentation does not apply to it, but script_orders does: 毛泽东 and 山田太郎 now land in last where 1.4 put them in first. Nothing splits, so the string looks identical and only the field changed -- the quietest form of this release's default-on change, and the doc had told readers there was nothing to check. Now stated plainly, with the Japanese case named, since a kanji name is affected exactly as a Chinese one is. The release log's first #271 bullet illustrated only the spaced case and inherited the same blind spot; it now covers the lone-token flip too. Its parse-output bullets also move under a Behavior Changes heading, matching the 2.0.0 block's structure. Also rewrites the locales.rst contributor bullet that opened "Sort the vocabulary before shipping it" -- leftover phrasing from the bullet it replaced, and actively contradicting the "do not alphabetize" rank-order convention that zh.py and surnames.py both document in the data itself. The bullet is about which LAYER the vocabulary belongs in, so it says that instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
diff --git a/docs/locales.rst b/docs/locales.rst
@@ -241,8 +241,8 @@ by ``tests/v2/test_locales.py``:
    the default must be one your ``DEVIATES`` predicate flags — no
    silent, undeclared side effects on names outside the pack's stated
    scope.
-#. Sort the vocabulary before shipping it, if the pack carries any.
-   Vocabulary that is *self-selecting* — able to match only text of
+#. Decide which layer the vocabulary belongs in, if the pack carries
+   any. Vocabulary that is *self-selecting* — able to match only text of
    the tradition it came from, the way a hangul surname can only ever
    match hangul — is default-safe, and belongs in the default lexicon
    (``nameparser/config/``) rather than in a pack: a pack nobody knows
diff --git a/docs/migrate.rst b/docs/migrate.rst
@@ -352,11 +352,29 @@ given name.
     >>> HumanName("김민준").last, HumanName("김민준").first
     ('김', '민준')
 
-1.4 read the first as ``first="毛"``/``last="泽东"`` and left the
-second whole in ``last``. ``Constants`` has no switch for either — the
-v1 configuration surface is frozen for 2.x — so the way out is the 2.0
-API: ``Parser(policy=Policy(script_orders={}, segment_scripts=()))``
-restores the old reading of both. Unspaced *Chinese* is unaffected
-either way, because splitting Han text is opt-in through a locale pack
-and ``HumanName`` cannot apply one: ``HumanName("毛泽东")`` still
-returns the whole string as ``last``.
+1.4 read the first as ``first="毛"``/``last="泽东"``, and left the
+second whole in ``first``.
+
+A third shape changes even though nothing splits it. 1.4 routed a lone
+token to ``first`` whatever it was, so an unspaced Chinese or Japanese
+name landed there entire; 2.1 reads it as native-script CJK and puts it
+in ``last`` instead. Nothing is segmented — Han splitting is opt-in
+through a locale pack and ``HumanName`` cannot apply one — but the
+field the whole string arrives in is different:
+
+.. doctest::
+
+    >>> HumanName("毛泽东").last, HumanName("毛泽东").first
+    ('毛泽东', '')
+    >>> HumanName("山田太郎").last, HumanName("山田太郎").first
+    ('山田太郎', '')
+
+Both were ``first`` under 1.4. If you feed unspaced CJK through
+``HumanName`` and read ``first``, that is the change most likely to
+reach you, and it is silent — the string is intact, just in the other
+field.
+
+``Constants`` has no switch for any of this — the v1 configuration
+surface is frozen for 2.x — so the way out is the 2.0 API:
+``Parser(policy=Policy(script_orders={}, segment_scripts=()))``
+restores 1.4's reading of all three shapes.
diff --git a/docs/release_log.rst b/docs/release_log.rst
@@ -2,10 +2,8 @@ Release Log
 ===========
 * 2.1.0 - Unreleased
 
-    **East Asian names**
+    **East Asian name support**
 
-    - Fix names written wholly in Han or Hangul parsing given-first: native-script CJK now reads family-first by default, through the new ``Policy.script_orders`` table, so ``"毛 泽东"`` gives family ``毛`` where 2.0 gave family ``泽东``. No language detection is involved — Chinese and Japanese both write the family name first in native script, so the script settles the order without anyone having to know which language it is. Latin-script and mixed-script names are never affected, and an explicit comma still wins. **Default-on: changes parse output for wholly-CJK names**, through ``HumanName`` as well as the 2.0 API (closes #271)
-    - Fix unspaced Korean names not splitting: the census surname list now ships as default vocabulary (``Lexicon.surnames``) with hangul segmentation on by default (``Policy.segment_scripts``), so ``"김민준"`` parses family ``김``, given ``민준`` instead of landing whole in the family name. Nothing but Korean is written in hangul and its surnames are a closed census set, which is what makes the split safe as a default rather than a pack. **Default-on**, and it reaches ``HumanName`` too. ``Policy(segment_scripts=())`` turns the split off; ``Policy(script_orders={})`` separately restores the positional reading; clearing both restores 2.0 behavior exactly (#271)
     - Add the Chinese locale pack ``locales.ZH`` — opt-in Han segmentation for unspaced names like ``毛泽东``, with the surname vocabulary it needs. A pack rather than a default because a Chinese surname list corrupts the Japanese names written in the same characters (``高橋一郎`` would split ``高`` + ``橋一郎``); Japanese needs the pluggable segmenter staged in #272. It sets no name order, since native-script Han already reads family-first without it. ``locales.available()`` is now ``('ru', 'tr_az', 'zh')`` (#271)
     - Add ``Lexicon.surnames``, ``Policy.script_orders``, ``Policy.segment_scripts``, the ``Script`` enum and the ``DEFAULT_SCRIPT_ORDERS`` constant to the public API. This is the first behavior nameparser keys on the script a name is written in; it is allowed only where the script itself settles a convention, never as a proxy for guessing the language (#271)
     - Add ``AmbiguityKind.SEGMENTATION``, reported when a surname split had a vocabulary-supported alternative: ``"남궁민수"`` is 남궁 + 민수 by the compound surname but 남 + 궁민수 by the single-syllable one, and longest-match had to pick. A name with only one possible split decided nothing and reports nothing (#271)
@@ -14,6 +12,11 @@ Release Log
 
     - Change the pickle compatibility of the 2.0 API's ``Policy`` and ``Lexicon``: the new fields change the field layout their guarded ``__setstate__`` checks, so a pickle written by 2.0.0 raises ``ValueError`` naming the missing fields instead of loading. Re-pickle after upgrading. ``HumanName`` pickles are unaffected — the facade pickles v1-shaped component state, not these objects
 
+    **Behavior Changes**
+
+    - Fix names written wholly in Han or Hangul parsing given-first: native-script CJK now reads family-first by default, through the new ``Policy.script_orders`` table, so ``"毛 泽东"`` gives family ``毛`` where 1.x gave family ``泽东``. A name that is a single unspaced token moves the same way — ``"毛泽东"`` and ``"山田太郎"`` now land in ``family``/``last`` where 1.x put them in ``given``/``first``, with the string itself untouched, which makes it the easiest form of this change to miss. No language detection is involved: Chinese and Japanese both write the family name first in native script, so the script settles the order without anyone having to know which language it is. Latin-script and mixed-script names are never affected, and an explicit comma still wins. **Default-on: changes parse output for wholly-CJK names**, through ``HumanName`` as well as the 2.0 API (closes #271)
+    - Fix unspaced Korean names not splitting: the census surname list now ships as default vocabulary (``Lexicon.surnames``) with hangul segmentation on by default (``Policy.segment_scripts``), so ``"김민준"`` parses family ``김``, given ``민준`` where 1.x returned the whole string as ``first``. Nothing but Korean is written in hangul and its surnames are a closed census set, which is what makes the split safe as a default rather than a pack. **Default-on**, and it reaches ``HumanName`` too. ``Policy(segment_scripts=())`` turns the split off; ``Policy(script_orders={})`` separately restores the positional reading; clearing both restores 2.0 behavior exactly (#271)
+
 * 2.0.0 - July 27, 2026
 
     Two release candidates preceded this release (rc1 on 2026-07-23,
