Skip to content

Commit 3280c04

Browse files
derek73claude
andcommitted
Restructure the East Asian names section: background before behavior
Lead with how the names are built (order, unspaced writing, closed surname sets), then the two automatic behaviors, then the boundaries -- replacing the casual 'come out right' / 'a step further' framing that assumed the reader already knew the conventions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 9ab84d1 commit 3280c04

1 file changed

Lines changed: 43 additions & 25 deletions

File tree

docs/usage.rst

Lines changed: 43 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -140,12 +140,22 @@ merging two, and is covered next.
140140
East Asian names
141141
-----------------
142142

143-
A name written wholly in Han (Chinese and Japanese characters) or
144-
Hangul needs no configuration to come out right: the script settles the
145-
convention by itself, so those names read family-first by default.
146-
Chinese and Japanese differ in most things but not this one — both put
147-
the family name first in native script — so nothing has to guess which
148-
language it is looking at.
143+
How these names are built, first, because the parsing rules follow
144+
from it. A Chinese, Japanese, or Korean name written in its own script
145+
puts the family name first: 毛泽东 is MAO Zedong, 山田太郎 is YAMADA
146+
Taro, 김민준 is KIM Minjun. The family name is short — one Han
147+
character or one hangul syllable, occasionally two — and comes from a
148+
small closed set, while given names are open-ended. And in native
149+
writing the parts are usually not separated at all: the whole name is
150+
one unbroken run of characters. A parser therefore has two distinct
151+
jobs here: assign family and given to the right fields, and, when the
152+
name arrives as a single token, find the boundary inside it.
153+
154+
Field assignment is automatic. A name written wholly in Han characters
155+
or hangul is assigned family-first, because every language written in
156+
those scripts orders names that way — Chinese and Japanese share
157+
little else, but they agree on this — so the assignment requires no
158+
knowledge of which language the name is in:
149159

150160
.. doctest::
151161

@@ -154,39 +164,47 @@ language it is looking at.
154164
>>> parse("山田 太郎").family
155165
'山田'
156166

157-
Korean names go a step further, because they are usually written with
158-
no space in them at all. The census surname list ships as default
159-
vocabulary, so an unspaced hangul name is split into its two parts:
167+
Splitting an unspaced name is also automatic, but only for Korean.
168+
Hangul is written by exactly one language, and Korean family names are
169+
a closed set small enough to ship: the census surname list is part of
170+
the default vocabulary, and the longest listed surname at the start of
171+
an unspaced hangul token becomes the family name:
160172

161173
.. doctest::
162174

163175
>>> minjun = parse("김민준")
164176
>>> minjun.family, minjun.given
165177
('김', '민준')
166178

167-
That works out of the box because nothing but Korean is written in
168-
hangul. The same trick on Han characters would have to know Chinese
169-
from Japanese first — a Chinese surname list splits ``高橋一郎`` after
170-
````, wrecking an ordinary Japanese name — so unspaced *Chinese* is
171-
opt-in through the ``zh`` locale pack, and Japanese waits on a
172-
segmenter of its own (`#272
173-
<https://github.com/derek73/python-nameparser/issues/272>`_):
179+
The same split is not automatic for Han text, because there the script
180+
does not identify the language: 高橋一郎 is a Japanese name whose
181+
family name is 高橋, but 高 alone is a common Chinese surname, so a
182+
Chinese surname list would split it in the wrong place. Declaring the
183+
language is up to you. When you know the data is Chinese, apply the
184+
``zh`` locale pack; Japanese needs a dictionary-backed segmenter and
185+
is tracked as `#272
186+
<https://github.com/derek73/python-nameparser/issues/272>`_ — until
187+
then an unspaced Japanese name stays whole, in the family field per
188+
the assignment rule above:
174189

175190
.. doctest::
176191

177192
>>> from nameparser import locales, parser_for
178193
>>> parser_for(locales.ZH).parse("毛泽东").family
179194
'毛'
180195

181-
A comma switches both behaviors off, on the reasoning ``name_order``
182-
already follows: someone who wrote one has said where the family name
183-
ends. When the vocabulary supported more than one split — ``남궁민수``
184-
is 남궁 + 민수 by the compound surname but 남 + 궁민수 by the
185-
single-syllable one — the longest match wins and the parse reports the
186-
choice as an ``AmbiguityKind.SEGMENTATION``, described under `When the
187-
parser had to guess`_; a name with only one possible split decided
188-
nothing and reports nothing. :doc:`customize` covers turning either
189-
behavior off.
196+
Three boundaries on all of the above. Romanized names ("Kim Min-jun")
197+
are Latin script and follow the ordinary positional rules — order
198+
genuinely varies in romanized data, so nothing script-based applies.
199+
A comma disables both behaviors, on the reasoning ``name_order``
200+
already follows: whoever wrote the comma has already said where the
201+
family name ends. And when an unspaced name has more than one
202+
vocabulary-supported split — ``남궁민수`` is 남궁 + 민수 by the
203+
two-syllable surname but 남 + 궁민수 by the single-syllable one — the
204+
longest surname wins and the parse records the decision as an
205+
``AmbiguityKind.SEGMENTATION``, described under `When the parser had
206+
to guess`_; a name with only one possible split reports nothing.
207+
:doc:`customize` covers turning either behavior off.
190208

191209
Aggregate views
192210
----------------

0 commit comments

Comments
 (0)