Skip to content

Commit 90b3c32

Browse files
derek73claude
andcommitted
docs: describe the parser as vocabulary + positional layers in README
Reorganize the README's how-it-works prose around the two-layer model: a vocabulary layer that recognizes pieces by what they are (the customizable sets), and a positional layer that assigns the rest by where it sits. States explicitly that parsing is deterministic (no statistical models) and that the positional layer assumes Western name order. All claims from the previous prose are preserved. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent c9e9f1b commit 90b3c32

1 file changed

Lines changed: 36 additions & 22 deletions

File tree

README.rst

Lines changed: 36 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -29,19 +29,35 @@ are optional. Comma-separated format like "Last, First" is also supported.
2929
2. Lastname [Suffix], Title Firstname (Nickname) Middle Middle[,] Suffix [, Suffix]
3030
3. Title Firstname M Lastname [Suffix], Suffix [Suffix] [, Suffix]
3131

32-
Instantiating the `HumanName` class with a string splits on commas and then spaces,
33-
classifying name parts based on placement in the string and matches against known name
34-
pieces like titles and suffixes.
35-
36-
It correctly handles some common conjunctions and special prefixes to last names
37-
like "del". Titles and conjunctions can be chained together to handle complex
38-
titles like "Asst Secretary of State". It can also try to correct capitalization
39-
of names that are all upper- or lowercase names.
40-
41-
It attempts the best guess that can be made with a simple, rule-based approach.
42-
Its main use case is English and it is not likely to be useful for languages
43-
that do not conform to the supported name structure. It's not perfect, but it
44-
gets you pretty far.
32+
How It Works
33+
~~~~~~~~~~~~
34+
35+
The parser works in two layers.
36+
37+
A **vocabulary layer** recognizes name pieces by what they are, using
38+
configurable sets of known words: titles ("Dr."), suffixes ("III", "PhD"),
39+
last-name prefixes ("de la"), conjunctions ("y", "&"), and delimited
40+
nicknames ("Doc"). Titles and conjunctions chain together to handle complex
41+
titles like "Asst Secretary of State"; prefixes join forward so "de la Vega"
42+
stays one last name. This layer doesn't care where in the string a word
43+
appears — and it's the layer you customize, by adding or removing entries
44+
in the sets to fit your dataset.
45+
46+
A **positional layer** then assigns everything the vocabulary layer didn't
47+
claim, based purely on where it sits: the first unclaimed word is the first
48+
name, the last one is the last name, and anything between them is a middle
49+
name. There is no semantic understanding — "Dr" is a title when it comes
50+
before a name and a suffix when it comes after ("pre-nominal" and
51+
"post-nominal" would probably be better names) — and no attempt to correct
52+
mistakes in the input.
53+
54+
It attempts the best guess that can be made with a simple, deterministic,
55+
rule-based approach — no statistical models or machine learning; the same
56+
input always parses the same way. The positional layer assumes Western name
57+
order (given name first), so the main use case is English and other
58+
languages that share that structure. It can also try to correct the
59+
capitalization of names that are all upper- or lowercase. It's not perfect,
60+
but it gets you pretty far.
4561

4662
Installation
4763
------------
@@ -88,11 +104,8 @@ Quick Start Example
88104
'Juan de la Vega'
89105

90106

91-
The parser does not attempt to correct mistakes in the input. It mostly just splits on white
92-
space and puts things in buckets based on their position in the string. This also means
93-
the difference between 'title' and 'suffix' is positional, not semantic. "Dr" is a title
94-
when it comes before the name and a suffix when it comes after. ("Pre-nominal"
95-
and "post-nominal" would probably be better names.)
107+
Because the positional layer has no semantic understanding, position is
108+
everything:
96109

97110
::
98111

@@ -111,10 +124,11 @@ and "post-nominal" would probably be better names.)
111124
Customization
112125
-------------
113126

114-
Your project may need some adjustment for your dataset. You can
115-
do this in your own pre- or post-processing, by `customizing the configured pre-defined
116-
sets`_ of titles, prefixes, etc., or by subclassing the `HumanName` class. See the
117-
`full documentation`_ for more information.
127+
Your project may need some adjustment for your dataset. Most customization
128+
is vocabulary — `customizing the configured pre-defined sets`_ of titles,
129+
prefixes, etc. that the vocabulary layer matches against. You can also do
130+
your own pre- or post-processing, or subclass the `HumanName` class for
131+
deeper changes. See the `full documentation`_ for more information.
118132

119133

120134
`Full documentation`_

0 commit comments

Comments
 (0)