Skip to content

Commit 717101e

Browse files
derek73claude
andcommitted
docs: concepts page reads plainly — accurate title, concrete walkthrough
Review feedback (Derek): 'How the parser thinks' implied nondeterminism -> 'How the parser works'; the 'four short essays' opener and 'pipeline, in one breath' phrasing replaced with direct statements; 'From string to name' now walks the canonical example concretely (8 tokens, Dr. span (0,3), family as a view joining de/la/Vega) and defines span before explaining what it prevents; the three-container bullets lead with the container names; the Dean paragraph returns to the v1 docs' problem-first framing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 895ecb3 commit 717101e

2 files changed

Lines changed: 47 additions & 37 deletions

File tree

docs/concepts.rst

Lines changed: 46 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,37 @@
1-
How the parser thinks
2-
======================
1+
How the parser works
2+
====================
33

4-
Four short essays on the model behind the 2.0 API: how a string
5-
becomes a :class:`~nameparser.ParsedName`, why configuration lives in
6-
exactly three places sorted by what varies, why parsers are plain
7-
values, and what the parser does when a name is genuinely ambiguous.
4+
``parse()`` turns a name string into a
5+
:class:`~nameparser.ParsedName`. This page explains the model behind
6+
that call: how a string becomes tokens and tokens become fields,
7+
where configuration lives and why it is split the way it is, why
8+
parsers are plain values, and what happens when a name is genuinely
9+
ambiguous. The task pages all build on these four ideas.
810

911
From string to name
1012
--------------------
1113

12-
The pipeline, in one breath: the original string becomes a sequence of
13-
tokens with spans (character positions into the original string),
14-
each span gets assigned a role — ``title``, ``given``, ``middle``,
15-
``family``, ``suffix``, ``nickname``, ``maiden`` — and every string you
16-
read off a :class:`~nameparser.ParsedName` (``.given``, ``.family``,
17-
``str(name)``, and so on) is a view computed over those roles at read
18-
time, not stored text.
19-
20-
Spans matter because of what they replace. v1's ``HumanName`` worked
21-
on plain lists of strings and re-found pieces by searching the list
22-
for a matching value — so a name with a repeated word could make the
23-
parser rewrite the wrong occurrence. That whole bug family (starting
24-
with `issue #100 <https://github.com/derek73/python-nameparser/issues/100>`_)
25-
traces back to value-based lookup. A token's span is a position, not a
26-
value, so nothing in the 2.0 pipeline is ever re-found by scanning for
27-
a string that looks like it.
14+
Every parse follows the same path: the input string is split into
15+
tokens, each token is assigned one of the seven roles — ``title``,
16+
``given``, ``middle``, ``family``, ``suffix``, ``nickname``,
17+
``maiden`` — and every string you read off the result is computed
18+
from those tokens at read time.
19+
20+
Parsing ``"Dr. Juan Q. Xavier de la Vega III"`` produces eight
21+
tokens. The first is ``Dr.`` with the ``title`` role; ``de``, ``la``,
22+
and ``Vega`` each carry the ``family`` role, which is why
23+
``name.family`` returns ``"de la Vega"`` — the field is a view that
24+
joins the family-role tokens in order, not a stored string.
25+
26+
Each token also records where it came from. A span is a pair of
27+
character positions bounding the token in the original string:
28+
``Dr.`` has span ``(0, 3)``, and ``name.original[0:3]`` is exactly
29+
``"Dr."``. Internally, spans let the pipeline refer to a token by
30+
position instead of by text. v1 re-found name pieces by searching for
31+
matching text, so a name with a repeated word could make the parser
32+
rewrite the wrong occurrence (`issue #100
33+
<https://github.com/derek73/python-nameparser/issues/100>`_ and its
34+
relatives); a position cannot be confused with a look-alike.
2835

2936
:class:`~nameparser.ParsedName` is frozen: there is no attribute
3037
assignment, ever. If a parse is almost right and you want to fix one
@@ -41,20 +48,23 @@ Every piece of nameparser configuration falls into exactly one of
4148
three places, and which one is decided by a single question: what does
4249
this setting vary with?
4350

44-
* varies by **language** → :class:`~nameparser.Lexicon` (vocabulary:
45-
titles, particles, suffixes, conjunctions, and the rest of the
46-
word lists the parser matches against)
47-
* varies by **data source or application** → :class:`~nameparser.Policy`
48-
(behavior switches: name order, patronymic rules, delimiters, strip
49-
flags — anything that changes how the pipeline runs, not what words
50-
it recognizes)
51-
* varies by **output destination** → a rendering argument (the
52-
``spec`` you pass to ``render(spec)``, or a keyword to
53-
``initials()``/``capitalized()``)
54-
55-
"Dean" being a title in your data is a fact about the language and
56-
domain the names come from (academic rosters, say), not about any one
57-
dataset or report — that's a :class:`~nameparser.Lexicon` entry. A CRM that always exports "Family, Given" strings is a fact
51+
* :class:`~nameparser.Lexicon` holds everything that varies by
52+
**language**: the vocabulary — titles, particles, suffixes,
53+
conjunctions, and the rest of the word lists the parser matches
54+
against.
55+
* :class:`~nameparser.Policy` holds everything that varies by
56+
**data source or application**: the behavior switches — name order,
57+
patronymic rules, delimiters, strip flags — anything that changes
58+
how the pipeline runs, not what words it recognizes.
59+
* Rendering arguments cover everything that varies by **output
60+
destination**: the ``spec`` you pass to ``render(spec)``, or a
61+
keyword to ``initials()``/``capitalized()``.
62+
63+
"Dean" is a common given name, so it is not in the default titles
64+
vocabulary — but in some data it is more common as a title. Which
65+
reading is right is a fact about the language and domain the names
66+
come from, not about any one dataset or report: that makes it a
67+
:class:`~nameparser.Lexicon` entry. A CRM that always exports "Family, Given" strings is a fact
5868
about that one data source, not about the language of the names in
5969
it — that's a :class:`~nameparser.Policy`. One particular report
6070
wanting names formatted as "Family, Given" while every other consumer

docs/usage.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,7 @@ ru``); see :doc:`locales`.
114114
Where next
115115
----------
116116

117-
* :doc:`concepts` — how the parser thinks
117+
* :doc:`concepts` — how the parser works
118118
* :doc:`customize` — your own vocabulary and behavior
119119
* :doc:`locales` — locale packs
120120
* :doc:`migrate` — coming from 1.x ``HumanName``

0 commit comments

Comments
 (0)