Skip to content

Commit 749b4c2

Browse files
kwagyemanclaude
andcommitted
i18n/potool: fix inline-markup whitespace-inside-delimiter boundary case
cjk_pad already separates markup from any non-whitespace neighbour (CJK, non-ASCII letters suffixed to a role, ASCII openers after a literal). It did NOT handle whitespace sitting immediately inside a delimiter -- e.g. a translator's `**) **` from restructured nested bold -- which docutils rejects as "start-string without end-string" yet _INLINE_RE matched as a valid span, so it slipped past both cjk_pad and the markup-invariant gate. Add _trim_inner_ws (called from cjk_pad): relocate leading/trailing whitespace from just inside ``literal``/**strong**/*emphasis* delimiters to just outside, collapsing into any adjacent whitespace so no double space is introduced. Visible text is unchanged; the markup now parses. Verified against docutils (broken -> 1 warning, fixed -> 0) and idempotent. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 68e0506 commit 749b4c2

1 file changed

Lines changed: 56 additions & 4 deletions

File tree

docs/tools/i18n/potool.py

Lines changed: 56 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -69,11 +69,63 @@ def untranslated_entries(po):
6969
_SEP = '\\ '
7070

7171

72+
def _trim_inner_ws(s):
73+
"""Relocate whitespace sitting *immediately inside* an inline-markup
74+
delimiter to just outside it. docutils requires a start-string to be
75+
followed by non-whitespace and an end-string to be preceded by
76+
non-whitespace, so a translator's ``**) **`` (from restructured nested
77+
bold, e.g. dropping the word between the delimiters) parses as
78+
"start-string without end-string". Moving the space out — ``**)** `` —
79+
fixes parsing while leaving the visible text identical. Applies to
80+
``literal``/**strong**/*emphasis* spans (the ones translators actually
81+
break this way); roles/links are left untouched."""
82+
spans = list(_INLINE_RE.finditer(s))
83+
if not spans:
84+
return s
85+
res, prev = '', 0
86+
for m in spans:
87+
res += s[prev:m.start()]
88+
span = m.group(0)
89+
rebuilt = lead = trail = None
90+
for od, cd in (('``', '``'), ('**', '**'), ('*', '*')):
91+
if span.startswith(od) and span.endswith(cd) and len(span) > len(od) + len(cd):
92+
inner = span[len(od):len(span) - len(cd)]
93+
core = inner.strip()
94+
if core: # skip degenerate all-whitespace spans (not fixable here)
95+
lead = inner[:len(inner) - len(inner.lstrip())]
96+
trail = inner[len(inner.rstrip()):]
97+
rebuilt = od + core + cd
98+
break
99+
if rebuilt is None:
100+
res += span
101+
else:
102+
# Relocate the inner edge whitespace outside, collapsing it into any
103+
# whitespace already adjacent so we never introduce a double space.
104+
if lead and not (res and res[-1].isspace()):
105+
res += lead
106+
res += rebuilt
107+
nxt = s[m.end()] if m.end() < len(s) else ''
108+
if trail and nxt != '' and not nxt.isspace():
109+
res += trail
110+
prev = m.end()
111+
res += s[prev:]
112+
return res
113+
114+
72115
def cjk_pad(s):
73-
"""Insert a reST escaped-space wherever inline markup butts directly
74-
against a CJK ideograph/kana/hangul/punctuation char, so docutils can
75-
parse the markup. Invisible in output. Idempotent: skips edges that are
76-
already preceded/followed by whitespace (incl. an existing escaped-space)."""
116+
"""Normalize inline-markup boundaries so docutils can parse them; invisible
117+
in output and idempotent.
118+
119+
Two boundary defects are handled:
120+
1. markup butting directly against a non-whitespace neighbour (a CJK
121+
ideograph/kana/hangul, CJK/fullwidth punctuation, a non-ASCII letter
122+
suffixed to a role like the German ``:doc:`Bild <t>`es``, or an ASCII
123+
opener like ``(`` right after a ``literal``) -- separated with a reST
124+
escaped-space ``\\ ``;
125+
2. whitespace sitting immediately inside a delimiter (``**) **``) -- moved
126+
outside via ``_trim_inner_ws`` before padding.
127+
"""
128+
s = _trim_inner_ws(s)
77129
spans = [(m.start(), m.end()) for m in _INLINE_RE.finditer(s)]
78130
if not spans:
79131
return s

0 commit comments

Comments
 (0)