pdf_io: support macOS TrueType fonts that only expose a (0,4) cmap subtable#1377
Open
hhh2210 wants to merge 2 commits into
Open
pdf_io: support macOS TrueType fonts that only expose a (0,4) cmap subtable#1377hhh2210 wants to merge 2 commits into
hhh2210 wants to merge 2 commits into
Conversation
…subtable
`otf_create_ToUnicode_stream()` probes a fixed list of cmap
(platform, encoding) pairs when building a ToUnicode CMap for a
TrueType/OpenType-backed CID font. The list was missing
(platform 0 = Unicode, encoding 4 = Unicode 2.0+, full repertoire),
so fonts whose *only* cmap subtable is (0,4) got no ToUnicode CMap at
all. Text set in such fonts then renders correctly but is not
extractable / searchable / copy-pasteable from the resulting PDF.
This affects real-world fonts: several macOS system CJK fonts ship a
single (0,4) format-12 subtable, e.g. STHeiti and STXihei. A Chinese
thesis compiled with tectonic produced a PDF whose bold/heading text
(set in STHeiti) could not be extracted, which breaks copy-paste and
plagiarism-check workflows.
Upstream dvipdfm-x already carries the { 0, 4 } entry in this table;
this change brings tectonic's vendored copy in line with it.
Verified on a real Chinese thesis: extractable CJK characters went
from 23957 to 26918, exactly matching the xelatex (TeX Live 2026)
output, and the STHeiti/STXihei font objects now carry /ToUnicode.
There was a problem hiding this comment.
Pull request overview
This pull request fixes missing PDF text-extraction metadata for certain TrueType/OpenType fonts whose only available cmap subtable is (platform=0, encoding=4), by ensuring Tectonic’s /ToUnicode generation logic considers that subtable.
Changes:
- Add
(0,4)to the prioritized list ofcmapplatform/encoding pairs probed when building a/ToUnicodeCMap. - This enables
/ToUnicodegeneration for fonts that previously matched no probed subtable despite having a supported format (4 or 12).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…0-4 cmap support)
Companion to the previous commit. Upstream dvipdfm-x fixed "support some
TrueType fonts in macOS which use 0-4 cmap" (texjporg, 2022-06-15, Clerk Ma)
by touching *two* files: tt_cmap.c (ToUnicode probe list) and cidtype2.c
(the CID-keyed embedding cmap probe list, `known_encodings[]`). tectonic's
vendored copy was missing both halves.
This adds the matching { 0, 4 } entry to `known_encodings[]` and bumps
KNOWN_ENCODINGS_MAX 9 -> 10, so the CID-keyed CIDFontType2 embedding path
(used outside the XeTeX glyph-by-GID route, e.g. CID-keyed fontmap usage)
can also find the cmap of macOS fonts whose only subtable is (platform 0,
encoding 4). index 10 > WIN_UCS_INDEX_MAX (1), so it is handled exactly like
the upstream change (non-Windows-UCS, via find_tocode_cmap with the UCS
names). Verbatim match of upstream.
hhh2210
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Text set in some fonts renders correctly but cannot be selected, searched, or copy-pasted from the output PDF, because no
/ToUnicodeCMap is written for those fonts.This happens with several macOS system CJK fonts. Compiling a Chinese thesis (
ctexwithfontset=mac) produced a PDF where the bold/heading text, set in STHeiti, could not be copied, while the body text (STSong/STKaiti) was fine. tectonic prints:The same document compiled with XeLaTeX from TeX Live 2026 is fully extractable, so the problem is specific to tectonic.
Root cause
The affected fonts expose only a
(platform 0 = Unicode, encoding 4 = Unicode 2.0+ full repertoire)cmap subtable. dvipdfm-x probes a fixed list of(platform, encoding)pairs when it reads a font's cmap, and that list was missing{ 0, 4 }in two spots:dpx-tt_cmap.c,cmap_plat_encs[], used byotf_create_ToUnicode_stream()to build the ToUnicode CMap. With no match, no ToUnicode is written and the text cannot be extracted.dpx-cidtype2.c,known_encodings[], used by the CID-keyedCIDFontType2embedding path to find the cmap for CID-to-GID / CID-to-code mapping. The{ 0, 4 }entry was absent andKNOWN_ENCODINGS_MAXwas one too small.Looking at the fonts shows why only some of them break:
(0,4) fmt12+(3,1) fmt4(3,1))(0,4) fmt12+(3,10) fmt12(3,10))(0,4) fmt12only(0,4) fmt12only(0,4) fmt12only3 of the 5 CJK fonts that
ctex'sfontset=macuses (黑体/细黑/仿宋) are(0,4)-only and all broke; 宋体/楷体 happen to carry a platform-3 subtable and slipped through.Fix
Add
{ 0, 4 }to both probe tables, matching upstream dvipdfm-x. This is the same change upstream made in "cidtype2.c, tt_cmap.c: Support some TrueType fonts in macOS which use 0-4 cmap" (texjporg/tex-jp-build, 2022-06-15, Clerk Ma), which tectonic's vendored copy never picked up. That commit touched both files, so this PR restores both halves. The format check (4 or 12) and theWIN_UCS_INDEX_MAXhandling are unchanged, so this only enables a case that was previously skipped.Verification
Built locally and recompiled the same Chinese thesis:
STHeitiandSTXiheifont objects in the output PDF now carry/ToUnicode.cidtype2.cchange there is no regression: extractable count stays at 26918, still 0 errors.Related (separate bugs, same area)
These are not fixed here, but they live in the same macOS CJK font-handling code and are worth linking:
xetex_layout(Option::unwrap()onNone), so compilation aborts before PDF output. This PR is a non-crashing PDF-output fix.\setmainfont{}on macOS arm64.