Add hi-IN , Ko-KR and pt-BR IPA tokenizer support#15567

Open

quapham wants to merge 22 commits intoNVIDIA-NeMo:mainfrom

quapham:hi_pt_BR_ipa

Commits on Apr 22, 2026

feat(tts): extend IPA tokenizer with hi-IN/en code-switching and pt-BR

quapham
authored and
XuesongYang
committed
feat(tts): remove ar-MSA locale as out of scope for this PR

quapham
authored and
XuesongYang
committed
Apply isort and black reformatting

quapham
authored and
XuesongYang
committed
Add Korean IPA support

quapham
authored and
XuesongYang
committed
Fix leftover merge markers in Korean IPA support

quapham
authored and
XuesongYang
committed
Apply isort and black reformatting

quapham
authored and
XuesongYang
committed
fix: add KOREAN_CHARS import

quapham
authored and
XuesongYang
committed
Apply suggestion from @Copilot

XuesongYang
and
Copilot
committed
Apply suggestion from @XuesongYang
XuesongYang
committed
Apply suggestion from @XuesongYang
XuesongYang
committed
Update tests/collections/common/tokenizers/text_to_speech/test_tts_tokenizers.py
XuesongYang
committed
Apply suggestion from @XuesongYang
XuesongYang
committed
Apply suggestion from @XuesongYang
XuesongYang
committed
Apply suggestion from @XuesongYang
XuesongYang
committed
Apply suggestion from @XuesongYang
XuesongYang
committed
Apply suggestion from @XuesongYang
XuesongYang
committed
WIP: save local changes before rebasing

quapham
authored and
XuesongYang
committed
fix: update IPAG2p typing and docs

quapham
authored and
XuesongYang
committed
Apply isort and black reformatting

quapham
authored and
XuesongYang
committed
bugfix: unit test of hindi
XuesongYang
committed
refactor: introduce a combined constant WORD_CHARS_ALL.
XuesongYang
committed

Commits on Apr 23, 2026

fix: backward-compatible punctuation for pt-BR and hi-IN tokenizers
XuesongYang
committed