Skip to content

Commit e8013f7

Browse files
committed
New config parameter to choose SentencePiece unicode normalization
1 parent fb7a09b commit e8013f7

18 files changed

Lines changed: 702768 additions & 6 deletions

File tree

configs/autogenerated/en-pl-example.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ experiment:
3636
spm-sample-size: 10_000_000
3737
spm-vocab-size: 32000
3838
spm-vocab-split: false
39+
spm-norm-rule: nmt_nfkc
3940
teacher-ensemble: 1
4041
teacher-mode: two-stage
4142
teacher-decoder: ctranslate2

0 commit comments

Comments
 (0)