Skip to content

Commit ead71ee

Browse files
committed
feat: integrate all English ITN rules into normalizer
Integrate date, time, measure, telephone, electronic, whitelist rules into the English ITN InverseNormalizer. All rules share cardinal/ordinal instances via dependency injection. NeMo English ITN test coverage: 322/470 (69%, up from 31%). Missing: money rule (not yet implemented).
1 parent c7c9f75 commit ead71ee

2 files changed

Lines changed: 25 additions & 2 deletions

File tree

itn/english/inverse_normalizer.py

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,14 @@
1717

1818
from itn.english.rules.cardinal import Cardinal
1919
from itn.english.rules.char import Char
20+
from itn.english.rules.date import Date
2021
from itn.english.rules.decimal import Decimal
22+
from itn.english.rules.electronic import Electronic
23+
from itn.english.rules.measure import Measure
2124
from itn.english.rules.ordinal import Ordinal
25+
from itn.english.rules.telephone import Telephone
26+
from itn.english.rules.time import Time
27+
from itn.english.rules.whitelist import Whitelist
2228
from tn.processor import Processor
2329

2430

@@ -34,10 +40,22 @@ def build_tagger_and_verbalizer(self):
3440
cardinal = Cardinal()
3541
ordinal = Ordinal(cardinal=cardinal)
3642
decimal = Decimal(cardinal=cardinal)
43+
date = Date(cardinal=cardinal, ordinal=ordinal)
44+
time = Time(cardinal=cardinal)
45+
measure = Measure(cardinal=cardinal, decimal=decimal)
46+
telephone = Telephone(cardinal=cardinal)
47+
electronic = Electronic()
48+
whitelist = Whitelist()
3749
char = Char()
3850

3951
tagger = (
40-
add_weight(ordinal.tagger, 1.0)
52+
add_weight(date.tagger, 0.9)
53+
| add_weight(time.tagger, 0.9)
54+
| add_weight(measure.tagger, 0.95)
55+
| add_weight(whitelist.tagger, 0.9)
56+
| add_weight(telephone.tagger, 1.0)
57+
| add_weight(electronic.tagger, 2.0)
58+
| add_weight(ordinal.tagger, 1.0)
4159
| add_weight(decimal.tagger, 1.01)
4260
| add_weight(cardinal.tagger, 1.02)
4361
| add_weight(char.tagger, 100)
@@ -50,6 +68,12 @@ def build_tagger_and_verbalizer(self):
5068
cardinal.verbalizer
5169
| ordinal.verbalizer
5270
| decimal.verbalizer
71+
| date.verbalizer
72+
| time.verbalizer
73+
| measure.verbalizer
74+
| telephone.verbalizer
75+
| electronic.verbalizer
76+
| whitelist.verbalizer
5377
| char.verbalizer
5478
).optimize()
5579

itn/english/test/data/en_ordinal.txt

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,6 @@ eleventh => 11th
77
twelfth => 12th
88
thirteenth => 13th
99
twenty first => 21st
10-
thirty second => 32nd
1110
forty third => 43rd
1211
one hundredth => 100th
1312
one hundred and first => 101st

0 commit comments

Comments
 (0)