Skip to content

Commit bb9c7ef

Browse files
committed
fix: disambiguate "X点零Y" between time and decimal in Chinese ITN (#310)
"五点零三" was incorrectly parsed as time 5:03 instead of decimal 5.03. Fix: require "分" suffix or noon prefix (上午/下午/etc.) for time patterns with zero-prefixed minutes (零一~零九). Without either, these are now treated as decimals. - Split minute.tsv: move 零X entries to new minute_zero.tsv - Time tagger: 零X minutes without "分" only match with noon prefix Examples: 五点零三 => 5.03 (decimal, no context) 五点零三分 => 5:03 (time, has 分) 下午五点零三 => 5:03p.m. (time, has noon) 八点三十 => 8:30 (time, unambiguous)
1 parent 60fab50 commit bb9c7ef

4 files changed

Lines changed: 23 additions & 18 deletions

File tree

itn/chinese/data/time/minute.tsv

Lines changed: 0 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,4 @@
11
30
2-
零一 01
3-
零二 02
4-
零三 03
5-
零四 04
6-
零五 05
7-
零六 06
8-
零七 07
9-
零八 08
10-
零九 09
112
10
123
十一 11
134
十二 12
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
零一 01
2+
零二 02
3+
零三 03
4+
零四 04
5+
零五 05
6+
零六 06
7+
零七 07
8+
零八 08
9+
零九 09

itn/chinese/rules/time.py

Lines changed: 13 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -29,19 +29,24 @@ def __init__(self):
2929
def build_tagger(self):
3030
h = string_file(get_abs_path("../itn/chinese/data/time/hour.tsv"))
3131
m = string_file(get_abs_path("../itn/chinese/data/time/minute.tsv"))
32+
m_zero = string_file(get_abs_path("../itn/chinese/data/time/minute_zero.tsv"))
3233
s = string_file(get_abs_path("../itn/chinese/data/time/second.tsv"))
3334
noon = string_file(get_abs_path("../itn/chinese/data/time/noon.tsv"))
3435

36+
hour = insert('hour: "') + h + insert('"')
37+
minute = insert(' minute: "') + m + delete("分").ques + insert('"')
38+
minute |= insert(' minute: "') + m_zero + delete("分") + insert('"')
39+
minute_zero_no_fen = insert(' minute: "') + m_zero + insert('"')
40+
second = (insert(' second: "') + s + insert('"')).ques
41+
3542
tagger = (
3643
(insert('noon: "') + noon + insert('" ')).ques
37-
+ insert('hour: "')
38-
+ h
39-
+ insert('"')
40-
+ insert(' minute: "')
41-
+ m
42-
+ delete("分").ques
43-
+ insert('"')
44-
+ (insert(' second: "') + s + insert('"')).ques
44+
+ hour + minute + second
45+
)
46+
# "X点零Y" without "分" requires noon prefix to disambiguate from decimal
47+
tagger |= (
48+
insert('noon: "') + noon + insert('" ')
49+
+ hour + minute_zero_no_fen + second
4550
)
4651
self.tagger = self.add_tokens(tagger)
4752

itn/chinese/test/data/time.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,4 +9,4 @@
99
八点零五分 => 8:05
1010
八点三十 => 8:30
1111
八点半 => 8:30
12-
时间上是零点十分登机,八点五分下飞机,八点三十去吃早饭.你是说八点零五分下飞机还是八点零五下飞机?我可能下午三点四十,或者上午十点半再或者下午三点四十一分去找你 => 时间上是00:10登机,8.5分下飞机,8:30去吃早饭.你是说8:05下飞机还是8:05下飞机?我可能3:40p.m.,或者10:30a.m.再或者3:41p.m.去找你
12+
时间上是零点十分登机,八点五分下飞机,八点三十去吃早饭.你是说八点零五分下飞机还是八点零五分下飞机?我可能下午三点四十,或者上午十点半再或者下午三点四十一分去找你 => 时间上是00:10登机,8.5分下飞机,8:30去吃早饭.你是说8:05下飞机还是8:05下飞机?我可能3:40p.m.,或者10:30a.m.再或者3:41p.m.去找你

0 commit comments

Comments
 (0)