Skip to content

Commit 80c25cd

Browse files
bbae0312pre-commit-ci[bot]
bbae0312
and
pre-commit-ci[bot]
committed
Korean TN for Date and Time (NVIDIA#316)
* feat(ko/date): Add date TN taggers, verbalizers, test cases, and post-processing fixes Signed-off-by: Jinwoo Bae <bbae7050@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(ko/date): update date tagger and sparrowhawk test Signed-off-by: Jinwoo Bae <bbae7050@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ko(TN): Date TN fixes & cleanup Signed-off-by: Jinwoo Bae <bbae7050@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ko(TN): Add Time tagger/verbalizer + tests Signed-off-by: Jinwoo Bae <bbae7050@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * ko(TN): Date — strict YYYY for delimited formats; define single-year 1–4 digit behavior Signed-off-by: Jinwoo Bae <bbae7050@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Jinwoo Bae <bbae7050@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Jinwoo Bae <bbae7050@gmail.com>
1 parent aacba0f commit 80c25cd

File tree

23 files changed

+1036
-29
lines changed

23 files changed

+1036
-29
lines changed

‎nemo_text_processing/text_normalization/ko/data/date/__init__.py‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎nemo_text_processing/text_normalization/ko/data/date/exceptions.tsv‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
6 유
2+
10 시

‎nemo_text_processing/text_normalization/ko/data/date/week.tsv‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
월 월요일
2+
화 화요일
3+
수 수요일
4+
목 목요일
5+
금 금요일
6+
토 토요일
7+
일 일요일
8+
공 공휴일

‎nemo_text_processing/text_normalization/ko/data/time/__init__.py‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.

‎nemo_text_processing/text_normalization/ko/data/time/division.tsv‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
오전
2+
오후
3+
새벽
4+
아침
5+
낮
6+
저녁
7+
밤

‎nemo_text_processing/text_normalization/ko/data/time/hour.tsv‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
1 한
2+
2 두
3+
3 세
4+
4 네
5+
5 다섯
6+
6 여섯
7+
7 일곱
8+
8 여덟
9+
9 아홉
10+
10 열
11+
11 열한
12+
12 열두

‎nemo_text_processing/text_normalization/ko/data/whitelist.tsv‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
Dr. 박사
2+
dr. 박사
3+
Mr. 씨
4+
mr. 씨
5+
Ms. 양
6+
ms. 양
7+
Mrs. 여사
8+
mrs. 여사
9+
mt. 산
10+
Mt. 산
11+
Prof. 교수
12+
prof. 교수
13+
sr. 시니어
14+
Sr. 시니어
15+
jr. 주니어
16+
Jr. 주니어
17+
rd. 로
18+
Rd. 로
19+
Ave. 대로
20+
ave. 대로
21+
no. 번호
22+
No. 번호
23+
( 왼쪽 괄호
24+
) 오른쪽 괄호
25+
+ 더하기
26+
- 마이너스
27+
Σ 시그마
28+
η 에타
29+
κ 카파
30+
ω 오메가
31+
σ 시그마
32+
α 알파
33+
ν 뉴
34+
δ 델타
35+
ι 이오타
36+
vs. 대
37+
Ph.D. 박사학위
38+
etc. 등

0 commit comments

Comments
 (0)