Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ The minimum requirement is now Python 3.9.
- Callers raise `FileNotFoundError` with download instructions when
a corpus path cannot be resolved (#1306)
- Migrate build backend to `hatchling` (#1311)
- Update HTTP links to HTTPS in source files and documentation

### Deprecated

Expand All @@ -63,6 +64,8 @@ The minimum requirement is now Python 3.9.
- Duplicated entries in Volubilis dictionary (#1200)
- Star imports (#1207)
- `requests` dependency (#1211)
- `pythainlp.util.is_native_thai` (deprecated since v5.0);
use `pythainlp.morpheme.is_native_thai` instead (#1315)

### Fixed

Expand Down
14 changes: 7 additions & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,21 +44,21 @@ Please refer to our
This is [to please Git][empty-line] and
[to keep up with POSIX standard][posix].

[pep8]: http://www.python.org/dev/peps/pep-0008/
[pep8]: https://peps.python.org/pep-0008/
[black]: https://github.com/ambv/black
[naming]: https://namingconvention.org/python/
[pep0498]: https://www.python.org/dev/peps/pep-0498/
[dead-codes]: http://www.codinghorror.com/blog/2008/07/coding-without-comments.html
[dead-codes]: https://blog.codinghorror.com/coding-without-comments/
[issues]: https://github.com/pythainlp/pythainlp/issues
[empty-line]: https://stackoverflow.com/questions/5813311/no-newline-at-end-of-file#5813359
[posix]: https://stackoverflow.com/questions/729692/why-should-text-files-end-with-a-newline

### Version Control System

- We use [Git](http://git-scm.com/) as our
[version control system](http://en.wikipedia.org/wiki/Revision_control),
- We use [Git](https://git-scm.com/) as our
[version control system](https://en.wikipedia.org/wiki/Revision_control),
so it may be a good idea to familiarize yourself with it.
- You can start with the [Pro Git book](http://git-scm.com/book/) (free!).
- You can start with the [Pro Git book](https://git-scm.com/book/) (free!).

### Commit Message

Expand All @@ -72,7 +72,7 @@ Please refer to our
and several checks automatically. Click the "Details" link
at the end of each check to see what needs to be fixed.

[gitflow]: http://nvie.com/posts/a-successful-git-branching-model/
[gitflow]: https://nvie.com/posts/a-successful-git-branching-model/

## Documentation

Expand Down Expand Up @@ -263,7 +263,7 @@ Thanks to all
(Selection of Important Sentences in Thai Text Summarization
Using a Hierarchical Model).
Retrieved from
[http://digi.library.tu.ac.th/thesis/st/0192/](https://digital.library.tu.ac.th/tu_dc/frontend/Info/item/dc:124897)
<https://digital.library.tu.ac.th/tu_dc/frontend/Info/item/dc:124897>
- **[Thai Discourse Treebank]** --
Ponrawee Prasertsom, Apiwat Jaroonpol, Attapol T. Rutherford;
The Thai Discourse Treebank: Annotating and Classifying Thai
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ please cite the software as follows:
> Phatthiyaphaibun, Wannaphong, Korakot Chaovavanich, Charin Polpanumas,
> Arthit Suriyawongkul, Lalita Lowphansirikul, and Pattarawat Chormai.
> “PyThaiNLP: Thai Natural Language Processing in Python”.
> Zenodo, 2 June 2024. <http://doi.org/10.5281/zenodo.3519354>.
> Zenodo, 2 June 2024. <https://doi.org/10.5281/zenodo.3519354>.

with this BibTeX entry:

Expand Down
2 changes: 1 addition & 1 deletion README_TH.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ PyThaiNLP ดาวน์โหลดข้อมูล (ดูแค็ตต
> Phatthiyaphaibun, Wannaphong, Korakot Chaovavanich, Charin Polpanumas,
> Arthit Suriyawongkul, Lalita Lowphansirikul, and Pattarawat Chormai.
> “PyThaiNLP: Thai Natural Language Processing in Python”.
> Zenodo, 2 June 2024. <http://doi.org/10.5281/zenodo.3519354>.
> Zenodo, 2 June 2024. <https://doi.org/10.5281/zenodo.3519354>.

โดยใช้รายการ BibTeX นี้:

Expand Down
4 changes: 2 additions & 2 deletions docs/api/soundex.rst
Original file line number Diff line number Diff line change
Expand Up @@ -90,9 +90,9 @@ References

.. [#metasound] Snae & Brückner. (2009). `Novel Phonetic Name Matching Algorithm with a Statistical Ontology for Analyzing Names Given in Accordance with Thai Astrology <https://pdfs.semanticscholar.org/3983/963e87ddc6dfdbb291099aa3927a0e3e4ea6.pdf>`_.

.. [#udom83] Wannee Udompanich (1983). `String searching for Thai alphabet using Soundex compression technique <http://cuir.car.chula.ac.th/handle/123456789/48471>`_. Master Thesis. Chulalongkorn University, Thailand. วรรณี อุดมพาณิชย์. `การใช้หลักคำพ้องเสียง เพื่อค้นหาชุดอักขระภาษาไทยที่ออกเสียงเหมือนกัน <http://cuir.car.chula.ac.th/handle/123456789/48471>`_. วิทยานิพนธ์ (วท.ม.)--จุฬาลงกรณ์มหาวิทยาลัย, 2526.
.. [#udom83] Wannee Udompanich (1983). `String searching for Thai alphabet using Soundex compression technique <https://cuir.car.chula.ac.th/handle/123456789/48471>`_. Master Thesis. Chulalongkorn University, Thailand. วรรณี อุดมพาณิชย์. `การใช้หลักคำพ้องเสียง เพื่อค้นหาชุดอักขระภาษาไทยที่ออกเสียงเหมือนกัน <https://cuir.car.chula.ac.th/handle/123456789/48471>`_. วิทยานิพนธ์ (วท.ม.)--จุฬาลงกรณ์มหาวิทยาลัย, 2526.

.. [#lk82] Vichit Lorchirachoonkul. 1982. `A Thai soundex system <https://doi.org/10.1016/0306-4573(82)90003-6>`_. Information Processing & Management, 18(5):243–255. วิชิต หล่อจีระชุณห์กุล และ เจริญ คุวินทร์พันธุ์. `โปรแกรมการสืบค้นคำไทยตามเสียงอ่าน (Thai Soundex) <http://guru.sanook.com/1520/>`_.
.. [#lk82] Vichit Lorchirachoonkul. 1982. `A Thai soundex system <https://doi.org/10.1016/0306-4573(82)90003-6>`_. Information Processing & Management, 18(5):243–255. วิชิต หล่อจีระชุณห์กุล และ เจริญ คุวินทร์พันธุ์. `โปรแกรมการสืบค้นคำไทยตามเสียงอ่าน (Thai Soundex) <https://guru.sanook.com/1520/>`_.

.. [#prayut_and_somchaip] Prayut Suwanvisat, Somchai Prasitjutrakul. Thai-English Cross-Language Transliterated Word Retrieval using Soundex Technique. In 1998 [cited 2022 Sep 8]. Available from: https://www.cp.eng.chula.ac.th/~somchai/spj/papers/ThaiText/ncsec98-clir.pdf.

Expand Down
4 changes: 2 additions & 2 deletions docs/api/spell.rst
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,11 @@ The `NorvigSpellChecker` class is a fundamental component of the `pythainlp.spel
DEFAULT_SPELL_CHECKER
~~~~~~~~~~~~~~~~~~~~~
.. autodata:: DEFAULT_SPELL_CHECKER
:annotation: = Default reference of the standard NorvigSpellChecker, using word list data from the Thai National Corpus: http://www.arts.chula.ac.th/ling/tnc/
:annotation: = Default reference of the standard NorvigSpellChecker, using word list data from the Thai National Corpus: https://www.arts.chula.ac.th/ling/tnc3/

The `DEFAULT_SPELL_CHECKER` is an reference to the `NorvigSpellChecker` class with default settings. It is pre-configured to use word list data from the Thai National Corpus, making it a reliable choice for general spell-checking tasks.

References
----------

.. [#norvig_spellchecker] Peter Norvig (2007). `How to Write a Spelling Corrector <http://norvig.com/spell-correct.html>`_.
.. [#norvig_spellchecker] Peter Norvig (2007). `How to Write a Spelling Corrector <https://norvig.com/spell-correct.html>`_.
2 changes: 1 addition & 1 deletion docs/api/tag.rst
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,7 @@ Named Entity tag Examples
ORGANIZATION กรมวิทยาศาสตร์การแพทย์, อย.
PERSON น.พ.จรัล, นางประนอม ทองจันทร์
PHONE 1200, 0 2670 8888
URL http://www.bangkokhealth.com/
URL https://www.bangkokhealth.com/
ZIP 10400, 11130
Money 2.7 ล้านบาท, 2,000 บาท
LAW พ.ร.บ.โรคระบาด พ.ศ.2499, รัฐธรรมนูญ
Expand Down
2 changes: 1 addition & 1 deletion docs/api/transliterate.rst
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ This section includes multiple transliteration engines designed to suit various
References
----------

.. [#rtgs_transcription] Nitaya Kanchanawan. (2006). `Romanization, Transliteration, and Transcription for the Globalization of the Thai Language. <http://www.royin.go.th/wp-content/uploads/royin-ebook/276/FileUpload/758_6484.pdf>`_
.. [#rtgs_transcription] Nitaya Kanchanawan. (2006). `Romanization, Transliteration, and Transcription for the Globalization of the Thai Language. <http://legacy.orst.go.th/wp-content/uploads/royin-ebook/276/FileUpload/758_6484.pdf>`_
The Journal of the Royal Institute of Thailand.

The `pythainlp.transliterate` module offers a comprehensive set of tools and engines for transliterating Thai text into Romanized form. Whether you need a simple transliteration, specific engines for accurate representation, or phonetic rendering, this module provides a wide range of options. Additionally, the module references a publication that highlights the significance of Romanization, Transliteration, and Transcription in making the Thai language accessible to a global audience.
2 changes: 1 addition & 1 deletion docs/api/wangchanberta.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,4 +47,4 @@ Modules
References
----------

.. [#Lowphansirikul_2021] Lowphansirikul L, Polpanumas C, Jantrakulchai N, Nutanong S. WangchanBERTa: Pretraining transformer-based Thai Language Models. `ArXiv:2101.09635 <http://arxiv.org/abs/2101.09635>`_ [Internet]. 2021 Jan 23 [cited 2021 Feb 27].
.. [#Lowphansirikul_2021] Lowphansirikul L, Polpanumas C, Jantrakulchai N, Nutanong S. WangchanBERTa: Pretraining transformer-based Thai Language Models. `ArXiv:2101.09635 <https://arxiv.org/abs/2101.09635>`_ [Internet]. 2021 Jan 23 [cited 2021 Feb 27].
2 changes: 1 addition & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# SPDX-License-Identifier: Apache-2.0
#
# Configuration file for the Sphinx documentation builder.
# http://www.sphinx-doc.org/en/master/config
# https://www.sphinx-doc.org/en/master/usage/configuration.html

import os
import sys
Expand Down
2 changes: 1 addition & 1 deletion docs/notes/license.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ License
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0
https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
Expand Down
2 changes: 1 addition & 1 deletion pythainlp/corpus/corpus_license.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ and released under
A Thai word list from ICU (International Components for Unicode) project
(icubrk_th.txt) is copyrighted by Unicode, Inc. and others.,
released under **Unicode License Agreement - Data Files and Software (2016)**
<http://www.unicode.org/copyright.html>
<https://www.unicode.org/copyright.html>

Original data:
<https://github.com/unicode-org/icu/blob/main/icu4c/source/data/brkitr/dictionaries/thaidict.txt>
Expand Down
2 changes: 1 addition & 1 deletion pythainlp/corpus/icubrk_th.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# SPDX-FileType: TEXT
# SPDX-FileCopyrightText: Copyright (C) 2016 and later: Unicode, Inc. and others.
# SPDX-FileCopyrightText: License & terms of use: http://www.unicode.org/copyright.html
# SPDX-FileCopyrightText: License & terms of use: https://www.unicode.org/copyright.html
# SPDX-FileCopyrightText: Copyright (c) 2006-2015 International Business Machines Corporation,
# SPDX-FileCopyrightText: Apple Inc., and others. All Rights Reserved.
# SPDX-License-Identifier: Unicode-DFS-2016
Expand Down
2 changes: 1 addition & 1 deletion pythainlp/corpus/wordnet.py
Original file line number Diff line number Diff line change
Expand Up @@ -436,7 +436,7 @@ def morphy(form: str, pos: Optional[str] = None) -> str:

def custom_lemmas(tab_file: Union[str, IO[str]], lang: str) -> None:
"""This function reads a custom tab file
(see: http://compling.hss.ntu.edu.sg/omw/)
(see: https://omwn.org/)
containing mappings of lemmas in the given language.

:param tab_file: Tab file as a file or file-like object
Expand Down
2 changes: 1 addition & 1 deletion pythainlp/morpheme/thaiwordcheck.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
References
----------
- ทีมงานทรูปลูกปัญญา 2015. ลักษณะของคำไทยแท้ \
http://www.trueplookpanya.com/learning/detail/30589-043067
https://www.trueplookpanya.com/learning/detail/30589-043067
- วารุณี บำรุงรส 2010. คำไทยแท้ https://www.gotoknow.org/posts/377619

"""
Expand Down
2 changes: 1 addition & 1 deletion pythainlp/soundex/udom83.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
using Soundex compression technique. Master Thesis
of Department of Computer Engineering Graduate
School, Chulalongkorn University, 1983.
http://cuir.car.chula.ac.th/handle/123456789/48471
https://cuir.car.chula.ac.th/handle/123456789/48471

Python implementation:
by Korakot Chaovavanich
Expand Down
2 changes: 1 addition & 1 deletion pythainlp/spell/pn.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Default spelling dictionary is based on Phupha: Thai Word Frequency Dataset,
filtered with Royal Society of Thailand word list.

Based on Peter Norvig's Python code from http://norvig.com/spell-correct.html
Based on Peter Norvig's Python code from https://norvig.com/spell-correct.html
"""

from __future__ import annotations
Expand Down
4 changes: 2 additions & 2 deletions pythainlp/tag/_tag_perceptron.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ class AveragedPerceptron:
"""An averaged perceptron, as implemented by Matthew Honnibal.

See more implementation details here:
http://honnibal.wordpress.com/2013/09/11/a-good-part-of-speechpos-tagger-in-about-200-lines-of-python/
https://honnibal.wordpress.com/2013/09/11/a-good-part-of-speechpos-tagger-in-about-200-lines-of-python/
"""

weights: dict[str, dict[str, float]]
Expand Down Expand Up @@ -104,7 +104,7 @@ class PerceptronTagger:
"""Greedy Averaged Perceptron tagger, as implemented by Matthew Honnibal.

See more implementation details here:
http://honnibal.wordpress.com/2013/09/11/a-good-part-of-speechpos-tagger-in-about-200-lines-of-python/
https://honnibal.wordpress.com/2013/09/11/a-good-part-of-speechpos-tagger-in-about-200-lines-of-python/

>>> from pythainlp.tag import PerceptronTagger
>>> tagger = PerceptronTagger()
Expand Down
2 changes: 1 addition & 1 deletion pythainlp/tokenize/tcc.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# SPDX-License-Identifier: Apache-2.0
"""The implementation of tokenizer according to Thai Character Clusters (TCCs)
rules proposed by `Theeramunkong et al. 2000. \
<http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.59.2548>`_
<https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.59.2548>`_

Credits:
* TCC: Jakkrit TeCho
Expand Down
2 changes: 1 addition & 1 deletion pythainlp/tokenize/tcc_p.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# SPDX-License-Identifier: Apache-2.0
"""The implementation of tokenizer according to Thai Character Clusters (TCCs)
rules proposed by `Theeramunkong et al. 2000. \
<http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.59.2548>`_
<https://doi.org/10.1145/355214.355225>`_
and improved rules that are used in newmm

Credits:
Expand Down
4 changes: 2 additions & 2 deletions pythainlp/translate/tokenization_small100.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# SPDX-FileType: SOURCE
# SPDX-License-Identifier: Apache-2.0

# Copyright (c) 2022 Idiap Research Institute, http://www.idiap.ch/
# Copyright (c) 2022 Idiap Research Institute, https://www.idiap.ch/
# Written by Alireza Mohammadshahi <alireza.mohammadshahi@idiap.ch>
# This is a modified version of https://github.com/huggingface/transformers/blob/main/src/transformers/models/m2m_100/tokenization_m2m_100.py
# which owns by Fariseq Authors and The HuggingFace Inc. team.
Expand All @@ -12,7 +12,7 @@
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
Expand Down
2 changes: 0 additions & 2 deletions pythainlp/util/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@
"find_keyword",
"find_profanity",
"ipa_to_rtgs",
"is_native_thai",
"isthai",
"isthaichar",
"longest_common_subsequence",
Expand Down Expand Up @@ -138,7 +137,6 @@
thai_word_tone_detector,
)
from pythainlp.util.thai_lunar_date import th_zodiac, to_lunar_date
from pythainlp.util.thaiwordcheck import is_native_thai
from pythainlp.util.time import thaiword_to_time, time_to_thaiword
from pythainlp.util.trie import Trie, dict_trie
from pythainlp.util.wordtonum import text_to_num, thaiword_to_num, words_to_num
Expand Down
2 changes: 1 addition & 1 deletion pythainlp/util/numtoword.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"""Convert number value to Thai read out

Adapted from
http://justmindthought.blogspot.com/2012/12/code-php.html
https://justmindthought.blogspot.com/2012/12/code-php.html
https://suksit.com/post/writing-bahttext-in-php/
"""

Expand Down
4 changes: 2 additions & 2 deletions pythainlp/util/strftime.py
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ def thai_strftime(

* Python
https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior
* C http://www.cplusplus.com/reference/ctime/strftime/
* C https://en.cppreference.com/w/cpp/chrono/c/strftime
* GNU https://metacpan.org/pod/POSIX::strftime::GNU
* Linux https://linux.die.net/man/3/strftime
* OpenBSD https://man.openbsd.org/strftime.3
Expand All @@ -195,7 +195,7 @@ def thai_strftime(
https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/strftime.3.html
* PHP https://secure.php.net/manual/en/function.strftime.php
* JavaScript's implementation https://github.com/samsonjs/strftime
* strftime() quick reference http://www.strftime.net/
* strftime() quick reference https://strftime.net/

:param datetime dt_obj: an instantiatetd object of
:mod:`datetime.datetime`
Expand Down
19 changes: 0 additions & 19 deletions pythainlp/util/thaiwordcheck.py

This file was deleted.

Loading