Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/deploy-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
python-version: "3.10"
- name: Install build tools and doc build tools
run: |
pip install --upgrade "pip<24.1" "setuptools>=65.0.2,<=73.0.1"
pip install --upgrade "pip<24.1" "setuptools>=69.0.0,<=73.0.1"
pip install boto smart_open sphinx sphinx-rtd-theme
# pip<24.1 because https://github.com/omry/omegaconf/pull/1195
# setuptools>=65.0.2 because https://github.com/pypa/setuptools/commit/d03da04e024ad4289342077eef6de40013630a44#diff-9ea6e1e3dde6d4a7e08c7c88eceed69ca745d0d2c779f8f85219b22266efff7fR1
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/pypi-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,7 @@ jobs:
SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL: True
run: |
python -m pip install --upgrade "pip<24.1" setuptools
python -m pip install -r https://raw.githubusercontent.com/PyThaiNLP/pythainlp/dev/docker_requirements.txt
python -m pip install pythainlp[full]
python -m pip install pythainlp[testing,full]
python -m nltk.downloader omw-1.4
- name: Test
run: |
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/unittest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ jobs:
cache: "pip"
- name: Install build tools
run: |
pip install --upgrade "pip<24.1" "setuptools>=65.0.2,<=73.0.1"
pip install --upgrade "pip<24.1" "setuptools>=69.0.0,<=73.0.1"
pip install coverage coveralls
# pip<24.1 because https://github.com/omry/omegaconf/pull/1195
# setuptools>=65.0.2 because https://github.com/pypa/setuptools/commit/d03da04e024ad4289342077eef6de40013630a44#diff-9ea6e1e3dde6d4a7e08c7c88eceed69ca745d0d2c779f8f85219b22266efff7fR1
Expand Down Expand Up @@ -70,11 +70,11 @@ jobs:
# If torch for the platform is not available in PyPI, use this command:
# pip install "<torch_wheel_url>"
# Get wheel URL from http://download.pytorch.org/whl/torch/
- name: Install dependencies from docker_requirements.txt
- name: Install testing dependencies
if: env.INSTALL_FULL_DEPS == 'true'
env:
SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL: True
run: pip install -r docker_requirements.txt
run: pip install ".[testing]"
- name: Install PyThaiNLP + dependencies (minimum)
if: matrix.python-version != env.PYTHON_VERSION_LATEST && matrix.python-version != env.PYTHON_VERSION_LATEST_2
run: pip install .
Expand Down
61 changes: 48 additions & 13 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
SPDX-FileCopyrightText: 2025 PyThaiNLP Project
SPDX-FileCopyrightText: 2025-2026 PyThaiNLP Project
SPDX-FileType: DOCUMENTATION
SPDX-License-Identifier: CC0-1.0
---
Expand Down Expand Up @@ -108,44 +108,79 @@ Make sure the tests pass on GitHub Actions.

See more in [tests/README.md](./tests/README.md)

## Installing and Building

### Installing for Development

Install PyThaiNLP in editable mode with core dependencies:

```sh
pip install -e .
```

Install with optional dependency groups:

```sh
# Install with compact set of dependencies (recommended for development)
pip install -e ".[compact]"

# Install with full dependencies
pip install -e ".[full]"

# Install with testing dependencies (pinned versions for reproducibility)
pip install -e ".[testing]"
```

See all available optional dependency groups in `pyproject.toml` under `[project.optional-dependencies]`.

### Building Distribution Packages

To build source distribution and wheel:

```sh
python -m build
```

This will create distribution packages in the `dist/` directory.

## Releasing

- We use [semantic versioning](https://semver.org/): MAJOR.MINOR.PATCH, with development build suffix: MAJOR.MINOR.PATCH-devBUILD
- We use [`bumpversion`](https://github.com/c4urself/bump2version/#installation) to manage versioning.
- `bumpversion [major|minor|patch|release|build]`
- We use [`bump-my-version`](https://github.com/callowayproject/bump-my-version) to manage versioning. The configuration is in `pyproject.toml` under `[tool.bumpversion]`.
- `bump-my-version bump [major|minor|patch|release|build]`
- Example:

```sh
#current_version = 2.3.3-dev0

bumpversion build
bump-my-version bump build
#current_version = 2.3.3-dev1

bumpversion build
bump-my-version bump build
#current_version = 2.3.3-dev2

bumpversion release
bump-my-version bump release
#current_version = 2.3.3-beta0

bumpversion release
bump-my-version bump release
#current_version = 2.3.3

bumpversion patch
bump-my-version bump patch
#current_version = 2.3.6-dev0

bumpversion minor
bump-my-version bump minor
#current_version = 2.3.1-dev0

bumpversion build
bump-my-version bump build
#current_version = 2.3.1-dev1

bumpversion major
bump-my-version bump major
#current_version = 3.0.0-dev0

bumpversion release
bump-my-version bump release
#current_version = 3.0.0-beta0

bumpversion release
bump-my-version bump release
#current_version = 3.0.0
```

Expand Down
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ RUN apt-get update && apt-get install -y --no-install-recommends build-essential
ENV VIRTUAL_ENV=/opt/venv
RUN python3 -m venv $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
RUN if [ -f docker_requirements.txt ]; then pip install -r docker_requirements.txt; fi
RUN pip install -e .[full] && pip cache purge
# Install PyThaiNLP with testing dependencies (replaces docker_requirements.txt)
RUN pip install -e ".[testing]" && pip cache purge
15 changes: 7 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -50,25 +50,24 @@ lint: ## check style with flake8
flake8 pythainlp tests

test: ## run tests quickly with the default Python
python setup.py test
python -m unittest discover

test-all: ## run tests on every Python version with tox
tox

coverage: ## check code coverage quickly with the default Python
coverage run --source pythainlp setup.py test
coverage run --source pythainlp -m unittest discover
coverage report -m
coverage html
$(BROWSER) htmlcov/index.html

release: clean ## package and upload a release
python setup.py sdist upload
python setup.py bdist_wheel upload
release: clean ## package and upload a release (deprecated - use GitHub Actions)
python -m build
python -m twine upload dist/*

dist: clean ## builds source and wheel package
python setup.py sdist
python setup.py bdist_wheel
python -m build
ls -l dist

install: clean ## install the package to the active Python's site-packages
python setup.py install
python -m pip install .
32 changes: 29 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ pip install pythainlp

## Getting Started

- PyThaiNLP requires Python 3.7+.
- PyThaiNLP requires Python 3.9+.
- Python 2.7 users can use PyThaiNLP 1.6. See [2.0 change log](https://github.com/PyThaiNLP/pythainlp/issues/118) | [Upgrading from 1.7](https://pythainlp.org/docs/2.0/notes/pythainlp-1_7-2_0.html) | [Upgrading ThaiNER from 1.7](https://github.com/PyThaiNLP/pythainlp/wiki/Upgrade-ThaiNER-from-PyThaiNLP-1.7-to-PyThaiNLP-2.0)
- [PyThaiNLP Get Started notebook](https://pythainlp.org/tutorials/notebooks/pythainlp_get_started.html) | [API document](https://pythainlp.org/docs) | [Tutorials](https://pythainlp.org/tutorials)
- [Official website](https://pythainlp.org/) | [PyPI](https://pypi.org/project/pythainlp/) | [Facebook page](https://www.facebook.com/pythainlp/)
Expand Down Expand Up @@ -81,17 +81,43 @@ Possible `extras`:

- `full` (install everything)
- `compact` (install a stable and small subset of dependencies)
- `abbreviation` (for Thai abbreviation support)
- `attacut` (to support attacut, a fast and accurate tokenizer)
- `benchmarks` (for [word tokenization benchmarking](tokenization-benchmark.md))
- `budoux` (for BudouX text segmentation)
- `coreference_resolution` (for coreference resolution)
- `dependency_parsing` (for dependency parsing)
- `el` (for entity linking)
- `esupar` (for esupar parser support)
- `generate` (for text generation)
- `icu` (for ICU, International Components for Unicode, support in transliteration and tokenization)
- `ipa` (for IPA, International Phonetic Alphabet, support in transliteration)
- `ml` (to support ULMFiT models for classification)
- `mt5` (for mT5 model support)
- `nlpo3` (for nlpo3 Thai word tokenizer)
- `onnx` (for ONNX model support)
- `oskut` (for OSKut Thai word tokenizer)
- `sefr_cut` (for SEFR CUT Thai word tokenizer)
- `spacy_thai` (for spaCy Thai language support)
- `spell` (for spelling correction)
- `ssg` (for sentence segmentation)
- `testing` (pinned versions for CI/CD reproducibility)
- `textaugment` (for text augmentation)
- `thai_nner` (for Thai named entity recognition)
- `thai2fit` (for Thai word vector)
- `thai2rom` (for machine-learnt romanization)
- `transformers_ud` (for Universal Dependencies with transformers)
- `translate` (for machine translation)
- `wangchanberta` (for WangchanBERTa model)
- `wangchanglm` (for WangchanGLM model)
- `word_approximation` (for word approximation)
- `wordnet` (for Thai WordNet API)
- `wsd` (for word sense disambiguation)
- `wtp` (for Where's the Point text segmentation)
- `wunsen` (for Wunsen spell checker)

For dependency details, look at the `extras` variable in
[`setup.py`](https://github.com/PyThaiNLP/pythainlp/blob/dev/setup.py).
For dependency details, look at the `[project.optional-dependencies]` section in
[`pyproject.toml`](https://github.com/PyThaiNLP/pythainlp/blob/dev/pyproject.toml).

## Data Directory

Expand Down
30 changes: 28 additions & 2 deletions README_TH.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,18 +86,44 @@ pip install pythainlp[extra1,extra2,...]

- `full` (ติดตั้งทุกอย่าง)
- `compact` (ติดตั้งไลบารีชุดเล็กที่ทดสอบแล้วว่าไม่ตีกันเองและติดตั้งได้ในทุกระบบปฏิบัติการ)
- `abbreviation` (สำหรับการย่อคำภาษาไทย)
- `attacut` (เพื่อสนับสนุน attacut ซึ่งเป็นตัวตัดคำที่ทำงานได้รวดเร็วและมีประสิทธิภาพ)
- `benchmarks` (สำหรับ [word tokenization benchmarking](tokenization-benchmark.md))
- `budoux` (สำหรับการแบ่งข้อความด้วย BudouX)
- `coreference_resolution` (สำหรับการหาคำที่อ้างอิงถึงกัน)
- `dependency_parsing` (สำหรับการวิเคราะห์โครงสร้างประโยค)
- `el` (สำหรับการเชื่อมโยงเอนทิตี)
- `esupar` (สำหรับการรองรับ esupar parser)
- `generate` (สำหรับการสร้างข้อความ)
- `icu` (สำหรับการรองรับ ICU หรือ International Components for Unicode ในการถอดเสียงเป็นอักษรและการตัดแบ่งคำ)
- `ipa` (สำหรับการรองรับ IPA หรือ International Phonetic Alphabet ในการถอดเสียงเป็นอักษร)
- `ml` (เพื่อให้สนับสนุนตัวแบบภาษา ULMFiT สำหรับการจำแนกข้อความ)
- `mt5` (สำหรับรองรับโมเดล mT5)
- `nlpo3` (สำหรับตัวตัดคำภาษาไทย nlpo3)
- `onnx` (สำหรับรองรับโมเดล ONNX)
- `oskut` (สำหรับตัวตัดคำภาษาไทย OSKut)
- `sefr_cut` (สำหรับตัวตัดคำภาษาไทย SEFR CUT)
- `spacy_thai` (สำหรับรองรับภาษาไทยใน spaCy)
- `spell` (สำหรับการแก้ไขคำสะกดผิด)
- `ssg` (สำหรับการแบ่งประโยค)
- `testing` (เวอร์ชันที่ปักหมุดสำหรับ CI/CD)
- `textaugment` (สำหรับการเพิ่มข้อมูลข้อความ)
- `thai_nner` (สำหรับการจดจำชื่อเฉพาะภาษาไทย)
- `thai2fit` (สำหรับ Thai word vector)
- `thai2rom` (สำหรับการถอดอักษรไทยเป็นอักษรโรมัน)
- `transformers_ud` (สำหรับ Universal Dependencies ด้วย transformers)
- `translate` (สำหรับการแปลภาษา)
- `wangchanberta` (สำหรับโมเดล WangchanBERTa)
- `wangchanglm` (สำหรับโมเดล WangchanGLM)
- `word_approximation` (สำหรับการประมาณคำ)
- `wordnet` (สำหรับ Thai WordNet API)
- `wsd` (สำหรับการแก้ความกำกวมของความหมายคำ)
- `wtp` (สำหรับการแบ่งข้อความด้วย Where's the Point)
- `wunsen` (สำหรับตัวตรวจการสะกดคำ Wunsen)

</details>

สำหรับโมดูลที่ต้องการ สามารถดูรายละเอียดได้ที่ตัวแปร `extras` ใน [`setup.py`](https://github.com/PyThaiNLP/pythainlp/blob/dev/setup.py).
สำหรับโมดูลที่ต้องการ สามารถดูรายละเอียดได้ที่ส่วน `[project.optional-dependencies]` ใน [`pyproject.toml`](https://github.com/PyThaiNLP/pythainlp/blob/dev/pyproject.toml).

## Command-line

Expand All @@ -117,7 +143,7 @@ thainlp help

## ผู้ใช้งาน Python 2

- PyThaiNLP สนับสนุน Python 3.7 ขึ้นไป บางความสามารถ สามารถใช้งานกับ Python 3 รุ่นก่อนหน้าได้ แต่ไม่ได้มีการทดสอบว่าใช้งานได้หรือไม่ อ่านเพิ่มเติม [1.7 -> 2.0 change log](https://github.com/PyThaiNLP/pythainlp/issues/118).
- PyThaiNLP สนับสนุน Python 3.9 ขึ้นไป บางความสามารถ สามารถใช้งานกับ Python 3 รุ่นก่อนหน้าได้ แต่ไม่ได้มีการทดสอบว่าใช้งานได้หรือไม่ อ่านเพิ่มเติม [1.7 -> 2.0 change log](https://github.com/PyThaiNLP/pythainlp/issues/118).
- [Upgrading from 1.7](https://pythainlp.org/docs/2.0/notes/pythainlp-1_7-2_0.html)
- [Upgrade ThaiNER from 1.7](https://github.com/PyThaiNLP/pythainlp/wiki/Upgrade-ThaiNER-from-PyThaiNLP-1.7-to-PyThaiNLP-2.0)
- ผู้ใช้งาน Python 2.7 สามารถใช้งาน PyThaiNLP 1.6
Expand Down
16 changes: 8 additions & 8 deletions codemeta.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"@type": "SoftwareSourceCode",
"name": "PyThaiNLP",
"description": "Thai Natural Language Processing in Python",
"version": "5.1.0",
"version": "5.2.0",
"author": [
{
"@type": "Person",
Expand Down Expand Up @@ -61,15 +61,15 @@
"issueTracker": "https://github.com/PyThaiNLP/pythainlp/issues",
"url": "https://pythainlp.org/",
"keywords": [
"NLP",
"natural language processing",
"Thai",
"Python",
"tokenization",
"text processing",
"linguistics",
"localization",
"computational linguistics",
"tokenization",
"word segmentation",
"NLP",
"Thai language",
"Thai NLP"
"ThaiNLP",
"Thai NLP",
"Thai language"
]
}
38 changes: 0 additions & 38 deletions docker_requirements.txt

This file was deleted.

Loading
Loading