Skip to content

Commit b536a8b

Browse files
bactbact
authored
Merge pull request #1248 from PyThaiNLP/copilot/scan-tests-extra-suite
Enable tests.extra with 4-tier test organization and consistent naming conventions
2 parents 7639867 + 425c4c3 commit b536a8b

31 files changed

Lines changed: 926 additions & 718 deletions

‎.github/copilot-instructions.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -163,6 +163,14 @@
163163
- [ ] API endpoints must use proper HTTP return codes.
164164
- [ ] Follow web best practices as recommended by OpenAPI, IETF, W3C, etc.
165165

166+
## Git
167+
168+
- [ ] Follow these guidelines for writing a good commit message:
169+
- How to Write a Git Commit Message
170+
<https://chris.beams.io/posts/git-commit/>
171+
- Commit Verbs 101: why I like to use this and why you should also like it.
172+
<https://chris.beams.io/posts/git-commit/>
173+
166174
## Python
167175

168176
- [ ] Defensive coding: always check for None/empty and handle exceptions

‎.github/workflows/deploy-docs.yml‎

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -35,11 +35,6 @@ jobs:
3535
# pip<24.1 because https://github.com/omry/omegaconf/pull/1195
3636
# setuptools>=65.0.2 because https://github.com/pypa/setuptools/commit/d03da04e024ad4289342077eef6de40013630a44#diff-9ea6e1e3dde6d4a7e08c7c88eceed69ca745d0d2c779f8f85219b22266efff7fR1
3737
# setuptools<=73.0.1 because https://github.com/pypa/setuptools/issues/4620
38-
#- name: Install dependencies
39-
# env:
40-
# SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL: True
41-
# run: |
42-
# if [ -f docker_requirements.txt ]; then pip install -r docker_requirements.txt; fi
4338
- name: Install PyThaiNLP
4439
run: pip install ".[docs]"
4540
- name: Build sphinx documentation

‎.github/workflows/pypi-test.yml‎

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -15,28 +15,36 @@ jobs:
1515
runs-on: ubuntu-latest
1616
strategy:
1717
matrix:
18-
# Ideally, this is the minimum supported version, as in requires-python in pyproject.toml
19-
python-version: ["3.9"]
18+
# Ideally, this should be the minimum supported version,
19+
# as in `requires-python` in pyproject.toml
20+
# But for now, we just test on the second-latest supported version
21+
# because some dependencies may not support the older versions.
22+
python-version: ["3.12"]
2023

2124
steps:
2225
- uses: actions/checkout@v6
26+
2327
- name: Set up Python ${{ matrix.python-version }}
2428
uses: actions/setup-python@v6
2529
with:
2630
python-version: ${{ matrix.python-version }}
31+
cache: "pip"
32+
2733
- name: Install dependencies
2834
env:
2935
SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL: True
3036
run: |
3137
python -m pip install --upgrade "pip<24.1" setuptools
32-
python -m pip install pythainlp[testing,full]
38+
python -m pip install ".[compact]"
3339
python -m nltk.downloader omw-1.4
40+
41+
# Download the sdist from PyPI, extract it, and run the tests inside it
42+
# (the binary distribution does not contain tests).
3443
- name: Test
3544
run: |
3645
set -euo pipefail
3746
mkdir -p pythainlp_test
3847
cd pythainlp_test
39-
# Download sdist from PyPI (the binary distribution does not contain tests)
4048
pip download --no-binary=:all: --no-dependencies pythainlp
4149
archive=$(ls -1 *.tar.gz | head -n1)
4250
echo "Found archive: $archive"
@@ -49,4 +57,4 @@ jobs:
4957
if [ -d data ] && [ -d tests ]; then
5058
mv data tests/
5159
fi
52-
python -m unittest discover -v
60+
python -m unittest tests.core tests.compact -v

‎.github/workflows/unittest.yml‎

Lines changed: 109 additions & 84 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# SPDX-FileCopyrightText: 2016-2026 PyThaiNLP Project
2-
# SPDX-License-Identifier: CC0-1.0
2+
# SPDX-License-Identifier: Apache-2.0
33

44
name: Unit test
55

@@ -32,11 +32,34 @@ jobs:
3232
strategy:
3333
fail-fast: false
3434
matrix:
35-
# Test latest and earliest versions of Python on every OSes
35+
# To save time and resources, and manage dependency compatibility,
36+
# we divide test cases into 3 groups
37+
# and run them in different configurations:
38+
# - Core (O): All supported Python versions on Ubuntu,
39+
# plus latest and earliest Python versions
40+
# on all supported OSes
41+
# - Compact (C): Second-latest and earliest supported Python versions
42+
# on all supported OSes, plus latest on Ubuntu
43+
# - Extra (X): Second-latest supported Python version on Ubuntu only
44+
#
45+
# | Python | Ubuntu | Windows | macOS |
46+
# |--------|---------|---------|-------|
47+
# | 3.14 | O+C | O | O |
48+
# | 3.13 | O+C+X | O+C | O+C |
49+
# | 3.12 | O | | |
50+
# | 3.11 | O | | |
51+
# | 3.10 | O | | |
52+
# | 3.9 | O+C | O+C | O+C |
53+
#
54+
# We should also consider to reduce the number of dependencies
55+
# to avoid conflicts between them.
56+
# See: https://github.com/PyThaiNLP/pythainlp/issues/935
3657
os: ["ubuntu-latest", "windows-latest", "macos-latest"]
37-
python-version: ["3.13", "3.9"] # Latest and earliest
58+
python-version: ["3.14", "3.9"] # Latest and earliest
3859
include:
3960
# Test the rest of Python versions only on Ubuntu
61+
- os: "ubuntu-latest"
62+
python-version: "3.13"
4063
- os: "ubuntu-latest"
4164
python-version: "3.12"
4265
- os: "ubuntu-latest"
@@ -46,87 +69,89 @@ jobs:
4669

4770
runs-on: ${{ matrix.os }}
4871
env:
49-
PYICU_WIN_VER: 2.14
50-
INSTALL_TORCH: false
72+
PYICU_WIN_VER: 2.15 # 2.15 is the last version that supports Python 3.9
5173
INSTALL_FULL_DEPS: false
52-
PYTHON_VERSION_LATEST: "3.13"
53-
PYTHON_VERSION_LATEST_2: "3.12"
74+
PYTHON_VERSION_LATEST: "3.14"
75+
PYTHON_VERSION_LATEST_2: "3.13" # Second-latest supported version
76+
PYTHON_VERSION_EARLIEST: "3.9"
5477

5578
steps:
56-
- name: Checkout
57-
uses: actions/checkout@v6
58-
- name: Set up Python ${{ matrix.python-version }}
59-
uses: actions/setup-python@v6
60-
with:
61-
python-version: ${{ matrix.python-version }}
62-
cache: "pip"
63-
- name: Install build tools
64-
run: |
65-
pip install --upgrade "pip<24.1" "setuptools>=69.0.0,<=73.0.1"
66-
pip install coverage coveralls
67-
# pip<24.1 because https://github.com/omry/omegaconf/pull/1195
68-
# setuptools>=65.0.2 because https://github.com/pypa/setuptools/commit/d03da04e024ad4289342077eef6de40013630a44#diff-9ea6e1e3dde6d4a7e08c7c88eceed69ca745d0d2c779f8f85219b22266efff7fR1
69-
# setuptools<=73.0.1 because https://github.com/pypa/setuptools/issues/4620
70-
- name: Install ICU (macOS)
71-
if: startsWith(matrix.os, 'macos-')
72-
run: |
73-
brew install icu4c
74-
PKG_CONFIG_PATH=$(brew --prefix)/opt/icu4c/lib/pkgconfig
75-
echo "PKG_CONFIG_PATH=${PKG_CONFIG_PATH}" >> "${GITHUB_ENV}"
76-
ICU_VER=$(pkg-config --modversion icu-i18n)
77-
echo "ICU_VER=${ICU_VER}"
78-
echo "ICU_VER=${ICU_VER}" >> "${GITHUB_ENV}"
79-
- name: Install PyICU (Windows)
80-
if: startsWith(matrix.os, 'windows-') && (matrix.python-version == '3.12' || matrix.python-version == '3.13')
81-
shell: powershell
82-
run: |
83-
$PYTHON_WIN_VER = "${{ matrix.python-version }}"
84-
$CP_VER = "cp" + $PYTHON_WIN_VER.Replace(".", "")
85-
$WHEEL_URL = "https://github.com/cgohlke/pyicu-build/releases/download/v${{ env.PYICU_WIN_VER }}/PyICU-${{ env.PYICU_WIN_VER }}-${CP_VER}-${CP_VER}-win_amd64.whl"
86-
pip install "$WHEEL_URL"
87-
# Get wheel URL from https://github.com/cgohlke/pyicu-build/releases
88-
- name: Install PyTorch
89-
if: env.INSTALL_TORCH == 'true'
90-
run: pip install torch
91-
# If torch for the platform is not available in PyPI, use this command:
92-
# pip install "<torch_wheel_url>"
93-
# Get wheel URL from http://download.pytorch.org/whl/torch/
94-
- name: Install testing dependencies
95-
if: env.INSTALL_FULL_DEPS == 'true'
96-
env:
97-
SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL: True
98-
run: pip install ".[testing]"
99-
- name: Install PyThaiNLP + dependencies (minimum)
100-
if: matrix.python-version != env.PYTHON_VERSION_LATEST && matrix.python-version != env.PYTHON_VERSION_LATEST_2
101-
run: pip install .
102-
- name: Install PyThaiNLP + dependencies (compact)
103-
if: matrix.python-version == env.PYTHON_VERSION_LATEST || matrix.python-version == env.PYTHON_VERSION_LATEST_2
104-
run: pip install ".[compact]"
105-
# If you want to install a safe small set of optional dependencies, use:
106-
# pip install ".[compact]"
107-
# We can gradually run more test cases by installing more optional
108-
# dependencies. But we should also consider to reduce the number
109-
# of dependencies to avoid the conflict between dependencies.
110-
# See: https://github.com/PyThaiNLP/pythainlp/issues/935
111-
- name: Unit test (core)
112-
if: matrix.python-version != env.PYTHON_VERSION_LATEST && matrix.python-version != env.PYTHON_VERSION_LATEST_2
113-
env:
114-
PYTHONIOENCODING: utf-8
115-
run: coverage run -m unittest tests.core
116-
- name: Unit test (core + compact)
117-
if: matrix.python-version == env.PYTHON_VERSION_LATEST || matrix.python-version == env.PYTHON_VERSION_LATEST_2
118-
env:
119-
PYTHONIOENCODING: utf-8
120-
run: coverage run -m unittest tests.core tests.compact
121-
# Only test "compact" set with the latest two stable Python versions.
122-
# Use 'unittest <test_module>' instead of 'unittest discover' to avoid
123-
# loading tests with dependencies more than expected.
124-
# Test cases loaded is defined in __init__.py in the tests directory.
125-
# See also tests/README.md
126-
- name: Coverage report
127-
if: matrix.os == 'ubuntu-latest' && matrix.python-version == env.PYTHON_VERSION_LATEST
128-
env:
129-
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
130-
COVERALLS_SERVICE_NAME: github
131-
run: coveralls
132-
# Only submit a report from the latest Python version on ubuntu-latest.
79+
- name: Checkout
80+
uses: actions/checkout@v6
81+
82+
- name: Set up Python ${{ matrix.python-version }}
83+
uses: actions/setup-python@v6
84+
with:
85+
python-version: ${{ matrix.python-version }}
86+
cache: "pip"
87+
88+
- name: Install build tools
89+
run: |
90+
pip install --upgrade "pip<24.1" "setuptools>=69.0.0,<=73.0.1"
91+
pip install coverage coveralls
92+
# pip<24.1 because https://github.com/omry/omegaconf/pull/1195
93+
# setuptools>=65.0.2 because https://github.com/pypa/setuptools/commit/d03da04e024ad4289342077eef6de40013630a44#diff-9ea6e1e3dde6d4a7e08c7c88eceed69ca745d0d2c779f8f85219b22266efff7fR1
94+
# setuptools<=73.0.1 because https://github.com/pypa/setuptools/issues/4620
95+
96+
- name: Install ICU (macOS)
97+
if: startsWith(matrix.os, 'macos-')
98+
run: |
99+
brew install icu4c
100+
PKG_CONFIG_PATH=$(brew --prefix)/opt/icu4c/lib/pkgconfig
101+
echo "PKG_CONFIG_PATH=${PKG_CONFIG_PATH}" >> "${GITHUB_ENV}"
102+
ICU_VER=$(pkg-config --modversion icu-i18n)
103+
echo "ICU_VER=${ICU_VER}"
104+
echo "ICU_VER=${ICU_VER}" >> "${GITHUB_ENV}"
105+
106+
- name: Install PyICU (Windows)
107+
if: startsWith(matrix.os, 'windows-') && (matrix.python-version == env.PYTHON_VERSION_LATEST_2 || matrix.python-version == env.PYTHON_VERSION_EARLIEST)
108+
shell: powershell
109+
# Get the wheel URL from https://github.com/cgohlke/pyicu-build/releases
110+
run: |
111+
$PYTHON_WIN_VER = "${{ matrix.python-version }}"
112+
$CP_VER = "cp" + $PYTHON_WIN_VER.Replace(".", "")
113+
$WHEEL_URL = "https://github.com/cgohlke/pyicu-build/releases/download/v${{ env.PYICU_WIN_VER }}/PyICU-${{ env.PYICU_WIN_VER }}-${CP_VER}-${CP_VER}-win_amd64.whl"
114+
pip install "$WHEEL_URL"
115+
116+
- name: Install PyThaiNLP + full testing dependencies
117+
if: env.INSTALL_FULL_DEPS == 'true'
118+
env:
119+
SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL: True
120+
run: pip install ".[full]"
121+
122+
- name: Test (core)
123+
if: ${{ (matrix.os == 'ubuntu-latest' && matrix.python-version != env.PYTHON_VERSION_LATEST_2 && matrix.python-version != env.PYTHON_VERSION_EARLIEST) || (matrix.os != 'ubuntu-latest' && matrix.python-version == env.PYTHON_VERSION_LATEST) }}
124+
env:
125+
PYTHONIOENCODING: utf-8
126+
run: |
127+
pip install .
128+
python -m unittest tests.core
129+
# Use 'unittest <test_module>' instead of 'unittest discover' to avoid
130+
# loading tests with dependencies more than expected.
131+
# Test cases loaded is defined in __init__.py in the tests directory.
132+
# See also tests/README.md
133+
134+
- name: Test (compact + core)
135+
if: ${{ ((matrix.python-version == env.PYTHON_VERSION_LATEST_2) || (matrix.python-version == env.PYTHON_VERSION_EARLIEST)) && !(matrix.os == 'ubuntu-latest' && matrix.python-version == env.PYTHON_VERSION_LATEST_2) }}
136+
env:
137+
PYTHONIOENCODING: utf-8
138+
run: |
139+
pip install ".[compact]"
140+
python -m unittest tests.core tests.compact
141+
142+
- name: Test (extra + compact + core)
143+
if: matrix.os == 'ubuntu-latest' && matrix.python-version == env.PYTHON_VERSION_LATEST_2
144+
env:
145+
PYTHONIOENCODING: utf-8
146+
run: |
147+
pip install ".[compact,extra]"
148+
coverage run -m unittest tests.core tests.compact tests.extra
149+
150+
# Only submit a report from the "extra" run, to get maximum coverage
151+
- name: Coverage report
152+
if: matrix.os == 'ubuntu-latest' && matrix.python-version == env.PYTHON_VERSION_LATEST_2
153+
env:
154+
COVERALLS_SERVICE_NAME: github
155+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
156+
PYTHONIOENCODING: utf-8
157+
run: coveralls

‎CONTRIBUTING.md‎

Lines changed: 31 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,11 @@ SPDX-License-Identifier: CC0-1.0
66

77
# Contributing to PyThaiNLP
88

9-
Hi! Thanks for your interest in contributing to [PyThaiNLP](https://github.com/PyThaiNLP/pythainlp).
9+
Hi! Thanks for your interest in contributing to
10+
[PyThaiNLP](https://github.com/PyThaiNLP/pythainlp).
1011

11-
Please refer to our [Contributor Covenant Code of Conduct](https://github.com/PyThaiNLP/pythainlp/blob/dev/CODE_OF_CONDUCT.md).
12+
Please refer to our
13+
[Contributor Covenant Code of Conduct](https://github.com/PyThaiNLP/pythainlp/blob/dev/CODE_OF_CONDUCT.md).
1214

1315
## Issue Report and Discussion
1416

@@ -52,8 +54,9 @@ Please refer to our [Contributor Covenant Code of Conduct](https://github.com/Py
5254

5355
### Version Control System
5456

55-
- We use [Git](http://git-scm.com/) as our [version control system](http://en.wikipedia.org/wiki/Revision_control),
56-
so it may be a good idea to familiarize yourself with it.
57+
- We use [Git](http://git-scm.com/) as our
58+
[version control system](http://en.wikipedia.org/wiki/Revision_control),
59+
so it may be a good idea to familiarize yourself with it.
5760
- You can start with the [Pro Git book](http://git-scm.com/book/) (free!).
5861

5962
### Commit Message
@@ -65,19 +68,24 @@ so it may be a good idea to familiarize yourself with it.
6568

6669
- We use the famous [gitflow][] to manage our branches.
6770
- When you create pull requests on GitHub, GitHub Actions will run tests
68-
and several checks automatically. Click the "Details" link at the end of
69-
each check to see what needs to be fixed.
71+
and several checks automatically. Click the "Details" link
72+
at the end of each check to see what needs to be fixed.
7073

7174
[gitflow]: http://nvie.com/posts/a-successful-git-branching-model/
7275

7376
## Documentation
7477

75-
- We use [Sphinx](https://www.sphinx-doc.org/en/master/) to generate API document
76-
automatically from "docstring" comments in source codes. This means the comment
77-
section in the source codes is important for the quality of documentation.
78-
- A docstring should start with one summary line, end with one line with a full stop (period),
79-
then be followed by a blank line before starting a new paragraph.
80-
- A commit to release branches (e.g. `2.2`, `2.1`) with a title **"(build and deploy docs)"** (without quotes) will trigger the system to rebuild the documentation files and upload them to the website <https://pythainlp.org/docs>.
78+
- We use [Sphinx](https://www.sphinx-doc.org/en/master/) to generate
79+
API document automatically from "docstring" comments in source codes.
80+
This means the comment section in the source codes is important for the
81+
quality of documentation.
82+
- A docstring should start with one summary line, end with one line with
83+
a full stop (period), then be followed by a blank line before starting
84+
a new paragraph.
85+
- A commit to release branches (e.g. `2.2`, `2.1`) with a title
86+
**"(build and deploy docs)"** (without quotes) will trigger the system
87+
to rebuild the documentation files and upload them to the website
88+
<https://pythainlp.org/docs>.
8189

8290
## Testing
8391

@@ -124,14 +132,12 @@ Install with optional dependency groups:
124132
# Install with compact set of dependencies (recommended for development)
125133
pip install -e ".[compact]"
126134

127-
# Install with full dependencies
128-
pip install -e ".[full]"
129-
130-
# Install with testing dependencies (pinned versions for reproducibility)
131-
pip install -e ".[testing]"
135+
# Install with extra set of dependencies (can be huge)
136+
pip install -e ".[compact,extra]"
132137
```
133138

134-
See all available optional dependency groups in `pyproject.toml` under `[project.optional-dependencies]`.
139+
See all available optional dependency groups in `pyproject.toml`
140+
under `[project.optional-dependencies]`.
135141

136142
### Building Distribution Packages
137143

@@ -145,8 +151,11 @@ This will create distribution packages in the `dist/` directory.
145151

146152
## Releasing
147153

148-
- We use [semantic versioning](https://semver.org/): MAJOR.MINOR.PATCH, with development build suffix: MAJOR.MINOR.PATCH-devBUILD
149-
- We use [`bump-my-version`](https://github.com/callowayproject/bump-my-version) to manage versioning. The configuration is in `pyproject.toml` under `[tool.bumpversion]`.
154+
- We use [semantic versioning](https://semver.org/): MAJOR.MINOR.PATCH,
155+
with development build suffix: MAJOR.MINOR.PATCH-devBUILD
156+
- We use [`bump-my-version`](https://github.com/callowayproject/bump-my-version)
157+
to manage versioning. The configuration is in `pyproject.toml`
158+
under `[tool.bumpversion]`.
150159
- `bump-my-version bump [major|minor|patch|release|build]`
151160
- Example:
152161

@@ -190,7 +199,8 @@ This will create distribution packages in the `dist/` directory.
190199

191200
[![Contributors](https://contributors-img.firebaseapp.com/image?repo=PyThaiNLP/pythainlp)](https://github.com/PyThaiNLP/pythainlp/graphs/contributors)
192201

193-
Thanks to all [contributors](https://github.com/PyThaiNLP/pythainlp/graphs/contributors). (Image made with [contributors-img](https://contributors-img.firebaseapp.com))
202+
Thanks to all [contributors](https://github.com/PyThaiNLP/pythainlp/graphs/contributors).
203+
(Image made with [contributors-img](https://contributors-img.firebaseapp.com))
194204

195205
### Development Leads
196206

‎Dockerfile‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,5 +12,4 @@ RUN apt-get update && apt-get install -y --no-install-recommends build-essential
1212
ENV VIRTUAL_ENV=/opt/venv
1313
RUN python3 -m venv $VIRTUAL_ENV
1414
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
15-
# Install PyThaiNLP with testing dependencies (replaces docker_requirements.txt)
16-
RUN pip install -e ".[testing]" && pip cache purge
15+
RUN pip install -e ".[full]" && pip cache purge

0 commit comments

Comments
 (0)