|
1 | | -# Test cases |
| 1 | +# Test suites and execution |
2 | 2 |
|
3 | | -The default test suite, triggered by the `unittest tests` command, encompasses |
4 | | -all test cases within the `tests.core` and `tests.compact` packages. |
5 | | -This suite is defined within the `__init__.py` file in this directory. |
| 3 | +The default test suite is executed using the `unittest tests` command. |
| 4 | +It includes all test cases within the `tests.core` and `tests.compact` |
| 5 | +packages, as defined in the `tests/__init__.py` file. |
6 | 6 |
|
7 | | -Tests are categorized into three groups: core, compact, and extra. |
| 7 | +To optimize CI/CD resource utilization and manage dependency overhead, |
| 8 | +tests are categorized into four tiers based on their resource |
| 9 | +requirements and complexity: "core", "compact", "extra", and "noauto". |
8 | 10 |
|
| 11 | +The following table outlines the automated test coverage across |
| 12 | +supported Python versions and operating systems: |
| 13 | + |
| 14 | +| Python | Ubuntu | Windows | macOS | |
| 15 | +| -------------- | ------- | ------- | ----- | |
| 16 | +| 3.14 (Latest) | O+C | O | O | |
| 17 | +| 3.13 | O+C+X | O+C | O+C | |
| 18 | +| 3.12 | O | | | |
| 19 | +| 3.11 | O | | | |
| 20 | +| 3.10 | O | | | |
| 21 | +| 3.9 (Earliest) | O+C | O+C | O+C | |
| 22 | + |
9 | 23 | ## Core tests (test_*.py) |
10 | 24 |
|
11 | 25 | - Run `unittest tests.core` |
12 | 26 | - Focus on core functionalities. |
13 | 27 | - Do not rely on external dependencies beyond the standard library. |
14 | | -- Test with all officially supported Python versions |
15 | | - (currently 3.9, 3.10, 3.11, 3.12, 3.13, and 3.14). |
| 28 | +- Tested on all supported operating systems and all active Python versions. |
| 29 | +- Test case class suffix: `TestCase` |
16 | 30 |
|
17 | 31 | ## Compact tests (testc_*.py) |
18 | 32 |
|
19 | 33 | - Run `unittest tests.compact` |
20 | | -- Test a limited set of functionalities that rely on a stable and small subset |
21 | | - of optional dependencies specified in `pyproject.toml`. |
| 34 | + - Need dependencies from `pip install "pythainlp[compact]"` |
| 35 | +- Test a limited set of functionalities that rely on a stable |
| 36 | + and small set of dependencies. |
22 | 37 | - These dependencies are `PyYAML`, `nlpo3`, `numpy`, `pyicu`, |
23 | 38 | `python-crfsuite`, and `requests`. |
24 | | -- Test with the latest two stable Python versions. |
| 39 | +- Tested on: |
| 40 | + - All OSes: earliest and second-latest supported Python versions |
| 41 | + - Ubuntu: additionally tested on the latest version |
| 42 | +- Test case class suffix: `TestCaseC` |
25 | 43 |
|
26 | 44 | ## Extra tests (testx_*.py) |
27 | 45 |
|
28 | 46 | - Run `unittest tests.extra` |
29 | | -- Explore functionalities that rely on optional dependencies specified in the |
30 | | - `project.optional-dependencies` section of `pyproject.toml`. |
| 47 | + - Need dependencies from `pip install "pythainlp[compact,extra]"` |
| 48 | +- Test more functionalities that rely on larger set of dependencies |
| 49 | + or one that require more time or computation. |
| 50 | +- Only tested on Ubuntu using the second-latest Python version. |
| 51 | +- Test case class suffix: `TestCaseX` |
31 | 52 |
|
32 | 53 | ## Noauto tests (testn_*.py) |
33 | 54 |
|
34 | 55 | - These dependencies might include huge libraries like `tensorflow`. |
35 | 56 | - Due to dependency complexities, these functionalities may not be tested |
36 | 57 | in the CI/CD pipeline. |
| 58 | +- Test case class suffix: `TestCaseN` |
37 | 59 |
|
38 | 60 | ## Robustness tests (test_robustness.py) |
39 | 61 |
|
|
0 commit comments