Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
54c4a29
Bump dieghernan/cff-validator from 4 to 5
dependabot[bot] Apr 30, 2026
fe467c9
Bump DavidAnson/markdownlint-cli2-action from 23.1.0 to 23.2.0
dependabot[bot] May 14, 2026
b6ad607
Bump github/codeql-action from 4 to 4.35.4
dependabot[bot] May 14, 2026
edf56c3
Merge pull request #1412 from PyThaiNLP/dependabot/github_actions/Dav…
bact May 14, 2026
9c6b6de
Bump actions/stale from 10 to 10.2.0
dependabot[bot] May 21, 2026
945f2d9
Merge pull request #1413 from PyThaiNLP/dependabot/github_actions/git…
bact May 22, 2026
e337400
Merge pull request #1424 from PyThaiNLP/dependabot/github_actions/act…
bact May 22, 2026
88c8c36
Merge pull request #1405 from PyThaiNLP/dependabot/github_actions/die…
bact May 22, 2026
a00adc3
Bump actions/stale from 10.2.0 to 10.3.0
dependabot[bot] May 28, 2026
2523fe5
Bump github/codeql-action from 4.35.4 to 4.35.5
dependabot[bot] May 28, 2026
51ad6b5
Merge pull request #1425 from PyThaiNLP/dependabot/github_actions/act…
wannaphong May 29, 2026
7b2328e
Merge pull request #1426 from PyThaiNLP/dependabot/github_actions/git…
wannaphong May 29, 2026
8a0ced6
Bump github/codeql-action from 4.35.5 to 4.36.0
dependabot[bot] Jun 4, 2026
6183ac0
Bump peaceiris/actions-gh-pages from 4 to 4.1.0
dependabot[bot] Jun 4, 2026
7178a41
Bump actions/checkout from 6 to 6.0.2
dependabot[bot] Jun 4, 2026
c421153
Merge pull request #1429 from PyThaiNLP/dependabot/github_actions/act…
bact Jun 4, 2026
d8bca18
Merge pull request #1428 from PyThaiNLP/dependabot/github_actions/pea…
bact Jun 5, 2026
246fca9
Merge pull request #1427 from PyThaiNLP/dependabot/github_actions/git…
bact Jun 6, 2026
32e7581
Add bandit security checks (resolves #1430)
Copilot Jun 10, 2026
b7130a7
Fix bandit workflow: remove **.yml from paths-ignore, add permissions…
Copilot Jun 10, 2026
3d5df0c
Fix B615 HuggingFace unsafe download warnings
Copilot Jun 10, 2026
8b84efd
Fix B615: pass revision to cached_file() in transformers_ud.py
Copilot Jun 10, 2026
3fd2c4d
Bump github/codeql-action from 4.36.0 to 4.36.2
dependabot[bot] Jun 11, 2026
76087b4
Bump actions/checkout from 6.0.2 to 6.0.3
dependabot[bot] Jun 11, 2026
1a6e0af
Merge pull request #1433 from PyThaiNLP/dependabot/github_actions/act…
bact Jun 11, 2026
f324027
Merge pull request #1432 from PyThaiNLP/dependabot/github_actions/git…
bact Jun 11, 2026
4e17b84
Update copyright year in bandit.yml
bact Jun 12, 2026
23b6535
Move B202/B310/B311/B614 from global skips to location-specific nosec…
Copilot Jun 12, 2026
c9fcfaf
Move B105/B107 from global skips to location-specific nosec comments
Copilot Jun 12, 2026
96cdae1
Update pyproject.toml
bact Jun 12, 2026
055ca78
Add # NOSONAR to all pseudo-random call sites to silence SonarQube S2…
Copilot Jun 13, 2026
38a2b09
Merge pull request #1431 from PyThaiNLP/copilot/issue-1430-add-bandit…
bact Jun 21, 2026
726be5b
Initial plan
Copilot Jun 22, 2026
a150bf4
Add th-pron transliteration module and core tests
Copilot Jun 22, 2026
6947396
Export th-pron helpers from transliterate package
Copilot Jun 22, 2026
2a0512d
Address validation feedback in th-pron transliterator
Copilot Jun 22, 2026
25a89c8
Refine th-pron transliteration edge handling
Copilot Jun 22, 2026
cee8a1b
Polish th-pron docs and imports
Copilot Jun 22, 2026
a327930
Rename th-pron entrypoint and clean imports
Copilot Jun 22, 2026
4949dc2
Add more test cases
wannaphong Jun 24, 2026
df82ac6
Fixed W292
wannaphong Jun 24, 2026
48d61a9
Update transliterate_wiktionary
wannaphong Jun 24, 2026
1146389
Merge branch 'dev' into copilot/add-module-th-pron
wannaphong Jun 24, 2026
9422238
Fixed test
wannaphong Jun 24, 2026
52c5f01
Fixed test
wannaphong Jun 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions .github/workflows/bandit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# SPDX-FileCopyrightText: 2026-present PyThaiNLP Project
# SPDX-License-Identifier: Apache-2.0

# Bandit is a tool designed to find common security issues in Python code.
# https://github.com/pycqa/bandit

name: Bandit

on:
push:
branches:
- dev
- main
paths-ignore:
- '**.cff'
- '**.json'
- '**.md'
- '**.rst'
- '**.txt'
- 'docs/**'
pull_request:
branches:
- dev
- main
paths-ignore:
- '**.cff'
- '**.json'
- '**.md'
- '**.rst'
- '**.txt'
- 'docs/**'

# Avoid duplicate runs for the same source branch and repository.
# For pull_request events, uses the source repo name from
# github.event.pull_request.head.repo.full_name; otherwise uses github.repository.
# For push events, uses the branch name from github.ref_name.
# For pull_request events, uses the source branch name from github.head_ref.
# This ensures events for the same repo and branch share the same group,
# and avoids cross-fork collisions when branch names are reused.
concurrency:
group: >-
${{ github.workflow }}-${{
github.event.pull_request.head.repo.full_name || github.repository
}}-${{ github.head_ref || github.ref_name }}
cancel-in-progress: true

jobs:
bandit:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- name: Checkout
uses: actions/checkout@v6.0.2

- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.x"
cache: "pip"

- name: Install bandit
run: pip install "bandit>=1.9.4"

- name: Run bandit
run: bandit -r pythainlp -c pyproject.toml
4 changes: 2 additions & 2 deletions .github/workflows/codemeta2cff.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
contents: write
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.3

- name: Generate CITATION.cff from codemeta.json
uses: caltechlibrary/codemeta2cff@99ce10831c3037a067c3d9551d455638e0b438ac #v0.3.3
Expand All @@ -37,7 +37,7 @@ jobs:
printf '\n```' >> $GITHUB_STEP_SUMMARY

- name: Validate CITATION.cff
uses: dieghernan/cff-validator@114aae53e1850c3757733beb60036941900e3dc3 #v4
uses: dieghernan/cff-validator@54c8e737314b948f25c0e5524a9d8ae3f39b207b #v5.0.1

- name: Commit and push updated CITATION.cff
uses: EndBug/add-and-commit@290ea2c423ad77ca9c62ae0f5b224379612c0321 #v10.0.0
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,11 +62,11 @@ jobs:

steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v6.0.3

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
uses: github/codeql-action/init@v4.36.2
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
Expand All @@ -77,7 +77,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v4
uses: github/codeql-action/autobuild@v4.36.2

# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
Expand All @@ -91,4 +91,4 @@ jobs:
# make release

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v4
uses: github/codeql-action/analyze@v4.36.2
2 changes: 1 addition & 1 deletion .github/workflows/corpus.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.3

- name: Set up Python
uses: actions/setup-python@v6
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/deploy-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.3
- name: Set up Python
uses: actions/setup-python@v6
with:
Expand All @@ -38,7 +38,7 @@ jobs:
cd docs && make html
cd ..
- name: Deploy documentation
uses: peaceiris/actions-gh-pages@v4
uses: peaceiris/actions-gh-pages@v4.1.0
with:
personal_token: ${{ secrets.PERSONAL_TOKEN }}
publish_dir: ./docs/_build/html
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.3
- name: Ruff
uses: astral-sh/ruff-action@v3
with:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/markdownlint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,11 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v6.0.3
with:
fetch-depth: 0

- uses: DavidAnson/markdownlint-cli2-action@v23.1.0
- uses: DavidAnson/markdownlint-cli2-action@v23.2.0
with:
globs: |
**/*.md
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/mypy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ jobs:
contents: read
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.3

- name: Set up Python
uses: actions/setup-python@v6
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/publish-versioned-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,19 +31,19 @@ jobs:
run: echo "version=${GITHUB_REF_NAME#v}" >> "$GITHUB_OUTPUT"

- name: Checkout pythainlp (for scripts)
uses: actions/checkout@v6
uses: actions/checkout@v6.0.3
with:
path: pythainlp

- name: Checkout dev-docs repository
uses: actions/checkout@v6
uses: actions/checkout@v6.0.3
with:
repository: PyThaiNLP/dev-docs
token: ${{ secrets.PERSONAL_TOKEN }}
path: dev-docs

- name: Checkout docs repository
uses: actions/checkout@v6
uses: actions/checkout@v6.0.3
with:
repository: PyThaiNLP/docs
token: ${{ secrets.PERSONAL_TOKEN }}
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/pypi-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ jobs:
build: ${{ steps.check_build_trigger.outputs.build }}
steps:
- name: Checkout source code
uses: actions/checkout@v6
uses: actions/checkout@v6.0.3
with:
ref: ${{ github.event.pull_request.head.sha }}
- id: check_build_trigger
Expand All @@ -71,7 +71,7 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.3

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pypi-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ jobs:
python-version: ["3.12"]

steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v6.0.3

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/stale.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:

steps:
- name: "Check PRs"
uses: actions/stale@v10
uses: actions/stale@v10.3.0
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
stale-pr-message: 'This PR is stale because it has been open for 30 days with no activity.'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/unittest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.3

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6
Expand Down
6 changes: 6 additions & 0 deletions docs/api/transliterate.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ Modules

The `transliterate` function serves as a versatile transliteration tool, offering a range of transliteration engines to choose from. It provides flexibility and customization for your transliteration needs.

.. autofunction:: transliterate_wiktionary
:noindex:

.. autofunction:: pronunciate
:noindex:

Expand All @@ -30,6 +33,9 @@ Modules

The `puan` function offers a unique transliteration feature known as "Puan." It provides a specialized transliteration method for Thai text and is an additional option for rendering Thai text into English characters.

.. autofunction:: get_word_dict
:noindex:

.. autoclass:: pythainlp.transliterate.wunsen.WunsenTransliterate
:members:

Expand Down
8 changes: 7 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# SPDX-FileCopyrightText: 2016-2026 PyThaiNLP Project
# SPDX-FileCopyrightText: 2016-present PyThaiNLP Project
# SPDX-FileType: SOURCE
# SPDX-License-Identifier: Apache-2.0

Expand Down Expand Up @@ -71,6 +71,7 @@ dependencies = [
## 1) Development ########################################

dev = [
"bandit>=1.9.4",
"black>=25.11.0",
"build>=1.0.0",
"bump-my-version>=1.2.6",
Expand Down Expand Up @@ -311,6 +312,11 @@ include = [
"README.md",
]

[tool.bandit]
# Skip tests that produce known false positives or are accepted risks.
# B110: Try/Except/Pass. Accepted pattern in optional-dependency loading.
skips = ["B110"]

[tool.bumpversion]
current_version = "5.3.4"
commit = true
Expand Down
6 changes: 3 additions & 3 deletions pythainlp/augment/lm/phayathaibert.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ def __init__(self) -> None:
)

self.tokenizer: AutoTokenizer = AutoTokenizer.from_pretrained(
_MODEL_NAME
_MODEL_NAME # nosec B615
)
self.model_for_masked_lm: AutoModelForMaskedLM = (
AutoModelForMaskedLM.from_pretrained(_MODEL_NAME)
AutoModelForMaskedLM.from_pretrained(_MODEL_NAME) # nosec B615
)
self.model: Pipeline = pipeline(
"fill-mask",
Expand All @@ -55,7 +55,7 @@ def generate(
input_text = self.processor.preprocess(sample_txt)
if sample:
# Non-cryptographic use, pseudo-random generator is acceptable here
random_word_idx = random.randint(0, 4) # noqa: S311
random_word_idx = random.randint(0, 4) # noqa: S311 # nosec B311 # NOSONAR
output = self.model(input_text)[random_word_idx]["sequence"]
else:
output = self.model(input_text)[word_rank]["sequence"]
Expand Down
2 changes: 1 addition & 1 deletion pythainlp/augment/lm/wangchanberta.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ def __init__(self) -> None:
self.target_tokenizer: type[CamembertTokenizer] = CamembertTokenizer
self.tokenizer: CamembertTokenizer = (
CamembertTokenizer.from_pretrained(
self.model_name, revision="main"
self.model_name, revision="main" # nosec B615
)
)
self.tokenizer.additional_special_tokens = [
Expand Down
26 changes: 18 additions & 8 deletions pythainlp/corpus/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ def get_corpus_db(url: str) -> Optional[_ResponseWrapper]:
try:
req = Request(url, headers={"User-Agent": _USER_AGENT})
# SSL certificate verification is enabled by default
with urlopen(req, timeout=10) as response:
with urlopen(req, timeout=10) as response: # nosec B310
corpus_db = _ResponseWrapper(response)
except HTTPError as http_err:
print(f"HTTP error occurred: {http_err}")
Expand Down Expand Up @@ -349,7 +349,7 @@ def _download(url: str, dst: str) -> int:

req = Request(url, headers={"User-Agent": _USER_AGENT})
# SSL certificate verification is enabled by default
with urlopen(req, timeout=10) as response:
with urlopen(req, timeout=10) as response: # nosec B310
file_size = int(response.info().get("Content-Length", -1))
with open(get_full_data_path(dst), "wb") as f:
pbar = None
Expand Down Expand Up @@ -383,7 +383,7 @@ def _check_hash(dst: str, md5: str) -> None:
with open(get_full_data_path(dst), "rb") as f:
content = f.read()
# MD5 is insecure but sufficient here
file_md5 = hashlib.md5(content).hexdigest() # noqa: S324
file_md5 = hashlib.md5(content).hexdigest() # noqa: S324 # nosec B324

if md5 != file_md5:
raise ValueError("Hash does not match expected.")
Expand Down Expand Up @@ -484,7 +484,7 @@ def _safe_extract_tar(tar: tarfile.TarFile, path: str) -> None:
f"Symlink {member.name} points outside extraction directory: {member.linkname}"
)

tar.extractall(path=path)
tar.extractall(path=path) # nosec B202


def _safe_extract_zip(zip_file: zipfile.ZipFile, path: str) -> None:
Expand Down Expand Up @@ -539,7 +539,7 @@ def _safe_extract_zip(zip_file: zipfile.ZipFile, path: str) -> None:
f"Symlink {member} points outside extraction directory: {link_target}"
)

zip_file.extractall(path=path)
zip_file.extractall(path=path) # nosec B202


def _version2int(v: str) -> int:
Expand Down Expand Up @@ -853,12 +853,17 @@ def make_safe_directory_name(name: str) -> str:
return safe_name


def get_hf_hub(repo_id: str, filename: str = "") -> str:
def get_hf_hub(
repo_id: str, filename: str = "", revision: Optional[str] = None
) -> str:
"""HuggingFace Hub in :mod:`pythainlp` data directory.

:param str repo_id: repo_id
:param str filename: filename (optional, default is empty string).
If empty, downloads entire snapshot.
:param Optional[str] revision: a git revision id, which can be a branch
name, a tag, or a commit hash (optional, default is ``None``).
Pin to a full commit hash for reproducible and secure downloads.
:return: path
:rtype: str
"""
Expand All @@ -876,10 +881,15 @@ def get_hf_hub(repo_id: str, filename: str = "") -> str:
root_project = safe_path_join(hf_root, name_dir)
if filename:
output_path = hf_hub_download(
repo_id=repo_id, filename=filename, local_dir=root_project
repo_id=repo_id,
filename=filename,
local_dir=root_project,
revision=revision,
)
else:
output_path = snapshot_download(
repo_id=repo_id, local_dir=root_project
repo_id=repo_id,
local_dir=root_project,
revision=revision,
)
return str(output_path)
Loading
Loading