Add Thai profanity detection utilities with custom words support

[Copilot]

What does this changes

Adds profanity detection and filtering capabilities for Thai text with three utility functions: contains_profanity(), find_profanity(), and censor_profanity(). Users can optionally provide custom profanity words in addition to the default list.

What was wrong

PyThaiNLP lacked profanity detection functionality for Thai language content moderation and filtering use cases.

How this fixes it

New corpus data:

• profanity_th.txt with 23 Thai profanity words
• thai_profanity_words() loader in corpus.common

New utility functions in pythainlp.util.profanity:

• contains_profanity(text, custom_words=None, engine="newmm") - Returns bool if profanity detected
• find_profanity(text, custom_words=None, engine="newmm") - Returns list of profanity words found
• censor_profanity(text, replacement="*", custom_words=None, engine="newmm") - Replaces profanity with censoring characters

Key features:

• Custom words support: Users can add their own profanity words via the custom_words parameter (accepts an optional set of strings, type: set[str] | None)
• Improved tokenization: Uses a custom dictionary that merges thai_words() and profanity words for better word segmentation
• Flexible censoring: Supports custom replacement characters

Usage:

from pythainlp.util import contains_profanity, find_profanity, censor_profanity

# Basic usage with default profanity list
contains_profanity("สวัสดีครับ")  # False
find_profanity("สวัสดี ควย ครับ")  # ['ควย']
censor_profanity("สวัสดี ควย ครับ")  # 'สวัสดี *** ครับ'
censor_profanity("ควย สัส", replacement="#")  # '### ###'

# Add custom profanity words
contains_profanity("คำใหม่", custom_words={"คำใหม่"})  # True
find_profanity("ควย และ คำใหม่", custom_words={"คำใหม่"})  # ['ควย', 'คำใหม่']
censor_profanity("ควย และ คำใหม่", custom_words={"คำใหม่"})  # '*** และ ******'

All functions use word tokenization with a merged custom dictionary (thai_words + profanity words) for accurate Thai text segmentation and support configurable tokenization engines.

Documentation:

• Added comprehensive API documentation for all three functions in docs/api/util.rst
• Functions documented in alphabetical order following project conventions

Compatibility updates:

• Updated type hints to use Python 3.9+ style (lowercase built-in types: frozenset[str], list[str], set[str] | None)
• Proper union types for optional parameters (set[str] | None instead of set[str] with None default)
• Merged with latest dev branch to ensure compatibility with current codebase
• Aligned with dev branch docstring formatting conventions
• Updated SPDX headers to 2026 for new files

Your checklist for this pull request

• Passed code styles and structures
• Passed code linting checks and unit test

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Profanity detection</issue_title>
<issue_description>Detect profanity in Thai language****</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Profanity detection #771

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[coveralls]

coverage: 52.629% (-0.2%) from 52.858%
when pulling 5d82c22 on copilot/add-profanity-detection-thai
into f59b908 on dev.

[wannaphong]

@copilot I'm reviewed and comment about the task.

[wannaphong]

@copilot Fix This branch has conflicts that must be resolved.

[Copilot]

@copilot Fix This branch has conflicts that must be resolved.

Fixed in commit a8b17f7. Successfully merged with dev branch and resolved all conflicts. The branch now uses Python 3.9+ style type hints (lowercase built-in types) and includes all latest changes from dev. All 11 tests pass.

[wannaphong]

@copilot Add docs to docs/api/util.rst

[Copilot]

@copilot Add docs to docs/api/util.rst

Added documentation in commit 7dd940d. All three profanity detection functions (contains_profanity, find_profanity, censor_profanity) are now documented in docs/api/util.rst in alphabetical order.

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud