Add exclude_words parameter to translation methods

[Copilot]

What do these changes do

Adds optional exclude_words parameter to all translation methods, allowing users to preserve specific words (proper nouns, technical terms, brand names) in their original form during translation.

What was wrong

The translation module had no mechanism to exclude specific words from translation. Users needed to preserve certain terms like brand names, technical jargon, or proper nouns but had to manually post-process translations.

How this fixes it

Core Implementation:

• Added _prepare_text_with_exclusions() and _restore_excluded_words() helper functions in core.py
• Uses unique placeholders (<<<PYTHAINLP_EXCLUDE_N>>>) to mark excluded words before translation
• Implements regex-based token boundary matching for space-separated languages (English, French)
• Token boundaries include whitespace and common punctuation (periods, commas, quotes, parentheses, etc.)
• Falls back to direct replacement for non-space languages (Thai, Chinese)
• Handles duplicates, overlapping words, and partial matches correctly
• Properly handles excluded words adjacent to punctuation marks

API Changes:

• Updated Translate.translate() with exclude_words: Optional[list[str]] = None parameter
• Updated all translator classes: EnThTranslator, ThEnTranslator, ThZhTranslator, ZhThTranslator, ThFrTranslator, Small100Translator

Type Annotations:

• Uses Optional[list[str]] for type annotations to maintain consistency with project-wide conventions
• Ensures compatibility with older static analysis tools
• Added TYPE_CHECKING imports with forward references for better type hint support
• Fixed import ordering to place standard library imports before third-party imports in TYPE_CHECKING blocks
• All type annotations verified with mypy

Testing:

• Added comprehensive test suite in tests/extra/testx_translate_helpers.py with 17 unit tests covering all edge cases
• Tests include coverage for punctuation handling (periods, commas, exclamations, quotes, parentheses)
• Test class follows the TestCaseX naming convention for extra tests (TranslateHelpersTestCaseX)
• Added to extra test suite package list for automated testing
• All tests pass after syncing with dev branch updates

Documentation:

• Updated docstring in _prepare_text_with_exclusions() to accurately describe behavior for text with and without spaces
• Clarified that token boundary matching uses both whitespace and common punctuation as delimiters
• Explained simple substring replacement is used only when entire text contains no spaces
• Added docstring examples showing usage with exclude_words parameter

Code Quality:

• Removed unused imports from translator modules
• Fixed import sorting in TYPE_CHECKING blocks to follow project conventions
• Fixed whitespace formatting issues
• All ruff linting checks pass
• All mypy type checking passes for modified files

Compatibility:

• Synced with latest dev branch multiple times to incorporate Python 3.13 compatibility fixes
• Synced with test suite refactoring from PR Enable tests.extra with 4-tier test organization and consistent naming conventions #1248 which introduced 4-tier test organization (core, compact, extra, noauto)
• Resolved merge conflicts while maintaining improved type hints from dev branch
• Verified all functionality remains unchanged after merge conflict resolution
• Code behavior is unchanged - only type hints, import ordering, and test organization were updated to match latest standards

Example Usage:

from pythainlp.translate import Translate

translator = Translate("th", "en")

# Preserve brand names and acronyms
result = translator.translate(
    "ฉันใช้ PyThaiNLP ในการทำ NLP",
    exclude_words=["PyThaiNLP", "NLP"]
)
# Output: "I use PyThaiNLP to do NLP"

# Works with punctuation
result = translator.translate(
    "I love cat.",
    exclude_words=["cat"]
)
# "cat" is preserved even with period

Your checklist for this pull request

• Passed code styles and structures
• Passed code linting checks and unit test

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Exclude words during translation with PyThaiNLP Translate</issue_title>
<issue_description>Hi all, was wondering if you guys have any method to exclude some words during the translation. </issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Exclude words during translation with PyThaiNLP Translate #792

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[wannaphong]

@copilot move test to tests/extra

[Copilot]

@copilot move test to tests/extra

Done. Moved to tests/extra/testx_translate_helpers.py in commit 76495b5.

[bact]

@copilot There are changes in dev branch to fix the errors occured in Python 3.13. Merge the changes and run the tests again.

[Copilot]

@copilot There are changes in dev branch to fix the errors occured in Python 3.13. Merge the changes and run the tests again.

Done. Merged dev branch in commit 1dd03cd. All 12 tests pass successfully after the merge.

[coveralls]

coverage: 65.045% (+0.3%) from 64.79%
when pulling e1a26a8 on copilot/exclude-words-in-translation
into f22c110 on dev.

[bact]

@copilot The dev branch is updated. Some code were refactored. Please resync with it.

The test suite got refactored and now we have 4-tier of test cases located in four test suites: core, compact, extra, noauto - depends on their dependency requirements.
Some test cases were reorganized and moved to other test suites.

See the test suite refactoring and naming conventions in this merged PR
#1248
and follow the new convention.

[bact]

@copilot the dev branch is updated, with numerous type annotations added. please resync and complete type annotations for changes that being made in this PR, then review the code again. there can be cases of unused imports or missing ones, or unused type-related comments fix that.

[Copilot]

@copilot the dev branch is updated, with numerous type annotations added. please resync and complete type annotations for changes that being made in this PR, then review the code again. there can be c...

Done. Resynced with dev branch and completed type annotation review in commit 97704b2.

Changes made:

1. Added class-level type annotations for EnThTranslator and ThEnTranslator instance variables (_tokenizer, _model_name, _model)
2. Fixed cuda() return value assignment in ThEnTranslator.__init__ (line 178) - changed self._model.cuda() to self._model = self._model.cuda()
3. Fixed cuda() return value assignment in ZhThTranslator.__init__ (line 129) - changed self.model_zhth.cuda() to self.model_zhth = self.model_zhth.cuda()

Verification:
✅ All 17 unit tests pass
✅ No unused imports found
✅ All type annotations complete and consistent (Optional[list[str]] used throughout)
✅ All exclude_words parameters properly typed
✅ No syntax errors in any modified files
✅ TYPE_CHECKING blocks properly organized

[bact]

@copilot the dev branch is updated, with numbers of type annotation fixes. Resync then recheck to avoid any type annotation reassignments/redundancy from this PR.
Verify the changes with mypy.
Then review the code again.

[Copilot]

@copilot the dev branch is updated, with numbers of type annotation fixes. Resync then recheck to avoid any type annotation reassignments/redundancy from this PR.
Verify the changes with mypy.
Then ...

Done. Resynced with dev branch and removed redundant type annotations in commit e1a26a8.

Changes made:

1. Merged latest dev branch (153 new commits with extensive type annotation updates)
2. Removed redundant class-level type annotations from EnThTranslator and ThEnTranslator in en_th.py - dev already has instance-level annotations
3. Removed redundant class-level annotations from ThZhTranslator and ZhThTranslator in zh_th.py - dev already has instance-level annotations
4. Kept class-level annotations in th_fr.py and small100.py - these already exist in dev and are not redundant

Verification:
✅ All 17 unit tests pass
✅ Ruff linting passes with no errors
✅ Mypy verification shows no errors in modified files
✅ No duplicate/redundant type annotations remaining
✅ All exclude_words functionality working correctly

[bact]

@wannaphong merge conflicts resolved.

[bact]

the dev branch is updated, with few type annotation fixes. Resync and review changes in this PR. @copilot

[Copilot]

the dev branch is updated, with few type annotation fixes. Resync and review changes in this PR. @copilot

Done. Merged latest dev branch in commit 519e8cd.

Verification:
✅ All 17 unit tests pass
✅ Ruff linting passes with no errors
✅ Mypy verification shows no issues in modified translate files
✅ No conflicts with exclude_words implementation
✅ Type annotations remain consistent using Optional[list[str]]

Changes in merge:

• 24 commits from dev branch including type annotation fixes
• No changes to translate module files
• Updates to build tools, documentation, and other modules
• All exclude_words functionality preserved and working correctly

[sonarqubecloud]

Quality Gate passed

Issues
7 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.