Add ClusterFuzzLite for continuous fuzz testing

.github/workflows/clusterfuzzlite.yml

@@ -0,0 +1,83 @@
+# SPDX-FileCopyrightText: 2026 PyThaiNLP Project
+# SPDX-License-Identifier: Apache-2.0
+
+name: ClusterFuzzLite
+
+on:
+  push:
+    branches:
+      - dev
+    paths-ignore:
+      - '**.cff'
+      - '**.json'
+      - '**.md'
+      - '**.rst'
+      - '**.txt'
+      - '**.yml'
+      - 'docs/**'
+  pull_request:
+    branches:
+      - dev
+    paths-ignore:
+      - '**.cff'
+      - '**.json'
+      - '**.md'
+      - '**.rst'
+      - '**.txt'
+      - '**.yml'
+      - 'docs/**'
+  schedule:
+    - cron: '0 6 * * *'  # Daily at 06:00 UTC
+
+# Avoid duplicate runs for the same source branch and repository.
+# For pull_request events, uses the source repo name from
+# github.event.pull_request.head.repo.full_name; otherwise uses github.repository.
+# For push events, uses the branch name from github.ref_name.
+# For pull_request events, uses the source branch name from github.head_ref.
+# This ensures events for the same repo and branch share the same group,
+# and avoids cross-fork collisions when branch names are reused.
+concurrency:
+  group: >-
+    ${{ github.workflow }}-${{
+      github.event.pull_request.head.repo.full_name || github.repository
+    }}-${{ github.head_ref || github.ref_name }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+  issues: write
+
+jobs:
+  fuzzing:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        sanitizer: [address]
+    steps:
+      - name: Build Fuzzers (${{ matrix.sanitizer }})
+        id: build
+        uses: google/clusterfuzzlite/actions/build_fuzzers@v1
+        with:
+          sanitizer: ${{ matrix.sanitizer }}
+          language: python
+          dockerfile-path: fuzz/Dockerfile
+
+      - name: Run Fuzzers (${{ matrix.sanitizer }})
+        id: run
+        uses: google/clusterfuzzlite/actions/run_fuzzers@v1
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          fuzz-seconds: 300
+          mode: 'code-change'
+          sanitizer: ${{ matrix.sanitizer }}
+          storage-repo: https://${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}.git
+          storage-repo-branch: gh-pages
+          storage-repo-branch-coverage: gh-pages
+
+      - name: Upload crash artifacts
+        if: failure() && steps.run.outcome == 'failure'
+        uses: actions/upload-artifact@v4
+        with:
+          name: ${{ matrix.sanitizer }}-artifacts
+          path: ./out/artifacts

fuzz/Dockerfile

@@ -0,0 +1,29 @@
+# SPDX-FileCopyrightText: 2026 PyThaiNLP Project
+# SPDX-FileType: SOURCE
+# SPDX-License-Identifier: Apache-2.0
+
+# Dockerfile for ClusterFuzzLite fuzzing
+# This extends the OSS-Fuzz base builder image for Python projects
+
+FROM gcr.io/oss-fuzz-base/base-builder-python
+
+# Install system dependencies
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends \
+    build-essential \
+    libicu-dev \
+    pkg-config && \
+    rm -rf /var/lib/apt/lists/*
+
+# Copy repository to $SRC/pythainlp
+COPY . $SRC/pythainlp
+
+# Set working directory
+WORKDIR $SRC/pythainlp
+
+# Install pythainlp in development mode with minimal dependencies
+# This installs the package without heavy ML dependencies to speed up builds
+RUN pip install --no-cache-dir -e .
+
+# Copy build script
+COPY fuzz/build.sh $SRC/

fuzz/README.md

@@ -0,0 +1,156 @@
+# PyThaiNLP Fuzz Testing
+
+This directory contains fuzz testing infrastructure using
+[ClusterFuzzLite](https://google.github.io/clusterfuzzlite/) and [Atheris](https://github.com/google/atheris).
+
+## Overview
+
+Fuzz testing helps discover edge cases, crashes, and potential security vulnerabilities by feeding random inputs to functions. This setup uses:
+
+- **ClusterFuzzLite**: Google's continuous fuzzing solution for GitHub projects
+- **Atheris**: Coverage-guided Python fuzzing engine
+- **AddressSanitizer**: Memory safety checks
+
+## Directory Structure
+
+```
+fuzz/
+├── Dockerfile                 # Docker image for ClusterFuzzLite fuzzing
+├── build.sh                   # Build script for compiling fuzzers
+├── fuzz_tokenize.py           # Fuzzer for word_tokenize()
+├── fuzz_util_normalize.py     # Fuzzer for normalize()
+└── README.md                  # This file
+```
+
+## Current Fuzzing Targets
+
+### 1. `fuzz_tokenize.py`
+Tests `pythainlp.tokenize.word_tokenize()` with random Unicode input to ensure:
+- No crashes on malformed input
+- Proper handling of edge cases
+- Memory safety
+
+### 2. `fuzz_util_normalize.py`
+Tests `pythainlp.util.normalize()` with random Unicode input to ensure:
+- No crashes on malformed input
+- Proper string normalization
+- Type safety
+
+## Local Testing
+
+To test fuzzers locally:
+
+```bash
+# Install atheris
+pip install atheris
+
+# Run a specific fuzzer for 60 seconds
+python fuzz/fuzz_tokenize.py -max_total_time=60
+
+# Run with specific corpus directory
+python fuzz/fuzz_tokenize.py corpus_dir/ -max_total_time=60
+```
+
+## CI/CD Integration
+
+Fuzzing runs automatically via GitHub Actions:
+- On pull requests to `dev` branch (focuses on code changes)
+- On push to `dev` branch
+- Daily at 06:00 UTC (full fuzzing run)
+
+Configuration: `.github/workflows/clusterfuzzlite.yml`
+
+## Adding New Fuzzers
+
+To add a new fuzzing target:
+
+1. Create a new file `fuzz/fuzz_<module_name>.py`:
+
+```python
+# SPDX-FileCopyrightText: 2026 PyThaiNLP Project
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileType: SOURCE
+"""Fuzzing harness for pythainlp.<module>.<function>()"""
+
+import sys
+import atheris
+import pythainlp.<module>
+
+
+def TestOneInput(data: bytes) -> None:
+    """Fuzz target for <function>."""
+    fdp = atheris.FuzzedDataProvider(data)
+
+    try:
+        # Generate test input
+        text = fdp.ConsumeUnicodeNoSurrogates(fdp.remaining_bytes())
+
+        # Call target function
+        result = pythainlp.<module>.<function>(text)
+
+        # Validate output
+        assert isinstance(result, <expected_type>)
+
+    except (ValueError, TypeError, UnicodeDecodeError):
+        # Expected exceptions
+        pass
+
+
+def main() -> None:
+    """Entry point for the fuzzer."""
+    atheris.Setup(sys.argv, TestOneInput)
+    atheris.Fuzz()
+
+
+if __name__ == "__main__":
+    main()
+```
+
+2. The fuzzer will be automatically discovered and built by `build.sh`
+
+3. No changes needed to GitHub Actions workflow
+
+## Expansion Plan
+
+Future fuzzing targets to consider:
+
+### High Priority
+- **spell/** - Spelling correction functions
+- **soundex/** - Phonetic encoding functions
+- **transliterate/** - Romanization functions
+
+### Medium Priority
+- **corpus/** - Data loading and corpus functions
+- **tag/** - Part-of-speech tagging
+- **parse/** - Parsing functions
+
+### Low Priority
+- **classify/** - Classification functions
+- **generate/** - Text generation functions
+- **summarize/** - Summarization functions
+
+## Troubleshooting
+
+### Fuzzer Crashes
+If a fuzzer finds a crash:
+1. Check the GitHub Actions artifacts for crash reports
+2. Reproduce locally: `python fuzz/fuzz_<name>.py <crash_file>`
+3. Fix the underlying issue in the target function
+4. Re-run fuzzer to verify fix
+
+### Performance Issues
+- Adjust fuzzing time in `.github/workflows/clusterfuzzlite.yml`
+- Default is 300 seconds (5 minutes) per fuzzer
+- For longer sessions, increase the value
+
+### False Positives
+- Update the exception handling in the fuzzer
+- Add expected exceptions to the `except` block
+- Document the reasoning in comments
+
+## Resources
+
+- [ClusterFuzzLite Documentation](https://google.github.io/clusterfuzzlite/)
+- [Atheris Documentation](https://github.com/google/atheris)
+- [OSS-Fuzz](https://github.com/google/oss-fuzz)
+- [libFuzzer Tutorial](https://github.com/google/fuzzing/blob/master/tutorial/libFuzzerTutorial.md)

fuzz/build.sh

@@ -0,0 +1,28 @@
+#!/bin/bash -eu
+# SPDX-FileCopyrightText: 2026 PyThaiNLP Project
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileType: SOURCE
+
+# Build script for ClusterFuzzLite fuzzing harnesses
+# This script installs atheris and compiles all fuzzing harnesses
+
+echo "Building PyThaiNLP fuzz targets..."
+
+# Install atheris for Python fuzzing
+pip install atheris
+
+# Find all fuzz_*.py files in the fuzz directory
+for fuzzer in "${SRC}/pythainlp/fuzz"/fuzz_*.py; do
+    fuzzer_basename=$(basename -s .py "$fuzzer")
+    fuzzer_package="fuzz.${fuzzer_basename}"
+
+    echo "Compiling ${fuzzer_basename}..."
+
+    # Compile fuzzer with atheris
+    python -m atheris.instrument_libfuzzer "${fuzzer}" "${OUT}/${fuzzer_basename}"
+
+    # Make fuzzer executable
+    chmod +x "${OUT}/${fuzzer_basename}"
+done
+
+echo "Build completed successfully!"

fuzz/fuzz_tokenize.py

@@ -0,0 +1,51 @@
+# SPDX-FileCopyrightText: 2026 PyThaiNLP Project
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileType: SOURCE
+"""Fuzzing harness for pythainlp.tokenize.word_tokenize()
+
+This fuzzer tests the word_tokenize function with random Unicode input
+to discover edge cases, crashes, and potential security issues.
+"""
+
+import sys
+
+import atheris
+
+import pythainlp.tokenize
+
+
+def TestOneInput(data: bytes) -> None:
+    """Fuzz target for word_tokenize.
+
+    :param bytes data: Random input bytes from the fuzzer
+    """
+    fdp = atheris.FuzzedDataProvider(data)
+
+    try:
+        # Generate random Unicode string
+        text = fdp.ConsumeUnicodeNoSurrogates(fdp.remaining_bytes())
+
+        # Test word_tokenize with default engine
+        result = pythainlp.tokenize.word_tokenize(text)
+
+        # Validate output type
+        assert isinstance(result, list), f"Expected list, got {type(result)}"
+        assert all(isinstance(token, str) for token in result), \
+            "All tokens should be strings"
+
+    except (ValueError, TypeError, UnicodeDecodeError):
+        # Expected exceptions - these are acceptable
+        pass
+    except Exception:
+        # Unexpected exceptions - re-raise for investigation
+        raise
+
+
+def main() -> None:
+    """Entry point for the fuzzer."""
+    atheris.Setup(sys.argv, TestOneInput)
+    atheris.Fuzz()
+
+
+if __name__ == "__main__":
+    main()