Skip to content

Declare free-threaded (no-GIL) support for the _cchardet C extension #55

Description

@wbarnha

Summary

On free-threaded CPython (3.13t / 3.14t), importing cchardet re-enables the GIL because the _cchardet extension does not declare that it can run without it. The wheels build and the test suite passes, but the free-threading benefit is lost until the module opts in.

Evidence

Seen in cibuildwheel's cp314t (free-threaded) test step — the wheels build and test green, but with:

<frozen importlib._bootstrap>:491: RuntimeWarning: The global interpreter lock (GIL) has been
enabled to load module 'cchardet._cchardet', which has not declared that it can run safely
without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run
with PYTHON_GIL=0 or -Xgil=0.

(cibuildwheel already produces cp314-cp314t manylinux + musllinux wheels once requires-python and cibuildwheel are current, so the only gap is the declaration.)

What's needed

  1. Thread-safety review of the extension (src/cchardet/_cchardet.pyx + the vendored uchardet):
    • detect() creates a fresh uchardet_t per call and frees it — no shared state, so it looks inherently safe for concurrent calls.
    • UniversalDetector holds per-instance state; sharing one instance across threads is the caller's responsibility (same as on GIL builds) and should just be documented, not synchronized.
    • Confirm the vendored uchardet has no mutable process-global state on the detection path.
    • Module-level Python state is only the read-only _MAC_LABEL_ALIASES dict — safe.
  2. Opt in via Cython. Cython 3.1+ supports the free-threading declaration; add the directive (e.g. # cython: freethreading_compatible = True) so the generated module sets Py_mod_gil = Py_MOD_GIL_NOT_USED. (We're already building with Cython 3.2.x.)
  3. Verify the RuntimeWarning is gone and the suite passes under a free-threaded interpreter (e.g. run the cp313t/cp314t test path, or PYTHON_GIL=0 python -m pytest).

Acceptance criteria

  • import cchardet on a free-threaded build no longer re-enables the GIL (no RuntimeWarning).
  • Test suite passes on cp313t/cp314t with the GIL disabled.
  • A short note in the README/docs on threading expectations for UniversalDetector (per-instance, not shared across threads).

Notes

  • Not a blocker for the freedesktop-uchardet swap (Switch to upstream freedesktop uchardet + its candidate API (#46) #50); that PR already builds and tests the cp314t wheels, they just run with the GIL re-enabled.
  • Low risk / self-contained, but the thread-safety review of the vendored C++ should be done before flipping the flag rather than assuming it's safe.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions