Skip to content

Replace distutils.util.strtobool in tests with a local helper#8298

Open
Sanjays2402 wants to merge 1 commit into
huggingface:mainfrom
Sanjays2402:fix/tests-distutils-strtobool
Open

Replace distutils.util.strtobool in tests with a local helper#8298
Sanjays2402 wants to merge 1 commit into
huggingface:mainfrom
Sanjays2402:fix/tests-distutils-strtobool

Conversation

@Sanjays2402

Copy link
Copy Markdown

What this fixes

tests/utils.py imports strtobool from distutils.util:

from distutils.util import strtobool

distutils was removed from the standard library in Python 3.12 (PEP 632). tests/utils.py is imported (via from .utils import ...) by 20 test modules, so on an interpreter without the distutils shim this raises at collection time and the entire test suite fails to run:

tests/utils.py:10: in <module>
    from distutils.util import strtobool
E   ModuleNotFoundError: No module named 'distutils'

When it triggers

  • setuptools absent on Python ≥ 3.12 — increasingly common with modern venvs, uv, and minimal CI images: hard ModuleNotFoundError, whole suite uncollectable.
  • setuptools present: it vendors a distutils shim so the import succeeds today, but the shim is deprecated and is removed on Python 3.14 — which this project already targets (setup.py advertises Programming Language :: Python :: 3.12/3.13/3.14, and CI runs on 3.14).

Either way, relying on distutils here is on borrowed time.

The fix

Replace the import with a small local strtobool that reproduces the exact semantics of distutils.util.strtobool:

def strtobool(value: str) -> int:
    value = value.lower()
    if value in {"y", "yes", "t", "true", "on", "1"}:
        return 1
    if value in {"n", "no", "f", "false", "off", "0"}:
        return 0
    raise ValueError(f"invalid truth value {value!r}")

This is the standard PEP 632 migration for strtobool (the stdlib provides no replacement). It's test-only — no change to shipped code.

Verification (local, Python 3.12.12, no setuptools)

Before — collection fails:

tests/utils.py:10: ModuleNotFoundError: No module named 'distutils'

After:

  • Semantics parity checked against the distutils spec for every truthy/falsy value (incl. mixed case) and the ValueError path — all match.
  • Test collection restored, and real tests run green:
python -m pytest tests/test_formatting.py tests/features/test_features.py
220 passed, 33 skipped
  • ruff check tests/utils.py → All checks passed!
  • ruff format --check tests/utils.py → already formatted.

Notes

  • The strtobool helper lives in tests/utils.py right next to its only caller (parse_flag_from_env), matching how the codebase already reads env flags like RUN_SLOW / RUN_REMOTE.
  • No production code touched; behavior for the four env flags (RUN_SLOW, RUN_REMOTE, RUN_LOCAL, RUN_PACKAGED) is unchanged.

`tests/utils.py` imported `strtobool` from `distutils.util`, but `distutils`
was removed from the standard library in Python 3.12 (PEP 632). On a
setuptools-free interpreter (increasingly common with modern venvs, uv, and
minimal CI images) this import raises `ModuleNotFoundError: No module named
'distutils'` at collection time, which breaks the entire test suite -- all 20
test modules that import from `tests.utils` fail to collect. When setuptools
is installed it vendors a `distutils` shim, but that shim is deprecated and is
being removed (it is gone on Python 3.14, which this project targets).

Replace the import with a small local `strtobool` that reproduces the exact
semantics of `distutils.util.strtobool` (truthy: y/yes/t/true/on/1; falsy:
n/no/f/false/off/0; ValueError otherwise). setup.py already advertises support
for Python 3.12, 3.13 and 3.14.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant