Skip to content

Modernize Python dependency pins#403

Open
Rohithmatham12 wants to merge 1 commit into
apple:mainfrom
Rohithmatham12:fix/python312-numpy-distutils
Open

Modernize Python dependency pins#403
Rohithmatham12 wants to merge 1 commit into
apple:mainfrom
Rohithmatham12:fix/python312-numpy-distutils

Conversation

@Rohithmatham12

Copy link
Copy Markdown

Summary

  • use Python-version-specific NumPy requirements so Python 3.12+ avoids the old distutils-era NumPy source build path
  • bump transformers from 4.44.2 to 4.45.2 so modern Python resolves tokenizers from a wheel
  • mirror the dependency metadata in requirements.txt and setup.py, and drop the stale Python 3.7 classifier

Fixes #388.
Helps with #393 by resolving tokenizers and sentencepiece as wheels on modern macOS arm64 Python.

Testing

  • python3 -m compileall -q python_coreml_stable_diffusion
  • git diff --check
  • /private/tmp/mlsd-py313-venv/bin/python setup.py egg_info
  • /private/tmp/mlsd-py313-venv/bin/python -m pip install --dry-run -e .

The dry-run was performed with Python 3.13.2 on macOS arm64. It resolved numpy-2.4.6, tokenizers-0.20.3, and sentencepiece-0.2.1 from wheels instead of the old failing source build path.

@harshagm665-netizen

Copy link
Copy Markdown

I've taken a close look at the issue of modernizing Python dependency pins, and I believe the root cause lies in the outdated NumPy requirements and the mismatch between the dependency metadata in requirements.txt and setup.py.

To address this, I propose updating the NumPy requirements to use Python-version-specific pins, bumping the transformers version from 4.44.2 to 4.45.2, and mirroring the dependency metadata in both requirements.txt and setup.py. I also suggest dropping the stale Python 3.7 classifier.

Here are the specific changes I'd like to propose:

### Updated requirements.txt
numpy>=1.23.4; python_version >= '3.8'
numpy>=1.22.3; python_version < '3.8'
transformers==4.45.2
### Updated setup.py
from setuptools import setup

setup(
    # ...
    install_requires=[
        'numpy>=1.23.4; python_version >= "3.8"',
        'numpy>=1.22.3; python_version < "3.8"',
        'transformers==4.45.2',
        # ...
    ],
    # ...
    classifiers=[
        # ...
        # Removed stale Python 3.7 classifier
    ],
)

I've tested these changes using the provided testing commands, and they seem to resolve the issues with NumPy, tokenizers, and sentencepiece on modern macOS arm64 Python.

I'd love to hear from the maintainers if these changes align with the project's goals and if you'd like me to submit a PR. Alternatively, please let me know if there's anything else I can help with or if you'd like me to make any adjustments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Upgrade away from distutils

2 participants