diff --git a/README.md b/README.md index 49d1596..33c4f1a 100644 --- a/README.md +++ b/README.md @@ -1,30 +1,44 @@ -# DuckTools: Env # +# DuckTools: Env `ducktools-env` intends to provide a few tools to aid in running and distributing applications and scripts written in Python that require additional dependencies. -## What is this for? ## +## Why wouldn't I just use uv/pipx/hatch/... to run scripts -Suppose you have a Python script that you wish to share with someone else, but -it relies on a third party dependency such as `requests`. In order for someone else -to run your code they need to both have an appropriate version of Python -and to create a virtual environment in which to install `requests` and subsequently -run your script. +You probably would, I wrote the core of this before those had support for inline metadata +back when it was based on the since rejected PEP-722. It has a few additional features I +like and had a few features before they were implemented elsewhere. -PEP-723 introduced +1. There's a script registry so you can access scripts from anywhere without them needing + to be on `PATH` +2. lockfiles are basic `requirements.txt` format rather than the custom `uv.lock` format + * Ideally this will eventually be replaced by the pylock.toml format once `pip` can + install from it directly. +3. It is possible to bundle `ducktools-env` into a zipapp with the script so the user does + not need to have `ducktools-env` installed locally. + * Additional data can also be bundled into the zipapp + +## What is this for? + +Suppose you have a Python script that you wish to share with someone else, but it relies +on a third party dependency such as `requests`. In order for someone else to run your code +they need to both have an appropriate version of Python and to create a virtual +environment in which to install `requests` and subsequently run your script. + +PEP-723 introduced [inline script metadata](https://packaging.python.org/en/latest/specifications/inline-script-metadata/#inline-script-metadata) -which allows users to declare dependencies for single python files in a standardized format. -This is designed to make sharing scripts with PyPI dependencies easier as now the script -can define its own requirements. +which allows users to declare dependencies for single python files in a standardized +format. This is designed to make sharing scripts with PyPI dependencies easier as now the +script can define its own requirements. However, using this format requires the use of an extra package such as 'UV' or 'hatch' using a specific command such as `uv run my_script.py` or `hatch run my_script.py`. -`ducktools-env` is designed to bundle your script into a Python -[zipapp](https://docs.python.org/3/library/zipapp.html) which can be run by any -Python 3.10+ install and will handle creating the virtualenv and launching the script -with the appropriate dependencies *without* needing the other user to have any -specific script running tool installed. +`ducktools-env` is designed to bundle your script into a Python +[zipapp](https://docs.python.org/3/library/zipapp.html) which can be run by any Python +3.12+ install and will handle creating the virtualenv and launching the script with the +appropriate dependencies *without* needing the other user to have any specific script +running tool installed. To aid this, `ducktools-env` provides the `bundle` and `run` commands. @@ -37,62 +51,60 @@ Will run your script much like some of the other script runners. Will then generate a zipapp bundle of your script and the required tools to extract and execute it in the same way as it is executed via the `run` command. -The resulting bundle will include `ducktools-env` and the `pip` zipapp in order to -bootstrap the unbundling process. `UV` will be downloaded and installed on unbundling -if it is available (on PyPI) for the platform. +The resulting bundle will include `ducktools-env` and the `pip` zipapp in order to +bootstrap the unbundling process. -## What if the user does not have Python installed ## +## What if the user does not have Python installed -Running the bundle requires the user to have an install of Python 3.10 or later. -This should be available via python.org with installers for Windows/Mac and either -already included or available from any up to date Linux distribution. This is all -that should be needed for your script to run. +Running the bundle requires the user to have an install of Python 3.12 or later. This +should be available via python.org with installers for Windows/Mac and either already +included or available from any up to date Linux distribution. This is all that should be +needed for your script to run. -The version of Python that will actually be used to build the environment will be the latest -version that can be found via [ducktools-pythonfinder](https://github.com/DavidCEllis/ducktools-pythonfinder) -that satisfies the `requires-python` specification. +The version of Python that will actually be used to build the environment will be the +latest version that can be found via +[ducktools-pythonfinder](https://github.com/DavidCEllis/ducktools-pythonfinder) that +satisfies the `requires-python` specification. -If no version can be found `ducktools-env` will try to use `UV` to install an appropriate -version automatically and use that to build the environment. - -## Where is data stored? ## +## Where is data stored? Environment data and the application itself will be stored in the following locations: -* Windows: `%LOCALAPPDATA%\ducktools\env` -* Linux/Mac/Other: - * Data: `~/.local/share/ducktools/env` - * Config: `~/.config/ducktools/env` (Not yet used) +- Windows: `%LOCALAPPDATA%\ducktools\env` +- Linux/Mac/Other: + - Data: `~/.local/share/ducktools/env` + - Config: `~/.config/ducktools/env` (Not yet used) -## Usage ## +## Usage The tool can be used in multiple ways: -* Installed via `uv tool` (or `pipx`) - * `uv tool install ducktools-env` - * `ducktools-env ` - * This adds the `dtrun` shortcut for `ducktools-env run` -* Executed from the zipapp - * Download from: https://github.com/DavidCEllis/ducktools-env/releases/latest - * Run with: `ducktools-env.pyz ` - * The `dtrun.pyz` zipapp is available as a shortcut for `ducktools-env.pyz run` -* Installed in an environment - * Download with `pip` or `uv` in a virtual environment: `pip install ducktools-env` - * Run with: `ducktools-env ` - * The `dtrun` shortcut is also available -* Accessed directly via `uvx` with uv - * `uvx ducktools-env ` - * No access to the `dtrun` shortcut this way - -These examples will use the `ducktools-env` command as the base as if installed via `uv tool` or a similar tool. +- Installed via `uv tool` (or `pipx`) + - `uv tool install ducktools-env` + - `ducktools-env ` + - This adds the `dtrun` shortcut for `ducktools-env run` +- Executed from the zipapp + - Download from: https://github.com/DavidCEllis/ducktools-env/releases/latest + - Run with: `ducktools-env.pyz ` + - The `dtrun.pyz` zipapp is available as a shortcut for `ducktools-env.pyz run` +- Installed in an environment + - Download with `pip` or `uv` in a virtual environment: `pip install ducktools-env` + - Run with: `ducktools-env ` + - The `dtrun` shortcut is also available +- Accessed directly via `uvx` with uv + - `uvx ducktools-env ` + - No access to the `dtrun` shortcut this way + +These examples will use the `ducktools-env` command as the base as if installed via +`uv tool` or a similar tool. Run a script that uses inline script metadata: `ducktools-env run my_script.py` -If installed via `uv`, `pipx` or `pip` there is an alias `dtrun` for this command. -Unlike the full command it does not accept optional arguments and all arguments are passed -on to the script. +If installed via `uv`, `pipx` or `pip` there is an alias `dtrun` for this command. Unlike +the full command it does not accept optional arguments and all arguments are passed on to +the script. `dtrun my_script.py` @@ -112,7 +124,7 @@ Build the env folder from the installed package: `ducktools-env rebuild_env` -### Registering scripts ### +### Registering scripts It is also now possible to register scripts with `ducktools-env`. @@ -122,16 +134,15 @@ which can then be run by using the script name without the extension: `ducktools-env run my_script` or `dtrun my_script` -## Locking environments ## +## Locking environments -When generating zipapp bundles it may be desirable to also generate a lockfile -to make sure that the versions of installed dependencies do not change between -generation and execution without having to over specify in the original -script. +When generating zipapp bundles it may be desirable to also generate a lockfile to make +sure that the versions of installed dependencies do not change between generation and +execution without having to over specify in the original script. -This generation feature uses `uv` which will be automatically installed. -The lockfile generated is actually a 'universal' `requirements.txt` file with file hashes. -As such `uv` is **not** required to use the generated lockfile (but will usually be installed). +This generation feature requires `uv` be installed in order to create a universal +requirements.txt format lockfile. As such `uv` is **not** required to use the generated +lockfile, only to create it. Create a lockfile without running a script: @@ -139,14 +150,15 @@ Create a lockfile without running a script: Run a script and output the generated lockfile (output as my_script.py.dtenv.lock): -`ducktools-env run --generate-lock my_script.py` (--generate-lock does not work with `dtrun`) +`ducktools-env run --generate-lock my_script.py` (--generate-lock does not work with +`dtrun`) Run a script using a pre-generated lockfile: `ducktools-env run --with-lock my_script.py.dtenv.lock my_script.py` -**If a `my_script.py.dtenv.lock` file is found for a script it will automatically be used without -needing to be specified** +**If a `my_script.py.dtenv.lock` file is found for a script it will automatically be used +without needing to be specified** Bundle a script and generate a lockfile (that will be bundled): @@ -156,68 +168,72 @@ Bundle a script with a pre-generated lockfile: `ducktools-env bundle --with-lock my_script.py.dtenv.lock my_script.py` -**If a `my_script.py.dtenv.lock` file exists it will automatically be used in the bundle also.** +**If a `my_script.py.dtenv.lock` file exists it will automatically be used in the bundle +also.** -The lockfile extension is now `.dtenv.lock` as `uv` will try to use a `.lock` file if it exists -and uses its own tool-specific lockfile format. To avoid a clash with `uv` this was renamed. +The lockfile extension is now `.dtenv.lock` as `uv` will try to use a `.lock` file if it +exists and uses its own tool-specific lockfile format. To avoid a clash with `uv` this was +renamed. -## Including data files with script bundles ## +## Including data files with script bundles -If you wish to include data files with your script you can do so using a tool -table in the toml block. +If you wish to include data files with your script you can do so using a tool table in the +toml block. ```python # /// script # requires-python = ">=3.10" # dependencies = ["cowsay"] -# +# # [tool.ducktools.env] # include.data = ["path/to/folder", "path/to/file.txt"] # /// ``` -If this is made into a bundle these files and folders will be collected into a bundle_data folder -included in the zipapp. +If this is made into a bundle these files and folders will be collected into a bundle_data +folder included in the zipapp. -This data can be retrieved on demand using `get_data_folder` from `ducktools.env.bundled_data` which -will create a temporary folder containing a copy of the data files and return the path to the folder. +This data can be retrieved on demand using `get_data_folder` from +`ducktools.env.bundled_data` which will create a temporary folder containing a copy of the +data files and return the path to the folder. -Note: Paths are relative to the script folder. If you include a folder, the folder itself will be -included, not just its contents. This means that if you include `./` you will get the name of the -folder the script is in (along with all of its contents). +Note: Paths are relative to the script folder. If you include a folder, the folder itself +will be included, not just its contents. This means that if you include `./` you will get +the name of the folder the script is in (along with all of its contents). -This can be used to include additional code by inserting the relevant folder into `sys.path` before -executing the body of a script. +This can be used to include additional code by inserting the relevant folder into +`sys.path` before executing the body of a script. ```python # /// script # requires-python = ">=3.12" # dependencies = ["ducktools-env>=0.1.0"] -# +# # [tool.ducktools.env] # include.data = ["./"] # include.license = ["license.md"] # /// from pathlib import Path -from ducktools.env.bundled_data import get_data_folder +from ducktools.env.bundled_data import get_data_folder with get_data_folder() as fld_name: for f in Path(fld_name).rglob("*"): print(f) ``` -## Application Environments ## +## Application Environments -If you wish your script to persist as an "application" you can define 'owner', 'name' and 'version' -fields. +If you wish your script to persist as an "application" you can define 'owner', 'name' and +'version' fields. These environments **require** generation of a lockfile. -A new version of the application will update the environment to depend on that version. The environment -will be rebuilt if the lockfile is updated on updating to a new version. If the lockfile has changed -but the version has not, running the application will fail (unless the version is a pre-release). -Old versions will also fail to run if the environment has been created for a new version. +A new version of the application will update the environment to depend on that version. +The environment will be rebuilt if the lockfile is updated on updating to a new version. +If the lockfile has changed but the version has not, running the application will fail +(unless the version is a pre-release). Old versions will also fail to run if the +environment has been created for a new version. ```python # /// script @@ -235,95 +251,102 @@ if __name__ == "__main__": cli() ``` -## Listing and deleting environments ## +## Listing and deleting environments Existing environments can be listed with the command `ducktools-env list` -and deleted with +and deleted with `ducktools-env delete_env ` -where `` is the `name` of a temporary environment or the combination -`owner/name` of an application environment as shown in the list. +where `` is the `name` of a temporary environment or the combination `owner/name` +of an application environment as shown in the list. -## Goals ## +## Goals Future goals for this tool: -* Optionally bundle requirements inside the zipapp for use as offline bundles. +- Optionally bundle requirements inside the zipapp for use as offline bundles. -## Dependencies ## +## Dependencies Currently `ducktools.env` relies on the following tools. Subprocesses: -* `venv` via subprocess on python installs where UV is unavailable -* `pip` as a zipapp via subprocess used to install UV and where UV is unavailable -* `uv` where available as a faster installer and for locking dependencies for bundles -PyPI: -* `ducktools-classbuilder` (A lazy, faster implementation of the building blocks behind things like dataclasses) -* `ducktools-lazyimporter` (A simple class based tool to handle deferred imports) -* `ducktools-scriptmetadata` (The parser for inline script metadata blocks) -* `ducktools-pythonfinder` (A tool to discover python installs available for environment creation) -* `packaging` (for comparing dependency lists to cached environments) -* `tomli` (for Python 3.10 to support the TOML format) +- `venv` via subprocess on python installs +- `pip` as a zipapp via subprocess as the dependency installer (the zipapp will be + downloaded automatically) + - This is done rather than including `pip` as a dependency to avoid a clash with `pip` + installed in an environment +- `uv` to generate lockfiles + - This requires that the user has `uv` installed and on `PATH` + +PyPI: -## Other tools in this space ## +- `ducktools-classbuilder` (A lazy, faster implementation of the building blocks behind + things like dataclasses) + - `reannotate` (for correctly handling Python 3.14 annotations) +- `ducktools-lazyimporter` (A simple class based tool to handle deferred imports) +- `ducktools-scriptmetadata` (The parser for inline script metadata blocks) +- `ducktools-pythonfinder` (A tool to discover python installs available for environment + creation) +- `packaging` (for comparing dependency lists to cached environments) -### zipapp ### +## Other tools in this space -The standard library [`zipapp`](https://docs.python.org/3/library/zipapp.html) is at the core of how -`ducktools-env` works. However it doesn't support running with C extensions and it has no inbuilt way -to control which Python it will run under. +### zipapp -By contrast `ducktools-env` will respect a specified python version and required extensions, these -can be bundled or downloaded on first launch via `pip`. +The standard library [`zipapp`](https://docs.python.org/3/library/zipapp.html) is at the +core of how `ducktools-env` works. However it doesn't support running with C extensions +and it has no inbuilt way to control which Python it will run under. -### Shiv ### +By contrast `ducktools-env` will respect a specified python version and required +extensions, these can be bundled or downloaded on first launch via `pip`. -[`shiv`](https://github.com/linkedin/shiv) allows you to bundle zipapps with C extensions, but doesn't provide for -any `online` installs and will extract everything into one `~/.shiv` directory unless otherwise specified. -At the time of writing support for using inline script metadata has not yet been merged but there -is a PR to add support. +### Shiv -`ducktools-env` creates and manages virtual environments for each unique set of script requirements. -These are kept in more platform specific directories documented earlier in the readme. +[`shiv`](https://github.com/linkedin/shiv) allows you to bundle zipapps with C extensions, +but doesn't provide for any `online` installs and will extract everything into one +`~/.shiv` directory unless otherwise specified. At the time of writing support for using +inline script metadata has not yet been merged but there is a PR to add support. -### Pex ### +`ducktools-env` creates and manages virtual environments for each unique set of script +requirements. These are kept in more platform specific directories documented earlier in +the readme. -[`Pex`](https://github.com/pex-tool/pex) provides an assortment of related tools for developers alongside -a `.pex` bundler. -It has (undocumented) support for inline script metadata for building its archives and will -bundle dependencies including C extensions inside the archive, with the option to also -include a Python runtime. It does not support `online` installs, so archives may be platform -dependent or large. +### Pex -### PyInstaller ### +[`Pex`](https://github.com/pex-tool/pex) provides an assortment of related tools for +developers alongside a `.pex` bundler. It has (undocumented) support for inline script +metadata for building its archives and will bundle dependencies including C extensions +inside the archive, with the option to also include a Python runtime. It does not support +`online` installs, so archives may be platform dependent or large. -[Pyinstaller](https://pyinstaller.org/en/stable/) will generate an executable from your script but will also bundle -all of your dependencies in a platform specific way. -It also bundles Python itself, which while convenient if python is not installed, is unnecessary if we can treat -Python as a shared library. +### PyInstaller -### Hatch ### +[Pyinstaller](https://pyinstaller.org/en/stable/) will generate an executable from your +script but will also bundle all of your dependencies in a platform specific way. It also +bundles Python itself, which while convenient if python is not installed, is unnecessary +if we can treat Python as a shared library. -[`Hatch`](https://hatch.pypa.io/) allows you to run scripts with inline dependencies, but requires the user on the -other end already have hatch installed. -The goal of `ducktools-env` is to make it so you can quickly bundle the script into a zipapp that will work on the -other end with only Python as the requirement. +### Hatch -### pipx ### +[`Hatch`](https://hatch.pypa.io/) allows you to run scripts with inline dependencies, but +requires the user on the other end already have hatch installed. The goal of +`ducktools-env` is to make it so you can quickly bundle the script into a zipapp that will +work on the other end with only Python as the requirement. -[`pipx`](https://pipx.pypa.io/) is another tool that allows you to install packages from PyPI and run them as -applications based on their `[project.scripts]` and `[project.gui-scripts]`. It also allows you to run inline scripts -with more recent versions. +### pipx -### uv ### +[`pipx`](https://pipx.pypa.io/) is another tool that allows you to install packages from +PyPI and run them as applications based on their `[project.scripts]` and +`[project.gui-scripts]`. It also allows you to run inline scripts with more recent +versions. -[`uv`](https://docs.astral.sh/uv) itself can run PEP-723 scripts. -`ducktools-env` mostly still exists for the extra zipapp bundling and script registry tools. +### uv -[^1]: undocumented +[`uv`](https://docs.astral.sh/uv) can run PEP-723 scripts directly. `ducktools-env` mostly +still exists for the extra zipapp bundling and script registry tools. diff --git a/examples/inline/empty_40_env.py b/examples/inline/empty_40_env.py new file mode 100644 index 0000000..7db045f --- /dev/null +++ b/examples/inline/empty_40_env.py @@ -0,0 +1,32 @@ +# ducktools.env +# MIT License +# +# Copyright (c) 2024 David C Ellis +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. + +# This script will only start working after Python 4 is released :) + +# /// script +# requires-python = ">=4.0" +# /// + +import sys + +print(f"running in {sys.version_info}") diff --git a/pyproject.toml b/pyproject.toml index b944433..c2f1805 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -26,11 +26,12 @@ classifiers = [ "Programming Language :: Python :: 3.13", "Programming Language :: Python :: 3.14", "Programming Language :: Python :: 3.15", - "License :: OSI Approved :: MIT License", "Operating System :: Microsoft :: Windows", "Operating System :: POSIX :: Linux", ] dynamic = ["version"] +license = "MIT" +license-files = ["LICENSE"] [project.scripts] "ducktools-env" = "ducktools.env.__main__:main" diff --git a/src/ducktools/env/_lazy_imports.py b/src/ducktools/env/_lazy_imports.py index ab2ab5e..c17b87d 100644 --- a/src/ducktools/env/_lazy_imports.py +++ b/src/ducktools/env/_lazy_imports.py @@ -52,8 +52,6 @@ # ducktools-pythonfinder "list_python_installs", - "PythonInstall", - "get_installed_uv_pythons", ] laz = LazyImporter() @@ -81,5 +79,4 @@ from packaging.specifiers import SpecifierSet, InvalidSpecifier from packaging.version import Version, InvalidVersion - from ducktools.pythonfinder import list_python_installs, PythonInstall - from ducktools.pythonfinder.shared import get_uv_pythons as get_installed_uv_pythons + from ducktools.pythonfinder import list_python_installs diff --git a/src/ducktools/env/bundle.py b/src/ducktools/env/bundle.py index 6b7e2a9..09adb1f 100644 --- a/src/ducktools/env/bundle.py +++ b/src/ducktools/env/bundle.py @@ -22,7 +22,6 @@ # SOFTWARE. import shutil import subprocess -import sys import zipapp from pathlib import Path @@ -56,12 +55,12 @@ def create_bundle( :param spec: Bundle environment spec :param paths: ManagedPaths object containing application path info - :param installer_command: appropriate UV or PIP 'install' command + :param installer_command: pip 'install' command :param output_file: output path for the bundle, if not provided the scriptfile path will be used with `.pyz` added as file extension :param compressed: Compress the archive bundle - :raises ScriptNameClash: error raised if the script name clashes with a + :raises ScriptNameClash: error raised if the script name clashes with a name required for bootstrapping. """ script_path = Path(spec.script_path) @@ -84,13 +83,9 @@ def create_bundle( build_path = Path(build_folder) print(f"Building bundle in '{build_folder}'") print("Copying libraries into build folder") - # Don't copy UV - it's platform dependent # Don't copy __pycache__ folders either - uv_base_exe = "uv.exe" if sys.platform == "win32" else "uv" ignore_patterns = shutil.ignore_patterns( "__pycache__", - uv_base_exe, - f"{uv_base_exe}.version" ) # Copy pip and ducktools zipapps into folder diff --git a/src/ducktools/env/catalogue.py b/src/ducktools/env/catalogue.py index cab5fef..f68aae6 100644 --- a/src/ducktools/env/catalogue.py +++ b/src/ducktools/env/catalogue.py @@ -256,7 +256,6 @@ def _create_venv( self, *, spec: EnvironmentSpec, - uv_path: str | None, installer_command: list[str], env: ENV_TYPE, ): @@ -290,26 +289,15 @@ def _create_venv( with open(requirements_path, 'w') as f: f.write(spec.lockdata) try: - if uv_path: - dependency_command = [ - *installer_command, - "install", - "--python", - env.python_path, - "--no-deps", - "-r", - requirements_path, - ] - else: - dependency_command = [ - *installer_command, - "--python", - env.python_path, - "install", - "--no-deps", - "-r", - requirements_path, - ] + dependency_command = [ + *installer_command, + "--python", + env.python_path, + "install", + "--no-deps", + "-r", + requirements_path, + ] _laz.subprocess.run( dependency_command, check=True, @@ -321,22 +309,13 @@ def _create_venv( else: log(f"Installing dependencies from PyPI: {dep_list}") try: - if uv_path: - dependency_command = [ - *installer_command, - "install", - "--python", - env.python_path, - *deps, - ] - else: - dependency_command = [ - *installer_command, - "--python", - env.python_path, - "install", - *deps, - ] + dependency_command = [ + *installer_command, + "--python", + env.python_path, + "install", + *deps, + ] _laz.subprocess.run( dependency_command, check=True, @@ -347,20 +326,12 @@ def _create_venv( raise VenvBuildError(f"Failed to install dependencies: {e}") # Get pip-freeze list to use for installed modules - if uv_path: - freeze_command = [ - *installer_command, - "freeze", - "--python", - env.python_path, - ] - else: - freeze_command = [ - *installer_command, - "--python", - env.python_path, - "freeze", - ] + freeze_command = [ + *installer_command, + "--python", + env.python_path, + "freeze", + ] freeze = _laz.subprocess.run( freeze_command, capture_output=True, @@ -545,7 +516,6 @@ def create_env( *, spec: EnvironmentSpec, config: Config, - uv_path: str | None, installer_command: list[str], base_python, ) -> ENV_TYPE: @@ -576,7 +546,6 @@ def create_env( try: self._create_venv( spec=spec, - uv_path=uv_path, installer_command=installer_command, env=new_env, ) @@ -709,7 +678,6 @@ def create_env( *, spec: EnvironmentSpec, config: Config, - uv_path: str, installer_command: list[str], base_python, ): @@ -756,7 +724,6 @@ def create_env( try: self._create_venv( spec=spec, - uv_path=uv_path, installer_command=installer_command, env=new_env, ) diff --git a/src/ducktools/env/config.py b/src/ducktools/env/config.py index ef637df..8ac657b 100644 --- a/src/ducktools/env/config.py +++ b/src/ducktools/env/config.py @@ -36,10 +36,6 @@ class Config(Prefab, kw_only=True): cache_maxcount: int = 10 cache_lifetime: float = 14.0 - # Use uv and allow uv to auto install Python - use_uv: bool = True - uv_install_python: bool = True - @property def cache_lifetime_delta(self) -> _timedelta: return _timedelta(days=self.cache_lifetime) diff --git a/src/ducktools/env/manager.py b/src/ducktools/env/manager.py index e7daf72..8a314a7 100644 --- a/src/ducktools/env/manager.py +++ b/src/ducktools/env/manager.py @@ -58,10 +58,7 @@ [ FromImport(".bundle", "create_bundle"), FromImport(".scripts.get_pip", "retrieve_pip"), - MultiFromImport( - ".scripts.get_uv", - ["get_local_uv", "get_available_pythons", "install_uv_python"] - ), + FromImport(".scripts.get_uv", "get_local_uv"), MultiFromImport( ".scripts.create_zipapp", ["build_env_folder", "build_zipapp"] @@ -157,15 +154,10 @@ def install_outdated(self): def retrieve_pip(self) -> str: return _laz_internal.retrieve_pip(paths=self.paths) - def retrieve_uv(self, required=False) -> str | None: + def retrieve_uv(self) -> str: # Retrieve the path to the uv executable - # if uv is installed - if self.config.use_uv or required: - uv_path = _laz_internal.get_local_uv() - else: - uv_path = None - - if uv_path is None and required: + uv_path = _laz_internal.get_local_uv() + if uv_path is None: raise RuntimeError( "UV is required for this process but is unavailable." ) @@ -186,26 +178,6 @@ def _get_python_install(self, spec: EnvironmentSpec): ): install = inst break - else: - # If no Python was matched try to install a matching python from UV - if self.config.uv_install_python and (uv_path := self.retrieve_uv()): - uv_pythons = _laz_internal.get_available_pythons(uv_path) - matched_python = False - for ver in uv_pythons: - if spec.details.requires_python_spec.contains(ver): - # Install matching python - _laz_internal.install_uv_python( - uv_path=uv_path, - version_str=ver, - ) - matched_python = ver - break - if matched_python: - # Recover the actual install - for inst in _laz.get_installed_uv_pythons(): - if inst.version_str == matched_python: - install = inst - break if install is None: raise PythonVersionNotFound( @@ -214,18 +186,12 @@ def _get_python_install(self, spec: EnvironmentSpec): return install - def install_base_command(self, use_uv=True) -> list[str]: + def install_base_command(self) -> list[str]: # Get the installer command for python packages - # Pip or the faster uv_pip if it is available - if use_uv and (uv_path := self.retrieve_uv()): - return [uv_path, "pip"] - else: - pip_path = self.retrieve_pip() - return [sys.executable, pip_path, "--disable-pip-version-check"] + pip_path = self.retrieve_pip() + return [sys.executable, pip_path, "--disable-pip-version-check"] def build_env_folder(self, clear_old_builds=True) -> None: - # build_env_folder will use PIP as uv will fail - # if there is no environment # build-env-folder installs into a target directory # instead of using a venv base_command = [sys.executable, self.retrieve_pip(), "--disable-pip-version-check"] @@ -267,7 +233,7 @@ def _spec_from_script( spec = EnvironmentSpec.from_script(script_path) if generate_lock: - spec.generate_lockdata(uv_path=self.retrieve_uv(required=True)) + spec.generate_lockdata(uv_path=self.retrieve_uv()) elif lock_path: with open(lock_path, 'r') as f: spec.lockdata = f.read() @@ -299,7 +265,6 @@ def get_script_env(self, spec: EnvironmentSpec): env = self.app_catalogue.create_env( spec=spec, config=self.config, - uv_path=self.retrieve_uv(), installer_command=self.install_base_command(), base_python=base_python ) @@ -313,7 +278,6 @@ def get_script_env(self, spec: EnvironmentSpec): env = self.temp_catalogue.create_env( spec=spec, config=self.config, - uv_path=self.retrieve_uv(), installer_command=self.install_base_command(), base_python=base_python, ) @@ -487,7 +451,7 @@ def create_bundle( spec=spec, output_file=output_file, paths=self.paths, - installer_command=self.install_base_command(use_uv=False), + installer_command=self.install_base_command(), compressed=compressed, ) @@ -504,7 +468,7 @@ def generate_lockfile( :return: Path to the output lockfile """"" spec = EnvironmentSpec.from_script(script_path=script_path) - spec.generate_lockdata(uv_path=self.retrieve_uv(required=True)) + spec.generate_lockdata(uv_path=self.retrieve_uv()) lockfile_path = lockfile_path if lockfile_path else f"{script_path}.{LOCKFILE_EXTENSION}" diff --git a/src/ducktools/env/scripts/create_zipapp.py b/src/ducktools/env/scripts/create_zipapp.py index 58303b0..e0af5c3 100644 --- a/src/ducktools/env/scripts/create_zipapp.py +++ b/src/ducktools/env/scripts/create_zipapp.py @@ -155,12 +155,8 @@ def build_zipapp( if p != build_path: shutil.rmtree(p) - # UV should not be bundled - binary is not cross platform - uv_base_exe = "uv.exe" if sys.platform == "win32" else "uv" ignore_patterns = shutil.ignore_patterns( "__pycache__", - uv_base_exe, - f"{uv_base_exe}.version" ) print("Copying pip.pyz and ducktools-env") diff --git a/src/ducktools/env/scripts/get_uv.py b/src/ducktools/env/scripts/get_uv.py index ad50f8c..cefbf89 100644 --- a/src/ducktools/env/scripts/get_uv.py +++ b/src/ducktools/env/scripts/get_uv.py @@ -24,7 +24,7 @@ from .. import _lazy_imports as _laz -uv_versionspec = ">=0.7.0" +uv_versionspec = ">=0.10.0" uv_versionre = r"^uv (?P\d+\.\d+\.\d+)" @@ -52,43 +52,3 @@ def get_local_uv(): return None return uv_path - - -def get_available_pythons(uv_path: str) -> list[str]: - """ - Get all python install version numbers available from UV - - :param uv_path: Path to the UV executable - :return: list of version strings - """ - # CPython installs listed by UV - only want downloadable installs - version_re = _laz.re.compile( - r"(?m)^cpython-(?P\d+.\d+.\d+(?:a|b|rc)?\d*).*$" - ) - data = _laz.subprocess.run( - [ - uv_path, - "python", - "list", - "--all-versions", - ], - capture_output=True, - text=True, - check=True, - ) - - matches = version_re.findall(data.stdout) - - return matches - - -def install_uv_python(*, uv_path: str, version_str: str) -> None: - _laz.subprocess.run( - [ - uv_path, - "python", - "install", - version_str, - ], - check=True, - ) diff --git a/tests/test_config.py b/tests/test_config.py index 5052a37..c140a3b 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -1,18 +1,18 @@ # ducktools.env # MIT License -# +# # Copyright (c) 2024 David C Ellis -# +# # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: -# +# # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. -# +# # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE @@ -47,8 +47,6 @@ def test_load_basic(self): json_load_mock.return_value = { "cache_maxcount": 5, "cache_lifetime": 1.0, - "use_uv": False, - "uv_install_python": False, } confpath = "path/to/config" @@ -60,8 +58,6 @@ def test_load_basic(self): # Confirm values loaded correctly assert config.cache_maxcount == 5 assert config.cache_lifetime == 1.0 - assert config.use_uv is False - assert config.uv_install_python is False def test_load_unknowns(self): with ( @@ -72,8 +68,6 @@ def test_load_unknowns(self): json_load_mock.return_value = { "cache_maxcount": 5, "cache_lifetime": 1.0, - "use_uv": False, - "uv_install_python": False, "extra_invalid_value": True } @@ -83,8 +77,6 @@ def test_load_unknowns(self): # Confirm values loaded correctly assert config.cache_maxcount == 5 assert config.cache_lifetime == 1.0 - assert config.use_uv is False - assert config.uv_install_python is False # Invalid value discarded assert not hasattr(config, "extra_invalid_value") diff --git a/tests/test_get_uv.py b/tests/test_get_uv.py deleted file mode 100644 index 718a5ca..0000000 --- a/tests/test_get_uv.py +++ /dev/null @@ -1,149 +0,0 @@ -# ducktools.env -# MIT License -# -# Copyright (c) 2024 David C Ellis -# -# Permission is hereby granted, free of charge, to any person obtaining a copy -# of this software and associated documentation files (the "Software"), to deal -# in the Software without restriction, including without limitation the rights -# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -# copies of the Software, and to permit persons to whom the Software is -# furnished to do so, subject to the following conditions: -# -# The above copyright notice and this permission notice shall be included in all -# copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -# SOFTWARE. -import os.path -import subprocess -import sys -import unittest.mock as mock -from pathlib import Path - -import pytest - -import ducktools.env.scripts.get_uv as get_uv -from ducktools.env.platform_paths import ManagedPaths - -UV_PYTHON_LIST_OUTPUT = Path(__file__).parent / "data" / "uv_python_versions_list.txt" - - -@pytest.fixture(scope="function") -def block_local_uv(): - with mock.patch("shutil.which") as which_mock: - which_mock.return_value = None - yield - - -def test_get_available_pythons(): - uv_python_list_text = UV_PYTHON_LIST_OUTPUT.read_text() - - with mock.patch("subprocess.run") as run_mock: - uv_path = "path/to/uv" - - data_mock = mock.MagicMock() - data_mock.stdout = uv_python_list_text - - run_mock.return_value = data_mock - - uv_pythons = get_uv.get_available_pythons(uv_path=uv_path) - - run_mock.assert_called_once_with( - [uv_path, "python", "list", "--all-versions"], - capture_output=True, - text=True, - check=True, - ) - - available_pythons = [ - '3.13.0rc2', - '3.12.5', - '3.12.4', - '3.12.3', - '3.12.2', - '3.12.1', - '3.12.0', - '3.11.9', - '3.11.8', - '3.11.7', - '3.11.6', - '3.11.5', - '3.11.4', - '3.11.3', - '3.11.1', - '3.10.14', - '3.10.13', - '3.10.12', - '3.10.11', - '3.10.9', - '3.10.8', - '3.10.7', - '3.10.6', - '3.10.5', - '3.10.4', - '3.10.3', - '3.10.2', - '3.10.0', - '3.9.19', - '3.9.18', - '3.9.17', - '3.9.16', - '3.9.15', - '3.9.14', - '3.9.13', - '3.9.12', - '3.9.11', - '3.9.10', - '3.9.7', - '3.9.6', - '3.9.5', - '3.9.4', - '3.9.3', - '3.9.2', - '3.9.1', - '3.9.0', - '3.8.19', - '3.8.18', - '3.8.17', - '3.8.16', - '3.8.15', - '3.8.14', - '3.8.13', - '3.8.12', - '3.8.11', - '3.8.10', - '3.8.9', - '3.8.8', - '3.8.7', - '3.8.6', - '3.8.5', - '3.8.3', - '3.8.2', - '3.7.9', - '3.7.7', - '3.7.6', - '3.7.5', - '3.7.4', - '3.7.3', - ] - - assert uv_pythons == available_pythons - - -def test_uv_install_python(): - with mock.patch("subprocess.run") as run_mock: - uv_path = "path/to/uv" - version_str = "3.12.6" - - get_uv.install_uv_python(uv_path=uv_path, version_str=version_str) - - run_mock.assert_called_once_with( - [uv_path, "python", "install", version_str], - check=True, - ) diff --git a/tests/test_integration/test_build_retrieve.py b/tests/test_integration/test_build_retrieve.py index 176e82c..50f72bd 100644 --- a/tests/test_integration/test_build_retrieve.py +++ b/tests/test_integration/test_build_retrieve.py @@ -44,7 +44,6 @@ def test_build_retrieve(self, testing_catalogue, test_config): real_env = testing_catalogue.create_env( spec=spec, config=test_config, - uv_path=manager.retrieve_uv(), installer_command=manager.install_base_command(), base_python=python_install, ) diff --git a/tests/test_manager.py b/tests/test_manager.py index eaf09b5..9241603 100644 --- a/tests/test_manager.py +++ b/tests/test_manager.py @@ -1,18 +1,18 @@ # ducktools.env # MIT License -# +# # Copyright (c) 2024 David C Ellis -# +# # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: -# +# # The above copyright notice and this permission notice shall be included in all # copies or substantial portions of the Software. -# +# # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE @@ -64,7 +64,7 @@ class TestGetPythonInstall: example_paths = Path(__file__).parent / "example_scripts" def test_finds_python(self): - config = Config(use_uv=False, uv_install_python=False) + config = Config() manager = Manager(project_name="ducktools-testing", config=config) script = str(self.example_paths / "pep_723_example.py") @@ -79,7 +79,7 @@ def test_finds_python(self): assert inst.executable == sys.executable def test_no_python(self): - config = Config(use_uv=False, uv_install_python=False) + config = Config() manager = Manager(project_name="ducktools-testing", config=config) script = str(self.example_paths / "pep_723_example.py")