Description
On Windows, defining a script under [project.scripts] (importantly, not under [tool.poetry.scripts]), and invoking it with poetry run myscript while the PATH environment variable is near or over ~8192 characters, results in:
'myscript' is not recognized as an internal or external command,
operable program or batch file.
This is the bug described in python/cpython#137254, caused by the following code:
|
kwargs["shell"] = True |
|
exe = subprocess.Popen(command, env=env, **kwargs) |
Notes
While inspecting the Poetry source code, I noticed a few things.
Scripts defined in [tool.poetry.scripts] get special treatment, where they also go through base_env.execute, but the call is made with python -c <inline script> based on the entrypoint's definition, rather than actually calling myscript.cmd (see RunCommand.handle(), RunCommand.run_script()). Scripts defined in [project.scripts] are not taken into account when the check is made, so they fallback to the regular invocation logic.
In Env.get_command_from_bin(), there is some Windows-specific logic that attempts to find <name>.exe in the venv's Scripts directory and resolve its full path. This approach would work. However, the entrypoint is actually <name>.cmd, and so it dodges this particular check.
I think the most reliable approach to resolve this issue would be to use shutil.which(bin, path=env.get("PATH")) in some capacity, as mentioned in the CPython issue. This ensures the script's full path is correctly resolved based on the virtual environment's path, before the call to subprocess.Popen, rather than delegating that logic to the shell. This approach might also allow to remove (or at least trim down) the logic in Env.get_command_from_bin() and Env._bin().
Reproducing
The difficulty in reproducing this issue is getting the PATH to be this long in the first place, as it cannot feasibly be done from cmd.exe, due to its own command length limitations. To work around that, here is a script which creates a ridiculously long path and invokes poetry run ... for the scripts as defined in the pyproject.toml below.
# do_poetry_run.py
import os, subprocess, shutil
fake_path = "C:\\NotAPath"
env = os.environ.copy()
path_elems = [env["PATH"]]
for _ in range(1000):
path_elems.append(fake_path)
env["PATH"] = os.pathsep.join(path_elems)
print("PATH length:", len(env["PATH"]))
poetry = shutil.which("poetry") or "poetry"
try:
subprocess.run([poetry, "run", "pepscript"], env=env)
except Exception as e:
print(f"Error running pepscript: {e}")
try:
subprocess.run([poetry, "run", "poetryscript"], env=env)
except Exception as e:
print(f"Error running poetryscript: {e}")
Workarounds
- Define the script in
tool.poetry.scripts rather than project.scripts
- Don't have an absurdly long PATH :)
Poetry Installation Method
pipx
Operating System
Windows 11
Poetry Version
2.1.3
Poetry Configuration
cache-dir = "C:\\Users\\martin.boisvert\\AppData\\Local\\pypoetry\\Cache"
data-dir = "C:\\Users\\martin.boisvert\\AppData\\Roaming\\pypoetry"
installer.max-workers = null
installer.no-binary = null
installer.only-binary = null
installer.parallel = true
installer.re-resolve = true
keyring.enabled = true
python.installation-dir = "{data-dir}\\python" # C:\Users\martin.boisvert\AppData\Roaming\pypoetry\python
requests.max-retries = 0
solver.lazy-wheel = true
system-git-client = false
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}\\virtualenvs" # C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs
virtualenvs.prompt = "{project_name}-py{python_version}"
virtualenvs.use-poetry-python = false
Python Sysconfig
sysconfig.log
Platform: "win-amd64"
Python version: "3.13"
Current installation scheme: "venv"
Paths:
data = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13"
include = "C:\Python313\Include"
platinclude = "C:\Python313\Include"
platlib = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Lib\site-packages"
platstdlib = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Lib"
purelib = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Lib\site-packages"
scripts = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Scripts"
stdlib = "C:\Python313\Lib"
Variables:
BINDIR = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Scripts"
BINLIBDEST = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Lib"
EXE = ".exe"
EXT_SUFFIX = ".cp313-win_amd64.pyd"
INCLUDEPY = "C:\Python313\Include"
LDLIBRARY = "python313.dll"
LIBDEST = "C:\Python313\Lib"
LIBDIR = "C:\Python313\libs"
LIBRARY = "python313.dll"
Py_GIL_DISABLED = "0"
SOABI = "cp313-win_amd64"
TZPATH = ""
VERSION = "313"
VPATH = "..\.."
abi_thread = ""
abiflags = ""
base = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13"
exec_prefix = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13"
implementation = "Python"
implementation_lower = "python"
installed_base = "C:\Python313"
installed_platbase = "C:\Python313"
platbase = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13"
platlibdir = "DLLs"
prefix = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13"
projectbase = "C:\Python313"
py_version = "3.13.3"
py_version_nodot = "313"
py_version_nodot_plat = "313"
py_version_short = "3.13"
srcdir = "C:\Python313"
userbase = "C:\Users\martin.boisvert\AppData\Roaming\Python"
Example pyproject.toml
[project]
name = "py-env-truncation"
version = "0.1.0"
description = ""
requires-python = ">=3.13"
[project.scripts]
pepscript = "py_env_truncation.__main__:main"
[tool.poetry.scripts]
poetryscript = "py_env_truncation.__main__:main"
[build-system]
requires = ["poetry-core>=2.0.0,<3.0.0"]
build-backend = "poetry.core.masonry.api"
Poetry Runtime Logs
poetry-runtime.log
C:\Work\Demos\py-env-truncation
(py-env-truncation-py3.13) λ do_poetry_run.py
PATH length: 14940
Loading configuration file C:\Users\martin.boisvert\AppData\Roaming\pypoetry\config.toml
Loading configuration file C:\Users\martin.boisvert\AppData\Roaming\pypoetry\auth.toml
Using virtualenv: C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\py-env-truncation-dNVbun6Y-py3.13
'pepscript' is not recognized as an internal or external command,
operable program or batch file.
Loading configuration file C:\Users\martin.boisvert\AppData\Roaming\pypoetry\config.toml
Loading configuration file C:\Users\martin.boisvert\AppData\Roaming\pypoetry\auth.toml
Using virtualenv: C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\py-env-truncation-dNVbun6Y-py3.13
Warning: 'poetryscript' is an entry point defined in pyproject.toml, but it's not installed as a script. You may get improper `sys.argv[0]`.
The support to run uninstalled scripts will be removed in a future release.
Run `poetry install` to resolve and get rid of this message.
Hello, world!
Description
On Windows, defining a script under
[project.scripts](importantly, not under[tool.poetry.scripts]), and invoking it withpoetry run myscriptwhile the PATH environment variable is near or over ~8192 characters, results in:This is the bug described in python/cpython#137254, caused by the following code:
poetry/src/poetry/utils/env/base_env.py
Lines 445 to 446 in b580e8a
Notes
While inspecting the Poetry source code, I noticed a few things.
Scripts defined in
[tool.poetry.scripts]get special treatment, where they also go throughbase_env.execute, but the call is made withpython -c <inline script>based on the entrypoint's definition, rather than actually callingmyscript.cmd(seeRunCommand.handle(),RunCommand.run_script()). Scripts defined in[project.scripts]are not taken into account when the check is made, so they fallback to the regular invocation logic.In
Env.get_command_from_bin(), there is some Windows-specific logic that attempts to find<name>.exein the venv's Scripts directory and resolve its full path. This approach would work. However, the entrypoint is actually<name>.cmd, and so it dodges this particular check.I think the most reliable approach to resolve this issue would be to use
shutil.which(bin, path=env.get("PATH"))in some capacity, as mentioned in the CPython issue. This ensures the script's full path is correctly resolved based on the virtual environment's path, before the call tosubprocess.Popen, rather than delegating that logic to the shell. This approach might also allow to remove (or at least trim down) the logic inEnv.get_command_from_bin()andEnv._bin().Reproducing
The difficulty in reproducing this issue is getting the PATH to be this long in the first place, as it cannot feasibly be done from cmd.exe, due to its own command length limitations. To work around that, here is a script which creates a ridiculously long path and invokes
poetry run ...for the scripts as defined in thepyproject.tomlbelow.Workarounds
tool.poetry.scriptsrather thanproject.scriptsPoetry Installation Method
pipx
Operating System
Windows 11
Poetry Version
2.1.3
Poetry Configuration
Python Sysconfig
sysconfig.log
Example pyproject.toml
Poetry Runtime Logs
poetry-runtime.log