Skip to content

poetry run can fail for scripts defined in [project.scripts] if PATH is too long #10482

@boimart1

Description

@boimart1

Description

On Windows, defining a script under [project.scripts] (importantly, not under [tool.poetry.scripts]), and invoking it with poetry run myscript while the PATH environment variable is near or over ~8192 characters, results in:

'myscript' is not recognized as an internal or external command,
operable program or batch file.

This is the bug described in python/cpython#137254, caused by the following code:

kwargs["shell"] = True
exe = subprocess.Popen(command, env=env, **kwargs)

Notes

While inspecting the Poetry source code, I noticed a few things.

Scripts defined in [tool.poetry.scripts] get special treatment, where they also go through base_env.execute, but the call is made with python -c <inline script> based on the entrypoint's definition, rather than actually calling myscript.cmd (see RunCommand.handle(), RunCommand.run_script()). Scripts defined in [project.scripts] are not taken into account when the check is made, so they fallback to the regular invocation logic.

In Env.get_command_from_bin(), there is some Windows-specific logic that attempts to find <name>.exe in the venv's Scripts directory and resolve its full path. This approach would work. However, the entrypoint is actually <name>.cmd, and so it dodges this particular check.

I think the most reliable approach to resolve this issue would be to use shutil.which(bin, path=env.get("PATH")) in some capacity, as mentioned in the CPython issue. This ensures the script's full path is correctly resolved based on the virtual environment's path, before the call to subprocess.Popen, rather than delegating that logic to the shell. This approach might also allow to remove (or at least trim down) the logic in Env.get_command_from_bin() and Env._bin().

Reproducing

The difficulty in reproducing this issue is getting the PATH to be this long in the first place, as it cannot feasibly be done from cmd.exe, due to its own command length limitations. To work around that, here is a script which creates a ridiculously long path and invokes poetry run ... for the scripts as defined in the pyproject.toml below.

# do_poetry_run.py
import os, subprocess, shutil

fake_path = "C:\\NotAPath"

env = os.environ.copy()
path_elems = [env["PATH"]]
for _ in range(1000):
    path_elems.append(fake_path)

env["PATH"] = os.pathsep.join(path_elems)
print("PATH length:", len(env["PATH"]))

poetry = shutil.which("poetry") or "poetry"
try:
    subprocess.run([poetry, "run", "pepscript"], env=env)
except Exception as e:
    print(f"Error running pepscript: {e}")

try:
    subprocess.run([poetry, "run", "poetryscript"], env=env)
except Exception as e:
    print(f"Error running poetryscript: {e}")

Workarounds

  • Define the script in tool.poetry.scripts rather than project.scripts
  • Don't have an absurdly long PATH :)

Poetry Installation Method

pipx

Operating System

Windows 11

Poetry Version

2.1.3

Poetry Configuration

cache-dir = "C:\\Users\\martin.boisvert\\AppData\\Local\\pypoetry\\Cache"
data-dir = "C:\\Users\\martin.boisvert\\AppData\\Roaming\\pypoetry"
installer.max-workers = null
installer.no-binary = null
installer.only-binary = null
installer.parallel = true
installer.re-resolve = true
keyring.enabled = true
python.installation-dir = "{data-dir}\\python"  # C:\Users\martin.boisvert\AppData\Roaming\pypoetry\python
requests.max-retries = 0
solver.lazy-wheel = true
system-git-client = false
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}\\virtualenvs"  # C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs
virtualenvs.prompt = "{project_name}-py{python_version}"
virtualenvs.use-poetry-python = false

Python Sysconfig

sysconfig.log
Platform: "win-amd64"
Python version: "3.13"
Current installation scheme: "venv"

Paths: 
    data = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13"
    include = "C:\Python313\Include"
    platinclude = "C:\Python313\Include"
    platlib = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Lib\site-packages"
    platstdlib = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Lib"
    purelib = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Lib\site-packages"
    scripts = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Scripts"   
    stdlib = "C:\Python313\Lib"

Variables:
    BINDIR = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Scripts"    
    BINLIBDEST = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13\Lib"    
    EXE = ".exe"
    EXT_SUFFIX = ".cp313-win_amd64.pyd"
    INCLUDEPY = "C:\Python313\Include"
    LDLIBRARY = "python313.dll"
    LIBDEST = "C:\Python313\Lib"
    LIBDIR = "C:\Python313\libs"
    LIBRARY = "python313.dll"
    Py_GIL_DISABLED = "0"
    SOABI = "cp313-win_amd64"
    TZPATH = ""
    VERSION = "313"
    VPATH = "..\.."
    abi_thread = ""
    abiflags = ""
    base = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13"
    exec_prefix = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13"       
    implementation = "Python"
    implementation_lower = "python"
    installed_base = "C:\Python313"
    installed_platbase = "C:\Python313"
    platbase = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13"
    platlibdir = "DLLs"
    prefix = "C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\poetry-gui-scripts-GiBywCFm-py3.13"
    projectbase = "C:\Python313"
    py_version = "3.13.3"
    py_version_nodot = "313"
    py_version_nodot_plat = "313"
    py_version_short = "3.13"
    srcdir = "C:\Python313"
    userbase = "C:\Users\martin.boisvert\AppData\Roaming\Python"

Example pyproject.toml

[project]
name = "py-env-truncation"
version = "0.1.0"
description = ""
requires-python = ">=3.13"

[project.scripts]
pepscript = "py_env_truncation.__main__:main"

[tool.poetry.scripts]
poetryscript = "py_env_truncation.__main__:main"

[build-system]
requires = ["poetry-core>=2.0.0,<3.0.0"]
build-backend = "poetry.core.masonry.api"

Poetry Runtime Logs

poetry-runtime.log
C:\Work\Demos\py-env-truncation
(py-env-truncation-py3.13) λ do_poetry_run.py
PATH length: 14940
Loading configuration file C:\Users\martin.boisvert\AppData\Roaming\pypoetry\config.toml
Loading configuration file C:\Users\martin.boisvert\AppData\Roaming\pypoetry\auth.toml
Using virtualenv: C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\py-env-truncation-dNVbun6Y-py3.13
'pepscript' is not recognized as an internal or external command,
operable program or batch file.
Loading configuration file C:\Users\martin.boisvert\AppData\Roaming\pypoetry\config.toml
Loading configuration file C:\Users\martin.boisvert\AppData\Roaming\pypoetry\auth.toml
Using virtualenv: C:\Users\martin.boisvert\AppData\Local\pypoetry\Cache\virtualenvs\py-env-truncation-dNVbun6Y-py3.13
Warning: 'poetryscript' is an entry point defined in pyproject.toml, but it's not installed as a script. You may get improper `sys.argv[0]`.

The support to run uninstalled scripts will be removed in a future release.

Run `poetry install` to resolve and get rid of this message.

Hello, world!

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugSomething isn't working as expectedstatus/triageThis issue needs to be triaged

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions