fix: find global promptfoo on Windows

Author: mldangelo

README.md

@@ -1,110 +1,176 @@
-# promptfoo - Python wrapper
-
-[![PyPI version](https://badge.fury.io/py/promptfoo.svg)](https://pypi.org/project/promptfoo/)
-[![Python versions](https://img.shields.io/pypi/pyversions/promptfoo.svg)](https://pypi.org/project/promptfoo/)
-[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
-
-Python wrapper for [promptfoo](https://www.promptfoo.dev) - the LLM testing, red teaming, and security evaluation framework.
-
-## What is promptfoo?
-
-Promptfoo is a TypeScript/Node.js tool for:
-
-- **LLM Testing & Evaluation** - Compare prompts, models, and RAG systems
-- **Red Teaming** - Automated vulnerability testing and adversarial attacks
-- **Security Scanning** - Detect prompt injection, jailbreaks, and data leaks
-- **CI/CD Integration** - Add automated AI security checks to your pipeline
+# Promptfoo: LLM evals & red teaming
+
+<p align="center">
+  <a href="https://pypi.org/project/promptfoo/"><img src="https://badge.fury.io/py/promptfoo.svg" alt="PyPI version"></a>
+  <a href="https://pypi.org/project/promptfoo/"><img src="https://img.shields.io/pypi/pyversions/promptfoo.svg" alt="Python versions"></a>
+  <a href="https://npmjs.com/package/promptfoo"><img src="https://img.shields.io/npm/dm/promptfoo" alt="npm downloads"></a>
+  <a href="https://github.com/promptfoo/promptfoo/blob/main/LICENSE"><img src="https://img.shields.io/github/license/promptfoo/promptfoo" alt="MIT license"></a>
+  <a href="https://discord.gg/promptfoo"><img src="https://github.com/user-attachments/assets/2092591a-ccc5-42a7-aeb6-24a2808950fd" alt="Discord"></a>
+</p>
+
+---
+
+> **📦 About this Python package**
+>
+> This is a lightweight wrapper that installs promptfoo via `pip`. It requires **Node.js 18+** and executes `npx promptfoo@latest` under the hood.
+>
+> **💡 If you have Node.js installed**, we recommend using `npx promptfoo@latest` directly for better performance:
+>
+> ```bash
+> npx promptfoo@latest init
+> npx promptfoo@latest eval
+> ```
+>
+> See the [main project](https://github.com/promptfoo/promptfoo) for the official npm package.
+>
+> **🐍 Use this pip wrapper when you:**
+>
+> - Need to install via `pip` for Python-only CI/CD environments
+> - Want to manage promptfoo with poetry/pipenv/pip alongside Python dependencies
+> - Work in environments where pip packages are easier to approve than npm
+
+---
+
+<p align="center">
+  <code>promptfoo</code> is a developer-friendly local tool for testing LLM applications. Stop the trial-and-error approach - start shipping secure, reliable AI apps.
+</p>
+
+<p align="center">
+  <a href="https://www.promptfoo.dev">Website</a> ·
+  <a href="https://www.promptfoo.dev/docs/getting-started/">Getting Started</a> ·
+  <a href="https://www.promptfoo.dev/docs/red-team/">Red Teaming</a> ·
+  <a href="https://www.promptfoo.dev/docs/">Documentation</a> ·
+  <a href="https://discord.gg/promptfoo">Discord</a>
+</p>
 
 ## Installation
 
 ### Requirements
 
 - **Python 3.9+** (for this wrapper)
-- **Node.js 18+** (to run the actual promptfoo CLI)
+- **Node.js 18+** (required to run promptfoo)
 
 ### Install from PyPI
 
 ```bash
 pip install promptfoo
 ```
 
-This Python package is a lightweight wrapper that calls the official promptfoo CLI via `npx`.
+### Alternative: Use npx (Recommended)
 
-### Verify Installation
+If you have Node.js installed, you can skip the wrapper and use npx directly:
 
 ```bash
-# Check that Node.js is installed
-node --version
-
-# Run promptfoo
-promptfoo --version
+npx promptfoo@latest init
+npx promptfoo@latest eval
 ```
 
+This is faster and gives you direct access to the latest version.
+
 ## Quick Start
 
 ```bash
-# Initialize a new project
+# Install
+pip install promptfoo
+
+# Initialize project
 promptfoo init
 
-# Run an evaluation
+# Run your first evaluation
 promptfoo eval
+```
 
-# Start red teaming
-promptfoo redteam run
+See [Getting Started](https://www.promptfoo.dev/docs/getting-started/) (evals) or [Red Teaming](https://www.promptfoo.dev/docs/red-team/) (vulnerability scanning) for more.
 
-# View results in the web UI
-promptfoo view
-```
+## What can you do with Promptfoo?
 
-## Usage
+- **Test your prompts and models** with [automated evaluations](https://www.promptfoo.dev/docs/getting-started/)
+- **Secure your LLM apps** with [red teaming](https://www.promptfoo.dev/docs/red-team/) and vulnerability scanning
+- **Compare models** side-by-side (OpenAI, Anthropic, Azure, Bedrock, Ollama, and [more](https://www.promptfoo.dev/docs/providers/))
+- **Automate checks** in [CI/CD](https://www.promptfoo.dev/docs/integrations/ci-cd/)
+- **Review pull requests** for LLM-related security and compliance issues with [code scanning](https://www.promptfoo.dev/docs/code-scanning/)
+- **Share results** with your team
 
-The `promptfoo` command behaves identically to the official Node.js CLI. All arguments are passed through:
+Here's what it looks like in action:
 
-```bash
-# Get help
-promptfoo --help
+![prompt evaluation matrix - web viewer](https://www.promptfoo.dev/img/claude-vs-gpt-example@2x.png)
 
-# Run tests
-promptfoo eval
+It works on the command line too:
 
-# Generate red team attacks
-promptfoo redteam generate
+![prompt evaluation matrix - command line](https://github.com/promptfoo/promptfoo/assets/310310/480e1114-d049-40b9-bd5f-f81c15060284)
 
-# Run vulnerability scans
-promptfoo redteam run
+It also can generate [security vulnerability reports](https://www.promptfoo.dev/docs/red-team/):
 
-# View results
-promptfoo view
+![gen ai red team](https://www.promptfoo.dev/img/riskreport-1@2x.png)
 
-# Export results
-promptfoo export --format json --output results.json
-```
+## Why Promptfoo?
 
-## How It Works
+- 🚀 **Developer-first**: Fast, with features like live reload and caching
+- 🔒 **Private**: LLM evals run 100% locally - your prompts never leave your machine
+- 🔧 **Flexible**: Works with any LLM API or programming language
+- 💪 **Battle-tested**: Powers LLM apps serving 10M+ users in production
+- 📊 **Data-driven**: Make decisions based on metrics, not gut feel
+- 🤝 **Open source**: MIT licensed, with an active community
+
+## How This Wrapper Works
 
 This Python package is a thin wrapper that:
 
-1. Checks if Node.js and npx are installed
-2. Executes `npx promptfoo@latest <your-args>`
+1. Checks if Node.js is installed
+2. Executes `npx promptfoo@latest <your-args>` (or uses globally installed promptfoo if available)
 3. Passes through all arguments and environment variables
 4. Returns the same exit code
 
-The actual promptfoo logic runs via the TypeScript package from npm.
+The actual promptfoo logic runs via the official TypeScript package from npm. All features and commands work identically.
+
+## Python-Specific Usage
+
+### With pip
+
+```bash
+pip install promptfoo
+promptfoo eval
+```
+
+### With poetry
+
+```bash
+poetry add --group dev promptfoo
+poetry run promptfoo eval
+```
+
+### With requirements.txt
+
+```bash
+echo "promptfoo>=0.2.0" >> requirements.txt
+pip install -r requirements.txt
+promptfoo eval
+```
 
-## Why a Python Wrapper?
+### In CI/CD (GitHub Actions example)
 
-Many Python developers prefer `pip install` over `npm install` for tools in their workflow. This wrapper allows you to:
+```yaml
+- name: Setup Node.js
+  uses: actions/setup-node@v4
+  with:
+    node-version: "20"
 
-- Install promptfoo alongside your Python dependencies
-- Use it in Python-based CI/CD pipelines
-- Manage it with standard Python tooling (pip, poetry, pipenv, etc.)
+- name: Install promptfoo
+  run: pip install promptfoo
 
-## Documentation
+- name: Run red team tests
+  run: promptfoo redteam run
+```
+
+## Learn More
 
-- **Website**: https://www.promptfoo.dev
-- **Docs**: https://www.promptfoo.dev/docs
-- **GitHub**: https://github.com/promptfoo/promptfoo
-- **Discord**: https://discord.gg/promptfoo
+- 📚 [Full Documentation](https://www.promptfoo.dev/docs/intro/)
+- 🔐 [Red Teaming Guide](https://www.promptfoo.dev/docs/red-team/)
+- 🎯 [Getting Started](https://www.promptfoo.dev/docs/getting-started/)
+- 💻 [CLI Usage](https://www.promptfoo.dev/docs/usage/command-line/)
+- 📦 [Main Project (npm)](https://github.com/promptfoo/promptfoo)
+- 🤖 [Supported Models](https://www.promptfoo.dev/docs/providers/)
+- 🔬 [Code Scanning Guide](https://www.promptfoo.dev/docs/code-scanning/)
 
 ## Troubleshooting
 
@@ -119,20 +185,43 @@ The wrapper needs Node.js to run. Install it:
 
 ### Slow First Run
 
-The first time you run `promptfoo`, npx will download the latest version from npm. Subsequent runs are fast.
+The first time you run `promptfoo`, npx downloads the latest version from npm (typically ~50MB). Subsequent runs use the cached version and are fast.
+
+To speed this up, install promptfoo globally:
+
+```bash
+npm install -g promptfoo
+```
+
+The Python wrapper will automatically use the global installation when available.
 
 ### Version Pinning
 
-By default, this wrapper uses `npx promptfoo@latest`. To pin a specific version, set the `PROMPTFOO_VERSION` environment variable:
+By default, this wrapper uses `npx promptfoo@latest`. To pin a specific version:
 
 ```bash
 export PROMPTFOO_VERSION=0.95.0
 promptfoo --version
 ```
 
-## Development
+Or install a specific version globally:
+
+```bash
+npm install -g promptfoo@0.95.0
+```
+
+## Contributing
+
+We welcome contributions! Check out our [contributing guide](https://www.promptfoo.dev/docs/contributing/) to get started.
+
+Join our [Discord community](https://discord.gg/promptfoo) for help and discussion.
+
+**For wrapper-specific issues**: Report them in this repository
+**For promptfoo features/bugs**: Report in the [main project](https://github.com/promptfoo/promptfoo)
 
-This is a minimal wrapper - the actual promptfoo source code lives in the main TypeScript repository.
+<a href="https://github.com/promptfoo/promptfoo/graphs/contributors">
+  <img src="https://contrib.rocks/image?repo=promptfoo/promptfoo" />
+</a>
 
 ## License
 

src/promptfoo/cli.py

@@ -42,6 +42,21 @@ def _normalize_path(path: str) -> str:
     return os.path.normcase(os.path.abspath(path))
 
 
+def _strip_quotes(path: str) -> str:
+    if len(path) >= 2 and path[0] == path[-1] and path[0] in ('"', "'"):
+        return path[1:-1]
+    return path
+
+
+def _split_path(path_value: str) -> list[str]:
+    entries = []
+    for entry in path_value.split(os.pathsep):
+        entry = _strip_quotes(entry.strip())
+        if entry:
+            entries.append(entry)
+    return entries
+
+
 def _resolve_argv0() -> Optional[str]:
     if not sys.argv:
         return None
@@ -56,21 +71,51 @@ def _resolve_argv0() -> Optional[str]:
     return None
 
 
+def _find_windows_promptfoo() -> Optional[str]:
+    candidates = []
+    for key in ("NPM_CONFIG_PREFIX", "npm_config_prefix"):
+        prefix = os.environ.get(key)
+        if prefix:
+            candidates.append(prefix)
+    appdata = os.environ.get("APPDATA")
+    if appdata:
+        candidates.append(os.path.join(appdata, "npm"))
+    localappdata = os.environ.get("LOCALAPPDATA")
+    if localappdata:
+        candidates.append(os.path.join(localappdata, "npm"))
+    for env_key in ("ProgramFiles", "ProgramFiles(x86)"):
+        program_files = os.environ.get(env_key)
+        if program_files:
+            candidates.append(os.path.join(program_files, "nodejs"))
+    for base in candidates:
+        for name in ("promptfoo.cmd", "promptfoo.exe"):
+            candidate = os.path.join(base, name)
+            if os.path.isfile(candidate):
+                return candidate
+    return None
+
+
 def _find_external_promptfoo() -> Optional[str]:
     promptfoo_path = shutil.which("promptfoo")
     if not promptfoo_path:
+        if os.name == "nt":
+            return _find_windows_promptfoo()
         return None
     argv0_path = _resolve_argv0()
     if argv0_path and _normalize_path(promptfoo_path) == argv0_path:
         wrapper_dir = _normalize_path(os.path.dirname(promptfoo_path))
         path_entries = [
             entry
-            for entry in os.environ.get("PATH", "").split(os.pathsep)
-            if entry and _normalize_path(entry) != wrapper_dir
+            for entry in _split_path(os.environ.get("PATH", ""))
+            if _normalize_path(entry) != wrapper_dir
         ]
-        if not path_entries:
-            return None
-        return shutil.which("promptfoo", path=os.pathsep.join(path_entries))
+        if path_entries:
+            candidate = shutil.which("promptfoo", path=os.pathsep.join(path_entries))
+            if candidate:
+                return candidate
+        if os.name == "nt":
+            return _find_windows_promptfoo()
+        return None
     return promptfoo_path