For contributions to promptfoo itself — new features, providers, bug fixes, and documentation — please go to the main project:
github.com/promptfoo/promptfoo
This repository is only the thin Python shim that lets people install promptfoo via pip. Issues here should be about:
- Installation failures via pip
- Node.js detection or environment problems
- Python shim behaviour on Windows/macOS/Linux
Requires Python 3.9+, Node.js 20+, and uv.
git clone https://github.com/promptfoo/promptfoo-python.git
cd promptfoo-python
uv sync --extra dev
uv run pytest -m 'not smoke' # fast unit tests
uv run pytest # all tests (requires Node.js)- Branch from
main - Follow Conventional Commits — release-please uses commit messages to version and changelog automatically
- Run checks before opening a PR:
uv run ruff check src/ --fix && uv run ruff format src/ uv run mypy src/promptfoo/ && uv run pyright src/promptfoo/ uv run pytest -m 'not smoke'
- Open a pull request against
main