chore: Replace Makefile with poethepoet task runner (#748)

vdusek · web-flow · commit 671e3239d65f · 2026-01-22T16:53:25.000+01:00
See apify/crawlee-python#1678 for details.
diff --git a/.github/workflows/_release_docs.yaml b/.github/workflows/_release_docs.yaml
@@ -68,13 +68,10 @@ jobs:
           python-version: ${{ env.PYTHON_VERSION }}
 
       - name: Install Python dependencies
-        run: make install-dev
+        run: uv run poe install-dev
 
-      - name: Build generated API reference
-        run: make build-api-reference
-
-      - name: Build Docusaurus docs
-        run: make build-docs
+      - name: Build docs
+        run: uv run poe build-docs
         env:
           APIFY_SIGNING_TOKEN: ${{ secrets.APIFY_SIGNING_TOKEN }}
 
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -3,12 +3,12 @@ repos:
     hooks:
       - id: lint-check
         name: Lint check
-        entry: make lint
+        entry: uv run poe lint
         language: system
         pass_filenames: false
 
       - id: type-check
         name: Type check
-        entry: make type-check
+        entry: uv run poe type-check
         language: system
         pass_filenames: false
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -8,20 +8,41 @@ For local development, it is required to have Python 3.10 (or a later version) i
 
 We use [uv](https://docs.astral.sh/uv/) for project management. Install it and set up your IDE accordingly.
 
+We use [Poe the Poet](https://poethepoet.natn.io/) as a task runner, similar to npm scripts in `package.json`.
+All tasks are defined in `pyproject.toml` under `[tool.poe.tasks]` and can be run with `uv run poe <task>`.
+
+### Available tasks
+
+| Task | Description |
+| ---- | ----------- |
+| `install-dev` | Install development dependencies |
+| `check-code` | Run lint, type-check, and unit-tests |
+| `lint` | Run linter |
+| `format` | Fix lint issues and format code |
+| `type-check` | Run type checker |
+| `unit-tests` | Run unit tests |
+| `unit-tests-cov` | Run unit tests with coverage |
+| `integration-tests` | Run integration tests |
+| `integration-tests-cov` | Run integration tests with coverage |
+| `build-docs` | Build documentation website |
+| `run-docs` | Run documentation website locally |
+| `build` | Build package |
+| `clean` | Remove build artifacts and clean caches |
+
 ## Dependencies
 
 To install this package and its development dependencies, run:
 
 ```sh
-make install-dev
+uv run poe install-dev
 ```
 
 ## Code checking
 
 To execute all code checking tools together, run:
 
 ```sh
-make check-code
+uv run poe check-code
 ```
 
 ### Linting
@@ -31,7 +52,7 @@ We utilize [ruff](https://docs.astral.sh/ruff/) for linting, which analyzes code
 To run linting:
 
 ```sh
-make lint
+uv run poe lint
 ```
 
 ### Formatting
@@ -41,7 +62,7 @@ Our automated code formatting also leverages [ruff](https://docs.astral.sh/ruff/
 To run formatting:
 
 ```sh
-make format
+uv run poe format
 ```
 
 ### Type checking
@@ -51,78 +72,50 @@ Type checking is handled by [ty](https://docs.astral.sh/ty/), verifying code aga
 To run type checking:
 
 ```sh
-make type-check
+uv run poe type-check
 ```
 
 ### Unit tests
 
-We employ pytest as our testing framework, equipped with various plugins. Check pyproject.toml for configuration details and installed plugins.
-
 We use [pytest](https://docs.pytest.org/) as a testing framework with many plugins. Check `pyproject.toml` for configuration details and installed plugins.
 
 To run unit tests:
 
 ```sh
-make unit-tests
+uv run poe unit-tests
 ```
 
-To run unit tests with HTML coverage report:
+To run unit tests with XML coverage report:
 
 ```sh
-make unit-tests-cov
+uv run poe unit-tests-cov
 ```
 
 ## Integration tests
 
-We have integration tests which build and run Actors using the Python SDK on the Apify Platform. To run these tests,
-you need to set the `APIFY_TEST_USER_API_TOKEN` environment variable to the API token of the Apify user you want to
-use for the tests, and then start them with `make integration-tests`.
-
-If you want to run the integration tests on a different environment than the main Apify Platform, you need to set
-the `APIFY_INTEGRATION_TESTS_API_URL` environment variable to the right URL to the Apify API you want to use.
-
-## Documentation
-
-We adhere to the [Google docstring format](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html) for documenting our codebase. Every user-facing class or method is documented. Documentation standards are enforced using [Ruff](https://docs.astral.sh/ruff/).
-
-Our API documentation is generated from these docstrings using [pydoc-markdown](https://pypi.org/project/pydoc-markdown/) with additional post-processing. Markdown files in the `docs/` folder complement the autogenerated content. Final documentation is rendered using [Docusaurus](https://docusaurus.io/) and published to GitHub Pages.
-
-To run the documentation locally, you need to have Node.js version 20 or higher installed. Once you have the correct version of Node.js, follow these steps:
-
-Navigate to the `website/` directory:
-
-```sh
-cd website/
-```
+We have integration tests which build and run Actors using the Python SDK on the Apify platform.
 
-Enable Corepack, which installs Yarn automatically:
+Prerequisites:
 
-```sh
-corepack enable
-```
+- Set `APIFY_TEST_USER_API_TOKEN` to your Apify API token
+- Optionally set `APIFY_INTEGRATION_TESTS_API_URL` to use a different Apify API environment
 
-Build the API reference:
+To run integration tests:
 
 ```sh
-./build_api_reference.sh
+uv run poe integration-tests
 ```
 
-Install the necessary dependencies:
-
-```sh
-yarn
-```
+## Documentation
 
-Start the project in development mode with Hot Module Replacement (HMR):
+We adhere to the [Google docstring format](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html) for documenting our codebase. Every user-facing class or method is documented. Documentation standards are enforced using [Ruff](https://docs.astral.sh/ruff/).
 
-```sh
-yarn start
-```
+Our API documentation is generated from these docstrings using [pydoc-markdown](https://pypi.org/project/pydoc-markdown/) with additional post-processing. Markdown files in the `docs/` folder complement the autogenerated content. Final documentation is rendered using [Docusaurus](https://docusaurus.io/) and published to GitHub Pages.
 
-Or using `make`:
+To run the documentation locally (requires Node.js):
 
 ```sh
-make run-doc
+uv run poe run-docs
 ```
 
 ## Release process
@@ -147,14 +140,14 @@ name = "apify"
 version = "x.z.y"
 ```
 
-4. Generate the distribution archives for the package:
+4. Build the package:
 
-```shell
-uv build
+```sh
+uv run poe build
 ```
 
-5. Set up the PyPI API token for authentication and upload the package to PyPI:
+5. Upload to PyPI:
 
-```shell
+```sh
 uv publish --token YOUR_API_TOKEN
 ```
diff --git a/Makefile b/Makefile
diff --git a/docs/02_concepts/12_pay_per_event.mdx b/docs/02_concepts/12_pay_per_event.mdx
@@ -43,7 +43,7 @@ When you plan to start using the pay-per-event pricing model for an Actor that i
 
 It is encouraged to test your monetization code on your machine before releasing it to the public. To tell your Actor that it should work in pay-per-event mode, pass it the `ACTOR_TEST_PAY_PER_EVENT` environment variable:
 
-```shell
+```sh
 ACTOR_TEST_PAY_PER_EVENT=true python -m youractor
 ```
 
diff --git a/pyproject.toml b/pyproject.toml
@@ -68,6 +68,7 @@ dev = [
     "crawlee[parsel]",
     "dycw-pytest-only<3.0.0",
     "griffe",
+    "poethepoet<1.0.0",
     "pre-commit<5.0.0",
     "pydoc-markdown<5.0.0",
     "pytest-asyncio<2.0.0",
@@ -216,3 +217,33 @@ exclude_lines = ["pragma: no cover", "if TYPE_CHECKING:", "assert_never()"]
 
 [tool.ipdb]
 context = 7
+
+# Run tasks with: uv run poe <task>
+[tool.poe.tasks]
+clean = "rm -rf .coverage .pytest_cache .ruff_cache .ty_cache build dist htmlcov"
+install-sync = "uv sync --all-extras"
+build = "uv build --verbose"
+publish-to-pypi = "uv publish --verbose --token ${APIFY_PYPI_TOKEN_CRAWLEE}"
+type-check = "uv run ty check"
+unit-tests = "uv run pytest --numprocesses=auto --verbose tests/unit"
+unit-tests-cov = "uv run pytest --numprocesses=auto --verbose --cov=src/apify --cov-report=xml:coverage-unit.xml tests/unit"
+integration-tests = "uv run pytest --numprocesses=${INTEGRATION_TESTS_CONCURRENCY:-1} --verbose tests/integration"
+integration-tests-cov = "uv run pytest --numprocesses=${INTEGRATION_TESTS_CONCURRENCY:-1} --verbose --cov=src/apify --cov-report=xml:coverage-integration.xml tests/integration"
+check-code = ["lint", "type-check", "unit-tests"]
+
+[tool.poe.tasks.install-dev]
+shell = "uv sync --all-extras && uv run pre-commit install"
+
+[tool.poe.tasks.lint]
+shell = "uv run ruff format --check && uv run ruff check"
+
+[tool.poe.tasks.format]
+shell = "uv run ruff check --fix && uv run ruff format"
+
+[tool.poe.tasks.build-docs]
+shell = "./build_api_reference.sh && npm ci && npm run build"
+cwd = "website"
+
+[tool.poe.tasks.run-docs]
+shell = "./build_api_reference.sh && npm ci && npm run start"
+cwd = "website"
diff --git a/tests/integration/README.md b/tests/integration/README.md
@@ -4,7 +4,7 @@ There are two different groups of integration tests in this repository:
 - Apify API integration tests. These test that the Apify SDK is correctly communicating with Apify API through Apify client.
 - Actor integration tests. These test that the Apify SDK can be used in Actors deployed to Apify platform. These are very high level tests, and they test communication with the API and correct interaction with the Apify platform.
 
-To run these tests, you need to set the `APIFY_TEST_USER_API_TOKEN` environment variable to the API token of the Apify user you want to use for the tests, and then start them with `make integration-tests`.
+To run these tests, you need to set the `APIFY_TEST_USER_API_TOKEN` environment variable to the API token of the Apify user you want to use for the tests, and then start them with `uv run poe integration-tests`.
 
 ## Apify API integration tests
 The tests are making real requests to the Apify API as opposed to the unit tests that are mocking such API calls. On the other hand they are faster than `Actor integration tests` as they do not require building and deploying the Actor. These test can be also fully debugged locally. Preferably try to write integration tests on this level if possible.
diff --git a/uv.lock b/uv.lock
