Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ repos:
hooks:
- id: lint-check
name: Lint check
entry: make lint
entry: uv run poe lint
language: system
pass_filenames: false

- id: type-check
name: Type check
entry: make type-check
entry: uv run poe type-check
language: system
pass_filenames: false
57 changes: 14 additions & 43 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,23 @@ For local development, it is required to have Python 3.10 (or a later version) i

We use [uv](https://docs.astral.sh/uv/) for project management. Install it and set up your IDE accordingly.

We use [Poe the Poet](https://poethepoet.natn.io/) as a task runner, similar to npm scripts in `package.json`.
All tasks are defined in `pyproject.toml` under `[tool.poe.tasks]` and can be run with `uv run poe <task>`.

## Dependencies

To install this package and its development dependencies, run:

```sh
make install-dev
uv run poe install-dev
```

## Code checking

To execute all code checking tools together, run:

```sh
make check-code
uv run poe check-code
```

### Linting
Expand All @@ -31,7 +34,7 @@ We utilize [ruff](https://docs.astral.sh/ruff/) for linting, which analyzes code
To run linting:

```sh
make lint
uv run poe lint
```

### Formatting
Expand All @@ -41,7 +44,7 @@ Our automated code formatting also leverages [ruff](https://docs.astral.sh/ruff/
To run formatting:

```sh
make format
uv run poe format
```

### Type checking
Expand All @@ -51,7 +54,7 @@ Type checking is handled by [ty](https://docs.astral.sh/ty/), verifying code aga
To run type checking:

```sh
make type-check
uv run poe type-check
```

### Unit tests
Expand All @@ -63,20 +66,20 @@ We use [pytest](https://docs.pytest.org/) as a testing framework with many plugi
To run unit tests:

```sh
make unit-tests
uv run poe unit-tests
```

To run unit tests with HTML coverage report:
To run unit tests with XML coverage report:

```sh
make unit-tests-cov
uv run poe unit-tests-cov
```

## Integration tests

We have integration tests which build and run Actors using the Python SDK on the Apify Platform. To run these tests,
you need to set the `APIFY_TEST_USER_API_TOKEN` environment variable to the API token of the Apify user you want to
use for the tests, and then start them with `make integration-tests`.
use for the tests, and then start them with `uv run poe integration-tests`.

If you want to run the integration tests on a different environment than the main Apify Platform, you need to set
the `APIFY_INTEGRATION_TESTS_API_URL` environment variable to the right URL to the Apify API you want to use.
Expand All @@ -87,42 +90,10 @@ We adhere to the [Google docstring format](https://sphinxcontrib-napoleon.readth

Our API documentation is generated from these docstrings using [pydoc-markdown](https://pypi.org/project/pydoc-markdown/) with additional post-processing. Markdown files in the `docs/` folder complement the autogenerated content. Final documentation is rendered using [Docusaurus](https://docusaurus.io/) and published to GitHub Pages.

To run the documentation locally, you need to have Node.js version 20 or higher installed. Once you have the correct version of Node.js, follow these steps:

Navigate to the `website/` directory:

```sh
cd website/
```

Enable Corepack, which installs Yarn automatically:

```sh
corepack enable
```

Build the API reference:

```sh
./build_api_reference.sh
```

Install the necessary dependencies:

```sh
yarn
```

Start the project in development mode with Hot Module Replacement (HMR):

```sh
yarn start
```

Or using `make`:
To run the documentation locally:

```sh
make run-doc
uv run poe run-docs
```

## Release process
Expand Down
71 changes: 0 additions & 71 deletions Makefile

This file was deleted.

31 changes: 31 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ dev = [
"crawlee[parsel]",
"dycw-pytest-only<3.0.0",
"griffe",
"poethepoet<1.0.0",
"pre-commit<5.0.0",
"pydoc-markdown<5.0.0",
"pytest-asyncio<2.0.0",
Expand Down Expand Up @@ -216,3 +217,33 @@ exclude_lines = ["pragma: no cover", "if TYPE_CHECKING:", "assert_never()"]

[tool.ipdb]
context = 7

# Run tasks with: uv run poe <task>
[tool.poe.tasks]
clean = "rm -rf .coverage .pytest_cache .ruff_cache .ty_cache build dist htmlcov"
install-sync = "uv sync --all-extras"
build = "uv build --verbose"
publish-to-pypi = "uv publish --verbose --token ${APIFY_PYPI_TOKEN_CRAWLEE}"
type-check = "uv run ty check"
unit-tests = "uv run pytest --numprocesses=auto --verbose tests/unit"
unit-tests-cov = "uv run pytest --numprocesses=auto --verbose --cov=src/apify --cov-report=xml:coverage-unit.xml tests/unit"
integration-tests = "uv run pytest --numprocesses=${INTEGRATION_TESTS_CONCURRENCY:-1} --verbose tests/integration"
integration-tests-cov = "uv run pytest --numprocesses=${INTEGRATION_TESTS_CONCURRENCY:-1} --verbose --cov=src/apify --cov-report=xml:coverage-integration.xml tests/integration"
check-code = ["lint", "type-check", "unit-tests"]

[tool.poe.tasks.install-dev]
shell = "uv sync --all-extras && uv run pre-commit install"

[tool.poe.tasks.lint]
shell = "uv run ruff format --check && uv run ruff check"

[tool.poe.tasks.format]
shell = "uv run ruff check --fix && uv run ruff format"

[tool.poe.tasks.build-docs]
shell = "./build_api_reference.sh && corepack enable && yarn && yarn build"
cwd = "website"

[tool.poe.tasks.run-docs]
shell = "./build_api_reference.sh && corepack enable && yarn && yarn start"
cwd = "website"
25 changes: 25 additions & 0 deletions uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading