Skip to content

Commit c6c4d46

Browse files
committed
Merge remote-tracking branch 'origin/master' into adapt-to-apify-client-v3
# Conflicts: # uv.lock
2 parents 9e1bb22 + a6c1a73 commit c6c4d46

24 files changed

Lines changed: 4527 additions & 3241 deletions

File tree

.github/workflows/_checks.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,7 @@ jobs:
110110
matrix.python-version == '3.14' &&
111111
env.CODECOV_TOKEN != ''
112112
}}
113-
uses: codecov/codecov-action@v6
113+
uses: codecov/codecov-action@v7
114114
env:
115115
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
116116
with:
@@ -170,7 +170,7 @@ jobs:
170170
matrix.python-version == '3.14' &&
171171
env.CODECOV_TOKEN != ''
172172
}}
173-
uses: codecov/codecov-action@v6
173+
uses: codecov/codecov-action@v7
174174
env:
175175
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
176176
with:

docs/01_introduction/quick-start.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,3 +106,4 @@ To see how you can integrate the Apify SDK with popular web scraping libraries,
106106
- [Crawlee](../guides/crawlee)
107107
- [Scrapy](../guides/scrapy)
108108
- [Running webserver](../guides/running-webserver)
109+
- [uv](../guides/uv)

docs/02_concepts/code/04_use_state.py

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,14 @@ async def main() -> None:
99
# On restart or migration, the state is loaded from the KVS.
1010
state = await Actor.use_state(default_value={'processed_items': 0})
1111

12-
# Resume from previous state
12+
# Resume from the persisted state (stored as JSON, so narrow the type).
1313
start_index = state['processed_items']
14+
if not isinstance(start_index, int):
15+
start_index = 0
1416
Actor.log.info(f'Resuming from item {start_index}')
1517

1618
# Do some work and update the state — it is persisted automatically
17-
for i in range(start_index, 100): # ty: ignore[invalid-argument-type]
19+
for i in range(start_index, 100):
1820
Actor.log.info(f'Processing item {i}...')
1921
state['processed_items'] = i + 1
2022
await asyncio.sleep(0.1)

docs/02_concepts/code/05_custom_proxy_function.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,11 @@
77

88
async def custom_new_url_function(
99
session_id: str | None = None,
10-
_: Request | None = None,
10+
request: Request | None = None,
1111
) -> str | None:
12+
# Pick a proxy URL based on the session and/or the request being proxied.
13+
if request is not None:
14+
Actor.log.debug(f'Selecting a proxy URL for {request.url}.')
1215
if session_id is not None:
1316
return f'http://my-custom-proxy-supporting-sessions.com?session-id={session_id}'
1417
return 'http://my-custom-proxy-not-supporting-sessions.com'
@@ -17,7 +20,7 @@ async def custom_new_url_function(
1720
async def main() -> None:
1821
async with Actor:
1922
proxy_cfg = await Actor.create_proxy_configuration(
20-
new_url_function=custom_new_url_function, # ty: ignore[invalid-argument-type]
23+
new_url_function=custom_new_url_function,
2124
)
2225

2326
if not proxy_cfg:

docs/03_guides/10_uv.mdx

Lines changed: 194 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,194 @@
1+
---
2+
id: uv
3+
title: Project management with uv
4+
description: Manage your Actor's Python version, dependencies, and virtual environment with the uv package and project manager.
5+
---
6+
7+
import CodeBlock from '@theme/CodeBlock';
8+
import Tabs from '@theme/Tabs';
9+
import TabItem from '@theme/TabItem';
10+
11+
import PyprojectExample from '!!raw-loader!./code/uv_project/pyproject.toml';
12+
import MainExample from '!!raw-loader!./code/uv_project/my_actor/main.py';
13+
import UnderscoreMainExample from '!!raw-loader!./code/uv_project/my_actor/__main__.py';
14+
import DockerfileExample from '!!raw-loader!./code/uv_project/Dockerfile';
15+
16+
In this guide, you'll learn how to use [uv](https://docs.astral.sh/uv/) to manage your Apify Actor projects. From creating a project and running it locally to building and deploying it on the Apify platform.
17+
18+
## Introduction
19+
20+
[uv](https://docs.astral.sh/uv/) is a modern project and package manager for Python. It replaces the combination of pip, virtualenv, and similar tools with a single binary that manages your project's Python version, virtual environment, and dependencies. It records the project metadata in the standard [`pyproject.toml`](https://packaging.python.org/en/latest/guides/writing-pyproject-toml/) file and the exact resolved versions of all dependencies in a [`uv.lock`](https://docs.astral.sh/uv/concepts/projects/sync/) lockfile.
21+
22+
The [Python Actor templates](https://apify.com/templates/categories/python) declare their dependencies in a `requirements.txt` file, which is the default approach for Actors. Using uv instead brings a few advantages:
23+
24+
- The lockfile guarantees that the dependencies installed in the Actor's Docker image are exactly the ones that you locally developed and tested against.
25+
- Dependency installation during the Docker build is significantly faster than with pip, especially with a warm cache.
26+
- During local development, a single tool manages your Python version, virtual environment, and dependencies. As a result, the project behaves the same on every developer's machine.
27+
28+
:::info Actor templates don't support uv yet
29+
30+
The [Apify Actor templates](https://apify.com/templates) currently support only pip with `requirements.txt`. Adding uv-based templates is planned. For updates, follow [apify/actor-templates#350](https://github.com/apify/actor-templates/issues/350).
31+
32+
:::
33+
34+
## Before you start
35+
36+
To follow along, install [uv](https://docs.astral.sh/uv/getting-started/installation/) and the [Apify CLI](https://docs.apify.com/cli/docs/installation) first.
37+
38+
## Create a new project
39+
40+
To create a new uv project and add the Apify SDK to its dependencies, use:
41+
42+
```bash
43+
uv init my-actor --bare
44+
cd my-actor
45+
uv python pin 3.14
46+
uv add apify
47+
```
48+
49+
Where:
50+
51+
- [`uv init`](https://docs.astral.sh/uv/reference/cli/#uv-init) with the `--bare` option creates the `pyproject.toml` project manifest.
52+
- `uv python pin` writes the project's Python version to the `.python-version` file. uv automatically downloads that Python version if it's not installed on your machine.
53+
- [`uv add`](https://docs.astral.sh/uv/reference/cli/#uv-add) records the dependency in `pyproject.toml`, resolves the exact versions of the whole dependency tree into `uv.lock`, and installs everything into the project's virtual environment in `.venv`.
54+
55+
The `uv add` command constrains the dependency to the latest version it resolved. You can edit the constraint as you see fit. The example Actor in this guide allows any version of the SDK within the current major one:
56+
57+
<CodeBlock className="language-toml" title="pyproject.toml">
58+
{PyprojectExample}
59+
</CodeBlock>
60+
61+
Note that the example has no `[build-system]` section. Without one, uv treats the project as a non-package ("virtual") project: it doesn't try to build and install the project itself, it only manages its dependencies. As a result, the Actor runs as a module straight from the source tree.
62+
63+
## Add the Actor scaffolding
64+
65+
For the project to be runnable as an Actor, it needs two more pieces: the source code as a runnable Python package, and the `.actor/` directory with the [Actor configuration](https://docs.apify.com/platform/actors/development/actor-definition/actor-json).
66+
67+
1. Create a `my_actor` package with the Actor's source code:
68+
69+
<Tabs>
70+
<TabItem value="my_actor/main.py" label="my_actor/main.py" default>
71+
<CodeBlock className="language-python">
72+
{MainExample}
73+
</CodeBlock>
74+
</TabItem>
75+
<TabItem value="my_actor/__main__.py" label="my_actor/__main__.py">
76+
<CodeBlock className="language-python">
77+
{UnderscoreMainExample}
78+
</CodeBlock>
79+
</TabItem>
80+
</Tabs>
81+
82+
1. Add an empty `my_actor/__init__.py` file, so that the directory is a regular Python package executable with `python -m my_actor`.
83+
84+
1. Add the Actor definition to `.actor/actor.json`:
85+
86+
```json title=".actor/actor.json"
87+
{
88+
"$schema": "https://apify.com/schemas/v1/actor.ide.json",
89+
"actorSpecification": 1,
90+
"name": "my-actor",
91+
"title": "My uv Actor",
92+
"description": "An Apify Actor with dependencies managed by uv.",
93+
"version": "0.1",
94+
"buildTag": "latest",
95+
"dockerfile": "../Dockerfile"
96+
}
97+
```
98+
99+
The `dockerfile` field points to the project's `Dockerfile`, which doesn't exist yet. You'll create it in the [Use uv in the Dockerfile](#use-uv-in-the-dockerfile) section.
100+
101+
The final project structure looks like this:
102+
103+
```text
104+
my-actor/
105+
├── .actor/
106+
│ └── actor.json
107+
├── my_actor/
108+
│ ├── __init__.py
109+
│ ├── __main__.py
110+
│ └── main.py
111+
├── .python-version
112+
├── Dockerfile
113+
├── pyproject.toml
114+
└── uv.lock
115+
```
116+
117+
Make sure to commit `uv.lock` and `.python-version` to version control, so that every developer's machine works with identical dependencies and the same Python version. The Actor's Docker build gets its Python interpreter from the base image instead, so keep the base image tag (`apify/actor-python:3.14`) in sync with `.python-version`.
118+
119+
## Run the Actor locally
120+
121+
If you've just cloned the project or skipped the `uv add` command, install the dependencies first:
122+
123+
```bash
124+
uv sync
125+
```
126+
127+
The [`uv sync`](https://docs.astral.sh/uv/reference/cli/#uv-sync) command creates the `.venv` virtual environment (if it doesn't exist yet) and installs the locked dependencies into it. Then, run the Actor with the Apify CLI:
128+
129+
```bash
130+
apify run
131+
```
132+
133+
The [`apify run`](https://docs.apify.com/cli/docs/reference#apify-run) command automatically detects the virtual environment in `.venv` and uses it to run the Actor as a module (`python -m my_actor`), with the environment set up to emulate the Apify platform locally. For example, the Actor input is read from `storage/key_value_stores/default/INPUT.json`.
134+
135+
## Use uv in the Dockerfile
136+
137+
On the Apify platform, the Actor runs as a Docker container built from the Dockerfile referenced in `.actor/actor.json`. The following Dockerfile installs the locked dependencies with uv on top of the [Apify Python base image](https://hub.docker.com/r/apify/actor-python):
138+
139+
<CodeBlock className="language-docker" title="Dockerfile">
140+
{DockerfileExample}
141+
</CodeBlock>
142+
143+
Note that:
144+
145+
- The uv binary is copied from its [official Docker image](https://docs.astral.sh/uv/guides/integration/docker/), pinned to a minor version line, so builds are reproducible and there is no need to install uv with pip.
146+
- `uv sync --locked --no-dev` installs the dependencies exactly as recorded in `uv.lock` and skips development dependencies. If the lockfile is missing or out of sync with `pyproject.toml`, the build fails instead of silently resolving different versions.
147+
- The dependencies are installed in a separate layer before the source code is copied, so editing your code doesn't invalidate the dependency layer, and rebuilds are fast.
148+
- Putting `.venv/bin` first on `PATH` makes `python` resolve to the project's virtual environment, both during the build and when the Actor runs.
149+
150+
Also create a `.dockerignore` file and exclude at least `.venv`, `.git`, and `storage` from the Docker build context. The local virtual environment must never be copied into the image, since it's recreated by `uv sync` during the build.
151+
152+
## Deploy to the Apify platform
153+
154+
Once the Actor works locally, log in and push it to the Apify platform:
155+
156+
```bash
157+
apify login
158+
apify push
159+
```
160+
161+
The [`apify push`](https://docs.apify.com/cli/docs/reference#apify-push) command uploads the project to the platform and builds the Docker image from the Dockerfile above. Thanks to the committed lockfile, the platform build installs exactly the dependency versions you ran locally.
162+
163+
## Manage dependencies
164+
165+
Day-to-day dependency management goes through uv as well:
166+
167+
```bash
168+
# Add a dependency (records it in pyproject.toml and updates uv.lock).
169+
uv add httpx
170+
171+
# Add a development-only dependency (skipped in the Docker build by --no-dev).
172+
uv add --dev ruff
173+
174+
# Remove a dependency.
175+
uv remove httpx
176+
177+
# Upgrade all dependencies to the latest versions allowed by pyproject.toml.
178+
uv lock --upgrade
179+
uv sync
180+
```
181+
182+
Whenever the dependencies change, commit the updated `uv.lock` together with `pyproject.toml`.
183+
184+
## Conclusion
185+
186+
In this guide, you learned how to use uv to manage Apify Actor projects. You can now create a uv project with the Apify SDK, run it locally with the Apify CLI, install the locked dependencies with uv in the Actor's Docker image, and deploy the whole project to the Apify platform with reproducible builds. If you have questions or need assistance, feel free to reach out on our [GitHub](https://github.com/apify/apify-sdk-python) or join our [Discord community](https://discord.com/invite/jyEM2PRvMU). Happy coding!
187+
188+
## Additional resources
189+
190+
- [uv: Official documentation](https://docs.astral.sh/uv/)
191+
- [uv: Working on projects](https://docs.astral.sh/uv/guides/projects/)
192+
- [uv: Using uv in Docker](https://docs.astral.sh/uv/guides/integration/docker/)
193+
- [Apify: Actor Dockerfile documentation](https://docs.apify.com/platform/actors/development/actor-definition/dockerfile)
194+
- [Apify templates: Python](https://apify.com/templates/categories/python)
Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# syntax=docker/dockerfile:1
2+
# First, specify the base Docker image.
3+
# You can see the Docker images from Apify at https://hub.docker.com/r/apify/.
4+
# You can also use any other image from Docker Hub.
5+
FROM apify/actor-python:3.14
6+
7+
# Add the uv binary from its official distroless image (pinned to the 0.11.x line).
8+
COPY --from=ghcr.io/astral-sh/uv:0.11 /uv /uvx /bin/
9+
10+
# Configure uv for container builds:
11+
# - compile installed packages to bytecode, so the Actor starts faster,
12+
# - copy packages instead of hardlinking, which avoids warnings with the cache mount,
13+
# - never download a managed Python, always reuse the base image's interpreter,
14+
# - put the project virtual environment first on PATH, so `python` resolves to it.
15+
ENV UV_COMPILE_BYTECODE=1 \
16+
UV_LINK_MODE=copy \
17+
UV_PYTHON_DOWNLOADS=0 \
18+
PATH="/usr/src/app/.venv/bin:$PATH"
19+
20+
# Install dependencies into the project virtual environment (.venv) as a separate
21+
# layer. The cache mount speeds up repeated builds, and the bind mounts make the
22+
# project metadata available without copying it into the image. This layer is
23+
# rebuilt only when uv.lock or pyproject.toml change - not on source code edits.
24+
RUN --mount=type=cache,target=/root/.cache/uv \
25+
--mount=type=bind,source=uv.lock,target=uv.lock \
26+
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \
27+
uv sync --locked --no-dev
28+
29+
# Next, copy the remaining files and directories with the source code.
30+
# Since we do this after installing the dependencies, quick rebuilds will be
31+
# really fast for most source file changes.
32+
COPY . ./
33+
34+
# Use compileall to ensure the runnability of the Actor Python code.
35+
RUN python -m compileall -q my_actor/
36+
37+
# Specify how to launch the source code of your Actor.
38+
CMD ["python", "-m", "my_actor"]

docs/03_guides/code/uv_project/my_actor/__init__.py

Whitespace-only changes.
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
import asyncio
2+
3+
from .main import main
4+
5+
if __name__ == '__main__':
6+
asyncio.run(main())
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
from apify import Actor
2+
3+
4+
async def main() -> None:
5+
async with Actor:
6+
actor_input = await Actor.get_input() or {}
7+
Actor.log.info('Actor input: %s', actor_input)
8+
await Actor.set_value('OUTPUT', 'Hello from a uv-managed Actor!')
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
[project]
2+
name = "my-actor"
3+
version = "0.1.0"
4+
description = "An Apify Actor with dependencies managed by uv."
5+
requires-python = ">=3.14"
6+
dependencies = [
7+
"apify>=3.0.0,<4.0.0",
8+
]

0 commit comments

Comments
 (0)