You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: document missing SDK features across concept pages
Add documentation for 13 previously undocumented SDK features including
Actor.use_state(), Actor.abort(), Actor.get_env(), ApifyRequestList,
storage alias parameter, tiered proxy URLs, EXIT event, advanced
ChargingManager API, ChargeResult from push_data, Actor.is_at_home(),
secret input fields, storage client architecture, and Actor
instantiation parameters.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: docs/02_concepts/01_actor_lifecycle.mdx
+9-1Lines changed: 9 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -43,7 +43,15 @@ When the Actor exits, either normally or due to an exception, the SDK performs a
43
43
</TabItem>
44
44
</Tabs>
45
45
46
-
You can also create an [`Actor`](https://docs.apify.com/sdk/python/reference/class/Actor) instance directly. This does not change its capabilities but allows you to specify optional parameters during initialization, such as disabling automatic `sys.exit()` calls or customizing timeouts. The choice between using a context manager or manual initialization depends on how much control you require over the Actor's startup and shutdown sequence.
46
+
You can also create an [`Actor`](https://docs.apify.com/sdk/python/reference/class/Actor) instance directly. This does not change its capabilities but allows you to specify optional parameters during initialization. The key parameters are:
47
+
48
+
-`configuration` — a custom [`Configuration`](https://docs.apify.com/sdk/python/reference/class/Configuration) instance to control storage paths, API URLs, and other settings.
49
+
-`configure_logging` — whether to set up default logging configuration (default `True`). Set to `False` if you configure logging yourself.
50
+
-`exit_process` — whether the Actor calls `sys.exit()` when the context manager exits. Defaults to `True`, except in IPython, Pytest, and Scrapy environments.
51
+
-`event_listeners_timeout` — maximum time to wait for Actor event listeners to complete before exiting.
52
+
-`cleanup_timeout` — maximum time to wait for cleanup tasks to finish (default 30 seconds).
53
+
54
+
The choice between using a context manager or manual initialization depends on how much control you require over the Actor's startup and shutdown sequence.
47
55
48
56
<TabsgroupId="request_queue">
49
57
<TabItemvalue="actor_instance_with_context_manager"label="Actor instance with context manager"default>
The Actor gets its [input](https://docs.apify.com/platform/actors/running/input) from the input record in its default [key-value store](https://docs.apify.com/platform/storage/key-value-store).
12
14
@@ -17,3 +19,17 @@ For example, if an Actor received a JSON input with two fields, `{ "firstNumber"
Actors commonly receive a list of URLs to process via their input. The <ApiLinkto="class/ApifyRequestList">`ApifyRequestList`</ApiLink> class (from `apify.request_loaders`) can parse the standard Apify input format for URL sources. It supports both direct URL objects (`{"url": "https://example.com"}`) and remote URL lists (`{"requestsFromUrl": "https://example.com/urls.txt"}`), where the remote file contains one URL per line.
The Apify platform supports [secret input fields](https://docs.apify.com/platform/actors/development/secret-input) that are encrypted before being stored. When you mark an input field as `"isSecret": true` in your Actor's [input schema](https://docs.apify.com/platform/actors/development/input-schema), the platform encrypts the value with the Actor's public key.
34
+
35
+
No special handling is needed in your code — when you call [`Actor.get_input`](../../reference/class/Actor#get_input), encrypted fields are automatically decrypted using the Actor's private key, which is provided by the platform via environment variables. You receive the plaintext values directly.
@@ -60,7 +62,7 @@ There are several methods for directly working with the default key-value store
60
62
-[`Actor.get_value('my-record')`](../../reference/class/Actor#get_value) reads a record from the default key-value store of the Actor.
61
63
-[`Actor.set_value('my-record', 'my-value')`](../../reference/class/Actor#set_value) saves a new value to the record in the default key-value store.
62
64
-[`Actor.get_input`](../../reference/class/Actor#get_input) reads the Actor input from the default key-value store of the Actor.
63
-
-[`Actor.push_data([{'result': 'Hello, world!'}, ...])`](../../reference/class/Actor#push_data) saves results to the default dataset of the Actor.
65
+
-[`Actor.push_data([{'result': 'Hello, world!'}, ...])`](../../reference/class/Actor#push_data) saves results to the default dataset of the Actor. When using the [pay-per-event pricing model](./pay-per-event), `push_data` returns a `ChargeResult` object that indicates whether the charge limit has been reached. You can also pass a `charged_event_name` parameter to charge for a custom event for each pushed item.
64
66
65
67
## Opening named and unnamed storages
66
68
@@ -70,6 +72,12 @@ The [`Actor.open_dataset`](../../reference/class/Actor#open_dataset), [`Actor.op
70
72
{OpeningStoragesExample}
71
73
</RunnableCodeBlock>
72
74
75
+
Besides `id` and `name`, the `open_*` methods also accept an `alias` parameter. An alias creates an unnamed storage scoped to the current Actor run — it does not persist across runs, but lets you reference the same storage within a single run using a human-readable label. The `alias` parameter is mutually exclusive with `id` and `name`.
Behind the scenes, the SDK uses storage clients to communicate with the storage backend. The SDK automatically selects the appropriate client based on the runtime environment:
187
+
188
+
-**`SmartApifyStorageClient`** (default on the Apify platform) — a hybrid client that writes to both the Apify API and the local filesystem for resilience.
189
+
-**`ApifyStorageClient`** — communicates directly with the Apify platform API for cloud storage.
190
+
-**`FileSystemStorageClient`** — stores data on the local filesystem (in the `storage/` directory). Used when running locally.
191
+
-**`MemoryStorageClient`** (from Crawlee) — stores data in memory. Useful for testing.
192
+
193
+
For most use cases, the default storage client selection is sufficient. All storage clients are available from the `apify.storage_clients` module. For details, see the <ApiLinkto="class/ApifyStorageClient">`ApifyStorageClient`</ApiLink> API reference.
The example above shows how to manually persist state using the `PERSIST_STATE` event. For most use cases, you can use the <ApiLinkto="class/Actor#use_state">`Actor.use_state`</ApiLink> method instead, which handles state persistence automatically.
97
+
98
+
`Actor.use_state` returns a dictionary that is automatically saved to the default key-value store at regular intervals and whenever a migration or shutdown occurs. You can modify the dictionary in place, and changes are persisted without any manual `set_value` calls.
99
+
100
+
You can optionally specify a `key` (the key-value store key under which the state is stored) and a `kvs_name` (the name of the key-value store to use). By default, the state is stored in the default key-value store under a default key.
The Apify SDK provides built-in proxy management through the <ApiLinkto="class/ProxyConfiguration">`ProxyConfiguration`</ApiLink> class, supporting both [Apify Proxy](https://apify.com/proxy) and custom proxy servers. Proxies are essential for web scraping to avoid [IP address blocking](https://en.wikipedia.org/wiki/IP_address_blocking) and distribute requests across multiple addresses.
@@ -81,6 +82,18 @@ Or you can pass it a method (accepting one optional argument, the session ID), t
81
82
{CustomProxyFunctionExample}
82
83
</RunnableCodeBlock>
83
84
85
+
### Tiered proxy rotation
86
+
87
+
<ApiLinkto="class/ProxyConfiguration">`ProxyConfiguration`</ApiLink> supports tiered proxy URLs via the `tiered_proxy_urls` parameter. This accepts a list of lists of proxy URLs, where each inner list represents a tier. The proxy rotator starts with the first (cheapest) tier and automatically escalates to higher tiers when lower-tier proxies get blocked. This is useful for optimizing proxy costs — you use cheap datacenter proxies for most requests and only switch to expensive residential proxies when necessary.
88
+
89
+
:::info
90
+
The `tiered_proxy_urls` parameter is only available when constructing `ProxyConfiguration` directly. It is not supported by `Actor.create_proxy_configuration()`.
To make selecting the proxies that the Actor uses easier, you can use an input field with the editor [`proxy` in your input schema](https://docs.apify.com/platform/actors/development/input-schema#object). This input will then be filled with a dictionary containing the proxy settings you or the users of your Actor selected for the Actor run.
The Apify SDK lets you start, call, and transform (metamorph) other Actors directly from your Actor code. This is useful for composing complex workflows from smaller, reusable Actors.
@@ -52,4 +53,14 @@ For example, imagine you have an Actor that accepts a hotel URL on input, and th
52
53
{InteractingMetamorphExample}
53
54
</RunnableCodeBlock>
54
55
56
+
## Aborting an Actor run
57
+
58
+
The [`Actor.abort`](../../reference/class/Actor#abort) method aborts a running Actor on the Apify platform. You can use it to cancel a long-running Actor that is no longer needed.
59
+
60
+
When you set `gracefully=True`, the platform sends `ABORTING` and `PERSIST_STATE` events to the target Actor, giving it time to save its state, and then force-stops it after 30 seconds. Without the `gracefully` flag, the Actor is stopped immediately.
The <ApiLinkto="class/Actor">`Actor`</ApiLink> class is configured through the <ApiLinkto="class/Configuration">`Configuration`</ApiLink> class, which reads its settings from environment variables. When running on the Apify platform or through the Apify CLI, configuration is automatic — manual setup is only needed for custom requirements.
@@ -33,4 +35,20 @@ This Actor run will not persist its local storages to the filesystem:
33
35
APIFY_PERSIST_STORAGE=0 apify run
34
36
```
35
37
38
+
## Reading the runtime environment
39
+
40
+
The <ApiLinkto="class/Actor#get_env">`Actor.get_env`</ApiLink> method returns a dictionary with all `APIFY_*` environment variables parsed into their typed values. This is useful for inspecting the Actor's runtime context, such as the Actor ID, run ID, or default storage IDs. Variables that are not set or are invalid will have a value of `None`.
The <ApiLinkto="class/Actor#is_at_home">`Actor.is_at_home`</ApiLink> method returns `True` when the Actor is running on the Apify platform, and `False` when running locally. This is useful for branching behavior based on the environment, such as using different storage backends or skipping proxy configuration during local development.
@@ -48,6 +49,23 @@ Alternatively, you can periodically check the remaining budget via <ApiLink to="
48
49
Always check the charge limit in your Actor, whether through `ChargeResult` return values or the `ChargingManager`. Without this check, your Actor will continue running and consuming platform resources after the budget is exhausted, producing no output.
49
50
:::
50
51
52
+
## Advanced charging management
53
+
54
+
The <ApiLinkto="class/ChargingManager">`ChargingManager`</ApiLink> (accessed via <ApiLinkto="class/Actor#get_charging_manager">`Actor.get_charging_manager()`</ApiLink>) provides methods for fine-grained budget control:
55
+
56
+
-`get_max_total_charge_usd()` — the configured budget limit for this run.
57
+
-`calculate_total_charged_amount()` — total USD charged so far.
58
+
-`calculate_max_event_charge_count_within_limit(event_name)` — how many more events of this type can be charged before reaching the limit.
59
+
-`get_charged_event_count(event_name)` — how many events of this type have been charged.
60
+
-`is_event_charge_limit_reached(event_name)` — whether the remaining budget is too low for even one more event of this type.
61
+
-`compute_chargeable()` — a dict of all event types and how many can still be charged.
62
+
63
+
These methods are useful for budget-aware crawling strategies, where you want to plan work based on the remaining budget rather than discovering the limit after the fact.
When you plan to start using the pay-per-event pricing model for an Actor that is already monetized with a different pricing model, your source code will need support both pricing models during the transition period enforced by the Apify platform. Arguably the most frequent case is the transition from the pay-per-result model which utilizes the `ACTOR_MAX_PAID_DATASET_ITEMS` environment variable to prevent returning unpaid dataset items. The following is an example how to handle such scenarios. The key part is the <ApiLinkto="class/ChargingManager#get_pricing_info">`ChargingManager.get_pricing_info()`</ApiLink> method which returns information about the current pricing model.
0 commit comments