Skip to content

Commit 165a3f6

Browse files
authored
feat: re-export apify-client errors from apify.errors (#990)
### Description Adds an `apify.errors` module that re-exports the `apify_client` error hierarchy, so SDK users have a single import location for every error the SDK can surface. The SDK raises these client exceptions as-is and does not wrap them. `Actor.call` and `Actor.call_task` keep returning the finished run regardless of status. Check `run.status` to react to a failed run. A new "Error handling" concept page maps every layer of exceptions an Actor can hit: - API client errors (`ApifyApiError` and its status-code subclasses). - The standard `RuntimeError`, `ValueError`, `TypeError`, and `ConnectionError` the SDK raises for misuse and invalid input. - Run-status checking after `Actor.call` and `Actor.call_task`. - The pay-per-event charge limit, which is a return value rather than an exception. - Short mention of errors while crawling and links Crawlee docs. ### Issue Closes #988
1 parent ff8c35f commit 165a3f6

7 files changed

Lines changed: 222 additions & 0 deletions

File tree

docs/02_concepts/13_exceptions.mdx

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
---
2+
id: error-handling
3+
title: Error handling
4+
description: The exceptions an Actor can raise and how to handle them
5+
---
6+
7+
import HandleCallErrorsSource from '!!raw-loader!roa-loader!./code/13_handle_call_errors.py';
8+
import RetryTimedOutSource from '!!raw-loader!roa-loader!./code/13_retry_timed_out.py';
9+
import ApiLink from '@theme/ApiLink';
10+
import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock';
11+
12+
When you run an Actor, exceptions come from a few layers: the Apify API client for failed API requests, the Apify SDK for misuse and invalid input, and the libraries you build on, such as Crawlee.
13+
14+
## Errors from the Apify API
15+
16+
Every SDK operation that talks to the Apify API can raise <ApiLink to="class/ApifyApiError">`ApifyApiError`</ApiLink>. Such operations include <ApiLink to="class/Actor#start">`Actor.start`</ApiLink>, <ApiLink to="class/Actor#call">`Actor.call`</ApiLink>, <ApiLink to="class/Actor#abort">`Actor.abort`</ApiLink>, <ApiLink to="class/Actor#metamorph">`Actor.metamorph`</ApiLink>, <ApiLink to="class/Actor#add_webhook">`Actor.add_webhook`</ApiLink>, charging, and all storage operations on datasets, key-value stores, and request queues. The SDK raises these client exceptions as-is, so you keep the HTTP status code, the error type, and the response data on the exception.
17+
18+
<ApiLink to="class/ApifyApiError">`ApifyApiError`</ApiLink> dispatches to a subclass based on the HTTP status code:
19+
20+
- <ApiLink to="class/UnauthorizedError">`UnauthorizedError`</ApiLink> (401) and <ApiLink to="class/ForbiddenError">`ForbiddenError`</ApiLink> (403) for an unauthorized or forbidden request.
21+
- <ApiLink to="class/NotFoundError">`NotFoundError`</ApiLink> (404) when the Actor, run, or storage doesn't exist.
22+
- <ApiLink to="class/ConflictError">`ConflictError`</ApiLink> (409) for a conflicting request.
23+
- <ApiLink to="class/RateLimitError">`RateLimitError`</ApiLink> (429) when the API rate limit is hit.
24+
- <ApiLink to="class/ServerError">`ServerError`</ApiLink> for any 5xx response.
25+
- <ApiLink to="class/InvalidRequestError">`InvalidRequestError`</ApiLink> (400) when the API rejects the request as malformed.
26+
27+
The client retries rate-limited and server errors on its own, so you only see <ApiLink to="class/RateLimitError">`RateLimitError`</ApiLink> or <ApiLink to="class/ServerError">`ServerError`</ApiLink> once those retries are exhausted. The `apify.errors` module re-exports the whole client error hierarchy, so you can import everything from one place:
28+
29+
```python
30+
from apify.errors import ApifyApiError, NotFoundError, RateLimitError
31+
```
32+
33+
To handle any API failure in one place, catch <ApiLink to="class/ApifyApiError">`ApifyApiError`</ApiLink>, then branch on the subclass or the HTTP `status_code`. To react to a specific failure, catch its subclass first:
34+
35+
<RunnableCodeBlock className="language-python" language="python">
36+
{HandleCallErrorsSource}
37+
</RunnableCodeBlock>
38+
39+
## Misuse and invalid input
40+
41+
The SDK raises standard Python exceptions when it's used incorrectly or given invalid input. These exceptions point to a bug or a bad argument in your code, so the fix is to correct the call rather than to catch the exception.
42+
43+
- [`RuntimeError`](https://docs.python.org/3/library/exceptions.html#RuntimeError) when an `Actor` method is used outside the `async with Actor:` block, either before initialization or after exit, or when the Actor is initialized twice.
44+
- [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError) for an invalid argument, such as a malformed `timeout`, an invalid proxy configuration, charging an automatically charged event by hand, or pushing data that is not JSON-serializable or is over the size limit.
45+
- [`TypeError`](https://docs.python.org/3/library/exceptions.html#TypeError) for an argument of the wrong type.
46+
- [`ConnectionError`](https://docs.python.org/3/library/exceptions.html#ConnectionError) when <ApiLink to="class/Actor#create_proxy_configuration">`Actor.create_proxy_configuration`</ApiLink> verifies Apify Proxy access and the proxy reports that you have none.
47+
48+
## Run failures
49+
50+
<ApiLink to="class/Actor#call">`Actor.call`</ApiLink> and <ApiLink to="class/Actor#call_task">`Actor.call_task`</ApiLink> wait for the run to finish and return it, whatever its final status. A finished run can be `SUCCEEDED`, `FAILED`, `ABORTED`, or `TIMED-OUT`, so check `run.status` before you rely on the run's output. A timed-out run is the one case where retrying can help, as long as you give it more time:
51+
52+
<RunnableCodeBlock className="language-python" language="python">
53+
{RetryTimedOutSource}
54+
</RunnableCodeBlock>
55+
56+
## The pay-per-event charge limit
57+
58+
Reaching the pay-per-event charge limit doesn't raise an error. Instead, the SDK caps charging and data pushing, while your Actor keeps running. When a single <ApiLink to="class/Actor#charge">`Actor.charge`</ApiLink> call crosses the limit, only the part that fits within the budget is billed, and `charged_count` on the returned <ApiLink to="class/ChargeResult">`ChargeResult`</ApiLink> reports how many events went through. <ApiLink to="class/Actor#push_data">`Actor.push_data`</ApiLink> behaves the same way when given a `charged_event_name`. It writes only the items that fit within the budget.
59+
60+
To detect the limit, check the `event_charge_limit_reached` field on the `ChargeResult`. It's a return value and not an exception, so you can read it in a tight charging loop and stop your work once the budget runs out. For details, see [Pay-per-event monetization](./pay-per-event).
61+
62+
## Errors while crawling
63+
64+
If your Actor runs a [Crawlee](https://crawlee.dev/python) crawler, failures inside request handlers surface as Crawlee exceptions. Crawlee handles the retries and session rotation around them, so a single failing request doesn't stop the crawl. API calls you make from inside a handler still raise <ApiLink to="class/ApifyApiError">`ApifyApiError`</ApiLink>. For how to handle those errors, see [Errors from the Apify API](#errors-from-the-apify-api).
65+
66+
## Conclusion
67+
68+
Most failures you handle at runtime are <ApiLink to="class/ApifyApiError">`ApifyApiError`</ApiLink> from the API client. Catch it to cover any API failure, and reach for a subclass or the HTTP `status_code` when you need finer control. The standard [`RuntimeError`](https://docs.python.org/3/library/exceptions.html#RuntimeError), [`ValueError`](https://docs.python.org/3/library/exceptions.html#ValueError), and [`TypeError`](https://docs.python.org/3/library/exceptions.html#TypeError) signal a bug or bad input, so correct the call rather than catch them. After <ApiLink to="class/Actor#call">`Actor.call`</ApiLink>, check `run.status` to react to a failed run, and let Crawlee handle the errors raised inside a crawler.
Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
import asyncio
2+
3+
from apify import Actor
4+
from apify.errors import ApifyApiError, NotFoundError
5+
6+
7+
async def main() -> None:
8+
async with Actor:
9+
try:
10+
run = await Actor.call('apify/web-scraper', run_input={'startUrls': []})
11+
except NotFoundError:
12+
# Catch a specific subclass first.
13+
Actor.log.error('The Actor to call does not exist.')
14+
return
15+
except ApifyApiError as exc:
16+
# Any other API failure, e.g. an invalid token or a server error.
17+
Actor.log.error(f'Calling the Actor failed: {exc} (HTTP {exc.status_code}).')
18+
return
19+
20+
# `Actor.call` returns the finished run whatever its status, so check it.
21+
if run.status != 'SUCCEEDED':
22+
Actor.log.error(f'Run {run.id} ended with status {run.status}.')
23+
return
24+
25+
Actor.log.info(f'Run {run.id} finished successfully.')
26+
27+
28+
if __name__ == '__main__':
29+
asyncio.run(main())
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
import asyncio
2+
from datetime import timedelta
3+
4+
from apify import Actor
5+
6+
7+
async def main() -> None:
8+
async with Actor:
9+
timeout = timedelta(minutes=5)
10+
max_attempts = 3
11+
12+
for attempt in range(1, max_attempts + 1):
13+
run = await Actor.call('apify/web-scraper', timeout=timeout)
14+
15+
if run.status != 'TIMED-OUT' or attempt == max_attempts:
16+
Actor.log.info(f'Run {run.id} ended with status {run.status}.')
17+
break
18+
19+
timeout *= 2
20+
Actor.log.warning(f'Timed out, retrying with timeout {timeout}.')
21+
22+
23+
if __name__ == '__main__':
24+
asyncio.run(main())

src/apify/_utils.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ def is_running_in_ipython() -> bool:
7474
'Actor',
7575
'Charging',
7676
'Configuration',
77+
'Errors',
7778
'Event data',
7879
'Event managers',
7980
'Events',

src/apify/errors.py

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
"""`apify.errors` re-exports the Apify API client's error hierarchy.
2+
3+
Callers get a single import location for every error raised by an operation that talks to the Apify API. The SDK
4+
raises these client exceptions as-is and does not wrap them in its own types. See
5+
https://docs.apify.com/api/client/python for the full client error reference.
6+
"""
7+
8+
from __future__ import annotations
9+
10+
from apify_client.errors import (
11+
ApifyApiError,
12+
ApifyClientError,
13+
ConflictError,
14+
ForbiddenError,
15+
InvalidRequestError,
16+
InvalidResponseBodyError,
17+
NotFoundError,
18+
RateLimitError,
19+
ServerError,
20+
UnauthorizedError,
21+
)
22+
23+
__all__ = [
24+
'ApifyApiError',
25+
'ApifyClientError',
26+
'ConflictError',
27+
'ForbiddenError',
28+
'InvalidRequestError',
29+
'InvalidResponseBodyError',
30+
'NotFoundError',
31+
'RateLimitError',
32+
'ServerError',
33+
'UnauthorizedError',
34+
]

tests/unit/test_errors.py

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
from __future__ import annotations
2+
3+
import apify_client.errors as client_errors
4+
5+
import apify.errors as sdk_errors
6+
7+
8+
def test_client_errors_are_re_exported() -> None:
9+
"""`apify.errors` re-exports the API client error hierarchy so callers have a single import location."""
10+
names = [
11+
'ApifyApiError',
12+
'ApifyClientError',
13+
'ConflictError',
14+
'ForbiddenError',
15+
'InvalidRequestError',
16+
'InvalidResponseBodyError',
17+
'NotFoundError',
18+
'RateLimitError',
19+
'ServerError',
20+
'UnauthorizedError',
21+
]
22+
assert set(sdk_errors.__all__) == set(names)
23+
for name in names:
24+
assert getattr(sdk_errors, name) is getattr(client_errors, name)

website/docusaurus.config.js

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ const GROUP_ORDER = [
99
'Actor',
1010
'Charging',
1111
'Configuration',
12+
'Errors',
1213
'Event data',
1314
'Event managers',
1415
'Events',
@@ -149,6 +150,47 @@ module.exports = {
149150
moduleShortcutsPath: join(__dirname, '/module_shortcuts.json'),
150151
},
151152
reexports: [
153+
// Errors
154+
{
155+
url: 'https://docs.apify.com/api/client/python/reference/class/ApifyApiError',
156+
group: 'Errors',
157+
},
158+
{
159+
url: 'https://docs.apify.com/api/client/python/reference/class/ApifyClientError',
160+
group: 'Errors',
161+
},
162+
{
163+
url: 'https://docs.apify.com/api/client/python/reference/class/ConflictError',
164+
group: 'Errors',
165+
},
166+
{
167+
url: 'https://docs.apify.com/api/client/python/reference/class/ForbiddenError',
168+
group: 'Errors',
169+
},
170+
{
171+
url: 'https://docs.apify.com/api/client/python/reference/class/InvalidRequestError',
172+
group: 'Errors',
173+
},
174+
{
175+
url: 'https://docs.apify.com/api/client/python/reference/class/InvalidResponseBodyError',
176+
group: 'Errors',
177+
},
178+
{
179+
url: 'https://docs.apify.com/api/client/python/reference/class/NotFoundError',
180+
group: 'Errors',
181+
},
182+
{
183+
url: 'https://docs.apify.com/api/client/python/reference/class/RateLimitError',
184+
group: 'Errors',
185+
},
186+
{
187+
url: 'https://docs.apify.com/api/client/python/reference/class/ServerError',
188+
group: 'Errors',
189+
},
190+
{
191+
url: 'https://docs.apify.com/api/client/python/reference/class/UnauthorizedError',
192+
group: 'Errors',
193+
},
152194
// Storages
153195
{
154196
url: 'https://crawlee.dev/python/api/class/Storage',

0 commit comments

Comments
 (0)