docs: add custom HTTP clients concept page and HTTPX guide (#658)

## Summary

- Basically documenting #416.

## Description

- Add concept page documenting the pluggable HTTP client architecture (default client config, abstract base classes, `HttpResponse` protocol, `call` method parameters)
- Add guide page with step-by-step HTTPX implementation example (sync + async)
- Add `ApiLink` component for linking to API reference from docs

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Author: vdusek

docs/02_concepts/10_custom_http_clients.mdx

@@ -0,0 +1,131 @@
+---
+id: custom-http-clients
+title: Custom HTTP clients
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+import CodeBlock from '@theme/CodeBlock';
+import ApiLink from '@site/src/components/ApiLink';
+
+import DefaultHttpClientAsyncExample from '!!raw-loader!./code/10_default_http_client_async.py';
+import DefaultHttpClientSyncExample from '!!raw-loader!./code/10_default_http_client_sync.py';
+
+import ArchitectureImportsExample from '!!raw-loader!./code/10_architecture_imports.py';
+
+import PluggingInAsyncExample from '!!raw-loader!./code/10_plugging_in_async.py';
+import PluggingInSyncExample from '!!raw-loader!./code/10_plugging_in_sync.py';
+
+The Apify API client uses a pluggable HTTP client architecture. By default, it ships with an [Impit](https://github.com/apify/impit)-based HTTP client that handles retries, timeouts, passing headers, and more. You can replace it with your own implementation for use cases like custom logging, proxying, request modification, or integrating with a different HTTP library.
+
+## Default HTTP client
+
+When you create an <ApiLink to="class/ApifyClient">`ApifyClient`</ApiLink> or <ApiLink to="class/ApifyClientAsync">`ApifyClientAsync`</ApiLink> instance, it automatically uses the built-in <ApiLink to="class/ImpitHttpClient">`ImpitHttpClient`</ApiLink> (or <ApiLink to="class/ImpitHttpClientAsync">`ImpitHttpClientAsync`</ApiLink>). This default client provides:
+
+- Automatic retries with exponential backoff for network errors, HTTP 429, and HTTP 5xx responses.
+- Configurable timeouts.
+- Preparing request data and headers according to the API requirements, including authentication.
+- Collecting requests statistics for monitoring and debugging.
+
+You can configure the default client through the <ApiLink to="class/ApifyClient">`ApifyClient`</ApiLink> or <ApiLink to="class/ApifyClientAsync">`ApifyClientAsync`</ApiLink> constructor:
+
+<Tabs>
+    <TabItem value="AsyncExample" label="Async client" default>
+        <CodeBlock className="language-python">
+            {DefaultHttpClientAsyncExample}
+        </CodeBlock>
+    </TabItem>
+    <TabItem value="SyncExample" label="Sync client">
+        <CodeBlock className="language-python">
+            {DefaultHttpClientSyncExample}
+        </CodeBlock>
+    </TabItem>
+</Tabs>
+
+## Architecture
+
+The HTTP client system is built on two key abstractions:
+
+- <ApiLink to="class/HttpClient">`HttpClient`</ApiLink> / <ApiLink to="class/HttpClientAsync">`HttpClientAsync`</ApiLink> - Abstract base classes that define the interface. Extend one of these to create a custom HTTP client by implementing the `call` method.
+- <ApiLink to="class/HttpResponse">`HttpResponse`</ApiLink> - A [runtime-checkable protocol](https://docs.python.org/3/library/typing.html#typing.runtime_checkable) that defines the expected response shape. Any object with the required attributes and methods satisfies the protocol — no inheritance needed.
+
+To plug in your custom implementation, use the <ApiLink to="class/ApifyClient#with_custom_http_client">`ApifyClient.with_custom_http_client`</ApiLink> class method.
+
+All of these are available as top-level imports from the `apify_client` package:
+
+<CodeBlock className="language-python">
+    {ArchitectureImportsExample}
+</CodeBlock>
+
+### The call method
+
+The `call` method receives all the information needed to make an HTTP request:
+
+- `method` - HTTP method (`GET`, `POST`, `PUT`, `DELETE`, etc.).
+- `url` - Full URL to make the request to.
+- `headers` - Additional headers to include.
+- `params` - Query parameters to append to the URL.
+- `data` - Raw request body (mutually exclusive with `json`).
+- `json` - JSON-serializable request body (mutually exclusive with `data`).
+- `stream` - Whether to stream the response body.
+- `timeout` - Timeout for the request as a `timedelta`.
+
+It must return an object satisfying the <ApiLink to="class/HttpResponse">`HttpResponse`</ApiLink> protocol.
+
+### The HTTP response protocol
+
+<ApiLink to="class/HttpResponse">`HttpResponse`</ApiLink> is not a concrete class. Any object with the following attributes and methods will work:
+
+| Property / Method | Description |
+|---|---|
+| `status_code: int` | HTTP status code |
+| `text: str` | Response body as text |
+| `content: bytes` | Raw response body |
+| `headers: Mapping[str, str]` | Response headers |
+| `json() -> Any` | Parse body as JSON |
+| `read() -> bytes` | Read entire response body |
+| `aread() -> bytes` | Read entire response body (async) |
+| `close() -> None` | Close the response |
+| `aclose() -> None` | Close the response (async) |
+| `iter_bytes() -> Iterator[bytes]` | Iterate body in chunks |
+| `aiter_bytes() -> AsyncIterator[bytes]` | Iterate body in chunks (async) |
+
+:::note
+Many HTTP libraries, including our default [Impit](https://github.com/apify/impit) or for example [HTTPX](https://www.python-httpx.org/) already satisfy this protocol out of the box.
+:::
+
+### Plugging it in
+
+Use the <ApiLink to="class/ApifyClient#with_custom_http_client">`ApifyClient.with_custom_http_client`</ApiLink> (or <ApiLink to="class/ApifyClientAsync#with_custom_http_client">`ApifyClientAsync.with_custom_http_client`</ApiLink>) class method to create a client with your custom implementation:
+
+<Tabs>
+    <TabItem value="AsyncExample" label="Async client" default>
+        <CodeBlock className="language-python">
+            {PluggingInAsyncExample}
+        </CodeBlock>
+    </TabItem>
+    <TabItem value="SyncExample" label="Sync client">
+        <CodeBlock className="language-python">
+            {PluggingInSyncExample}
+        </CodeBlock>
+    </TabItem>
+</Tabs>
+
+After that, all API calls made through the client will go through your custom HTTP client.
+
+:::warning
+When using a custom HTTP client, you are responsible for constructing the request, handling retries, timeouts, and errors yourself. The default retry logic is not applied.
+:::
+
+## Use cases
+
+Custom HTTP clients might be useful when you need to:
+
+- **Use a different HTTP library** - Swap Impit for [httpx](https://www.python-httpx.org/), [requests](https://requests.readthedocs.io/), or [aiohttp](https://docs.aiohttp.org/).
+- **Route through a proxy** - Add proxy support or request routing.
+- **Implement custom retry logic** - Use different backoff strategies or retry conditions.
+- **Log requests and responses** - Track API calls for debugging or auditing.
+- **Modify requests** - Add custom fields, modify the body, or change headers.
+- **Collect custom metrics** - Measure request latency, track error rates, or count API calls.
+
+For a step-by-step walkthrough of building a custom HTTP client, see the [Using HTTPX as the HTTP client](/api/client/python/docs/guides/custom-http-client-httpx) guide.

docs/02_concepts/code/10_architecture_imports.py

@@ -0,0 +1,5 @@
+from apify_client import (
+    HttpClient,
+    HttpClientAsync,
+    HttpResponse,
+)

docs/02_concepts/code/10_default_http_client_async.py

@@ -0,0 +1,14 @@
+from datetime import timedelta
+
+from apify_client import ApifyClientAsync
+
+TOKEN = 'MY-APIFY-TOKEN'
+
+
+async def main() -> None:
+    client = ApifyClientAsync(
+        token=TOKEN,
+        max_retries=4,
+        min_delay_between_retries=timedelta(milliseconds=500),
+        timeout=timedelta(seconds=360),
+    )

docs/02_concepts/code/10_default_http_client_sync.py

@@ -0,0 +1,14 @@
+from datetime import timedelta
+
+from apify_client import ApifyClient
+
+TOKEN = 'MY-APIFY-TOKEN'
+
+
+def main() -> None:
+    client = ApifyClient(
+        token=TOKEN,
+        max_retries=4,
+        min_delay_between_retries=timedelta(milliseconds=500),
+        timeout=timedelta(seconds=360),
+    )

docs/02_concepts/code/10_plugging_in_async.py

@@ -0,0 +1,30 @@
+from datetime import timedelta
+from typing import Any
+
+from apify_client import ApifyClientAsync, HttpClientAsync, HttpResponse
+
+TOKEN = 'MY-APIFY-TOKEN'
+
+
+class MyHttpClientAsync(HttpClientAsync):
+    """Custom async HTTP client."""
+
+    async def call(
+        self,
+        *,
+        method: str,
+        url: str,
+        headers: dict[str, str] | None = None,
+        params: dict[str, Any] | None = None,
+        data: str | bytes | bytearray | None = None,
+        json: Any = None,
+        stream: bool | None = None,
+        timeout: timedelta | None = None,
+    ) -> HttpResponse: ...
+
+
+async def main() -> None:
+    client = ApifyClientAsync.with_custom_http_client(
+        token=TOKEN,
+        http_client=MyHttpClientAsync(),
+    )

docs/02_concepts/code/10_plugging_in_sync.py

@@ -0,0 +1,30 @@
+from datetime import timedelta
+from typing import Any
+
+from apify_client import ApifyClient, HttpClient, HttpResponse
+
+TOKEN = 'MY-APIFY-TOKEN'
+
+
+class MyHttpClient(HttpClient):
+    """Custom sync HTTP client."""
+
+    def call(
+        self,
+        *,
+        method: str,
+        url: str,
+        headers: dict[str, str] | None = None,
+        params: dict[str, Any] | None = None,
+        data: str | bytes | bytearray | None = None,
+        json: Any = None,
+        stream: bool | None = None,
+        timeout: timedelta | None = None,
+    ) -> HttpResponse: ...
+
+
+def main() -> None:
+    client = ApifyClient.with_custom_http_client(
+        token=TOKEN,
+        http_client=MyHttpClient(),
+    )

docs/03_guides/05_custom_http_client.mdx

@@ -0,0 +1,61 @@
+---
+id: custom-http-client-httpx
+title: Using HTTPX as the HTTP client
+---
+
+import ApiLink from '@site/src/components/ApiLink';
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+import CodeBlock from '@theme/CodeBlock';
+
+import CustomHttpClientAsyncExample from '!!raw-loader!./code/05_custom_http_client_async.py';
+import CustomHttpClientSyncExample from '!!raw-loader!./code/05_custom_http_client_sync.py';
+
+This guide shows how to replace the default <ApiLink to="class/ImpitHttpClient">`ImpitHttpClient`</ApiLink> and <ApiLink to="class/ImpitHttpClientAsync">`ImpitHttpClientAsync`</ApiLink> with one based on [HTTPX](https://www.python-httpx.org/). The same approach works for any HTTP library — see [Custom HTTP clients](/api/client/python/docs/concepts/custom-http-clients) for the underlying architecture.
+
+## Why HTTPX?
+
+You might want to use [HTTPX](https://www.python-httpx.org/) instead of the default [Impit](https://github.com/apify/impit)-based client for reasons like:
+
+- You already use HTTPX in your project and want a single HTTP stack.
+- You need HTTPX-specific features.
+- You want fine-grained control over connection pooling or proxy routing.
+
+## Implementation
+
+The implementation involves two steps:
+
+1. **Extend <ApiLink to="class/HttpClient">`HttpClient`</ApiLink> (sync) or <ApiLink to="class/HttpClientAsync">`HttpClientAsync`</ApiLink> (async)** and implement the `call` method that delegates to HTTPX.
+2. **Pass it to <ApiLink to="class/ApifyClient#with_custom_http_client">`ApifyClient.with_custom_http_client`</ApiLink>** to create a client that uses your implementation.
+
+The `call` method receives parameters like `method`, `url`, `headers`, `params`, `data`, `json`, `stream`, and `timeout`. Map them to the corresponding HTTPX arguments — most map directly, except `data` which becomes HTTPX's `content` parameter and `timeout` which needs conversion from `timedelta` to seconds.
+
+A convenient property of HTTPX is that its `httpx.Response` object already satisfies the <ApiLink to="class/HttpResponse">`HttpResponse`</ApiLink> protocol, so you can return it directly without wrapping.
+
+<Tabs>
+    <TabItem value="AsyncExample" label="Async client" default>
+        <CodeBlock className="language-python">
+            {CustomHttpClientAsyncExample}
+        </CodeBlock>
+    </TabItem>
+    <TabItem value="SyncExample" label="Sync client">
+        <CodeBlock className="language-python">
+            {CustomHttpClientSyncExample}
+        </CodeBlock>
+    </TabItem>
+</Tabs>
+
+:::warning
+When using a custom HTTP client, you are responsible for handling retries, timeouts, and error handling yourself. The built-in retry logic with exponential backoff is part of the default <ApiLink to="class/ImpitHttpClient">`ImpitHttpClient`</ApiLink> and is not applied to custom implementations.
+:::
+
+## Going further
+
+The example above is minimal on purpose. In a production setup, you might want to extend it with:
+
+- **Retry logic** - Use [HTTPX's event hooks](https://www.python-httpx.org/advanced/event-hooks/) or utilize library like [tenacity](https://tenacity.readthedocs.io/) to retry failed requests.
+- **Custom headers** - You can add headers in the `call` method before delegating to HTTPX.
+- **Connection lifecycle** - Close the underlying `httpx.Client` when done by adding a `close()` method to your custom client.
+- **Proxy support** - You can pass `proxy=...` when creating the `httpx.Client`.
+- **Metrics collection** - Track request latency, error rates, or other metrics by adding instrumentation in the `call` method.
+- **Logging** - Log requests and responses for debugging or auditing purposes.

docs/03_guides/code/05_custom_http_client_async.py

@@ -0,0 +1,61 @@
+from __future__ import annotations
+
+import asyncio
+from typing import TYPE_CHECKING, Any
+
+import httpx
+
+from apify_client import ApifyClientAsync, HttpClientAsync, HttpResponse
+
+if TYPE_CHECKING:
+    from datetime import timedelta
+
+TOKEN = 'MY-APIFY-TOKEN'
+
+
+class HttpxClientAsync(HttpClientAsync):
+    """Custom async HTTP client using HTTPX library."""
+
+    def __init__(self) -> None:
+        super().__init__()
+        self._client = httpx.AsyncClient()
+
+    async def call(
+        self,
+        *,
+        method: str,
+        url: str,
+        headers: dict[str, str] | None = None,
+        params: dict[str, Any] | None = None,
+        data: str | bytes | bytearray | None = None,
+        json: Any = None,
+        stream: bool | None = None,
+        timeout: timedelta | None = None,
+    ) -> HttpResponse:
+        timeout_secs = timeout.total_seconds() if timeout else 0
+
+        # httpx.Response satisfies the HttpResponse protocol,
+        # so it can be returned directly.
+        return await self._client.request(
+            method=method,
+            url=url,
+            headers=headers,
+            params=params,
+            content=data,
+            json=json,
+            timeout=timeout_secs,
+        )
+
+
+async def main() -> None:
+    client = ApifyClientAsync.with_custom_http_client(
+        token=TOKEN,
+        http_client=HttpxClientAsync(),
+    )
+
+    actor = await client.actor('apify/hello-world').get()
+    print(actor)
+
+
+if __name__ == '__main__':
+    asyncio.run(main())