Skip to content

Commit 919e84e

Browse files
committed
Add curll
1 parent c691e6b commit 919e84e

2 files changed

Lines changed: 93 additions & 0 deletions

File tree

browsers/curl.mdx

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
---
2+
title: "Curl"
3+
description: "Send HTTP requests through a browser session's Chromium network stack so cookies, proxies, and TLS match real browsers"
4+
---
5+
6+
Browser curl lets you run HTTP requests through Kernel browsers, automatically attaching the browser's cookie jar, transport fingerprint, [stealth and proxy settings](/bot-detection/overview), and other defaults that browsers send. This allows you to efficiently fetch resources from websites while getting the benefits of real browsers, including bot anti-detection and session cookie state.
7+
8+
## Why this beats server-side `fetch()`
9+
10+
When you call `fetch()` or `httpx`, you're on a different TLS stack, IP, cookie store, and header profile than the browsers your agents (or automations) are using. Browser curl runs requests directly through a Kernel browser's networking stack.
11+
12+
### What curl requests inherit from Kernel browsers
13+
14+
- **Cookies and storage policy** — Same cookie jar as the profile, including Chromium inclusion rules, `Set-Cookie` persistence, session cookies, encryption, `SameSite`, partitioning, and content settings. Response `Set-Cookie` headers update that same profile.
15+
- **Transport fingerprint** — BoringSSL TLS, ALPN, HTTP/2, HTTP/3/QUIC where enabled, Alt-Svc, connection reuse, Happy Eyeballs, certificate verification, CT/HSTS policy.
16+
- **Network configuration** — Proxy and PAC settings, DNS and Secure DNS, SSL and cert policy, cache, HTTP server properties, network quality signals.
17+
- **Default client behavior** — Chromium user agent, `Accept-Language`, brotli/zstd support, and typical fetch metadata behavior.
18+
- **Chromium request lifecycle** — Browser-managed redirects, decompression, HTTP authentication and client certificate eligibility, and cache behavior.
19+
20+
## Streaming browser requests
21+
22+
Streaming requests go through the browser's Chromium network stack, and the SDK mirrors platform-native HTTP ergonomics.
23+
24+
You can stream or read the body incrementally, which avoids buffering the full payload in the browser for large responses.
25+
26+
<CodeGroup>
27+
```typescript Typescript/Javascript
28+
import Kernel from '@onkernel/sdk';
29+
30+
const kernel = new Kernel();
31+
const browser = await kernel.browsers.create({});
32+
33+
const response: Response = await kernel.browsers.fetch(browser.session_id, 'https://example.com', {
34+
method: 'GET',
35+
});
36+
```
37+
38+
```python Python
39+
import httpx
40+
41+
from kernel import Kernel
42+
43+
const response: httpx.Response = client.browsers.request(
44+
browser.session_id,
45+
"GET",
46+
"https://example.com",
47+
);
48+
```
49+
</CodeGroup>
50+
51+
## Buffered browser curl
52+
53+
Buffered browser curl calls the HTTP API and returns a single JSON envelope: `status`, `headers`, `body`, and `duration_ms`. Use it when the response is small enough to hold in memory and you want one structured object back—typical APIs, HTML snippets, and JSON payloads.
54+
55+
<Info>
56+
Buffered curl loads the **entire** response body into the browser process before it returns to you. Requesting a very large payload can exhaust memory and cause OOMs. For big downloads or unknown sizes, use [streaming browser requests](#streaming-browser-requests) instead.
57+
</Info>
58+
59+
<CodeGroup>
60+
```typescript Typescript/Javascript
61+
import Kernel from '@onkernel/sdk';
62+
63+
const kernel = new Kernel();
64+
const browser = await kernel.browsers.create({});
65+
66+
const buffered = await kernel.browsers.curl(browser.session_id, {
67+
url: 'https://example.com',
68+
method: 'GET',
69+
});
70+
```
71+
72+
```python Python
73+
from kernel import Kernel
74+
75+
client = Kernel()
76+
browser = client.browsers.create()
77+
78+
buffered = client.browsers.curl(browser.session_id, url="https://example.com", method="GET")
79+
```
80+
</CodeGroup>
81+
82+
## Concurrency limits
83+
84+
Browser curl concurrency is constrained by Chromium's internal networking limits:
85+
86+
- **HTTP/1.x (direct)** — About 6 sockets per host group and up to 256 active sockets per pool.
87+
- **Proxied chains** — On the order of tens of sockets per proxy chain (Chromium clamps configured values into a bounded range).
88+
- **HTTP stream pool** — Similar per-group and per-pool behavior to HTTP/1.x direct connections.
89+
- **HTTP/2** — Roughly 100 concurrent streams per session initially, updated from the server's `SETTINGS_MAX_CONCURRENT_STREAMS`, with an upper cap in Chromium (on the order of 256).
90+
- **HTTP/3 / QUIC** — Stream creation can queue when peer or open-stream limits are hit.
91+
92+
If you're issuing many parallel curls from one browser, you're sharing those pools with navigation, XHR, and other session traffic. Latency may vary, as Chromium may queue requests when it reaches its limits.

docs.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,7 @@
8181
"browsers/termination",
8282
"browsers/standby",
8383
"browsers/headless",
84+
"browsers/curl",
8485
"info/projects"
8586
]
8687
},

0 commit comments

Comments
 (0)