Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 8 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -357,29 +357,27 @@ await runloop.devboxes.create({...}, {

### HTTP/2 transport

On Node.js, the SDK can send requests over HTTP/2, which multiplexes many concurrent requests over a small number of TLS connections instead of opening a connection per request. Enable it with the `http2` option:
On Node.js, the SDK sends requests over HTTP/2 **by default**, multiplexing many concurrent requests over a small number of TLS connections instead of opening a connection per request. The transport is built on Node's native `node:http2` and manages a bounded pool of persistent H2 sessions per origin, auto-scaling with load. On the web, Deno, and other runtimes the platform `fetch` already speaks HTTP/2, so this option is a no-op there.

To tune the pool — for example to raise the number of connections for a high-concurrency workload — pass options as `http2`:

<!-- prettier-ignore -->
```ts
const runloop = new RunloopSDK({
http2: true,
http2: { maxConnections: 20 },
});
```

Requests are routed through an [undici](https://github.com/nodejs/undici) connection pool with HTTP/2 enabled, falling back to HTTP/1.1 for origins that don't negotiate h2 via ALPN. It is intended for HTTP/2-capable origins such as the Runloop API. This transport uses undici and therefore **requires Node.js >= 20.18.1** (see Requirements).
Comment thread
jrvb-rl marked this conversation as resolved.

`http2: true` uses a default bounded pool. To control the pool yourself — for example to raise the number of connections or multiplexed streams for a high-concurrency workload — pass a configured undici `Dispatcher` (such as an `Agent`) instead. The SDK uses it verbatim and does not manage its lifecycle, the same way it treats a custom `httpAgent`:
To opt out and use the HTTP/1.1 `node-fetch` transport, set `http2: false`:

<!-- prettier-ignore -->
```ts
import { Agent } from 'undici';

const runloop = new RunloopSDK({
http2: new Agent({ allowH2: true, connections: 8, pipelining: 100 }),
http2: false,
});
```

The `httpAgent` option does not apply to the HTTP/2 transport (undici has no Node `http.Agent` concept); set `http2` to a `Dispatcher` to tune connections. A one-time warning is emitted if both `http2` and `httpAgent` are provided.
The `httpAgent` option only applies to the HTTP/1.1 transport. A Node `http.Agent` configures HTTP/1.1 socket pooling (keep-alive, max sockets), which has no equivalent under HTTP/2 — the H2 transport multiplexes over its own managed connections — so `httpAgent` has no effect there. To tune HTTP/2 connection behavior, use `http2: { … }` instead. To keep an existing `httpAgent` working, passing one without an explicit `http2` value keeps the client on HTTP/1.1 (with a one-time warning); pass `http2: false` to select HTTP/1.1 explicitly and silence the warning. If both `httpAgent` and `http2` are set, `httpAgent` is ignored (also warned once). A custom `fetch` always takes precedence over `http2`.

## Semantic versioning

Expand All @@ -400,7 +398,7 @@ TypeScript >= 4.5 is supported.
The following runtimes are supported:

- Web browsers (Up-to-date Chrome, Firefox, Safari, Edge, and more)
- Node.js 20.18.1 LTS or later ([non-EOL](https://endoflife.date/nodejs)) versions. (Raised from 18 because the SDK now depends on undici 7 on Node; the HTTP/2 transport needs the undici >= 7.23.0 crash fix, and the package pins `^7.26.0`.)
- Node.js 20.18.1 LTS or later ([non-EOL](https://endoflife.date/nodejs)) versions.
- Deno v1.28.0 or higher.
- Bun 1.0 or later.
- Cloudflare Workers.
Expand Down
4 changes: 2 additions & 2 deletions src/_shims/index-deno.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ type _fetch = typeof fetch;
export { _fetch as fetch };

// The platform `fetch` already negotiates HTTP/2 at the transport layer, so
// `{ http2: ... }` is a no-op on Deno — reuse the global fetch and ignore any passed
// dispatcher (undici dispatchers are a Node-only concept).
// `{ http2: ... }` is a no-op on Deno — reuse the global fetch and ignore any
// passed pool options (the `node:http2` transport is Node-only).
export const makeHttp2Fetch = () => _fetch;

const _Request = Request;
Expand Down
10 changes: 5 additions & 5 deletions src/_shims/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,12 @@ export const fetch: SelectType<typeof manual.fetch, typeof auto.fetch>;

/**
* Build an HTTP/2-capable `fetch`, used when the client is constructed with
* `{ http2: ... }`. In Node this is the undici adapter (`Agent({ allowH2: true })`);
* the optional `dispatcher` lets the caller pass a configured undici `Dispatcher`
* (the `http2: <Dispatcher>` passthrough), defaulting to the SDK's bounded pool. On
* the web it returns the platform `fetch`, which already negotiates HTTP/2.
* `{ http2: ... }`. On Node this is the native `node:http2` transport; the
* optional `options` tune its connection pool (the `http2: <H2FetchOptions>`
* passthrough), defaulting to the SDK's shared bounded pool. On the web it
* returns the platform `fetch`, which already negotiates HTTP/2.
*/
export function makeHttp2Fetch(dispatcher?: any): typeof fetch;
export function makeHttp2Fetch(options?: any): typeof fetch;

// @ts-ignore
export type Request = SelectType<manual.Request, auto.Request>;
Expand Down
10 changes: 5 additions & 5 deletions src/_shims/registry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@ export interface Shims {
fetch: any;
/**
* Build an HTTP/2-capable `fetch`, used when the client is constructed with
* `{ http2: ... }`. In Node this is the undici adapter (`Agent({ allowH2: true })`);
* the optional `dispatcher` lets the caller pass a configured undici `Dispatcher`
* (the `http2: <Dispatcher>` passthrough), defaulting to the SDK's bounded pool. On
* the web the platform `fetch` already negotiates HTTP/2, so the argument is ignored.
* `{ http2: ... }`. On Node this is the native `node:http2` transport; the
* optional `options` tune its connection pool (the `http2: <H2FetchOptions>`
* passthrough), defaulting to the SDK's shared bounded pool. On the web the
* platform `fetch` already negotiates HTTP/2, so the argument is ignored.
*/
makeHttp2Fetch: (dispatcher?: any) => any;
makeHttp2Fetch: (options?: any) => any;
Request: any;
Response: any;
Headers: any;
Expand Down
4 changes: 2 additions & 2 deletions src/_shims/web-runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ export function getRuntime({ manuallyImported }: { manuallyImported?: boolean }
kind: 'web',
fetch: _fetch,
// The platform `fetch` already negotiates HTTP/2 at the transport layer, so
// `{ http2: ... }` is a no-op on the web — reuse the global fetch and ignore any
// passed dispatcher (undici dispatchers are a Node-only concept).
// `{ http2: ... }` is a no-op on the web — reuse the global fetch and ignore
// any passed pool options (the `node:http2` transport is Node-only).
makeHttp2Fetch: () => _fetch,
Request: _Request,
Response: _Response,
Expand Down
43 changes: 16 additions & 27 deletions src/index.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// File generated from our OpenAPI spec by Stainless. See CONTRIBUTING.md for details.

import { type Agent, makeHttp2Fetch } from './_shims/index';
import { type Agent } from './_shims/index';
import { resolveHttp2Fetch } from './lib/http2-transport';
import * as Core from './core';
import * as Errors from './error';
import * as Pagination from './pagination';
Expand Down Expand Up @@ -294,14 +295,20 @@ export interface ClientOptions {
/**
* Send requests over HTTP/2 using native `node:http2` connection pools.
*
* - `true` uses the SDK's default bounded HTTP/2 pool.
* HTTP/2 is the default transport on Node.js: it multiplexes many concurrent
* requests over a small number of TLS connections instead of opening one
* connection per request.
*
* - `true` / omitted uses the SDK's default bounded HTTP/2 pool.
* - Pass `H2FetchOptions` to tune pool size, timeouts, etc.
* - `false` opts out and uses the HTTP/1.1 `node-fetch` transport.
*
* On the HTTP/2 path the `httpAgent` option is not used — the H2 transport
* manages its own persistent connections. A one-time warning is emitted if
* both `http2` and `httpAgent` are set.
* manages its own persistent connections. Passing an `httpAgent` without an
* explicit `http2` value keeps the client on HTTP/1.1 (with a one-time
* warning); pass `http2: false` to opt out silently.
*
* @default false
* @default true
*/
http2?: boolean | import('./lib/h2-transport').H2FetchOptions | undefined;

Expand Down Expand Up @@ -341,11 +348,6 @@ export interface ClientOptions {
* console.log(result.exitCode);
* ```
*/
// Emitted at most once per process when `http2` and `httpAgent` are combined (see
// the constructor). Module-scoped flag mirrors the `fileFromPathWarned` pattern in
// _shims/node-runtime.ts.
let http2HttpAgentWarned = false;

export class Runloop extends Core.APIClient {
bearerToken: string;

Expand All @@ -359,7 +361,7 @@ export class Runloop extends Core.APIClient {
* @param {number} [opts.timeout=30 seconds] - The maximum amount of time (in milliseconds) the client will wait for a response before timing out.
* @param {number} [opts.httpAgent] - An HTTP agent used to manage HTTP(s) connections.
* @param {Core.Fetch} [opts.fetch] - Specify a custom `fetch` function implementation.
* @param {boolean | H2FetchOptions} [opts.http2=false] - Send requests over HTTP/2 (Node only; ignored when `fetch` is provided). `true` uses the default bounded pool; pass H2FetchOptions to tune.
* @param {boolean | H2FetchOptions} [opts.http2=true] - Send requests over HTTP/2. Enabled by default on Node.js; pass `false` to use HTTP/1.1, or H2FetchOptions to tune the pool. Has no effect on browsers/Deno/Bun (their platform `fetch` already uses HTTP/2) or when a custom `fetch` is provided.
* @param {number} [opts.maxRetries=5] - The maximum number of times the client will retry a request.
* @param {Core.Headers} opts.defaultHeaders - Default headers to include with every request to the API.
* @param {Core.DefaultQuery} opts.defaultQuery - Default query parameters to include with every request to the API.
Expand All @@ -381,28 +383,15 @@ export class Runloop extends Core.APIClient {
baseURL: baseURL || `https://api.runloop.ai`,
};

// `httpAgent` does not apply to the HTTP/2 transport — it manages its own
// persistent connections. Warn once instead of silently ignoring.
if (!options.fetch && options.http2 && options.httpAgent && !http2HttpAgentWarned) {
http2HttpAgentWarned = true;
console.warn(
'[runloop] `httpAgent` is ignored when `http2` is set: the HTTP/2 transport manages ' +
'its own connections. To tune the H2 pool, pass options as `http2` ' +
'(e.g. `http2: { maxConnections: 20 }`).',
);
}

super({
baseURL: options.baseURL!,
baseURLOverridden: baseURL ? baseURL !== 'https://api.runloop.ai' : false,
timeout: options.timeout ?? 30000 /* 30 seconds */,
httpAgent: options.httpAgent,
maxRetries: options.maxRetries,
fetch:
options.fetch ??
(options.http2 ?
makeHttp2Fetch(typeof options.http2 === 'object' ? options.http2 : undefined)
: undefined),
// HTTP/2 is the default transport on Node; a custom `fetch` always wins.
// See `resolveHttp2Fetch` for the `http2` / `httpAgent` resolution.
fetch: options.fetch ?? resolveHttp2Fetch(options),
});

const customHeadersEnv = Core.readEnv('RUNLOOP_CUSTOM_HEADERS');
Expand Down
37 changes: 32 additions & 5 deletions src/lib/h2-transport/index.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
/**
* A high-performance HTTP/2 fetch adapter built on Node's native `node:http2`.
*
* Replaces undici for HTTP/2 transport. Manages a pool of persistent H2 sessions
* per origin, multiplexing requests as concurrent streams. Each session handles up
* to the server-advertised MAX_CONCURRENT_STREAMS; the pool auto-scales between
* minConnections and maxConnections as load requires.
* Manages a pool of persistent H2 sessions per origin, multiplexing requests as
* concurrent streams. Each session handles up to the server-advertised
* MAX_CONCURRENT_STREAMS; the pool auto-scales between minConnections and
* maxConnections as load requires.
*
* Usage:
* const fetch = createH2Fetch({ maxConnections: 20 });
Expand All @@ -17,9 +17,16 @@

import { Readable } from 'node:stream';
import { H2Pool, type H2PoolOptions } from './pool';
import type { H2Response } from './response';
import { MultipartBody } from '../../_shims/MultipartBody';
import { Response } from 'node-fetch';
import { type Fetch } from '../../core';

// Statuses that must not carry a body per the Fetch spec; node-fetch's Response
// rejects a non-null body for these. (101 is an HTTP/1.1 upgrade status and can't
// occur over HTTP/2, so it's omitted.)
const NULL_BODY_STATUSES = new Set([204, 205, 304]);

const MIN_NODE_MAJOR = 18;

function checkNodeVersion(): void {
Expand Down Expand Up @@ -63,6 +70,25 @@ async function normalizeBody(body: unknown): Promise<string | Buffer | null> {
return String(body);
}

/**
* Adapt an internal {@link H2Response} to a standard node-fetch `Response`, so
* callers get the same type the default (HTTP/1.1) transport returns — including
* `instanceof Response`. Body streaming is preserved: the H2 body is a web
* `ReadableStream`, which node-fetch consumes as a Node `Readable`.
*/
function toFetchResponse(h2: H2Response): Response {
const headers: Record<string, string> = {};
for (const [key, value] of h2.headers.entries()) headers[key] = value;

const body = NULL_BODY_STATUSES.has(h2.status) ? null : Readable.fromWeb(h2.body as any);
const response = new Response(body as any, { status: h2.status, headers });

// node-fetch derives `url` from the request internals it never saw; expose the
// real request URL (`Response.url` is a prototype getter, shadowed here).
Object.defineProperty(response, 'url', { value: h2.url, writable: true, configurable: true });
return response;
}

/**
* Build a fetch adapter backed by native HTTP/2 connection pools.
*
Expand Down Expand Up @@ -109,7 +135,8 @@ export function createH2Fetch(options?: H2PoolOptions): H2Fetch {
}

const pool = getPool(parsed.origin);
return pool.request(path, method.toUpperCase(), reqHeaders, body, signal) as any;
const h2resp = await pool.request(path, method.toUpperCase(), reqHeaders, body, signal);
return toFetchResponse(h2resp) as any;
}) as H2Fetch;

h2Fetch.close = async () => {
Expand Down
9 changes: 9 additions & 0 deletions src/lib/h2-transport/session.ts
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,10 @@ export class H2Session {
// Enlarge the session-level flow-control window so the server can push
// large responses without stalling on WINDOW_UPDATE round trips.
session.setLocalWindowSize(DEFAULT_INITIAL_WINDOW_SIZE);
// An idle pooled session must not keep the process alive; each in-flight
// request re-refs the socket (see `request`) and it is unref'd again once
// the stream count returns to zero.
session.unref();
resolve();
});

Expand Down Expand Up @@ -140,6 +144,9 @@ export class H2Session {

const stream = this._session!.request(h2Headers);
this._activeStreams++;
// Keep the process alive while this request is outstanding; idle sessions
// are unref'd (see `connect` and `cleanup`).
this._session!.ref();

let settled = false;
let cleaned = false;
Expand All @@ -155,6 +162,7 @@ export class H2Session {
stream.on('error', () => {});
stream.destroy();
this._activeStreams--;
if (this._activeStreams === 0) this._session?.unref();
reject(createAbortError());
return;
}
Expand All @@ -165,6 +173,7 @@ export class H2Session {
if (cleaned) return;
cleaned = true;
this._activeStreams--;
if (this._activeStreams === 0) this._session?.unref();
if (signal) signal.removeEventListener('abort', onAbort);
if (this._state === SessionState.DRAINING && this._activeStreams === 0) {
this._close();
Expand Down
87 changes: 87 additions & 0 deletions src/lib/http2-transport.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
import { makeHttp2Fetch } from '../_shims/index';
import { type Fetch } from '../core';
import type { H2FetchOptions } from './h2-transport';

/**
* Transport-selection logic for the client, kept out of the generated
* `src/index.ts` (which holds only a one-line call-site hook).
*
* This selection matters only on Node.js. There the SDK's default transport is
* `node-fetch`, which speaks HTTP/1.1, so using HTTP/2 means swapping in the
* dedicated `node:http2` transport that {@link makeHttp2Fetch} builds. On
* browsers, Deno, and Bun the platform `fetch` already negotiates HTTP/2, so the
* shim's {@link makeHttp2Fetch} simply returns that platform `fetch` and this
* whole resolution is effectively a no-op.
*/

// A single HTTP/2 transport shared by every client that uses the default pool,
// so short-lived clients don't each open (and leak) their own H2 sessions.
// Clients that pass explicit `H2FetchOptions` get a dedicated pool, since they
// asked to tune it.
let sharedDefaultFetch: Fetch | undefined;

function getSharedDefaultFetch(): Fetch {
return (sharedDefaultFetch ??= makeHttp2Fetch());
}

// Each warning is emitted at most once per process. Module-scoped flags mirror
// the `fileFromPathWarned` pattern in _shims/node-runtime.ts.
let httpAgentIgnoredWarned = false;
let httpAgentFallbackWarned = false;

function warnHttpAgentIgnored(): void {
if (httpAgentIgnoredWarned) return;
httpAgentIgnoredWarned = true;
console.warn(
'[runloop] `httpAgent` is ignored when `http2` is set: the HTTP/2 transport manages ' +
'its own connections. To tune the H2 pool, pass options as `http2` ' +
'(e.g. `http2: { maxConnections: 20 }`).',
);
}

function warnHttpAgentFallback(): void {
if (httpAgentFallbackWarned) return;
httpAgentFallbackWarned = true;
console.warn(
'[runloop] HTTP/2 is the default transport, but `httpAgent` is set, so this client ' +
'uses HTTP/1.1. Remove `httpAgent` to use HTTP/2, or pass `http2: false` to select ' +
'HTTP/1.1 explicitly and silence this warning.',
);
}

export interface Http2TransportOptions {
http2?: boolean | H2FetchOptions | undefined;
httpAgent?: unknown;
}

/**
* Resolve the `fetch` implementation for the transport, or `undefined` to fall
* back to the default HTTP/1.1 (`node-fetch`) transport. Only called when the
* caller did not supply a custom `fetch` (a custom `fetch` always wins).
*
* - `http2: false` → HTTP/1.1.
* - `http2: true` / omitted → the shared default HTTP/2 pool.
* - `http2: H2FetchOptions` → a dedicated HTTP/2 pool tuned with those options.
* - a bare `httpAgent` (no explicit `http2`) → HTTP/1.1, so existing agents keep
* working; warns once since it overrides the HTTP/2 default.
Comment thread
jrvb-rl marked this conversation as resolved.
*
* `httpAgent` never applies to the HTTP/2 path: a Node `http.Agent` configures
* HTTP/1.1 socket pooling, which has no equivalent in HTTP/2's multiplexed model
* (its knobs live on `H2FetchOptions`). So the H2 branches ignore it, warning
* once, rather than pretending to honor it.
*/
export function resolveHttp2Fetch(options: Http2TransportOptions): Fetch | undefined {
if (options.http2 === false) return undefined;

if (options.http2) {
if (options.httpAgent) warnHttpAgentIgnored();
return typeof options.http2 === 'object' ? makeHttp2Fetch(options.http2) : getSharedDefaultFetch();
}

if (options.httpAgent) {
warnHttpAgentFallback();
return undefined;
}

return getSharedDefaultFetch();
}
Loading