Skip to content

Commit c7b54be

Browse files
justin808claude
andauthored
[Pro] Document renderer warmup/pool/keep-alive for streamed RSC; fix stale pool docs (#4240) (#4253)
## Why The dominant suspects for the streamed RSC `responseEnd` tail (Discover median +3.2% vs Inertia in the 2026-06-24 hosted run) are Node renderer round-trip overhead, cold/under-warmed workers, and per-request Rails↔renderer connection setup. The issue asked for documented, measured guidance on the levers that move that tail: warmup, pool sizing, and keep-alive. Closes #4240. ## Key finding: keep-alive is already implemented, but the docs said otherwise While investigating, I found the configuration doc was **materially stale**. `configuration-pro.md` stated: > `renderer_http_pool_size` — *"Currently has no effect. The async-http adapter creates a new client per request, so this pool limit is never reached… planned persistent connection support (see issue #3283)."* The code tells a different story. `RendererHttpClient#with_client` / `#scheduler_scoped_client` reuse a persistent `Async::HTTP::Client` across requests within a `Fiber.scheduler`, with an inline comment: *"Connection pool (limit) is now effective — multiple streams share pooled connections."* The streaming helper runs inside `Sync {}`, so **the streaming path already gets HTTP/2 keep-alive / connection reuse** — issue #3283 is done. `renderer_http_keep_alive_timeout` is already deprecated/ignored in `configuration.rb`. So the keep-alive "investigation" deliverable resolves to: **the capability exists; the docs were wrong.** This PR corrects them and consolidates the tuning guidance. ## What changed (docs only) `docs/oss/configuration/configuration-pro.md`: - **Corrected** the `renderer_http_pool_size` comment to describe the implemented behavior (persistent HTTP/2 reuse within the streaming `Fiber.scheduler`; the limit bounds concurrent multiplexed streams). - **Added** a "Renderer Performance Tuning for Streamed RSC" section: 1. **Warmup** — pre-warm every worker so the first measured render isn't cold (cross-references the existing [health-checks `/ready` cold-start contract](docs/oss/building-features/node-renderer/health-checks.md), incl. the `workersCount > 1` fan-out caveat). 2. **Pool sizing** — size `renderer_http_pool_size` (Rails) against `workersCount` (renderer) and Puma concurrency, with a comparison table. 3. **Keep-alive** — document that connection reuse is automatic on the streaming path and that `renderer_http_keep_alive_timeout` is deprecated. Measurement ties back to the streamed `Server-Timing` metrics (`ror_renderer_prepare`, `ror_stream_shell`) added for #4239. ## Acceptance criteria status - [x] Documented renderer warmup and pool-sizing guidance for streamed RSC routes. - [x] Verified keep-alive / pooled connection between Rails and the renderer — confirmed in code (`scheduler_scoped_client`, persistent `Async::HTTP::Client`), and the stale doc claiming otherwise is corrected. - [ ] **Re-run the demo benchmark to close the Discover `responseEnd` regression / improve p95** — requires the separate [`react-on-rails-demo-gumroad-rsc`](https://github.com/shakacode/react-on-rails-demo-gumroad-rsc) hosted environment. Maintainer handoff; this PR delivers the guidance + corrected knobs the demo needs. ## Testing Docs-only change. Locally: `node script/generate-llms-full.mjs --validate` ✓, `script/check-docs-sidebar origin/main` ✓ (no new pages), Prettier ✓. ## Decision log - **Non-blocking:** Add a new warmup knob/rake task vs. document. **Decision:** document — warmup is a deploy-orchestration concern already served by the `/ready` health endpoint and the render path; a redundant knob would add surface without value. Keep-alive and pool-size knobs already exist. **Review later:** add a first-class warmup rake task if users ask. - **Non-blocking:** CHANGELOG entry placed under **Changed** (not Added) since the core change is a doc correction. Also avoids a merge conflict with the #4239/#4238 batch entries under Added. ## Related Streamed-RSC performance batch with #4251 (Server-Timing, #4239) and #4252 (compression, #4238). No source-file overlap — this PR touches only `configuration-pro.md` + CHANGELOG. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Updated streamed React Server Components guidance to clarify how `renderer_http_pool_size` limits concurrent async-http TCP connection pooling (effective on the streaming path via HTTP/2 reuse) and how scheduler lifetime impacts connection reuse/concurrency. * Added “Renderer Performance Tuning for Streamed RSC” notes covering streamed `responseEnd` tail-latency contributors and tuning levers (renderer worker pre-warming, coordinated worker/pool sizing, and `ssr_timeout` as idle socket timeout), plus Rails↔renderer keep-alive behavior. * Revised upgrade and changelog entries: `renderer_http_keep_alive_timeout` is deprecated/ignored (with deprecation warning when non-`nil`), and renderer connection drops surface immediately via the pro error path (with no implicit transport retry). <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 67d12aa commit c7b54be

5 files changed

Lines changed: 143 additions & 41 deletions

File tree

CHANGELOG.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -270,8 +270,7 @@ After a release, run `/update-changelog` in Claude Code to analyze commits, writ
270270
#### Breaking Changes
271271

272272
- **[Pro]** **Node Renderer now requires Ruby 3.3+ for the async-http transport**: The `react-on-rails-pro` gem now requires Ruby `>= 3.3` (raised from `>= 3.0`) because `async-http` depends on Ruby 3.3 features. Upgrade Ruby before moving to this release. See `docs/pro/updating.md` for the full upgrade guide. [PR 3320](https://github.com/shakacode/react_on_rails/pull/3320) by [AbanoubGhadban](https://github.com/AbanoubGhadban).
273-
- **[Pro]** **Async Rails server deployments need to stay on the HTTPX renderer until support is added**: Falcon and async-rails deployments are not currently supported with the new async-http renderer client because calling the renderer from inside an existing Async reactor without an `Async::Task.current?` context can create a nested reactor. Keep those deployments on the previous HTTPX renderer client until support is explicitly added. See `docs/pro/updating.md` for the full upgrade guide. [PR 3320](https://github.com/shakacode/react_on_rails/pull/3320) by [AbanoubGhadban](https://github.com/AbanoubGhadban).
274-
- **[Pro]** **`config.renderer_http_pool_size` now limits per-request HTTP/2 streams**: Existing numeric values now cap concurrent HTTP/2 streams for each request-scoped renderer client instead of sizing a persistent process-wide connection pool. Setting a non-default value emits a warning so the changed meaning is visible during upgrades; setting `nil` keeps the default stream limit and does not make the request-scoped client unlimited. Persistent connection reuse is tracked in [Issue 3283](https://github.com/shakacode/react_on_rails/issues/3283). See `docs/pro/updating.md` for the full upgrade guide. [PR 3320](https://github.com/shakacode/react_on_rails/pull/3320) by [AbanoubGhadban](https://github.com/AbanoubGhadban).
273+
- **[Pro]** **`config.renderer_http_pool_size` now limits async-http connections per renderer client**: Existing numeric values now cap concurrent async-http connections for each renderer client instead of sizing a persistent process-wide connection pool. HTTP/2 may multiplex request streams over those pooled connections. Setting `nil` keeps the default connection limit and does not make the async-http client unlimited. Persistent connection reuse is automatic when a long-lived `Fiber.scheduler` is present. See `docs/pro/updating.md` for the full upgrade guide. [PR 3320](https://github.com/shakacode/react_on_rails/pull/3320) by [AbanoubGhadban](https://github.com/AbanoubGhadban).
275274

276275
#### Added
277276

@@ -294,6 +293,7 @@ After a release, run `/update-changelog` in Claude Code to analyze commits, writ
294293
- **Allow trusted pnpm 10 build scripts in contributor installs**: The root workspace now allowlists required native dependency postinstall checks for `@swc/core` and `unrs-resolver`, so `pnpm install` under pnpm 10 no longer skips those trusted build hooks. [PR 3421](https://github.com/shakacode/react_on_rails/pull/3421) by [justin808](https://github.com/justin808).
295294
- **Release publishing now checks `origin/main` CI status before shipping**: `rake release` now inspects GitHub Checks for `origin/main` before publishing, blocking stable releases on any visible failing or missing checks and prereleases on required checks, with an explicit override path for maintainers. [PR 3407](https://github.com/shakacode/react_on_rails/pull/3407) by [justin808](https://github.com/justin808).
296295
- **[Pro]** **Updated Pino in the Node Renderer**: Raised the `react-on-rails-pro-node-renderer` `pino` dependency range to `^9.14.0 || ^10.1.0`, aligning with the current Fastify dependency. [PR 3401](https://github.com/shakacode/react_on_rails/pull/3401) by [alexeyr-ci2](https://github.com/alexeyr-ci2).
296+
- **[Pro]** **Async Rails server deployments use scheduler-scoped renderer clients**: Falcon and async-rails deployments can use the async-http renderer client when a long-lived `Fiber.scheduler` is already running; renderer clients are reused within that scheduler. Standard Puma streaming uses a per-request scheduler and cleans up the client when the response ends. See `docs/pro/updating.md` for the full upgrade guide. [PR 3320](https://github.com/shakacode/react_on_rails/pull/3320) by [AbanoubGhadban](https://github.com/AbanoubGhadban).
297297
- **[Pro]** **Per-scheduler persistent HTTP connections for Node Renderer**: `RendererHttpClient` now reuses HTTP/2 connections across requests within the same Fiber scheduler (Falcon, async Puma), eliminating per-request TCP+TLS+HTTP/2 handshake overhead. Standalone requests (no outer scheduler) continue using ephemeral connections with guaranteed cleanup. The internal connection pool automatically recovers from broken connections without manual eviction. [PR 3428](https://github.com/shakacode/react_on_rails/pull/3428) by [AbanoubGhadban](https://github.com/AbanoubGhadban).
298298
- **[Pro]** **Migrated Node Renderer HTTP transport from HTTPX to `async-http`**: React on Rails Pro now uses `async-http` (`~> 0.95`) with `io-endpoint` (`~> 0.17`) for all Rails→Node Renderer requests (render, streaming render, asset upload), replacing the previous HTTPX adapter and the custom `httpx_stream_bidi_patch.rb`. The new `RendererHttpClient` is a request-scoped client (one client per Rails request — no persistent process-wide pool) and integrates with the length-prefixed wire protocol introduced in [PR 2903](https://github.com/shakacode/react_on_rails/pull/2903). HTTP/2 bidirectional streaming for async props is now provided by `post_bidi` on the new adapter. **Action required for upgraders:**
299299
- **`config.ssr_timeout`** is now a per-read socket timeout applied to each renderer socket read, rather than a task-level timeout wrapping the entire request.
@@ -361,7 +361,7 @@ After a release, run `/update-changelog` in Claude Code to analyze commits, writ
361361

362362
#### Deprecated
363363

364-
- **[Pro]** **`config.renderer_http_keep_alive_timeout` is deprecated**: The setting now has no effect because async-http renderer clients are scoped to individual requests. Setting it emits a deprecation warning; remove the configuration during upgrade. See `docs/pro/updating.md` for the full upgrade guide. [PR 3320](https://github.com/shakacode/react_on_rails/pull/3320) by [AbanoubGhadban](https://github.com/AbanoubGhadban).
364+
- **[Pro]** **`config.renderer_http_keep_alive_timeout` is deprecated**: The setting now has no effect because async-http manages renderer client lifecycle through scheduler-scoped clients when a `Fiber.scheduler` is already running and per-request clients otherwise. Explicitly setting it to a non-`nil` value emits a deprecation warning; leaving it unset or setting it to `nil` is accepted silently. Remove non-`nil` configuration during upgrade. See `docs/pro/updating.md` for the full upgrade guide. [PR 3320](https://github.com/shakacode/react_on_rails/pull/3320) by [AbanoubGhadban](https://github.com/AbanoubGhadban).
365365

366366
#### Removed
367367

docs/oss/configuration/configuration-pro.md

Lines changed: 52 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -82,9 +82,9 @@ ReactOnRailsPro.configure do |config|
8282
# config.renderer_password = ENV["RENDERER_PASSWORD"]
8383

8484
# Set the `ssr_timeout` configuration so the Rails server will not wait more than this many seconds
85-
# for a SSR request to return once issued. With the async-http renderer client, this is applied as
85+
# for a renderer socket read once issued. With the async-http renderer client, this is applied as
8686
# the per-read socket timeout on the renderer connection. Increase this value for long-running
87-
# streaming SSR responses.
87+
# streaming SSR responses with legitimate gaps between chunks.
8888
config.ssr_timeout = 5
8989

9090
# If false, then crash if no backup rendering when the remote renderer is not available
@@ -94,10 +94,15 @@ ReactOnRailsPro.configure do |config|
9494
# Default for `renderer_use_fallback_exec_js` is true.
9595
config.renderer_use_fallback_exec_js = false
9696

97-
# Currently has no effect. The async-http adapter creates a new client per request,
98-
# so this pool limit is never reached. The setting is kept for forward-compatibility
99-
# with planned persistent connection support (see issue #3283).
100-
# Setting a non-default value emits a warning explaining this.
97+
# Maximum number of concurrent async-http connections per client to the Node renderer.
98+
# HTTP/2 may multiplex multiple request streams over those pooled connections.
99+
# When a Fiber.scheduler already exists before the renderer request enters `Sync {}`, the
100+
# client is reused across requests within that long-lived scheduler (persistent connection /
101+
# keep-alive; see the tuning section below), so this limit bounds connection concurrency for
102+
# streamed renders sharing one client.
103+
# Otherwise the adapter uses a request-scoped client. Under standard Puma (no pre-existing scheduler), `Sync {}`
104+
# creates that scheduler/client for the request and cleans it up when the request ends.
105+
# See "Renderer Performance Tuning for Streamed RSC" below.
101106
# Default for `renderer_http_pool_size` is 10
102107
config.renderer_http_pool_size = 10
103108

@@ -172,6 +177,47 @@ ReactOnRailsPro.configure do |config|
172177
end
173178
```
174179

180+
## Renderer Performance Tuning for Streamed RSC
181+
182+
The dominant contributors to a streamed route's `responseEnd` tail are Node renderer round-trip overhead, cold or under-warmed renderer workers, and per-request connection setup between Rails and the renderer. Three levers address these (see [issue #4240](https://github.com/shakacode/react_on_rails/issues/4240)). Measure changes with before/after page-load timing for the streamed route, `Server-Timing` for the early Rails/renderer phases that are known before the first stream write, the inline RSC stream performance marks described in the [Streaming SSR guide](../../pro/streaming-ssr.md), and renderer logs or tracing for cold-worker behavior. Inline marks remain the source for late payload, flush, hydration, and stream-drain timing because `ActionController::Live` commits headers on the first stream write.
183+
184+
### 1. Warm up renderer workers
185+
186+
Each worker compiles its bundle on its **first** render request, so the first measured render after a deploy is cold. Pre-warm every worker before serving real traffic so cold-start cost does not land on a user (or skew a benchmark):
187+
188+
- Send a warm-up render (or hit `/ready` after a warm-up render) to each replica during deploy.
189+
- With `workersCount > 1`, a single warm-up render is not enough. Route warm-up traffic so it reaches **every** worker under your actual load-balancing policy; sticky or hash-based routing may require replica-local hooks or an explicit fan-out endpoint. **Do not gate all traffic on `/ready` without a separate warm-up path** — if no render requests reach the renderer, `/ready` never flips to 200 (deadlock).
190+
191+
Full warm-up patterns (Kubernetes probes, `postStart` hooks, the `/ready` cold-start contract) are in [Node renderer health checks → Gating traffic on `/ready`](../building-features/node-renderer/health-checks.md#gating-traffic-on-ready).
192+
193+
### 2. Size the worker pool and the connection pool together
194+
195+
Two independent limits gate renderer throughput:
196+
197+
| Setting | Where | Default | Governs |
198+
| --------------------------------------------------------------------------------------------------- | -------- | -------- | ----------------------------------------------------------------- |
199+
| [`workersCount`](../building-features/node-renderer/js-configuration.md) / `RENDERER_WORKERS_COUNT` | Renderer | CPUs − 1 | How many renders the renderer can execute concurrently. |
200+
| `renderer_http_pool_size` | Rails | 10 | Max concurrent async-http connections per client to the renderer. |
201+
202+
Guidance:
203+
204+
- With Falcon or another long-lived scheduler, keep `renderer_http_pool_size` close to (and generally not far above) `workersCount`; streamed renders sharing that scheduler use the same async-http client, and HTTP/2 may multiplex request streams over the pooled connections. Sending many more concurrent renderer requests than there are workers just queues renders at the renderer while adding connection and scheduling overhead.
205+
- Under standard Puma streaming, `Sync {}` creates a request-scoped client, so `renderer_http_pool_size` only bounds concurrent async-http connections inside one streamed response. Use a value near the number of renderer calls one response can overlap; scale `workersCount` and renderer replicas for cross-request concurrency.
206+
- Account for your Rails concurrency: with many Puma threads/workers all streaming, a renderer with only one or two workers becomes the bottleneck. Scale `workersCount` (and renderer replicas) to your real concurrent streamed-render load.
207+
- Tune `ssr_timeout` for legitimate long gaps between streamed chunks — it applies as a per-read socket timeout, so it fires when a single read from the renderer blocks for `ssr_timeout` seconds. It is not a total response-duration cap; avoid masking renderer hangs with an unnecessarily high value.
208+
209+
### 3. Rails ↔ renderer keep-alive (persistent on Falcon/async scheduler; per-request on standard Puma)
210+
211+
Connection reuse is automatic when the renderer request runs under a long-lived `Fiber.scheduler`, such as Falcon or Puma configured with an async scheduler. In that setup, the async-http client is stored on the scheduler and reused across streaming requests, so HTTP/2 connections stay alive and renders multiplex over them instead of paying TCP handshake and H2 connection setup per request ([issue #3283](https://github.com/shakacode/react_on_rails/issues/3283)). No React on Rails configuration is required to enable this.
212+
213+
Under standard Puma, the streaming helper's `Sync {}` block creates a per-request scheduler. The async-http client is cleaned up when that streaming response ends, so connection reuse does not persist across consecutive Rails requests. The benefit is still meaningful inside a single streamed response: renderer calls in that response can share the same client lifecycle and `renderer_http_pool_size` still bounds concurrent async-http connections within that request.
214+
215+
Call the renderer from the normal Rails request path. The adapter chooses scheduler-scoped reuse whenever a `Fiber.scheduler` already exists before it enters `Sync {}`; custom middleware or background code that installs a scheduler with an unclear lifecycle can therefore keep renderer clients alive longer than intended. Keep those calls inside the request's scheduler lifecycle, or use the standard path where `Sync {}` creates and cleans up the per-request client.
216+
217+
`config.renderer_http_keep_alive_timeout` is **deprecated** and ignored: the async-http adapter manages connection lifecycle automatically (connections are reused within the scheduler and cleaned up when it ends). Explicitly setting it to a non-`nil` value in your `configure` block emits a deprecation warning; leaving it unset or setting it to `nil` is accepted silently. If you previously set it to `30` (the old default), remove the line from your `configure` block entirely.
218+
219+
To confirm reuse, compare before/after `responseEnd` timing and streamed RSC performance marks, and trace renderer sockets when you need to distinguish long-lived scheduler reuse from standard Puma's per-request scheduler cleanup.
220+
175221
## Need Help?
176222

177223
- **Pro Features:** [React on Rails Pro](../../pro/react-on-rails-pro.md)

docs/pro/updating.md

Lines changed: 18 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -392,22 +392,27 @@ Before upgrading:
392392
read timeout on each renderer socket. It no longer wraps the entire request as a single task-level timeout.
393393
- Treat `config.renderer_http_pool_timeout` as the TCP connect timeout. After the socket connects, individual reads
394394
are bounded by `ssr_timeout`.
395-
- Treat `config.renderer_http_pool_size` as a per-request HTTP/2 stream limit, not as a persistent process-wide
396-
connection pool size. The current async-http adapter opens a request-scoped client for each renderer request and
397-
does not reuse TCP connections between Rails requests, so high-latency networks or very high request rates can see
398-
extra connection and HTTP/2 handshake overhead compared with HTTPX. Setting this value now emits a warning to make
399-
the changed meaning visible during upgrades. Setting it to `nil` keeps the default stream limit; it does not make
400-
the request-scoped async-http client unlimited. Persistent async-http connection reuse is tracked in
401-
[Issue 3283](https://github.com/shakacode/react_on_rails/issues/3283).
395+
- Treat `config.renderer_http_pool_size` as the per-client async-http connection-pool limit, not as the old
396+
process-wide persistent pool size. With a long-lived `Fiber.scheduler` (for example Falcon or Puma configured with an
397+
async scheduler), the client is reused across renderer requests within that scheduler and the setting bounds
398+
concurrent async-http connections for streamed renders sharing one client. HTTP/2 may multiplex multiple request
399+
streams over those pooled connections. Under standard Puma streaming, `Sync {}` creates a per-request scheduler and
400+
cleans up the client when that streaming response ends, so reuse does not persist across consecutive Rails requests.
401+
Setting it to `nil` keeps the default connection limit; it does not make the async-http client unlimited.
402402
- Expect renderer connection drops to surface immediately as `ReactOnRailsPro::Error`/connection failures. HTTPX
403403
previously performed one implicit transport retry for some connection drops; the async-http adapter uses
404404
`retries: 0` and leaves retry policy to the existing bundle-upload retry loop and caller behavior.
405-
- Run the node renderer client from the normal synchronous Rails request path. Async Rails servers or middleware that
406-
call the renderer from inside an existing Async reactor without an `Async::Task.current?` context are not currently
407-
supported because the sync fallback may create a nested reactor. Keep Falcon/async-rails deployments on the previous
408-
HTTPX renderer client until this support is explicitly added.
409-
- `config.renderer_http_keep_alive_timeout` remains accepted for compatibility, but it has no effect because
410-
async-http clients are currently scoped to individual requests. Setting it now emits a deprecation warning.
405+
- Run the node renderer client from the normal Rails request path. **Note for Falcon/async-rails users:** the earlier
406+
advisory to keep Falcon deployments on the HTTPX renderer client is superseded; HTTPX has been removed and async-http
407+
now handles Falcon natively. Async Rails servers (Falcon, Puma with an async scheduler) are supported: the async-http
408+
client uses scheduler-scoped connection reuse automatically when a `Fiber.scheduler` already exists before the adapter
409+
enters `Sync {}`. Middleware and background code should call the renderer from a scheduler with a deliberate request or
410+
service lifecycle; a custom scheduler-only context can keep renderer clients alive longer than intended.
411+
- `config.renderer_http_keep_alive_timeout` is deprecated and ignored, but remains accepted during upgrade because
412+
async-http manages connection lifecycle through its scheduler-scoped clients and ephemeral request clients. Explicitly
413+
setting it to a non-`nil` value in your `configure` block emits a deprecation warning; leaving it unset or setting it
414+
to `nil` is accepted silently. If you previously set it to `30` (the old default), remove the line from your
415+
`configure` block entirely.
411416

412417
#### Upgrading to 16.4.0 or later
413418

0 commit comments

Comments
 (0)