Skip to content

Commit 33c0ccc

Browse files
justin808justin808
authored
[codex] Add ShakaPerf RSC FOUC release gate (#3617)
## Summary Adds a ShakaPerf-backed release gate for the React Server Components FOUC regression case. The gate boots the Pro dummy app in GitHub Actions, runs the node renderer and Rails server, then checks the RSC page with ShakaPerf visual-regression tests. The ShakaPerf test asserts two things that are intended to catch the flash-of-unstyled-content class of regression directly: - the RSC stylesheet link is present before hydration when client JS is blocked - the first visible RSC CSS probe is already styled, including the expected computed background color The workflow uploads ShakaPerf reports and server logs as artifacts for release-gate debugging. ## Validation - `pnpm install --frozen-lockfile` - `pnpm exec shaka-perf --version` - `pnpm exec prettier --check package.json pnpm-lock.yaml .github/workflows/shakaperf-release-gates.yml test/shakaperf/rsc-fouc/README.md test/shakaperf/rsc-fouc/abtests.config.ts test/shakaperf/rsc-fouc/ab-tests/rsc-fouc-release-gate.abtest.ts` - `actionlint .github/workflows/shakaperf-release-gates.yml` - ShakaPerf positive fixture run: both FOUC checks passed and generated report artifacts - ShakaPerf negative fixture run: intentionally delayed the RSC stylesheet; ShakaPerf failed on missing first-paint stylesheet and unstyled first-visible probe - Autoreview: clean after fixes - Pre-commit hooks: markdown links, trailing newlines, Prettier, ESLint - Pre-push hooks: branch lint, markdown links ## Notes I verified the ShakaPerf config and failure mode locally with fixtures. The full Pro dummy app release workflow still needs to run in GitHub Actions with `REACT_ON_RAILS_PRO_LICENSE` available. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > The release path can block tagging and npm/gem publish if the new gate fails; the workflow depends on Pro license secrets and a full dummy-app stack, but it does not change runtime library code shipped to apps. > > **Overview** > Adds a **manual ShakaPerf release gate** for React Server Components flash-of-unstyled-content on the Pro dummy route `/rsc_posts_page_over_http`. A new workflow boots the Pro dummy app (node renderer + Rails), runs `shaka-perf compare` with absolute visreg assertions (RSC stylesheet before hydration with JS blocked, first-visible probe styled), and uploads reports and server logs. > > **Tooling:** pins `shaka-perf` / `shaka-shared` at the repo root (pnpm `onlyBuiltDependencies`), ignores `compare-results/`, and extends **Knip** so the workflow and `test/shakaperf/**` count as used. The **release rake** path dispatches this workflow after the version-bump push and waits for success before tagging and publishing (overridable via `RELEASE_CI_STATUS_OVERRIDE`). > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 2465a15. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added automated visual/regression release-gate tests and config to detect CSS FOUC in React Server Components, including two experiment checks. * **Chores** * Added a manual/tag-triggered release-gate workflow that runs comparisons, collects reports/logs, and uploads artifacts. * Integrated the gate into the release flow and updated tooling and dev-dependency allowlists; ignored compare-results output. * **Documentation** * Added README documenting the new ShakaPerf gate and local run/reporting instructions. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent 00202ba commit 33c0ccc

10 files changed

Lines changed: 3748 additions & 203 deletions

File tree

‎.github/workflows/shakaperf-release-gates.yml‎

Lines changed: 210 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,210 @@
1+
name: ShakaPerf Release Gates
2+
3+
permissions:
4+
contents: read
5+
6+
on:
7+
workflow_dispatch:
8+
9+
concurrency:
10+
group: shakaperf-release-gates-${{ github.ref }}
11+
cancel-in-progress: false
12+
13+
env:
14+
NODE_VERSION: 22.12.0
15+
RUBY_VERSION: 3.3.7
16+
17+
jobs:
18+
rsc-fouc:
19+
name: RSC FOUC ShakaPerf Gate
20+
runs-on: ubuntu-22.04
21+
timeout-minutes: 45
22+
env:
23+
RAILS_ENV: test
24+
NODE_ENV: test
25+
# Dummy value for RAILS_ENV=test; no real credentials are encrypted or persisted.
26+
SECRET_KEY_BASE: dummy-secret-key-base-for-shakaperf-rsc-fouc # gitleaks:allow
27+
SHAKAPERF_CONTROL_URL: http://127.0.0.1:3000
28+
SHAKAPERF_EXPERIMENT_URL: http://127.0.0.1:3000
29+
steps:
30+
- uses: actions/checkout@v4
31+
with:
32+
persist-credentials: false
33+
34+
- name: Check Pro license secret
35+
env:
36+
REACT_ON_RAILS_PRO_LICENSE: ${{ secrets.REACT_ON_RAILS_PRO_LICENSE_V2 }}
37+
run: |
38+
if [[ -z "${REACT_ON_RAILS_PRO_LICENSE}" ]]; then
39+
echo "::error::REACT_ON_RAILS_PRO_LICENSE_V2 secret is not set; cannot run the Pro dummy app ShakaPerf gate"
40+
exit 1
41+
fi
42+
43+
- name: Setup Ruby
44+
uses: ./.github/actions/setup-ruby
45+
with:
46+
ruby-version: ${{ env.RUBY_VERSION }}
47+
bundler-version: 2.5.4
48+
49+
- name: Setup Node
50+
uses: actions/setup-node@v4
51+
with:
52+
node-version: ${{ env.NODE_VERSION }}
53+
54+
- name: Setup pnpm
55+
uses: pnpm/action-setup@v4
56+
57+
- name: Get pnpm store directory
58+
shell: bash
59+
run: echo "STORE_PATH=$(pnpm store path --silent)" >> "$GITHUB_ENV"
60+
61+
- name: Setup pnpm cache
62+
uses: actions/cache@v4
63+
with:
64+
path: ${{ env.STORE_PATH }}
65+
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
66+
restore-keys: |
67+
${{ runner.os }}-pnpm-store-
68+
69+
- name: Print system information
70+
run: |
71+
echo "Linux release: "; cat /etc/issue
72+
echo "Current user: "; whoami
73+
echo "Current directory: "; pwd
74+
echo "Ruby version: "; ruby -v
75+
echo "Node version: "; node -v
76+
echo "pnpm version: "; pnpm --version
77+
echo "Bundler version: "; bundle --version
78+
79+
- name: Install Node modules with pnpm
80+
run: pnpm install --frozen-lockfile
81+
82+
- name: Install Ruby Gems for Pro package
83+
uses: ./.github/actions/setup-bundle
84+
with:
85+
working-directory: react_on_rails_pro
86+
ruby-version: ${{ env.RUBY_VERSION }}
87+
88+
- name: Install Ruby Gems for Pro dummy app
89+
uses: ./.github/actions/setup-bundle
90+
with:
91+
working-directory: react_on_rails_pro/spec/dummy
92+
ruby-version: ${{ env.RUBY_VERSION }}
93+
94+
- name: Build Pro JavaScript packages
95+
run: |
96+
pnpm --filter react-on-rails run build
97+
pnpm --filter react-on-rails-pro run build
98+
pnpm --filter react-on-rails-pro-node-renderer run build
99+
100+
- name: Prepare Pro dummy app
101+
working-directory: react_on_rails_pro/spec/dummy
102+
env:
103+
REACT_ON_RAILS_PRO_LICENSE: ${{ secrets.REACT_ON_RAILS_PRO_LICENSE_V2 }}
104+
run: |
105+
bundle exec rake react_on_rails:generate_packs
106+
pnpm run build:test
107+
bundle exec rails db:prepare
108+
109+
- name: Cache Playwright Chromium
110+
uses: actions/cache@v4
111+
with:
112+
path: ~/.cache/ms-playwright
113+
key: ${{ runner.os }}-playwright-${{ hashFiles('**/pnpm-lock.yaml') }}
114+
restore-keys: |
115+
${{ runner.os }}-playwright-
116+
117+
- name: Install Playwright Chromium
118+
working-directory: react_on_rails_pro/spec/dummy
119+
run: pnpm exec playwright install --with-deps chromium
120+
121+
- name: Start Pro dummy app
122+
shell: bash
123+
env:
124+
REACT_ON_RAILS_PRO_LICENSE: ${{ secrets.REACT_ON_RAILS_PRO_LICENSE_V2 }}
125+
NODE_OPTIONS: --dns-result-order=ipv4first
126+
run: |
127+
set -euo pipefail
128+
129+
mkdir -p "$RUNNER_TEMP/shakaperf-logs"
130+
131+
cd react_on_rails_pro/spec/dummy
132+
RENDERER_PORT=3800 pnpm run node-renderer > "$RUNNER_TEMP/shakaperf-logs/node-renderer.log" 2>&1 &
133+
node_renderer_pid="$!"
134+
echo "$node_renderer_pid" > "$RUNNER_TEMP/shakaperf-node-renderer.pid"
135+
136+
# RSCPostsPageOverHTTP fetches http://localhost:3000/api/... inside the node renderer.
137+
PORT=3000 REACT_RENDERER_URL=http://127.0.0.1:3800 bundle exec rails s -b 127.0.0.1 -p 3000 \
138+
> "$RUNNER_TEMP/shakaperf-logs/rails.log" 2>&1 &
139+
rails_pid="$!"
140+
echo "$rails_pid" > "$RUNNER_TEMP/shakaperf-rails.pid"
141+
142+
for attempt in {1..60}; do
143+
echo "Waiting for RSC FOUC probe server (${attempt}/60)..."
144+
# /info proves the renderer socket responds, /empty proves Rails responds,
145+
# and the RSC page proves both sides are warm enough for the gate.
146+
if curl --connect-timeout 2 --max-time 5 -fsS "http://127.0.0.1:3800/info" > /dev/null &&
147+
curl --connect-timeout 2 --max-time 5 -fsS "$SHAKAPERF_CONTROL_URL/empty" > /dev/null &&
148+
curl --connect-timeout 2 --max-time 5 -fsS "$SHAKAPERF_CONTROL_URL/rsc_posts_page_over_http" > /dev/null
149+
then
150+
# All probes are up; exit this step and leave the background servers running for ShakaPerf.
151+
exit 0
152+
fi
153+
154+
if ! kill -0 "$node_renderer_pid" 2>/dev/null; then
155+
echo "::error::Node renderer died before becoming ready"
156+
tail -n 100 "$RUNNER_TEMP/shakaperf-logs/node-renderer.log" || true
157+
exit 1
158+
fi
159+
160+
if ! kill -0 "$rails_pid" 2>/dev/null; then
161+
echo "::error::Rails server died before becoming ready"
162+
tail -n 100 "$RUNNER_TEMP/shakaperf-logs/rails.log" || true
163+
exit 1
164+
fi
165+
166+
sleep 2
167+
done
168+
169+
echo "::error::Timed out waiting for renderer /info, $SHAKAPERF_CONTROL_URL/empty, and /rsc_posts_page_over_http"
170+
tail -n 200 "$RUNNER_TEMP/shakaperf-logs/node-renderer.log" || true
171+
tail -n 200 "$RUNNER_TEMP/shakaperf-logs/rails.log" || true
172+
exit 1
173+
174+
- name: Run RSC FOUC ShakaPerf gate
175+
run: |
176+
# This release gate compares the candidate app with itself. The assertions
177+
# in the abtest are absolute, so they fail even when both sides are broken.
178+
pnpm exec shaka-perf compare \
179+
--categories visreg \
180+
--config test/shakaperf/rsc-fouc/abtests.config.ts \
181+
--filter test/shakaperf/rsc-fouc/ab-tests/rsc-fouc-release-gate.abtest.ts \
182+
--controlURL "$SHAKAPERF_CONTROL_URL" \
183+
--experimentURL "$SHAKAPERF_EXPERIMENT_URL" \
184+
--full-report-zip
185+
186+
- name: Upload ShakaPerf report
187+
if: always()
188+
uses: actions/upload-artifact@v4
189+
with:
190+
name: rsc-fouc-shakaperf-report
191+
path: compare-results/
192+
retention-days: 30
193+
194+
- name: Upload ShakaPerf server logs
195+
if: always()
196+
uses: actions/upload-artifact@v4
197+
with:
198+
name: rsc-fouc-shakaperf-server-logs
199+
path: ${{ runner.temp }}/shakaperf-logs/
200+
retention-days: 30
201+
202+
- name: Stop Pro dummy app
203+
if: always()
204+
shell: bash
205+
run: |
206+
for pidfile in "$RUNNER_TEMP/shakaperf-rails.pid" "$RUNNER_TEMP/shakaperf-node-renderer.pid"; do
207+
if [ -f "$pidfile" ]; then
208+
kill "$(cat "$pidfile")" 2>/dev/null || true
209+
fi
210+
done

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@ react_on_rails_pro/spec/dummy/**/*.res.js
7777

7878
# Performance test results
7979
/bench_results
80+
/compare-results/
8081

8182
# Generated by ROR FS-based Registry
8283
generated

‎knip.ts‎

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,14 @@ const config: KnipConfig = {
55
workspaces: {
66
// Root workspace - manages the monorepo and global tooling
77
'.': {
8-
entry: ['eslint.config.ts', 'jest.config.base.js', 'benchmarks/k6.ts'],
9-
project: ['*.{js,mjs,ts}'],
8+
entry: [
9+
'eslint.config.ts',
10+
'jest.config.base.js',
11+
'benchmarks/k6.ts',
12+
'.github/workflows/shakaperf-release-gates.yml',
13+
'test/shakaperf/**/*.ts',
14+
],
15+
project: ['*.{js,mjs,ts}', 'test/shakaperf/**/*.ts'],
1016
ignoreBinaries: [
1117
// Has to be installed globally
1218
'yalc',
@@ -49,15 +55,26 @@ const config: KnipConfig = {
4955
// Used for package validation but not directly imported
5056
'@arethetypeswrong/cli',
5157
'publint',
58+
// Used by the release gate workflow via `pnpm exec shaka-perf`; Knip does
59+
// not detect that CLI usage from the GitHub Actions shell command.
60+
'shaka-perf',
5261
],
5362
},
5463

64+
// Create React on Rails app package workspace
65+
'packages/create-react-on-rails-app': {
66+
entry: ['bin/create-react-on-rails-app.js!', 'src/index.ts!'],
67+
project: ['src/**/*.ts', 'tests/**/*.ts', 'jest.config.js'],
68+
ignore: ['lib/**', 'node_modules/**'],
69+
},
70+
5571
// React on Rails core package workspace
5672
'packages/react-on-rails': {
5773
entry: [
5874
'src/ReactOnRails.full.ts!',
5975
'src/ReactOnRails.client.ts!',
6076
'src/base/full.rsc.ts!',
77+
'src/capabilities/ssr.rsc.ts!',
6178
'src/context.ts!',
6279
],
6380
project: ['src/**/*.[jt]s{x,}!', 'tests/**/*.[jt]s{x,}', '!lib/**'],
@@ -127,6 +144,8 @@ const config: KnipConfig = {
127144
'src/getReactServerComponent.server.ts!',
128145
'src/transformRSCNodeStream.ts!',
129146
'src/tanstack-router.ts!',
147+
'src/cache/index.stub.ts!',
148+
'src/registerDefaultRSCProvider.client.tsx!',
130149
],
131150
project: ['src/**/*.[jt]s{x,}!', 'tests/**/*.[jt]s{x,}', '!lib/**'],
132151
ignore: [

‎package.json‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,8 @@
7171
"react-dom": "~19.0.4",
7272
"react-on-rails-rsc": "19.0.5-rc.6",
7373
"redux": "^4.2.1",
74+
"shaka-perf": "0.1.3",
75+
"shaka-shared": "0.1.3",
7476
"size-limit": "^12.0.0",
7577
"stylelint": "^16.14.0",
7678
"stylelint-config-standard-scss": "^13.1.0",
@@ -160,11 +162,14 @@
160162
"When adding native or compiled dependencies, run pnpm ignored-builds and add trusted packages here.",
161163
"protobufjs: intentionally NOT listed - its postinstall is a version-scheme warning, not a build step.",
162164
"@swc/core: postinstall verifies the platform-specific native binary loads.",
165+
"shaka-perf: postinstall records the Node binary used by the CLI and builds its socketpair helper.",
166+
"shaka-perf is exact-pinned; re-audit its postinstall before upgrading to a new stable release.",
163167
"unrs-resolver: napi-postinstall check, used by eslint-import-resolver-typescript.",
164168
"protobufjs: intentionally omitted; its postinstall reports version-scheme warnings, not a build step."
165169
],
166170
"onlyBuiltDependencies": [
167171
"@swc/core",
172+
"shaka-perf",
168173
"unrs-resolver"
169174
]
170175
}

0 commit comments

Comments
 (0)