Skip to content

Commit e26d659

Browse files
justin808claude
justin808
and
claude
authored
fix(ci): stop running benchmarks on docs-only and CI-config-only PRs
## Why Bencher kept posting benchmark results on PRs that can't possibly affect runtime performance. Two reported cases: - **[#3717](#3717 (comment) — a markdown/agent-tooling PR whose only non-`.md` file was a new `.claude/skills` symlink. - **[#3697](#3697 (comment) — a CI-config-only PR (`.github/workflows/pro-integration-tests.yml` + the detector's own test harness). Both slipped through because the change detector (`script/ci-changes-detector`) is the single gate feeding every test workflow **and** the benchmark matrix. ### Root cause 1 — non-`.md` repo metadata hits the catch-all (#3717) The extensionless `.claude/skills` symlink matched no detector category → fell into the `*` catch-all ("run everything for safety") → `non_runtime_only=false` → full suite + benchmarks. Same class as #3597. ### Root cause 2 — benchmarks piggyback on the test run-flags (#3697) `benchmark.yml` derived which suites to run from the test `run_*` flags. Those flags are intentionally inflated by CI plumbing: the CI-infrastructure catch-all (`script/`, `bin/`, actions) runs the full suite, and the suite-specific workflow YAMLs map to test categories (`pro-integration-tests.yml` → `PRO_DUMMY`, `node-renderer-tests.yml` → `PRO_NODE_RENDERER`, …). None of that moves runtime performance, but it all dragged in benchmarks. ## Fix **Commit 1 — agent/editor tooling is non-runtime.** Categorize `.claude/**`, `.agents/**`, `.cursor/**` alongside docs/`internal/**`/issue-templates, so agent-tooling PRs are `non_runtime_only` (no tests, no benchmarks). **Commit 2 — decouple benchmark selection from test selection.** - The detector tracks `BENCH_CORE` / `BENCH_PRO` / `BENCH_PRO_NODE_RENDERER`, set **only** by genuine product-source arms (gem/app/package/dummy code) and the uncategorized catch-all. CI plumbing (new `CI_INFRA_CHANGED` flag) and the suite-workflow YAMLs (`is_benchmark_irrelevant_path` guard) set no `BENCH` flag — they still run their **tests** but never benchmarks. The targeted per-workflow test mapping (`ci-optimization.md`) is preserved. - The detector emits `run_core_benchmarks` / `run_pro_benchmarks` / `run_pro_node_renderer_benchmarks`; `benchmark.yml` reads them directly instead of re-deriving from `run_*`. The mapping mirrors the old derivation **exactly** for every genuine-source category, so legitimate benchmark runs are unchanged — only CI-plumbing-triggered runs are dropped. - `push`-to-main, `workflow_dispatch`, and the `benchmark` PR label still run suites regardless (the label is the escape hatch for validating `benchmark.yml`/workflow edits). ## Verification - Replayed both PRs' exact file sets against the patched detector: #3717 → `non_runtime_only: true`; #3697 → all three `run_*_benchmarks: false` while tests still run. - Confirmed legit benchmark behavior is unchanged: core source → all 3 suites; node-renderer source → pro + node-renderer; pro-ruby → pro only; a suite-workflow-only edit → its tests but no benchmark. - `script/ci-changes-detector-test.bash`: **43 run, 0 failed** (added regressions for both root causes). `actionlint` clean on `benchmark.yml`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * CI now separates benchmark gating from test-selection, treats additional agent/editor/tooling paths as docs/non-runtime-only, treats CI-infrastructure edits as infra-only (forcing tests but suppressing benchmarks), and exposes per-suite benchmark run flags in CI outputs while preventing docs-only when CI infra changes. * **Tests** * Added regression tests covering docs/tooling paths, CI-infra-only edits, suite-specific gating, combined infra+runtime edits, and node-renderer benchmark behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes central CI gating logic for all workflows and benchmarks; behavior is heavily regression-tested but misclassification could skip needed benchmarks or still run Bencher on non-runtime PRs. > > **Overview** > Stops Bencher from running on PRs that only touch docs, agent tooling, or CI config by tightening **what counts as non-runtime** and **when benchmarks are selected**. > > **`script/ci-changes-detector`** now treats `.claude/**`, `.agents/**`, and `.cursor/**` like docs (fixes extensionless paths such as `.claude/skills` hitting the catch-all). CI plumbing uses a dedicated **`CI_INFRA_CHANGED`** flag: it still forces the full **test** matrix but does **not** set benchmark flags. Benchmark gating is decoupled from test `run_*` flags via **`BENCH_*`** flags and new outputs **`run_core_benchmarks`**, **`run_pro_benchmarks`**, and **`run_pro_node_renderer_benchmarks`**; workflow YAML under `.github/workflows/*` is excluded from benchmark relevance via **`is_benchmark_irrelevant_path`**. > > **`.github/workflows/benchmark.yml`** drops inline benchmark derivation and reads those detector outputs directly. **`ci-commands.yml`** aligns the docs-only heuristic with the same agent/editor paths. > > Regression coverage in **`script/ci-changes-detector-test.bash`** covers agent tooling, CI-only vs workflow-only edits, mixed CI + source, and node-renderer benchmarks. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 92ba2bd. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> ## Codex Decision Log - Full-matrix validation: accepted CodeRabbit's current-head nit and added the existing `full-ci` label so detector/benchmark-gating changes receive full CI coverage before merge. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent ccc7609 commit e26d659

4 files changed

Lines changed: 267 additions & 46 deletions

File tree

‎.github/workflows/benchmark.yml‎

Lines changed: 5 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -110,37 +110,8 @@ jobs:
110110
with:
111111
docs-only: ${{ steps.detect.outputs.docs_only }}
112112
previous-sha: ${{ github.event.before }}
113-
- name: Set benchmark outputs
114-
id: benchmark-outputs
115-
if: steps.detect.outputs.non_runtime_only != 'true'
116-
run: |
117-
RUN_CORE_BENCHMARKS=false
118-
RUN_PRO_BENCHMARKS=false
119-
RUN_PRO_NODE_RENDERER_BENCHMARKS=false
120-
121-
if [ "${{ steps.detect.outputs.run_ruby_tests }}" = "true" ] ||
122-
[ "${{ steps.detect.outputs.run_js_tests }}" = "true" ] ||
123-
[ "${{ steps.detect.outputs.run_dummy_tests }}" = "true" ]; then
124-
RUN_CORE_BENCHMARKS=true
125-
RUN_PRO_BENCHMARKS=true
126-
fi
127-
128-
if [ "${{ steps.detect.outputs.run_pro_tests }}" = "true" ] ||
129-
[ "${{ steps.detect.outputs.run_pro_dummy_tests }}" = "true" ]; then
130-
RUN_PRO_BENCHMARKS=true
131-
fi
132-
133-
if [ "${{ steps.detect.outputs.run_pro_node_renderer_tests }}" = "true" ]; then
134-
RUN_PRO_BENCHMARKS=true
135-
RUN_PRO_NODE_RENDERER_BENCHMARKS=true
136-
fi
137-
138-
{
139-
echo "run_core_benchmarks=$RUN_CORE_BENCHMARKS"
140-
echo "run_pro_benchmarks=$RUN_PRO_BENCHMARKS"
141-
echo "run_pro_node_renderer_benchmarks=$RUN_PRO_NODE_RENDERER_BENCHMARKS"
142-
} >> "$GITHUB_OUTPUT"
143-
shell: bash
113+
# script/ci-changes-detector owns benchmark-suite selection via
114+
# run_core_benchmarks / run_pro_benchmarks / run_pro_node_renderer_benchmarks.
144115
- name: Set benchmark matrices
145116
id: benchmark-matrices
146117
# Always runs. GitHub Actions evaluates `strategy.matrix` for downstream jobs at
@@ -157,9 +128,9 @@ jobs:
157128
BENCHMARK_PULL_REQUEST_HEAD_REPO: ${{ github.event.pull_request.head.repo.full_name || '' }}
158129
BENCHMARK_PULL_REQUEST_LABELS: ${{ github.event_name == 'pull_request' && toJSON(github.event.pull_request.labels.*.name) || '[]' }}
159130
GITHUB_REPOSITORY: ${{ github.repository }}
160-
RUN_CORE_BENCHMARKS: ${{ steps.benchmark-outputs.outputs.run_core_benchmarks || 'false' }}
161-
RUN_PRO_BENCHMARKS: ${{ steps.benchmark-outputs.outputs.run_pro_benchmarks || 'false' }}
162-
RUN_PRO_NODE_RENDERER_BENCHMARKS: ${{ steps.benchmark-outputs.outputs.run_pro_node_renderer_benchmarks || 'false' }}
131+
RUN_CORE_BENCHMARKS: ${{ steps.detect.outputs.run_core_benchmarks || 'false' }}
132+
RUN_PRO_BENCHMARKS: ${{ steps.detect.outputs.run_pro_benchmarks || 'false' }}
133+
RUN_PRO_NODE_RENDERER_BENCHMARKS: ${{ steps.detect.outputs.run_pro_node_renderer_benchmarks || 'false' }}
163134
run: |
164135
BENCHMARK_MATRIX=$(ruby benchmarks/generate_matrix.rb)
165136
export BENCHMARK_MATRIX

‎.github/workflows/ci-commands.yml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -171,7 +171,10 @@ jobs:
171171
filename === '.lychee.toml' ||
172172
filename.startsWith('docs/') ||
173173
filename.startsWith('internal/') ||
174-
filename.startsWith('.github/ISSUE_TEMPLATE/');
174+
filename.startsWith('.github/ISSUE_TEMPLATE/') ||
175+
filename.startsWith('.claude/') ||
176+
filename.startsWith('.agents/') ||
177+
filename.startsWith('.cursor/');
175178
}
176179
177180
function isDocsOnly(files) {

‎script/ci-changes-detector‎

Lines changed: 137 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -81,8 +81,19 @@ PRO_JS_CHANGED=false
8181
PRO_DUMMY_CHANGED=false
8282
PRO_NODE_RENDERER_CHANGED=false
8383
UNCATEGORIZED_CHANGED=false
84+
CI_INFRA_CHANGED=false
8485
SOURCE_COMMENT_ONLY_CHANGED=false
8586
PRO_SOURCE_COMMENT_ONLY_CHANGED=false
87+
# Benchmark relevance, tracked separately from the test-selection flags above.
88+
# Benchmarks measure runtime performance, so they must fire ONLY when genuine
89+
# product SOURCE changes — never on CI plumbing (scripts, actions) or the
90+
# suite-specific workflow YAML files, which change how the suite runs but cannot
91+
# move performance (the Bencher-on-a-config-PR noise in #3697). These three drive
92+
# run_core_benchmarks / run_pro_benchmarks / run_pro_node_renderer_benchmarks,
93+
# which benchmark.yml reads directly.
94+
BENCH_CORE_CHANGED=false
95+
BENCH_PRO_CHANGED=false
96+
BENCH_PRO_NODE_RENDERER_CHANGED=false
8697

8798
comment_line_is_runtime_directive() {
8899
local line="$1"
@@ -243,6 +254,18 @@ source_file_comment_only_change() {
243254
[ "$saw_changed_line" = true ]
244255
}
245256

257+
# Suite-specific workflow YAML files are test selectors, not benchmark inputs.
258+
# Some of them live in source/spec arms so the right tests still run, but a
259+
# workflow edit cannot move runtime performance (#3697). Plain CI plumbing
260+
# (script/, bin/, .github/actions/, .lefthook.yml) lands in the CI-infrastructure
261+
# arm instead, which sets no BENCH_* flag at all.
262+
is_benchmark_irrelevant_path() {
263+
case "$1" in
264+
.github/workflows/*) return 0 ;;
265+
*) return 1 ;;
266+
esac
267+
}
268+
246269
# Analyze each changed file
247270
while IFS= read -r file; do
248271
# A here-string yields a single empty line for empty input, so skip blank lines —
@@ -260,7 +283,12 @@ while IFS= read -r file; do
260283
# (see the #3474 regression and the benchmarks-on-a-docs-PR case in #3597).
261284
# - internal/** : planning, contributor-info, testimonials, history docs
262285
# - .github/ISSUE_TEMPLATE/** : GitHub issue templates (repo metadata, not CI infra)
263-
*.md|*.mdx|*.markdown|*.rst|*.txt|docs/*|internal/*|internal/**/*|.github/ISSUE_TEMPLATE/*|.github/ISSUE_TEMPLATE/**/*|.lychee.toml)
286+
# - .claude/** .agents/** .cursor/** : agent/editor tooling (skills, prompts,
287+
# hooks, workflows). These run locally for coding agents, never in GitHub
288+
# Actions CI, so they don't affect the runtime, tests, or benchmarks. The
289+
# non-extension .claude/skills symlink is what slipped through the catch-all
290+
# and ran benchmarks on the docs-only PR #3717.
291+
*.md|*.mdx|*.markdown|*.rst|*.txt|docs/*|internal/*|internal/**/*|.github/ISSUE_TEMPLATE/*|.github/ISSUE_TEMPLATE/**/*|.claude/*|.claude/**/*|.agents/*|.agents/**/*|.cursor/*|.cursor/**/*|.lychee.toml)
264292
# Don't change DOCS_ONLY flag, just continue
265293
;;
266294

@@ -271,6 +299,11 @@ while IFS= read -r file; do
271299
SOURCE_COMMENT_ONLY_CHANGED=true
272300
else
273301
GENERATORS_CHANGED=true
302+
# Generator source feeds the core dummy app the core+pro suites benchmark.
303+
if ! is_benchmark_irrelevant_path "$file"; then
304+
BENCH_CORE_CHANGED=true
305+
BENCH_PRO_CHANGED=true
306+
fi
274307
fi
275308
;;
276309

@@ -281,6 +314,11 @@ while IFS= read -r file; do
281314
SOURCE_COMMENT_ONLY_CHANGED=true
282315
else
283316
RUBY_CORE_CHANGED=true
317+
# Core gem code underlies every suite (Pro depends on core, the node
318+
# renderer renders core output), so it benchmarks all three.
319+
BENCH_CORE_CHANGED=true
320+
BENCH_PRO_CHANGED=true
321+
BENCH_PRO_NODE_RENDERER_CHANGED=true
284322
fi
285323
;;
286324

@@ -291,6 +329,13 @@ while IFS= read -r file; do
291329
SOURCE_COMMENT_ONLY_CHANGED=true
292330
else
293331
RSPEC_CHANGED=true
332+
# Mirrors the existing benchmark behavior: a core gem-spec change runs the
333+
# core+pro suites (run_ruby_tests fed them). The gem-tests.yml workflow in
334+
# this arm is guarded out — editing it can't move performance.
335+
if ! is_benchmark_irrelevant_path "$file"; then
336+
BENCH_CORE_CHANGED=true
337+
BENCH_PRO_CHANGED=true
338+
fi
294339
fi
295340
;;
296341

@@ -314,6 +359,11 @@ while IFS= read -r file; do
314359
SOURCE_COMMENT_ONLY_CHANGED=true
315360
else
316361
JS_CHANGED=true
362+
# Core JS package code drives the core+pro suites' bundles.
363+
if ! is_benchmark_irrelevant_path "$file"; then
364+
BENCH_CORE_CHANGED=true
365+
BENCH_PRO_CHANGED=true
366+
fi
317367
fi
318368
;;
319369

@@ -324,6 +374,12 @@ while IFS= read -r file; do
324374
SOURCE_COMMENT_ONLY_CHANGED=true
325375
else
326376
SPEC_DUMMY_CHANGED=true
377+
# The core suite benchmarks this dummy app; the pro suite shares core. The
378+
# integration-tests.yml workflow in this arm is guarded out.
379+
if ! is_benchmark_irrelevant_path "$file"; then
380+
BENCH_CORE_CHANGED=true
381+
BENCH_PRO_CHANGED=true
382+
fi
327383
fi
328384
;;
329385

@@ -334,6 +390,9 @@ while IFS= read -r file; do
334390
PRO_SOURCE_COMMENT_ONLY_CHANGED=true
335391
else
336392
PRO_RUBY_CORE_CHANGED=true
393+
if ! is_benchmark_irrelevant_path "$file"; then
394+
BENCH_PRO_CHANGED=true
395+
fi
337396
fi
338397
;;
339398

@@ -344,6 +403,9 @@ while IFS= read -r file; do
344403
PRO_SOURCE_COMMENT_ONLY_CHANGED=true
345404
else
346405
PRO_JS_CHANGED=true
406+
# Pro JS is bundled by both the Pro Rails suite and the node renderer.
407+
BENCH_PRO_CHANGED=true
408+
BENCH_PRO_NODE_RENDERER_CHANGED=true
347409
fi
348410
;;
349411

@@ -354,6 +416,10 @@ while IFS= read -r file; do
354416
PRO_SOURCE_COMMENT_ONLY_CHANGED=true
355417
else
356418
PRO_RSPEC_CHANGED=true
419+
# pro-gem-tests.yml in this arm is guarded out (a workflow edit can't move perf).
420+
if ! is_benchmark_irrelevant_path "$file"; then
421+
BENCH_PRO_CHANGED=true
422+
fi
357423
fi
358424
;;
359425

@@ -364,6 +430,12 @@ while IFS= read -r file; do
364430
PRO_SOURCE_COMMENT_ONLY_CHANGED=true
365431
else
366432
PRO_DUMMY_CHANGED=true
433+
# The Pro Rails suite benchmarks this dummy app. pro-integration-tests.yml
434+
# in this arm is guarded out — that workflow edit (PR #3697) is exactly
435+
# what spuriously triggered the Pro benchmark.
436+
if ! is_benchmark_irrelevant_path "$file"; then
437+
BENCH_PRO_CHANGED=true
438+
fi
367439
fi
368440
;;
369441

@@ -374,6 +446,12 @@ while IFS= read -r file; do
374446
PRO_SOURCE_COMMENT_ONLY_CHANGED=true
375447
else
376448
PRO_NODE_RENDERER_CHANGED=true
449+
# Node renderer source drives the node-renderer suite and the Pro suite.
450+
# node-renderer-tests.yml in this arm is guarded out.
451+
if ! is_benchmark_irrelevant_path "$file"; then
452+
BENCH_PRO_CHANGED=true
453+
BENCH_PRO_NODE_RENDERER_CHANGED=true
454+
fi
377455
fi
378456
;;
379457

@@ -389,18 +467,19 @@ while IFS= read -r file; do
389467
PRO_LINT_CONFIG_CHANGED=true
390468
;;
391469

392-
# CI infrastructure files (scripts, workflows, CI configs)
393-
# Changes to CI infrastructure should trigger tests to validate the changes work
470+
# CI infrastructure files (scripts, workflows, CI configs).
471+
# These change HOW the suite runs but never the gem/package runtime, so they
472+
# trigger the full TEST suite to validate the change (see the CI_INFRA_CHANGED
473+
# run-flag block below) WITHOUT triggering benchmarks: performance cannot move
474+
# when only CI plumbing changes, so a benchmark run here is pure Bencher noise
475+
# (see the CI-workflow-edit case in #3697). The 'benchmark' PR label still
476+
# forces a run for workflow self-tests (benchmarks/generate_matrix.rb honors
477+
# labels independently of these change-detection flags).
478+
# NOTE: keep this as its own flag rather than setting the source *_CHANGED
479+
# flags directly — that conflation is exactly what made benchmarks run here.
394480
script/*|script/**/*|bin/*|bin/**/*|.github/workflows/*|.github/actions/*|.github/actions/**/*|.lefthook.yml)
395481
DOCS_ONLY=false
396-
RUBY_CORE_CHANGED=true # Trigger all tests for CI infrastructure changes
397-
JS_CHANGED=true
398-
SPEC_DUMMY_CHANGED=true
399-
GENERATORS_CHANGED=true
400-
PRO_RUBY_CORE_CHANGED=true
401-
PRO_JS_CHANGED=true
402-
PRO_DUMMY_CHANGED=true
403-
PRO_NODE_RENDERER_CHANGED=true
482+
CI_INFRA_CHANGED=true
404483
;;
405484

406485
# Catch-all: any file not explicitly categorized above
@@ -409,6 +488,11 @@ while IFS= read -r file; do
409488
*)
410489
DOCS_ONLY=false
411490
UNCATEGORIZED_CHANGED=true
491+
# Unknown file: benchmark every suite too (safety, matching the full-suite
492+
# test run below). A new runtime path must never silently skip benchmarks.
493+
BENCH_CORE_CHANGED=true
494+
BENCH_PRO_CHANGED=true
495+
BENCH_PRO_NODE_RENDERER_CHANGED=true
412496
;;
413497
esac
414498
done <<< "$CHANGED_FILES"
@@ -430,6 +514,7 @@ elif [ "$SOURCE_COMMENT_ONLY_CHANGED" = true ] || [ "$PRO_SOURCE_COMMENT_ONLY_CH
430514
[ "$PRO_JS_CHANGED" = false ] &&
431515
[ "$PRO_DUMMY_CHANGED" = false ] &&
432516
[ "$PRO_NODE_RENDERER_CHANGED" = false ] &&
517+
[ "$CI_INFRA_CHANGED" = false ] &&
433518
[ "$UNCATEGORIZED_CHANGED" = false ]; then
434519
NON_RUNTIME_ONLY=true
435520
fi
@@ -456,6 +541,9 @@ if [ "$DOCS_ONLY" = true ]; then
456541
"run_pro_tests:false"
457542
"run_pro_dummy_tests:false"
458543
"run_pro_node_renderer_tests:false"
544+
"run_core_benchmarks:false"
545+
"run_pro_benchmarks:false"
546+
"run_pro_node_renderer_benchmarks:false"
459547
"benchmarks_changed:false"
460548
)
461549

@@ -507,6 +595,7 @@ echo "Changed file categories:"
507595
[ "$PRO_LINT_CONFIG_CHANGED" = true ] && echo -e "${YELLOW} • React on Rails Pro lint configuration${NC}"
508596
[ "$SOURCE_COMMENT_ONLY_CHANGED" = true ] && echo -e "${YELLOW} • Source comments only${NC}"
509597
[ "$PRO_SOURCE_COMMENT_ONLY_CHANGED" = true ] && echo -e "${YELLOW} • React on Rails Pro source comments only${NC}"
598+
[ "$CI_INFRA_CHANGED" = true ] && echo -e "${YELLOW} • CI infrastructure (full test suite, no benchmarks)${NC}"
510599
[ "$UNCATEGORIZED_CHANGED" = true ] && echo -e "${YELLOW} • Uncategorized changes (running full suite for safety)${NC}"
511600

512601
echo ""
@@ -584,6 +673,24 @@ if [ "$PRO_NODE_RENDERER_CHANGED" = true ] || [ "$PRO_JS_CHANGED" = true ] || [
584673
RUN_PRO_NODE_RENDERER_TESTS=true
585674
fi
586675

676+
# CI infrastructure changes run the full TEST suite to validate the change. Unlike
677+
# the uncategorized catch-all below, they set no BENCH_* flag, so the benchmark
678+
# outputs stay false: CI plumbing can't move runtime performance, so a Bencher run
679+
# here is noise (#3697). This block requests every test job while benchmarks stay
680+
# off.
681+
if [ "$CI_INFRA_CHANGED" = true ]; then
682+
RUN_LINT=true
683+
RUN_RUBY_TESTS=true
684+
RUN_JS_TESTS=true
685+
RUN_DUMMY_TESTS=true
686+
RUN_GENERATORS=true
687+
RUN_E2E_TESTS=true
688+
RUN_PRO_LINT=true
689+
RUN_PRO_TESTS=true
690+
RUN_PRO_DUMMY_TESTS=true
691+
RUN_PRO_NODE_RENDERER_TESTS=true
692+
fi
693+
587694
# If we encounter a change that isn't explicitly categorized, default to running everything
588695
if [ "$UNCATEGORIZED_CHANGED" = true ]; then
589696
RUN_LINT=true
@@ -598,6 +705,16 @@ if [ "$UNCATEGORIZED_CHANGED" = true ]; then
598705
RUN_PRO_NODE_RENDERER_TESTS=true
599706
fi
600707

708+
# Benchmark suite selection (consumed by .github/workflows/benchmark.yml). One
709+
# output per suite, driven purely by the BENCH_* source-relevance flags set in
710+
# the categorization above — never by the test run_* flags, which CI plumbing and
711+
# the suite-workflow YAML files inflate. This keeps Bencher off docs-only (#3717)
712+
# and CI-config-only (#3697) PRs. On push to main / workflow_dispatch the suites
713+
# run regardless (generate_matrix.rb), so these only gate PR runs.
714+
RUN_CORE_BENCHMARKS=$BENCH_CORE_CHANGED
715+
RUN_PRO_BENCHMARKS=$BENCH_PRO_CHANGED
716+
RUN_PRO_NODE_RENDERER_BENCHMARKS=$BENCH_PRO_NODE_RENDERER_CHANGED
717+
601718
[ "$RUN_LINT" = true ] && echo " ✓ Lint (Ruby + JS)"
602719
[ "$RUN_RUBY_TESTS" = true ] && echo " ✓ RSpec gem tests"
603720
[ "$RUN_JS_TESTS" = true ] && echo " ✓ JS unit tests"
@@ -608,6 +725,9 @@ fi
608725
[ "$RUN_PRO_TESTS" = true ] && echo " ✓ React on Rails Pro RSpec unit tests (Ruby + JS)"
609726
[ "$RUN_PRO_DUMMY_TESTS" = true ] && echo " ✓ React on Rails Pro Dummy app integration tests"
610727
[ "$RUN_PRO_NODE_RENDERER_TESTS" = true ] && echo " ✓ React on Rails Pro Node Renderer tests"
728+
[ "$RUN_CORE_BENCHMARKS" = true ] && echo " ✓ Core benchmarks"
729+
[ "$RUN_PRO_BENCHMARKS" = true ] && echo " ✓ React on Rails Pro benchmarks"
730+
[ "$RUN_PRO_NODE_RENDERER_BENCHMARKS" = true ] && echo " ✓ React on Rails Pro Node Renderer benchmarks"
611731

612732
# Export as GitHub Actions outputs if running in CI
613733
if [ -n "${GITHUB_OUTPUT:-}" ]; then
@@ -624,6 +744,9 @@ if [ -n "${GITHUB_OUTPUT:-}" ]; then
624744
echo "run_pro_tests=$RUN_PRO_TESTS"
625745
echo "run_pro_dummy_tests=$RUN_PRO_DUMMY_TESTS"
626746
echo "run_pro_node_renderer_tests=$RUN_PRO_NODE_RENDERER_TESTS"
747+
echo "run_core_benchmarks=$RUN_CORE_BENCHMARKS"
748+
echo "run_pro_benchmarks=$RUN_PRO_BENCHMARKS"
749+
echo "run_pro_node_renderer_benchmarks=$RUN_PRO_NODE_RENDERER_BENCHMARKS"
627750
echo "benchmarks_changed=$BENCHMARKS_CHANGED"
628751
} >> "$GITHUB_OUTPUT"
629752
fi
@@ -644,6 +767,9 @@ if [ "${CI_JSON_OUTPUT:-}" = "1" ]; then
644767
"run_pro_tests": $RUN_PRO_TESTS,
645768
"run_pro_dummy_tests": $RUN_PRO_DUMMY_TESTS,
646769
"run_pro_node_renderer_tests": $RUN_PRO_NODE_RENDERER_TESTS,
770+
"run_core_benchmarks": $RUN_CORE_BENCHMARKS,
771+
"run_pro_benchmarks": $RUN_PRO_BENCHMARKS,
772+
"run_pro_node_renderer_benchmarks": $RUN_PRO_NODE_RENDERER_BENCHMARKS,
647773
"benchmarks_changed": $BENCHMARKS_CHANGED
648774
}
649775
EOF

0 commit comments

Comments
 (0)