Summary
The Pro dummy-app integration test jobs in .github/workflows/pro-integration-tests.yml are gated on the run_pro_tests change-detector flag, which does not include PRO_DUMMY_CHANGED. As a result, a PR that touches only files under react_on_rails_pro/spec/dummy/** (and nothing in Pro lib/JS/node-renderer or core Ruby) will have these integration jobs skipped — the dummy specs never run for that PR.
A dedicated flag, run_pro_dummy_tests, already exists for exactly this purpose and does include PRO_DUMMY_CHANGED, but it is currently only consumed by benchmark.yml, never by pro-integration-tests.yml.
Evidence
script/ci-changes-detector:575-577 — RUN_PRO_TESTS=true only when PRO_RUBY_CORE_CHANGED || PRO_RSPEC_CHANGED || PRO_JS_CHANGED || RUBY_CORE_CHANGED || PRO_NODE_RENDERER_CHANGED. No PRO_DUMMY_CHANGED.
script/ci-changes-detector:579-581 — RUN_PRO_DUMMY_TESTS=true does include PRO_DUMMY_CHANGED.
.github/workflows/pro-integration-tests.yml:50 — the detect-changes job exposes only run_pro_tests as an output (not run_pro_dummy_tests).
.github/workflows/pro-integration-tests.yml:96,196,372 — the dummy-app build/test jobs (including rspec-dummy-app-node-renderer, which runs cd spec/dummy && bundle exec rspec at line 317) gate on run_pro_tests == 'true'.
run_pro_dummy_tests is referenced only in .github/workflows/benchmark.yml:129.
Impact
Regression guards that live entirely under spec/dummy/** are not self-protecting. For example, a future PR editing only react_on_rails_pro/spec/dummy/spec/requests/posts_page_spec.rb or react_on_rails_pro/spec/dummy/spec/db/seeds_spec.rb (both added in #3615) would not run those specs in CI. They run today only because #3615 also edits .github/workflows/benchmark.yml, which trips the CI-infrastructure catch-all that forces all flags on.
Suggested fix
Wire the dummy-app integration jobs to run_pro_dummy_tests instead of (or in addition to) run_pro_tests:
- Add
run_pro_dummy_tests: ${{ steps.detect.outputs.run_pro_dummy_tests }} to the detect-changes job outputs in pro-integration-tests.yml.
- Update the
if: gating on the dummy-app jobs (e.g. rspec-dummy-app-node-renderer) to also consider run_pro_dummy_tests == 'true'.
This is a pre-existing CI design gap, not introduced by #3615. Filing as a follow-up so the dummy specs added there are durably protected.
Discovered during review of #3615.
Summary
The Pro dummy-app integration test jobs in
.github/workflows/pro-integration-tests.ymlare gated on therun_pro_testschange-detector flag, which does not includePRO_DUMMY_CHANGED. As a result, a PR that touches only files underreact_on_rails_pro/spec/dummy/**(and nothing in Pro lib/JS/node-renderer or core Ruby) will have these integration jobs skipped — the dummy specs never run for that PR.A dedicated flag,
run_pro_dummy_tests, already exists for exactly this purpose and does includePRO_DUMMY_CHANGED, but it is currently only consumed bybenchmark.yml, never bypro-integration-tests.yml.Evidence
script/ci-changes-detector:575-577—RUN_PRO_TESTS=trueonly whenPRO_RUBY_CORE_CHANGED || PRO_RSPEC_CHANGED || PRO_JS_CHANGED || RUBY_CORE_CHANGED || PRO_NODE_RENDERER_CHANGED. NoPRO_DUMMY_CHANGED.script/ci-changes-detector:579-581—RUN_PRO_DUMMY_TESTS=truedoes includePRO_DUMMY_CHANGED..github/workflows/pro-integration-tests.yml:50— thedetect-changesjob exposes onlyrun_pro_testsas an output (notrun_pro_dummy_tests)..github/workflows/pro-integration-tests.yml:96,196,372— the dummy-app build/test jobs (includingrspec-dummy-app-node-renderer, which runscd spec/dummy && bundle exec rspecat line 317) gate onrun_pro_tests == 'true'.run_pro_dummy_testsis referenced only in.github/workflows/benchmark.yml:129.Impact
Regression guards that live entirely under
spec/dummy/**are not self-protecting. For example, a future PR editing onlyreact_on_rails_pro/spec/dummy/spec/requests/posts_page_spec.rborreact_on_rails_pro/spec/dummy/spec/db/seeds_spec.rb(both added in #3615) would not run those specs in CI. They run today only because #3615 also edits.github/workflows/benchmark.yml, which trips the CI-infrastructure catch-all that forces all flags on.Suggested fix
Wire the dummy-app integration jobs to
run_pro_dummy_testsinstead of (or in addition to)run_pro_tests:run_pro_dummy_tests: ${{ steps.detect.outputs.run_pro_dummy_tests }}to thedetect-changesjob outputs inpro-integration-tests.yml.if:gating on the dummy-app jobs (e.g.rspec-dummy-app-node-renderer) to also considerrun_pro_dummy_tests == 'true'.This is a pre-existing CI design gap, not introduced by #3615. Filing as a follow-up so the dummy specs added there are durably protected.
Discovered during review of #3615.