Skip to content

CI: Pro dummy integration tests skip on Pro-dummy-only changes (gated on run_pro_tests, not run_pro_dummy_tests) #3633

Description

@justin808

Summary

The Pro dummy-app integration test jobs in .github/workflows/pro-integration-tests.yml are gated on the run_pro_tests change-detector flag, which does not include PRO_DUMMY_CHANGED. As a result, a PR that touches only files under react_on_rails_pro/spec/dummy/** (and nothing in Pro lib/JS/node-renderer or core Ruby) will have these integration jobs skipped — the dummy specs never run for that PR.

A dedicated flag, run_pro_dummy_tests, already exists for exactly this purpose and does include PRO_DUMMY_CHANGED, but it is currently only consumed by benchmark.yml, never by pro-integration-tests.yml.

Evidence

  • script/ci-changes-detector:575-577RUN_PRO_TESTS=true only when PRO_RUBY_CORE_CHANGED || PRO_RSPEC_CHANGED || PRO_JS_CHANGED || RUBY_CORE_CHANGED || PRO_NODE_RENDERER_CHANGED. No PRO_DUMMY_CHANGED.
  • script/ci-changes-detector:579-581RUN_PRO_DUMMY_TESTS=true does include PRO_DUMMY_CHANGED.
  • .github/workflows/pro-integration-tests.yml:50 — the detect-changes job exposes only run_pro_tests as an output (not run_pro_dummy_tests).
  • .github/workflows/pro-integration-tests.yml:96,196,372 — the dummy-app build/test jobs (including rspec-dummy-app-node-renderer, which runs cd spec/dummy && bundle exec rspec at line 317) gate on run_pro_tests == 'true'.
  • run_pro_dummy_tests is referenced only in .github/workflows/benchmark.yml:129.

Impact

Regression guards that live entirely under spec/dummy/** are not self-protecting. For example, a future PR editing only react_on_rails_pro/spec/dummy/spec/requests/posts_page_spec.rb or react_on_rails_pro/spec/dummy/spec/db/seeds_spec.rb (both added in #3615) would not run those specs in CI. They run today only because #3615 also edits .github/workflows/benchmark.yml, which trips the CI-infrastructure catch-all that forces all flags on.

Suggested fix

Wire the dummy-app integration jobs to run_pro_dummy_tests instead of (or in addition to) run_pro_tests:

  1. Add run_pro_dummy_tests: ${{ steps.detect.outputs.run_pro_dummy_tests }} to the detect-changes job outputs in pro-integration-tests.yml.
  2. Update the if: gating on the dummy-app jobs (e.g. rspec-dummy-app-node-renderer) to also consider run_pro_dummy_tests == 'true'.

This is a pre-existing CI design gap, not introduced by #3615. Filing as a follow-up so the dummy specs added there are durably protected.

Discovered during review of #3615.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions