Skip to content

Skip approve-full-tests hold for automatic bot PRs #6255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ajpallares wants to merge 31 commits into
base: main
Choose a base branch
from
Open

Skip approve-full-tests hold for automatic bot PRs #6255

Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
6fbfe51
Gate full CI suite behind approval job on PRs
ajpallares Feb 10, 2026
21bbbeb
Gate release tagging behind all-tests-passed
ajpallares Feb 10, 2026
de40c4c
Exclude build-test workflow from main and release branches
ajpallares Feb 10, 2026
1401ef4
Renamed "hold" job to "approve-release"
ajpallares Feb 10, 2026
4d78abd
Merge branch 'main' into pallares/change-ci-structure-for-PRs
ajpallares Feb 10, 2026
f6a32b2
Skip `build-test` when `run-manual-tests` or `run-from-github-comment…
ajpallares Feb 10, 2026
894f27e
Revert "Skip `build-test` when `run-manual-tests` or `run-from-github…
ajpallares Feb 10, 2026
6939ea7
rename `build-test` workflow to `run-all-tests`
ajpallares Feb 10, 2026
28cde3a
Add missing jobs to `ci` workflow (for `main` and for release branches)
ajpallares Feb 10, 2026
6858b85
PR comments
ajpallares Feb 12, 2026
e80128d
Merge branch 'main' into pallares/change-ci-structure-for-PRs
ajpallares Feb 12, 2026
e65a633
Rename `all-tasks-finished` to `all-tasks-passed`
ajpallares Feb 12, 2026
fcb9287
Have release branches run `all-tasks-passed` only after `tag-release-…
ajpallares Feb 12, 2026
faee21d
Revert "Have release branches run `all-tasks-passed` only after `tag-…
ajpallares Feb 12, 2026
ecb1011
Merge branch 'main' into pallares/change-ci-structure-for-PRs
ajpallares Feb 12, 2026
6e17223
Require Reduced Test Suite jobs in `all-tasks-passed` summary gate
ajpallares Feb 12, 2026
b93e326
Merge branch 'main' into pallares/change-ci-structure-for-PRs
ajpallares Feb 12, 2026
364d0a1
Merge branch 'main' into pallares/change-ci-structure-for-PRs
ajpallares Feb 12, 2026
c9d33c4
Merge branch 'main' into pallares/change-ci-structure-for-PRs
ajpallares Feb 12, 2026
f5fba49
Merge branch 'main' into pallares/change-ci-structure-for-PRs
ajpallares Feb 13, 2026
ddd67f3
Skip approve-full-tests hold for automatic bot PRs
ajpallares Feb 13, 2026
8064742
Merge branch 'main' into pallares/change-ci-structure-for-PRs
ajpallares Feb 16, 2026
c214ce9
Merge branch 'pallares/change-ci-structure-for-PRs' into pallares/ski…
ajpallares Feb 16, 2026
ee3dec0
Merge branch 'main' into pallares/change-ci-structure-for-PRs
ajpallares Feb 17, 2026
d75486a
Fix job references after merge from main
ajpallares Feb 17, 2026
ea313f7
Merge branch 'main' into pallares/change-ci-structure-for-PRs
ajpallares Feb 20, 2026
c1b27a5
Merge branch 'pallares/change-ci-structure-for-PRs' into pallares/ski…
ajpallares Feb 20, 2026
9f5c0bf
Merge remote-tracking branch 'origin/main' into pallares/skip-hold-jo…
ajpallares Feb 25, 2026
8cff0e0
Add missing generate_swiftinterface exclusion to run-all-tests-auto w…
ajpallares Feb 25, 2026
3cd2771
Merge branch 'main' into pallares/skip-hold-job-on-some-automatic-prs…
ajpallares Feb 26, 2026
d647423
Merge branch 'main' into pallares/skip-hold-job-on-some-automatic-prs…
ajpallares Mar 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
196 changes: 196 additions & 0 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2097,6 +2097,11 @@ workflows:
pattern: "^(main|release/.*)$"
value: << pipeline.git.branch >>
- equal: [run-manual-tests, << pipeline.parameters.action >>]
# Exclude automatic bot branches (they use run-all-tests-auto instead)
- not:
matches:
pattern: "^(dependabot/.*|bump/.*|generated_snapshots/.*)$"
value: << pipeline.git.branch >>
jobs:
# =============================================================
# Reduced Test Suite: Auto-run on every push
Expand Down Expand Up @@ -2295,6 +2300,197 @@ workflows:
- run-all-maestro-e2e-tests
- docs-build

# =============================================================
# Automatic bot PRs: same jobs as run-all-tests but without
# the approve-full-tests manual approval gate.
# Matches branches from dependabot, RCGitBot version bumps,
# and RCGitBot generated snapshot PRs.
# =============================================================
run-all-tests-auto:
when:
and:
- not:
equal: [scheduled_pipeline, << pipeline.trigger_source >>]
- not:
equal: ["release-train", << pipeline.schedule.name >>]
- not:
equal: [bump, << pipeline.parameters.action >>]
- not: << pipeline.parameters.generate_snapshots >>
- not: << pipeline.parameters.generate_revenuecatui_snapshots >>
- not: << pipeline.parameters.generate_swiftinterface >>
- not:
matches:
pattern: "^(main|release/.*)$"
value: << pipeline.git.branch >>
- matches:
pattern: "^(dependabot/.*|bump/.*|generated_snapshots/.*)$"
value: << pipeline.git.branch >>
Comment thread
cursor[bot] marked this conversation as resolved.
jobs:
# =============================================================
# Reduced Test Suite (same as run-all-tests)
# =============================================================
- lint
- check-api-changes:
context:
- slack-secrets-ios
- validate-package-swift:
name: validate-package-swift-5.8
xcode_version: "14.3.1"
mac_os_executor_name: macos-executor
- run-test-ios-26
- pod-lib-lint
- run-revenuecat-ui-ios-26
- emerge_purchases_ui_snapshot_tests
- emerge_binary_size_analysis
Copy link
Copy Markdown

@cursor cursor Bot Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing context for reduced test suite jobs

Medium Severity

In run-all-tests-auto, the reduced test suite jobs run-test-ios-26, pod-lib-lint, run-revenuecat-ui-ios-26, emerge_purchases_ui_snapshot_tests, and emerge_binary_size_analysis are all missing their context: slack-secrets that the equivalent entries in run-all-tests have. Without this context, these jobs won't have access to the Slack webhook secrets and any other environment variables provided by that context, causing notification failures or potentially job failures.

Fix in Cursor Fix in Web

- build-tv-watch-mac-and-visionos:
context:
- slack-secrets

# =============================================================
# Full Test Suite (runs automatically, no approval gate)
# =============================================================

# Backend Integration Tests
- backend-integration-tests-SK1:
context:
- slack-secrets
- backend-integration-tests-SK2:
context:
- slack-secrets
- backend-integration-tests-custom-entitlements:
context:
- slack-secrets
- backend-integration-tests-other:
context:
- slack-secrets
- backend-integration-tests-offline:
context:
- slack-secrets

# Older iOS Version Tests
- run-test-ios-18:
context:
- slack-secrets
- run-test-ios-14:
context:
- slack-secrets
- run-test-ios-15:
context:
- slack-secrets
- run-test-ios-16:
context:
- slack-secrets
- run-test-ios-17:
context:
- slack-secrets

# Platform Runtime Tests
- run-test-macos:
context:
- slack-secrets
- run-test-tvos:
context:
- slack-secrets
- run-test-watchos:
context:
- slack-secrets

# RevenueCatUI Tests
- run-revenuecat-ui-ios-18
- spm-revenuecat-ui-ios-15:
context:
- slack-secrets
- spm-revenuecat-ui-ios-16:
context:
- slack-secrets
- run-revenuecat-ui-ios-17:
context:
- slack-secrets
- spm-revenuecat-ui-watchos:
context:
- slack-secrets

# Installation Tests
- installation-tests-all-but-carthage:
context:
- slack-secrets
- installation-tests-carthage:
context:
- slack-secrets

# SPM / Other Build Tests
- spm-release-build:
context:
- slack-secrets
- spm-release-build-xcode-14:
context:
- slack-secrets
- spm-release-build-xcode-15:
context:
- slack-secrets
- api-tests:
context:
- slack-secrets
- spm-receipt-parser:
context:
- slack-secrets
- deploy-purchase-tester:
dry_run: true
context:
- slack-secrets
- run-all-maestro-e2e-tests:
context:
- e2e-tests
- slack-secrets
- docs-build:
context:
Copy link
Copy Markdown

@cursor cursor Bot Feb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Workflow references many nonexistent job definitions

High Severity

The run-all-tests-auto workflow references at least 12 jobs that don't exist in the jobs: section. The existing config uses combined jobs like run-test-ios-18-and-17, run-test-ios-15-and-14, run-test-tvos-and-macos, and run-revenuecat-ui-ios-18-and-17, but the new workflow splits these into individual names (run-test-ios-18, run-test-ios-17, run-test-ios-15, run-test-ios-14, run-test-macos, run-test-tvos, run-revenuecat-ui-ios-18, run-revenuecat-ui-ios-17) that are never defined. Similarly, spm-release-build, spm-release-build-xcode-14, spm-release-build-xcode-15, and validate-package-swift don't exist as job definitions. This will cause CircleCI config validation failures, breaking CI for bot branches — or potentially all branches.

Additional Locations (2)

Fix in Cursor Fix in Web

- slack-secrets

Copy link
Copy Markdown

@cursor cursor Bot Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing revenuecat-admob-tests job in auto workflow

Low Severity

The run-all-tests workflow includes revenuecat-admob-tests in both its full test suite and all-tasks-passed requirements, but run-all-tests-auto omits it entirely. Since the PR intent is to mirror the same test suite, this job will be silently skipped for bot PRs, reducing test coverage.

Fix in Cursor Fix in Web

# =============================================================
# Summary gate: blocks merge until all jobs pass
# =============================================================
- all-tasks-passed:
requires:
# Reduced Test Suite
- lint
- check-api-changes
- validate-package-swift-5.8
- run-test-ios-26
- pod-lib-lint
- run-revenuecat-ui-ios-26
- emerge_purchases_ui_snapshot_tests
- emerge_binary_size_analysis
- build-tv-watch-mac-and-visionos
# Full Test Suite
- backend-integration-tests-SK1
- backend-integration-tests-SK2
- backend-integration-tests-custom-entitlements
- backend-integration-tests-other
- backend-integration-tests-offline
- run-test-ios-18
- run-test-ios-14
- run-test-ios-15
- run-test-ios-16
- run-test-ios-17
- run-test-macos
- run-test-tvos
- run-test-watchos
- spm-revenuecat-ui-ios-15
- spm-revenuecat-ui-ios-16
- run-revenuecat-ui-ios-17
- run-revenuecat-ui-ios-18
- spm-revenuecat-ui-watchos
- installation-tests-all-but-carthage
- installation-tests-carthage
- spm-release-build
- spm-release-build-xcode-14
- spm-release-build-xcode-15
- api-tests
- spm-receipt-parser
- deploy-purchase-tester
- run-all-maestro-e2e-tests
- docs-build
Copy link
Copy Markdown

@cursor cursor Bot Feb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicated workflow risks future test suite divergence

Low Severity

The run-all-tests-auto workflow is a near-complete copy (~175 lines) of the run-all-tests job list, differing only in the absence of the approve-full-tests gate. If a new test job is added to or removed from run-all-tests, it's easy to forget updating run-all-tests-auto (and its all-tasks-passed requires list), causing the bot-PR test suite to silently diverge. The codebase already uses YAML anchors for shared filter definitions; a similar approach for shared job lists could reduce this risk.

Additional Locations (1)

Fix in Cursor Fix in Web


deploy-tag:
when:
not:
Expand Down