Skip to content

[build] delete .skipped-tests file entries#17613

Merged
titusfortner merged 3 commits into
SeleniumHQ:trunkfrom
titusfortner:skipped_tests
Jun 2, 2026
Merged

[build] delete .skipped-tests file entries#17613
titusfortner merged 3 commits into
SeleniumHQ:trunkfrom
titusfortner:skipped_tests

Conversation

@titusfortner
Copy link
Copy Markdown
Member

@titusfortner titusfortner commented Jun 2, 2026

💥 What does this PR do?

Removes the .skipped-tests deselection list and the reference to it in the RBE CI build script.
I still think we shouldn't use this, but there's no harm keeping it and just removing references we don't need for now.

We should either tag targets as skip-rbe or guard them against RBE runs in test runners going forward

🔧 Implementation Notes

I didn't have to fix anything everything passes with the simple deletion.

🤖 AI assistance

  • AI assisted (complete below)
    • Tool(s): Claude Code
    • What was generated: the commit (file deletion + ci-build.sh edit) and this PR description
    • I reviewed all AI output and can explain the change

🔄 Types of changes

  • Cleanup (formatting, renaming)

@qodo-code-review
Copy link
Copy Markdown
Contributor

Review Summary by Qodo

Remove .skipped-tests file and CI build script reference

✨ Enhancement

Grey Divider

Walkthroughs

Description
• Remove .skipped-tests file containing hardcoded test exclusions
• Simplify CI build script by eliminating test deselection logic
• Remove unnecessary shellcheck disable comment from build script
• Tests should use skip-rbe tags or guards instead
Diagram
flowchart LR
  A["CI Build Script"] -->|previously used| B[".skipped-tests file"]
  B -->|contained| C["26 test exclusions"]
  A -->|now simplified| D["Direct test execution"]
  E["Future approach"] -->|use| F["skip-rbe tags or guards"]

Loading

Grey Divider

File Changes

1. scripts/github-actions/ci-build.sh ✨ Enhancement +1/-2

Simplify CI build script test execution

• Remove shellcheck disable comment for SC2046
• Simplify bazel test command by removing .skipped-tests file reference
• Remove test deselection logic that read and transformed skipped tests
• Streamline CI build to run all tests without exclusion list

scripts/github-actions/ci-build.sh


2. .skipped-tests ✨ Enhancement +0/-26

Remove hardcoded test exclusion list

• Delete entire file containing 26 hardcoded test exclusions
• Remove exclusions for Chrome, Edge, Firefox, Grid, Remote, and BiDi tests
• Eliminate centralized test deselection mechanism

.skipped-tests


Grey Divider

Qodo Logo

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented Jun 2, 2026

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (1) 📎 Requirement gaps (0) 🎨 UX issues (0)

Grey Divider


Action required

1. Deleted .skipped-tests breaks CI 📘 Rule violation ☼ Reliability
Description
This PR removes or empties the .skipped-tests file, but scripts/github-actions/ci-build.sh still
reads it (and expands it into the Bazel test command), which will either fail in CI when the file is
missing or silently turn the skip-list mechanism into a misleading no-op. This creates an
unvalidated missing-artifact dependency and increases maintenance risk because future changes may
incorrectly assume the skip list still controls test deselection.
Code

.skipped-tests[1]

Evidence
Compliance ID 10 requires CI/automation to validate required preconditions/artifacts and fail
safely; however, the CI build script still performs cat .skipped-tests (via command substitution
like $(cat .skipped-tests | tr '\n' ' ')) and appends the result to the bazel test invocation.
The PR changes .skipped-tests such that it is deleted or has all entries removed, so the script
either breaks at runtime due to the missing file (no precondition check) or continues running with
an empty expansion that makes the skip-list behavior ineffective and confusing to maintain.

scripts/github-actions/ci-build.sh[18-21]
scripts/github-actions/ci-build.sh[17-22]
Best Practice: Learned patterns

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The PR deletes or empties `.skipped-tests`, but `scripts/github-actions/ci-build.sh` still reads it and expands its contents into the Bazel test command, causing CI to either fail at runtime when the file is absent or silently make the skip-list mechanism a misleading no-op.

## Issue Context
Compliance ID 10 expects CI/automation to validate required artifacts and fail safely/clearly. The CI script currently uses command substitution (e.g., `$(cat .skipped-tests | tr '\n' ' ')`) to incorporate the skip list into the `bazel test` invocation, but the file no longer contains entries (or is removed), so the behavior becomes either a missing-artifact dependency with an unclear failure mode or a confusing, ineffective mechanism.

## Fix Focus Areas
- scripts/github-actions/ci-build.sh[17-22]
- .skipped-tests[1-26]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. Skipped tests now run ✓ Resolved 🐞 Bug ☼ Reliability
Description
scripts/github-actions/ci-build.sh now runs bazel test --config=rbe-ci //... without the prior
explicit deselection mechanism, so targets not tagged skip-rbe will start executing on RBE CI.
Many Selenium browser tests/macros add requires-network but do not add skip-rbe, while the repo
documents RBE network limitations, so this change can reintroduce CI flakes/failures.
Code

scripts/github-actions/ci-build.sh[R17-20]

Evidence
RBE CI now runs //... with no explicit per-target exclusions; RBE’s tag filter only excludes
skip-rbe, while browser tests are tagged requires-network via COMMON_TAGS and are not
automatically tagged skip-rbe, and the repo explicitly notes RBE network limitations. Therefore,
removing the deselection mechanism changes which tests run on RBE and can reintroduce
previously-avoided RBE incompatibilities.

scripts/github-actions/ci-build.sh[15-20]
.bazelrc.remote[35-37]
common/browsers.bzl[1-8]
java/private/selenium_test.bzl[22-56]
java/test/org/openqa/selenium/chrome/BUILD.bazel[26-47]
rb/spec/tests.bzl[181-257]
py/BUILD.bazel[64-76]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`ci-build.sh` now runs all tests (`//...`) under `--config=rbe-ci` without excluding the targets that were previously being deselected for RBE CI. Because `test:rbe` only filters `-skip-rbe`, any RBE-incompatible targets that are not tagged `skip-rbe` will now execute on RBE.

## Issue Context
- `test:rbe` uses `--test_tag_filters=-skip-rbe` and does not exclude `requires-network`.
- Selenium browser test rules commonly include `requires-network` (via `COMMON_TAGS`) and are frequently generated by macros (Java `selenium_test`, Ruby `rb_integration_test`), so the safest replacement for the removed deselection list is to tag the formerly-deselected targets (or split suites) with `skip-rbe`.

## Fix Focus Areas
- scripts/github-actions/ci-build.sh[15-20]
- .bazelrc.remote[35-37]
- common/browsers.bzl[1-8]
- java/private/selenium_test.bzl[22-56]
- java/test/org/openqa/selenium/chrome/BUILD.bazel[26-47]
- rb/spec/tests.bzl[181-257]
- py/BUILD.bazel[64-76]

### Acceptable fixes (pick one)
1) **Preferred:** Add `skip-rbe` tags to the specific targets that were previously deselected (may require splitting macros/suites so only those targets get the tag).
2) Temporarily **restore** an explicit exclusion mechanism in CI until tagging is completed.
3) If intentional, update RBE filtering logic to exclude the appropriate tag(s) (e.g., `requires-network`)—but only if that matches desired CI coverage.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

@selenium-ci selenium-ci added the B-build Includes scripting, bazel and CI integrations label Jun 2, 2026
@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented Jun 2, 2026

Code review by qodo was updated up to the latest commit fb69818

Comment thread .skipped-tests
@titusfortner titusfortner merged commit dd6ec2a into SeleniumHQ:trunk Jun 2, 2026
18 checks passed
@titusfortner titusfortner changed the title [build] Remove .skipped-tests file and its CI build script reference [build] delete .skipped-tests file entries Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

B-build Includes scripting, bazel and CI integrations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants