dev-cmd/tests: add except flag to skip tests#23089
Conversation
There was a problem hiding this comment.
Pull request overview
Adds an --except flag to brew tests to allow skipping specific test specs or directories, mirroring the existing --only selection behavior and updating Sorbet RBI accordingly.
Changes:
- Add
--except=CLI flag tobrew testsand exclude matching spec files/directories from the run list. - Update option conflict rules to prevent combining
--exceptwith--changed/--only. - Extend the Sorbet DSL RBI for
Homebrew::DevCmd::Tests::Argswith the newexceptaccessor.
Reviewed changes
Copilot reviewed 1 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| Library/Homebrew/dev-cmd/tests.rb | Adds --except flag parsing and applies exclusions to the computed RSpec file list. |
| Library/Homebrew/sorbet/rbi/dsl/homebrew/dev_cmd/tests.rbi | Updates the generated RBI to include args.except typing. |
Files not reviewed (1)
- Library/Homebrew/sorbet/rbi/dsl/homebrew/dev_cmd/tests.rbi: File type not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
| if (except = args.except) | ||
| except_tests = except.split(",").flat_map do |test_name| | ||
| tests = Dir.glob("test/{#{test_name},#{test_name}/**/*}_spec.rb") | ||
| raise UsageError, "Invalid `--except` argument: #{test_name}" if tests.blank? | ||
|
|
||
| tests | ||
| end | ||
| files -= except_tests | ||
| end |
| if (except = args.except) | ||
| except_tests = except.split(",").flat_map do |test_name| | ||
| tests = Dir.glob("test/{#{test_name},#{test_name}/**/*}_spec.rb") | ||
| raise UsageError, "Invalid `--except` argument: #{test_name}" if tests.blank? | ||
|
|
||
| tests | ||
| end | ||
| files -= except_tests | ||
| end |
MikeMcQuaid
left a comment
There was a problem hiding this comment.
Thanks! I'm not sure this really makes sense as an approach, though. Instead can we skip those environments/tests where we're getting failures.
It also might be that it doesn't make sense to run brew tests for Portable Ruby when building in non-Tier 1 environments or we use --only to run a small subset instead.
|
Closing in favor of: |
brewcommands to reproduce the bug?brew lgtm(style, typechecking and tests) locally?Will likely use this to bypass any remaining test failures in portable Ruby PR to fix up our workflows. Many failures seem to be environment differences that are difficult to reconcile (e.g. Tier 3 vs Tier 1, container setup, etc).
Flag name was chosen based on
brew auditwhich also has--onlyand--except.The handling is based as
--onlyflag (other than not providing line number support):brew/Library/Homebrew/dev-cmd/tests.rb
Lines 76 to 82 in 85c5ba0
Needed to manually update RBI as
brew tc --updatekept failing with stacktrace.