Skip to content

Commit f6ec9be

Browse files
committed
Pluggable parallel-test-runner adapter interface
Coordination with parallel test runners (picking the "final" worker, waiting for siblings, knowing how many resultsets to expect) used to hard-code the parallel_tests gem. Env-var-only runners like parallel_rspec hit the wrong branch of `final_result_process?` and every worker thought it was the final one. They clobbered each other's resultsets. Introduce `SimpleCov::ParallelAdapters`, a small registry and selection layer with a four-method contract (`active?`, `first_worker?`, `wait_for_siblings`, `expected_worker_count`). `ParallelTestsAdapter` wraps the historical gem API. `GenericAdapter` handles the env-var convention without needing any specific gem. Adapters are tried in registration order, and the first whose `active?` returns true is chosen. parallel_rspec works out of the box, and users with custom runners can plug in their own adapter via: SimpleCov::ParallelAdapters.register MyAdapter Resolves #1065.
1 parent 98a47e0 commit f6ec9be

9 files changed

Lines changed: 622 additions & 72 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ Unreleased
2525
* `# :nocov:` toggle comments (and the configurable `SimpleCov.nocov_token` / `SimpleCov.skip_token`) are deprecated in favor of the new `# simplecov:disable` / `# simplecov:enable` directives. Each file that still uses `# :nocov:` emits a one-time deprecation warning to stderr at load time pointing at the recommended replacement, and any call to `SimpleCov.nocov_token` or `SimpleCov.skip_token` (getter or setter) likewise warns. The directive will be removed in a future release.
2626

2727
## Enhancements
28+
* Added `SimpleCov::ParallelAdapters` — a pluggable adapter interface for parallel test runners. SimpleCov's coordination with parallel test runners (deciding which worker does final-result work, waiting for siblings, knowing how many resultsets to expect) now routes through an adapter chain rather than hard-coding the `parallel_tests` gem's API. Two adapters ship: `ParallelTestsAdapter` wraps the historical grosser/parallel_tests gem (precise, gem-API-based); `GenericAdapter` handles any runner that follows the `TEST_ENV_NUMBER` / `PARALLEL_TEST_GROUPS` env-var convention without shipping a Ruby API. The practical impact: **parallel_rspec (and any similar env-var-only runner) now works out of the box** — previously every worker thought it was the "final" one and they clobbered each other's resultsets. Custom runners can register their own adapter via `SimpleCov::ParallelAdapters.register MyAdapter`, where `MyAdapter` subclasses `SimpleCov::ParallelAdapters::Base` and overrides the four contract methods (`active?`, `first_worker?`, `wait_for_siblings`, `expected_worker_count`). See #1065.
2829
* Added `SimpleCov.ignore_branches` for opting out of synthetic `:else` branches that Ruby's `Coverage` library reports for constructs with no literal `else` keyword — exhaustive `case/in` pattern matches, `case/when` without `else`, `||=` / `&&=`, and `if` / `unless` without `else`. Variadic; only `:implicit_else` is supported today, with room for future synthetic branch types. Calling it without (or before) `enable_coverage :branch` is harmless — the setting is stored and applies once branch coverage is enabled. Explicit `else` arms still count. See #1033.
2930
* Added `SimpleCov.cover` for declaring a positive coverage scope (the long-requested allowlist counterpart to `add_filter`). Accepts string globs, Regexps, blocks, or arrays of those; multiple calls union. When any `cover` matcher is configured the report drops every source file that doesn't match at least one of them, and string-glob matchers also expand on disk so files that exist but were never required during the run still appear in the report (at 0% coverage). Resolves the long-standing requests in #696 and #869. The companion `SimpleCov.no_default_skips` opts out of the filters that `SimpleCov.start` installs (hidden files, `vendor/bundle/`, test directories) so users who want to opt out wholesale don't have to call `clear_filters` themselves.
3031
* `SimpleCov.formatter false` (and the equivalent `SimpleCov.formatters []`) now opts out of formatting entirely instead of raising `ConfigurationError`. `SimpleCov::Result#format!` returns `nil` when no formatter is configured. Intended for worker processes in big parallel CI runs (hundreds of jobs) where only a final `SimpleCov.collate` step needs a report — every other worker just drops its `.resultset.json` and exits without paying for HTML or multi-formatter output. See #964.

README.md

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1215,6 +1215,63 @@ $ simplecov uncovered --threshold 90 --top 5
12151215
`--json` emits the same rows as a JSON array (an empty array when
12161216
nothing is below the threshold), useful for piping into a CI gate.
12171217

1218+
## Parallel-test-runner adapters
1219+
1220+
SimpleCov coordinates with parallel test runners through a small pluggable
1221+
adapter interface (`SimpleCov::ParallelAdapters`). Two adapters ship out of
1222+
the box:
1223+
1224+
- **`ParallelTestsAdapter`** — wraps the
1225+
[grosser/parallel_tests](https://github.com/grosser/parallel_tests) gem
1226+
and uses its `ParallelTests.first_process?` /
1227+
`ParallelTests.wait_for_other_processes_to_finish` APIs for precise
1228+
worker coordination.
1229+
- **`GenericAdapter`** — catch-all for any runner that follows the
1230+
`TEST_ENV_NUMBER` / `PARALLEL_TEST_GROUPS` env-var convention but
1231+
doesn't ship a Ruby API (parallel_rspec, knapsack-style splitters,
1232+
custom CI sharding scripts). Activates when `TEST_ENV_NUMBER` is set
1233+
and no more-specific adapter is.
1234+
1235+
Adapters are tried in registration order; the first whose `active?`
1236+
returns `true` is chosen for the run. With both built-ins this means
1237+
parallel_tests users get the precise gem-based path, and parallel_rspec
1238+
(or any env-var-only runner) gets the polling-based fallback without any
1239+
configuration change. See #1065.
1240+
1241+
### Registering a custom adapter
1242+
1243+
If you use a parallel runner that uses different env vars or has its own
1244+
synchronization API, define a class that inherits from
1245+
`SimpleCov::ParallelAdapters::Base` and register it:
1246+
1247+
```ruby
1248+
# In your spec_helper.rb / test_helper.rb (before SimpleCov.start)
1249+
class MyRunnerAdapter < SimpleCov::ParallelAdapters::Base
1250+
def self.active?
1251+
!ENV["MY_RUNNER_PID"].nil?
1252+
end
1253+
1254+
def self.first_worker?
1255+
ENV["MY_RUNNER_PID"].to_i == 1
1256+
end
1257+
1258+
def self.wait_for_siblings
1259+
MyRunner.barrier! # if your runner provides a sync primitive
1260+
end
1261+
1262+
def self.expected_worker_count
1263+
ENV["MY_RUNNER_WORKERS"].to_i
1264+
end
1265+
end
1266+
1267+
SimpleCov::ParallelAdapters.register MyRunnerAdapter
1268+
```
1269+
1270+
Custom adapters are inserted at the front of the selection chain so they
1271+
take precedence over the built-ins. `Base` provides safe no-op defaults
1272+
for any method you don't override (single-process semantics: `active?`
1273+
returns `false`, `first_worker?` returns `true`, etc.).
1274+
12181275
## Merging resultsets from parallel CI workers
12191276

12201277
CI matrices that produce one `.resultset.json` per worker can stitch

lib/simplecov.rb

Lines changed: 30 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,11 @@ def start_tracking
131131
::Process.respond_to?(:_fork)
132132
# simplecov:enable
133133

134-
make_parallel_tests_available
134+
# Trigger adapter selection now so the (possibly lazy)
135+
# parallel_tests gem load happens at start_tracking time rather
136+
# than mid-suite. `current` is memoized; subsequent calls are
137+
# cheap.
138+
SimpleCov::ParallelAdapters.current
135139

136140
@result = nil
137141
self.pid = Process.pid
@@ -407,53 +411,44 @@ def result_exit_status(result)
407411
# @api private
408412
#
409413
def final_result_process?
410-
return true unless defined?(ParallelTests) && ENV["TEST_ENV_NUMBER"]
411-
412-
# Pick the *first* started process to do the final-result work, not
413-
# the last. Two reasons: (1) the parallel_tests README explicitly
414-
# recommends `ParallelTests.first_process?` for "run something
415-
# once after every worker finishes" hooks, so user code that
416-
# implements its own `wait_for_other_processes_to_finish` (e.g. in
417-
# an RSpec `after(:suite)`) overwhelmingly waits in the first
418-
# process — picking the same side avoids the cross-process
419-
# deadlock #922 reported. (2) `first_process?` naturally handles
420-
# the PARALLEL_TEST_GROUPS=1 case: parallel_tests sets the first
421-
# worker's TEST_ENV_NUMBER to "" and `first_process?` tests for
422-
# that empty string, so a single-group run correctly returns true
423-
# without needing the previous `GROUPS.to_i <= 1` workaround
424-
# (#1066). Users who waited in the LAST process now hit the
425-
# symmetric deadlock and must migrate to `first_process?`; the
426-
# CHANGELOG calls this out.
427-
ParallelTests.first_process?
414+
adapter = SimpleCov::ParallelAdapters.current
415+
return true unless adapter
416+
417+
adapter.first_worker?
428418
end
429419

430420
#
431421
# @api private
432422
#
433423
# simplecov:disable
434-
# Methods below only fire under parallel_tests; not reachable from a
435-
# single-process rspec run. Cucumber's test_projects exercise the
436-
# parallel_tests integration end-to-end in subprocesses, but those
437-
# subprocesses don't merge their Coverage data back into the parent
438-
# this dogfood report measures.
424+
# Methods below only fire under a parallel test runner; not reachable
425+
# from a single-process rspec run. Cucumber's test_projects exercise
426+
# the parallel_tests integration end-to-end in subprocesses, but
427+
# those subprocesses don't merge their Coverage data back into the
428+
# parent this dogfood report measures.
439429
def wait_for_other_processes
440-
return unless defined?(ParallelTests) && final_result_process?
430+
adapter = SimpleCov::ParallelAdapters.current
431+
return unless adapter && final_result_process?
441432

442-
ParallelTests.wait_for_other_processes_to_finish
433+
# Native synchronization first (adapters that wrap a runner with a
434+
# real "wait" primitive — parallel_tests' `wait_for_other_processes_to_finish`
435+
# — implement this; adapters without a native API no-op and rely
436+
# on the polling fallback below).
437+
adapter.wait_for_siblings
443438

444-
# ParallelTests signals "done" before at_exit handlers finish, so other
445-
# processes may still be writing their results. Poll the resultset until
446-
# all parallel groups have reported or a timeout is reached.
447-
wait_for_parallel_results
439+
# The native wait can return before sibling at_exit handlers finish
440+
# writing resultsets, and adapters without a native wait have nothing
441+
# else. Either way, poll the resultset cache until all expected
442+
# workers have reported or a timeout is reached.
443+
wait_for_parallel_results(adapter.expected_worker_count)
448444
end
449445
# simplecov:enable
450446

451447
# @api private
452-
def wait_for_parallel_results
453-
expected = ENV["PARALLEL_TEST_GROUPS"]&.to_i
454-
return unless expected && expected > 1 # simplecov:disable branch — only false in real parallel_tests run
448+
def wait_for_parallel_results(expected)
449+
return unless expected > 1 # simplecov:disable branch — only false in real parallel runs
455450

456-
# simplecov:disable — only fires when ENV is set with >1 group
451+
# simplecov:disable — only fires under multi-worker parallel runs
457452
deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + 10
458453
loop do
459454
resultset = SimpleCov::ResultMerger.read_resultset
@@ -622,35 +617,6 @@ def defer_to_minitest_after_run
622617
Minitest.after_run { SimpleCov.at_exit_behavior }
623618
end
624619

625-
# Auto-require `parallel_tests` when it's installed AND the env vars
626-
# it sets are present, so `wait_for_other_processes` and friends can
627-
# call `ParallelTests.first_process?` later. `parallel_tests` is an
628-
# optional dependency (see https://github.com/grosser/parallel_tests/issues/772),
629-
# and `TEST_ENV_NUMBER` / `PARALLEL_TEST_GROUPS` are commonly set for
630-
# other reasons (custom subprocess coordination, CI sharding), so a
631-
# missing gem is treated as "user isn't using parallel_tests" — silently
632-
# skip rather than warn. Users who want to override the auto-detect can
633-
# set `SimpleCov.parallel_tests true` (force on) or `false` (force off).
634-
# See #1018.
635-
def make_parallel_tests_available
636-
return if defined?(ParallelTests) # simplecov:disable — only true after a previous load
637-
return if SimpleCov.parallel_tests == false # simplecov:disable — only fires when user opts out
638-
# simplecov:disable — false outside parallel_tests
639-
return unless SimpleCov.parallel_tests || probably_running_parallel_tests?
640-
641-
# simplecov:disable — only fires under a real parallel_tests setup
642-
require "parallel_tests"
643-
rescue LoadError
644-
# The gem isn't installed; the env vars were set for some other
645-
# reason. Stay quiet — warning here regressed users who use those
646-
# env vars for their own subprocess coordination (see #1018).
647-
# simplecov:enable
648-
end
649-
650-
def probably_running_parallel_tests?
651-
ENV.fetch("TEST_ENV_NUMBER", nil) && ENV.fetch("PARALLEL_TEST_GROUPS", nil)
652-
end
653-
654620
# JRuby coverage data is unreliable unless full-trace mode is enabled.
655621
# @see https://github.com/jruby/jruby/issues/1196
656622
# @see https://github.com/simplecov-ruby/simplecov/issues/420
@@ -690,6 +656,7 @@ def warn_if_jruby_full_trace_disabled
690656
require_relative "simplecov/last_run"
691657
require_relative "simplecov/lines_classifier"
692658
require_relative "simplecov/result_merger"
659+
require_relative "simplecov/parallel_adapters"
693660
require_relative "simplecov/command_guesser"
694661
require_relative "simplecov/version"
695662
require_relative "simplecov/result_adapter"

lib/simplecov/parallel_adapters.rb

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# frozen_string_literal: true
2+
3+
require_relative "parallel_adapters/base"
4+
require_relative "parallel_adapters/parallel_tests"
5+
require_relative "parallel_adapters/generic"
6+
7+
module SimpleCov
8+
# Registry + selection for parallel-test-runner adapters. An adapter
9+
# answers a small fixed set of questions on SimpleCov's behalf:
10+
#
11+
# - `active?` — are WE the runner in charge for this process?
12+
# - `first_worker?` — should this process do the final-result work?
13+
# - `wait_for_siblings` — block until siblings finish (optional)
14+
# - `expected_worker_count` — how many workers total
15+
#
16+
# `SimpleCov::ParallelAdapters::Base` provides safe no-op defaults; two
17+
# adapters ship out of the box:
18+
#
19+
# - `ParallelTestsAdapter` — wraps the grosser/parallel_tests gem
20+
# (precise sync + first-process detection via the gem's own API).
21+
# - `GenericAdapter` — env-var-only detection for runners that follow
22+
# the parallel_tests `TEST_ENV_NUMBER` convention but don't ship a
23+
# Ruby API (parallel_rspec, custom CI sharding, knapsack-style
24+
# splitters). See https://github.com/simplecov-ruby/simplecov/issues/1065.
25+
#
26+
# Users can plug in additional adapters:
27+
#
28+
# SimpleCov::ParallelAdapters.register MyRunnerAdapter
29+
#
30+
# An adapter just needs to be a class responding to the four methods
31+
# above. Subclass `SimpleCov::ParallelAdapters::Base` to inherit the
32+
# no-op defaults and override only what you need (the contract methods
33+
# are defined as class methods, so plain inheritance is what carries
34+
# them through; `extend Base` won't pick them up).
35+
module ParallelAdapters
36+
module_function
37+
38+
# Adapters in selection order. ParallelTestsAdapter first (most
39+
# specific — uses the gem's own API when the gem is loaded); then
40+
# GenericAdapter as the env-var fallback. User-registered adapters
41+
# are prepended (#register puts new entries at the front) so
42+
# downstream code can override the built-ins by registering a more
43+
# specific match.
44+
def adapters
45+
@adapters ||= [ParallelTestsAdapter, GenericAdapter]
46+
end
47+
48+
# Register a custom adapter. Newly registered adapters are inserted
49+
# at the front of the selection list so a custom adapter for a
50+
# specific runner takes precedence over the built-in ParallelTests
51+
# and Generic adapters.
52+
#
53+
# class MyRunnerAdapter < SimpleCov::ParallelAdapters::Base
54+
# def self.active? = ENV["MY_RUNNER_PID"]
55+
# def self.first_worker? = ENV["MY_RUNNER_PID"].to_i == 1
56+
# def self.expected_worker_count = ENV["MY_RUNNER_WORKERS"].to_i
57+
# end
58+
#
59+
# SimpleCov::ParallelAdapters.register MyRunnerAdapter
60+
def register(adapter)
61+
reset_current!
62+
adapters.unshift(adapter) unless adapters.include?(adapter)
63+
adapter
64+
end
65+
66+
# The adapter SimpleCov should consult for this process — the first
67+
# registered adapter whose `active?` returns true. Returns nil when
68+
# no adapter is active (i.e., we're not running under any recognized
69+
# parallel test runner), in which case the caller should treat the
70+
# process as single-worker.
71+
def current
72+
return @current if defined?(@current)
73+
74+
@current = adapters.find(&:active?)
75+
end
76+
77+
# Clear the memoized `current` selection. Primarily for tests that
78+
# mutate env vars between examples; production runs are single-shot.
79+
def reset_current!
80+
remove_instance_variable(:@current) if defined?(@current)
81+
end
82+
end
83+
end
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# frozen_string_literal: true
2+
3+
module SimpleCov
4+
module ParallelAdapters
5+
# Default no-op implementations for a parallel-test-runner adapter.
6+
# Real adapters subclass and override what they need; everything else
7+
# falls back to "behave like a single-process run."
8+
#
9+
# Adapters are classes (used as singletons, never instantiated) — they
10+
# answer a small fixed set of questions about whether THIS worker
11+
# process is the one that should do final-result work, and provide an
12+
# optional hook for waiting on sibling workers.
13+
#
14+
# @see SimpleCov::ParallelAdapters for the registry and selection.
15+
class Base
16+
class << self
17+
# Should this adapter be selected for the current process? Adapters
18+
# are tried in registration order; the first one whose `active?`
19+
# returns true is chosen. Inactive adapters return `false`.
20+
def active?
21+
false
22+
end
23+
24+
# Among the parallel workers in this run, should THIS worker do
25+
# the final-result work (wait for siblings, merge resultsets,
26+
# run threshold checks, format the report)? Default is `true`
27+
# for the single-process case.
28+
def first_worker?
29+
true
30+
end
31+
32+
# Optional: block until sibling workers have finished writing
33+
# their resultsets. An adapter that wraps a parallel-test runner
34+
# with a native synchronization primitive (e.g., `parallel_tests`'s
35+
# `wait_for_other_processes_to_finish`) implements this for
36+
# lower latency; otherwise SimpleCov polls the resultset cache
37+
# as a fallback (see `SimpleCov.wait_for_parallel_results`).
38+
def wait_for_siblings
39+
# No-op default; polling fallback handles correctness.
40+
end
41+
42+
# How many parallel workers are participating in this run. Used
43+
# by the polling fallback to know how many resultset entries to
44+
# expect. Defaults to 1 (single-process).
45+
def expected_worker_count
46+
1
47+
end
48+
end
49+
end
50+
end
51+
end
Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# frozen_string_literal: true
2+
3+
require_relative "base"
4+
5+
module SimpleCov
6+
module ParallelAdapters
7+
# Catch-all adapter for parallel test runners that follow the
8+
# `TEST_ENV_NUMBER` / `PARALLEL_TEST_GROUPS` env-var convention but
9+
# don't ship a Ruby API for SimpleCov to hook (parallel_rspec,
10+
# knapsack-style splitters, custom CI sharding scripts). Activates
11+
# when `TEST_ENV_NUMBER` is set; doesn't require any specific gem to
12+
# be loaded.
13+
#
14+
# Heuristic for `first_worker?`: the worker whose `TEST_ENV_NUMBER`
15+
# is `""` (parallel_tests/parallel_rspec convention) or `"1"`
16+
# (zero-based runners that start at 1). Any other value is treated
17+
# as a non-first worker.
18+
#
19+
# `wait_for_siblings` is inherited from Base as a no-op — without a
20+
# runner-provided API the only synchronization available is polling
21+
# the resultset cache, which `SimpleCov.wait_for_parallel_results`
22+
# does after the no-op returns.
23+
class GenericAdapter < Base
24+
class << self
25+
def active?
26+
!ENV.fetch("TEST_ENV_NUMBER", nil).nil?
27+
end
28+
29+
# parallel_tests sets the first worker's TEST_ENV_NUMBER to "";
30+
# parallel_rspec inherits that. Runners that number from 1 use
31+
# "1" for the first worker. Both shapes match.
32+
def first_worker?
33+
["", "1"].include?(ENV.fetch("TEST_ENV_NUMBER", nil))
34+
end
35+
36+
def expected_worker_count
37+
ENV["PARALLEL_TEST_GROUPS"]&.to_i || 1
38+
end
39+
end
40+
end
41+
end
42+
end

0 commit comments

Comments
 (0)