Model RepeatTestSuite as a TestSuite, mirroring DataProviderTestSuite

Previously, RepeatTestSuite was declared as a leaf Test even though it structurally represents a group of test cases (the N repetitions of a test method). This produced two inconsistencies:

* TestSuite::addTest() registered the suite under the first repetition's id (Class::method (repetition 1 of N)) rather than the group id, so other repetitions appeared as no members of the group.

* Event\TestSuite\TestSuiteBuilder::process() treated the suite as a leaf and surfaced only tests[0]->valueObjectForEvents() into the parent's TestCollection. The collection therefore had one entry per repetition group while count() reported N: the two disagreed.

Both issues are removed by treating RepeatTestSuite the same way DataProviderTestSuite is treated:

* RepeatTestSuite now extends Framework\TestSuite and is constructed via RepeatTestSuite::fromTests($name, $tests, $failureThreshold). It overrides run() to retain the failure-threshold/abort semantics and delegates provides(), requires(), sortId(), and setDependencies() to its first child / all children.

* A dedicated event-level value object TestSuiteForRepeatedTestMethod is introduced alongside TestSuiteForTestMethodWithDataProvider. It exposes className(), methodName(), file(), line(), and an isForRepeatedTestMethod() predicate on the base Event\TestSuite\TestSuite.

* Event\TestSuite\TestSuiteBuilder::from() detects RepeatTestSuite and returns the new value object; process() recurses through it like any other framework TestSuite, so each repetition's TestMethod event value object now appears in the parent's TestCollection.

* The special-case branches in Framework\TestSuite::addTest() and Runner\Filter\NameFilterIterator::accept() are removed. The inherited instanceof self branch handles registration; the existing TestSuite branch in the filter recurses into children, which then match individually.

* Runner\TestResult\Collector::testSuiteFinished() learns about the new value object and, when no repetition of the method failed, records the method as passed via PassedTests::testMethodPassed(), mirroring the data-provider handler.

As a consequence of RepeatTestSuite being a real TestSuite, Test Suite Started / Test Suite Finished events are now emitted around each repetition group. The JUnit XML logger correspondingly produces a nested <testsuite> element per repeated method, matching how it already renders data-provider suites.

Author: sebastianbergmann

src/Event/Value/TestSuite/TestSuite.php

@@ -78,4 +78,12 @@ public function isForTestMethodWithDataProvider(): bool
     {
         return false;
     }
+
+    /**
+     * @phpstan-assert-if-true TestSuiteForRepeatedTestMethod $this
+     */
+    public function isForRepeatedTestMethod(): bool
+    {
+        return false;
+    }
 }

src/Event/Value/TestSuite/TestSuiteBuilder.php

@@ -14,6 +14,8 @@
 use function count;
 use function explode;
 use function method_exists;
+use function strpos;
+use function substr;
 use PHPUnit\Event\Code\Test;
 use PHPUnit\Event\Code\TestCollection;
 use PHPUnit\Event\RuntimeException;
@@ -67,6 +69,44 @@ public static function from(FrameworkTestSuite $testSuite): TestSuite
             );
         }
 
+        if ($testSuite instanceof RepeatTestSuite) {
+            $name = $testSuite->name();
+
+            $separatorPosition = strpos($name, '::');
+
+            assert($separatorPosition !== false);
+
+            $className  = substr($name, 0, $separatorPosition);
+            $methodName = substr($name, $separatorPosition + 2);
+
+            $hashPosition = strpos($methodName, '#');
+
+            if ($hashPosition !== false) {
+                $methodName = substr($methodName, 0, $hashPosition);
+            }
+
+            assert($className !== '' && class_exists($className));
+            assert($methodName !== '' && method_exists($className, $methodName));
+
+            $reflector = new ReflectionMethod($className, $methodName);
+
+            $file = $reflector->getFileName();
+            $line = $reflector->getStartLine();
+
+            assert($file !== false);
+            assert($line !== false);
+
+            return new TestSuiteForRepeatedTestMethod(
+                $name,
+                $testSuite->count(),
+                TestCollection::fromArray($tests),
+                $className,
+                $methodName,
+                $file,
+                $line,
+            );
+        }
+
         if ($testSuite->isForTestClass()) {
             $testClassName = $testSuite->name();
 
@@ -108,12 +148,6 @@ private static function process(FrameworkTestSuite $testSuite, array &$tests): v
                 continue;
             }
 
-            if ($test instanceof RepeatTestSuite) {
-                $tests[] = $test->valueObjectForEvents();
-
-                continue;
-            }
-
             if ($test instanceof TestCase || $test instanceof PhptTestCase) {
                 $tests[] = $test->valueObjectForEvents();
             }

src/Event/Value/TestSuite/TestSuiteForRepeatedTestMethod.php

@@ -0,0 +1,78 @@
+<?php declare(strict_types=1);
+/*
+ * This file is part of PHPUnit.
+ *
+ * (c) Sebastian Bergmann <sebastian@phpunit.de>
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+namespace PHPUnit\Event\TestSuite;
+
+use PHPUnit\Event\Code\TestCollection;
+
+/**
+ * @immutable
+ *
+ * @no-named-arguments Parameter names are not covered by the backward compatibility promise for PHPUnit
+ */
+final readonly class TestSuiteForRepeatedTestMethod extends TestSuite
+{
+    /**
+     * @var class-string
+     */
+    private string $className;
+
+    /**
+     * @var non-empty-string
+     */
+    private string $methodName;
+    private string $file;
+    private int $line;
+
+    /**
+     * @param non-empty-string $name
+     * @param class-string     $className
+     * @param non-empty-string $methodName
+     */
+    public function __construct(string $name, int $size, TestCollection $tests, string $className, string $methodName, string $file, int $line)
+    {
+        parent::__construct($name, $size, $tests);
+
+        $this->className  = $className;
+        $this->methodName = $methodName;
+        $this->file       = $file;
+        $this->line       = $line;
+    }
+
+    /**
+     * @return class-string
+     */
+    public function className(): string
+    {
+        return $this->className;
+    }
+
+    /**
+     * @return non-empty-string
+     */
+    public function methodName(): string
+    {
+        return $this->methodName;
+    }
+
+    public function file(): string
+    {
+        return $this->file;
+    }
+
+    public function line(): int
+    {
+        return $this->line;
+    }
+
+    public function isForRepeatedTestMethod(): true
+    {
+        return true;
+    }
+}

src/Framework/RepeatTestSuite.php

@@ -9,54 +9,73 @@
  */
 namespace PHPUnit\Framework;
 
-use function count;
+use function assert;
 use PHPUnit\Event;
-use PHPUnit\Event\Code\TestMethod;
 use PHPUnit\Event\NoPreviousThrowableException;
+use PHPUnit\TestRunner\TestResult\Facade as TestResultFacade;
+use SebastianBergmann\CodeCoverage\InvalidArgumentException;
+use SebastianBergmann\CodeCoverage\UnintentionallyCoveredCodeException;
 
 /**
  * @no-named-arguments Parameter names are not covered by the backward compatibility promise for PHPUnit
  *
  * @internal This class is not covered by the backward compatibility promise for PHPUnit
  */
-final class RepeatTestSuite implements Reorderable, SelfDescribing, Test
+final class RepeatTestSuite extends TestSuite
 {
-    /**
-     * @var non-empty-list<TestCase>
-     */
-    private array $tests;
-
     /**
      * @var positive-int
      */
-    private int $failureThreshold;
+    private int $failureThreshold = 1;
 
     /**
+     * @param non-empty-string         $name
      * @param non-empty-list<TestCase> $tests
      * @param positive-int             $failureThreshold
      */
-    public function __construct(array $tests, int $failureThreshold = 1)
+    public static function fromTests(string $name, array $tests, int $failureThreshold): self
     {
-        $this->tests            = $tests;
-        $this->failureThreshold = $failureThreshold;
-    }
+        $suite = self::empty($name);
 
-    public function count(): int
-    {
-        return count($this->tests);
+        $suite->failureThreshold = $failureThreshold;
+
+        foreach ($tests as $test) {
+            $suite->addTest($test);
+        }
+
+        return $suite;
     }
 
     /**
-     * @throws Event\InvalidArgumentException
+     * @throws Event\RuntimeException
      * @throws Exception
+     * @throws InvalidArgumentException
      * @throws NoPreviousThrowableException
+     * @throws UnintentionallyCoveredCodeException
      */
     public function run(): void
     {
+        if ($this->isEmpty()) {
+            return;
+        }
+
+        $emitter                       = Event\Facade::emitter();
+        $testSuiteValueObjectForEvents = Event\TestSuite\TestSuiteBuilder::from($this);
+
+        $emitter->testSuiteStarted($testSuiteValueObjectForEvents);
+
         $failureCount         = 0;
         $lastFailedRepetition = 0;
 
-        foreach ($this->tests as $test) {
+        foreach ($this as $test) {
+            assert($test instanceof TestCase);
+
+            if (TestResultFacade::shouldStop()) {
+                $emitter->testRunnerExecutionAborted();
+
+                break;
+            }
+
             if ($failureCount >= $this->failureThreshold) {
                 $test->markSkippedForRepeatAbort($lastFailedRepetition);
 
@@ -70,57 +89,61 @@ public function run(): void
                 $lastFailedRepetition = $test->repetition();
             }
         }
+
+        $emitter->testSuiteFinished($testSuiteValueObjectForEvents);
     }
 
-    public function sortId(): string
+    /**
+     * @param list<ExecutionOrderDependency> $dependencies
+     */
+    public function setDependencies(array $dependencies): void
     {
-        return $this->tests[0]->sortId();
+        foreach ($this->tests() as $test) {
+            assert($test instanceof TestCase);
+
+            $test->setDependencies($dependencies);
+        }
     }
 
     /**
      * @return list<ExecutionOrderDependency>
      */
     public function provides(): array
     {
-        return $this->tests[0]->provides();
+        $tests = $this->tests();
+
+        if ($tests === []) {
+            return [];
+        }
+
+        assert($tests[0] instanceof TestCase);
+
+        return $tests[0]->provides();
     }
 
     /**
      * @return list<ExecutionOrderDependency>
      */
     public function requires(): array
     {
-        return $this->tests[0]->requires();
-    }
+        $tests = $this->tests();
 
-    /**
-     * @param list<ExecutionOrderDependency> $dependencies
-     */
-    public function setDependencies(array $dependencies): void
-    {
-        foreach ($this->tests as $test) {
-            $test->setDependencies($dependencies);
+        if ($tests === []) {
+            return [];
         }
-    }
 
-    /**
-     * @return non-empty-string
-     */
-    public function name(): string
-    {
-        return $this->tests[0]::class . '::' . $this->tests[0]->nameWithDataSet();
-    }
+        assert($tests[0] instanceof TestCase);
 
-    /**
-     * @return non-empty-string
-     */
-    public function toString(): string
-    {
-        return $this->name();
+        return $tests[0]->requires();
     }
 
-    public function valueObjectForEvents(): TestMethod
+    public function sortId(): string
     {
-        return $this->tests[0]->valueObjectForEvents();
+        $tests = $this->tests();
+
+        assert($tests !== []);
+        assert($tests[0] instanceof TestCase);
+
+        return $tests[0]->sortId();
     }
 }

src/Framework/TestBuilder.php

@@ -192,7 +192,11 @@ private function buildDataProviderTestSuite(string $methodName, string $classNam
                 }
 
                 $dataProviderTestSuite->addTest(
-                    new RepeatTestSuite($tests, $failureThreshold),
+                    RepeatTestSuite::fromTests(
+                        $className . '::' . $methodName . '#' . $_dataName,
+                        $tests,
+                        $failureThreshold,
+                    ),
                     $groups,
                 );
             } else {
@@ -240,7 +244,11 @@ private function buildRepeatTestSuite(string $className, string $methodName, int
             $tests[] = $test;
         }
 
-        return new RepeatTestSuite($tests, $failureThreshold);
+        return RepeatTestSuite::fromTests(
+            $className . '::' . $methodName,
+            $tests,
+            $failureThreshold,
+        );
     }
 
     /**

src/Framework/TestSuite.php

@@ -158,28 +158,6 @@ public function addTest(Test $test, array $groups = []): void
             return;
         }
 
-        if ($test instanceof RepeatTestSuite) {
-            $this->tests[] = $test;
-
-            $this->clearCaches();
-
-            $id = $test->valueObjectForEvents()->id();
-
-            if ($this->containsOnlyVirtualGroups($groups)) {
-                $groups[] = 'default';
-            }
-
-            foreach ($groups as $group) {
-                if (!isset($this->groups[$group])) {
-                    $this->groups[$group] = [$id];
-                } else {
-                    $this->groups[$group][] = $id;
-                }
-            }
-
-            return;
-        }
-
         assert($test instanceof TestCase || $test instanceof PhptTestCase);
 
         $this->tests[] = $test;