Skip to content

Commit 8b6c8c1

Browse files
staabmstaabm
authored
Merge branch '2.1.x' into less-work55
2 parents 8c51d01 + f387269 commit 8b6c8c1

35 files changed

+1827
-45
lines changed

‎.claude/skills/regression-test/SKILL.md‎

Lines changed: 186 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
---
2+
name: regression-test
3+
description: Add a regression test for an already-fixed PHPStan bug given a GitHub issue number
4+
argument-hint: "[issue-number]"
5+
allowed-tools: Read, Grep, Glob, Bash(curl *), Bash(gh *), Bash(git *), Bash(make tests), Bash(vendor/bin/phpunit *), Bash(php *)
6+
---
7+
8+
# Adding a regression test for a fixed PHPStan bug
9+
10+
You are given a GitHub issue number. Your goal is to write a regression test that locks in the fix so the bug cannot silently resurface.
11+
12+
## Step 1 — Gather context from the issue
13+
14+
Fetch the GitHub issue and its comments:
15+
16+
```
17+
gh issue view <number> --repo phpstan/phpstan --json title,body,comments
18+
```
19+
20+
In the comments, look for:
21+
22+
1. **phpstan-bot's commit link** — this identifies the commit that fixed the bug.
23+
2. **PHPStan Playground links** — URLs matching `https://phpstan.org/r/<UUID>`. These contain the reproducing code and the analysis results before and after the fix.
24+
25+
For each playground link, fetch the sample data:
26+
27+
```
28+
curl -s 'https://api.phpstan.org/sample?id=<UUID>'
29+
```
30+
31+
The API returns JSON with:
32+
- `code` — the PHP code that was analysed
33+
- `level` — the PHPStan rule level used
34+
- `config.strictRules`, `config.bleedingEdge`, `config.treatPhpDocTypesAsCertain` — configuration flags
35+
- `versionedErrors` — array of `{phpVersion, errors: [{line, message, identifier}]}`
36+
37+
The playground sample gives you the reproducing code and tells you what errors PHPStan produces (or should no longer produce) at each line. Use this to determine the correct test type and expected outcomes.
38+
39+
## Step 2 — Decide which kind of test to write
40+
41+
### Type-inference test (NSRT)
42+
43+
Use when the bug is about **wrong inferred type, missing type narrowing, or incorrect type after control flow** — i.e., PHPStan reports the wrong type for an expression but no specific rule error is involved.
44+
45+
- Create `tests/PHPStan/Analyser/nsrt/bug-<number>.php`
46+
- Use `assertType()` and optionally `assertNativeType()` to pin the correct types
47+
- The file is **auto-discovered** — no test-case class changes are needed
48+
49+
Example (`tests/PHPStan/Analyser/nsrt/bug-12875.php`):
50+
51+
```php
52+
<?php declare(strict_types = 1); // lint >= 8.0
53+
54+
namespace Bug12875;
55+
56+
use function PHPStan\Testing\assertType;
57+
58+
interface HasFoo
59+
{
60+
public function foo(): int;
61+
}
62+
63+
interface HasBar
64+
{
65+
public function bar(): int;
66+
}
67+
68+
class HelloWorld
69+
{
70+
/**
71+
* @param "foo"|"bar" $method
72+
* @param ($method is "foo" ? HasFoo : HasBar) $a
73+
* @param ($method is "foo" ? HasFoo : HasBar) $b
74+
*/
75+
public function add(string $method, HasFoo|HasBar $a, HasFoo|HasBar $b): void
76+
{
77+
assertType('int', $a->{$method}());
78+
assertType('int', $b->{$method}());
79+
80+
$addInArrow = fn () => assertType('int', $a->{$method}());
81+
82+
$addInAnonymous = function () use ($a, $b, $method): void {
83+
assertType('int', $a->{$method}());
84+
assertType('int', $b->{$method}());
85+
};
86+
}
87+
}
88+
```
89+
90+
### Rule test
91+
92+
Use when the bug is about a **false positive** (rule reported an error that shouldn't exist) or a **false negative** (rule failed to report an error that should exist).
93+
94+
1. **Find the relevant rule test case.** Search for the rule's identifier or class name:
95+
```
96+
grep -r 'identifier' tests/PHPStan/Rules/ --include='*Test.php'
97+
```
98+
Or look at the error identifier from the playground output (e.g., `argument.type` points to a rule in `Rules/Methods/` or `Rules/Functions/`).
99+
100+
2. **Create the test data file** at `tests/PHPStan/Rules/<Category>/data/bug-<number>.php`.
101+
102+
3. **Add a test method** in the corresponding `*Test.php` rule test case.
103+
104+
Example — test data (`tests/PHPStan/Rules/Methods/data/bug-11470.php`):
105+
106+
```php
107+
<?php declare(strict_types = 1);
108+
109+
namespace Bug11470;
110+
111+
use DateTimeImmutable;
112+
113+
interface HelloWorld
114+
{
115+
public function sayHello(): dateTimeImmutable;
116+
}
117+
118+
interface HelloWorld2
119+
{
120+
public function sayHello(dateTimeImmutable $a): void;
121+
}
122+
123+
interface HelloWorld3
124+
{
125+
public function sayHello(): ?dateTimeImmutable;
126+
}
127+
```
128+
129+
Example — test method in `ExistingClassesInTypehintsRuleTest.php`:
130+
131+
```php
132+
public function testBug11470(): void
133+
{
134+
$this->analyse([__DIR__ . '/data/bug-11470.php'], [
135+
[
136+
'Class DateTimeImmutable referenced with incorrect case: dateTimeImmutable.',
137+
9,
138+
],
139+
[
140+
'Class DateTimeImmutable referenced with incorrect case: dateTimeImmutable.',
141+
14,
142+
],
143+
[
144+
'Class DateTimeImmutable referenced with incorrect case: dateTimeImmutable.',
145+
19,
146+
],
147+
]);
148+
}
149+
```
150+
151+
For **false-positive** fixes the expected-errors array is usually empty `[]`, meaning the code should analyse clean.
152+
153+
## Step 3 — Write the test
154+
155+
- Use the reproducing code from the playground sample as the basis for the test file.
156+
- Put it in the correct `namespace Bug<number>;`.
157+
- Add `use function PHPStan\Testing\assertType;` for NSRT tests.
158+
- If the playground code requires a minimum PHP version, add `// lint >= 8.0` (or whichever version) on the first line.
159+
- For rule tests, determine the expected errors from the playground `versionedErrors`. If the fix eliminated a false positive, expect `[]`. If the fix added a missing error, list the expected `[message, line]` pairs.
160+
161+
## Step 4 — Verify the test
162+
163+
Run the specific test to confirm it passes with the current (fixed) code:
164+
165+
```bash
166+
# NSRT test — run the full NSRT suite or a filtered subset:
167+
vendor/bin/phpunit tests/PHPStan/Analyser/NodeScopeResolverTest.php --filter 'bug-<number>'
168+
169+
# Rule test — run the specific test class:
170+
vendor/bin/phpunit tests/PHPStan/Rules/<Category>/<TestClass>.php --filter testBug<number>
171+
```
172+
173+
## Step 5 — Confirm the test catches a regression
174+
175+
Temporarily revert the fixing commit and verify the test fails:
176+
177+
```bash
178+
git stash
179+
git revert --no-commit <fixing-commit-hash>
180+
vendor/bin/phpunit <test-command-from-step-4>
181+
# Expect failure
182+
git checkout .
183+
git stash pop
184+
```
185+
186+
If the test passes even with the fix reverted, the test is not correctly covering the bug — revisit the assertions or the reproducing code.

0 commit comments

Comments
 (0)