Skip to content

Commit d2144f9

Browse files
asgrimasgrim
authored
Merge pull request #553 from asgrim/add-infection-mutation-testing
Add mutation testing with Infection
2 parents 6cd155d + cbbe697 commit d2144f9

File tree

6 files changed

+216
-23
lines changed

6 files changed

+216
-23
lines changed

‎.github/ISSUE_TEMPLATE/bug_report.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
---
2+
name: Bug report
3+
about: Create a report to help us improve
4+
title: ''
5+
labels: ''
6+
assignees: ''
7+
8+
---
9+
10+
**What are you trying to do?**
11+
Please describe what you are trying to do
12+
13+
**What platform, and PIE version are you using?**
14+
(add the output of `pie show -v`, and information about which OS you're using etc.)
15+
16+
**Steps to reproduce the issue**
17+
1. ...
18+
2. ...
19+
3. etc...
20+
21+
**What do you expect to happen?**
22+
Describe what you expect to see or happen.
23+
24+
**What is actually happening**
25+
Describe what is actually happening.
26+
27+
**Complete logs/output**
28+
Complete error messages and/or logs, including stack traces (hint: use `-v` for verbose), but _please redact sensitive information_.
29+
30+
**Anything else?**
31+
other information that might be helpful...

‎.github/workflows/continuous-integration.yml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -203,6 +203,24 @@ jobs:
203203
- name: Run phpstan
204204
run: vendor/bin/phpstan
205205

206+
mutation-testing:
207+
runs-on: ubuntu-latest
208+
steps:
209+
- name: Setup PHP
210+
uses: shivammathur/setup-php@v2
211+
with:
212+
php-version: 8.4
213+
extensions: intl, sodium, zip
214+
tools: infection
215+
env:
216+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
217+
- uses: actions/checkout@v6
218+
- uses: ramsey/composer-install@v3
219+
- name: Run infection
220+
env:
221+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
222+
run: sudo infection --min-msi=68 --min-covered-msi=68 --threads=max --show-mutations=max --no-progress
223+
206224
build-assets:
207225
needs:
208226
- unit-tests

‎.gitignore‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ behat.yml
88
box.json
99
box.phar
1010
pie.phar
11+
infection.phar
1112
/docs-package/
1213
CHANGELOG-*.md
13-
/.idea/
14+
/.idea/

‎CONTRIBUTING.md‎

Lines changed: 2 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,10 @@
11
# Contributing to PIE
22

3-
Thank you for taking an interest in contributing to PIE.
4-
53
> [!IMPORTANT]
64
> The TL;DR version of this is please either [open an issue](https://github.com/php/pie/issues) if you think you have
75
> found a bug, or [create a discussion](https://github.com/php/pie/discussions) if you have an idea for a new feature,
86
> **BEFORE** starting work on any code changes. Submitting a pull request without checking first may be duplicating
97
> work that is already in progress!
108
11-
## Reporting bugs
12-
13-
If you think you have a bug, please [open an issue](https://github.com/php/pie/issues), and include:
14-
15-
- What platform and PIE version you're using
16-
- The steps to reproduce the issue - please use the `-v` verbose flag (or higher)
17-
- Complete error messages and/or logs, including stack traces (hint: use `-v` for verbose), but please redact sensitive information
18-
19-
## Submitting PRs
20-
21-
Please **do not** just submit PRs for features or ideas without discussing them first. **Start by
22-
[creating a discussion](https://github.com/php/pie/discussions) first** if there is not already an open discussion or
23-
issue. If there is already an open discussion or issue, please comment and wait for feedback before starting any work.
24-
This is because the work may already be in progress or being investigated already. The PIE project is actively being
25-
developed, and many features are already in discussion or being developed, so if you do not discuss with us first, you
26-
may be duplicating work already in progress.
27-
28-
> [!TIP]
29-
> We try to stay on top of issues already being worked on with the `maintainer investigating` purple label. If you see
30-
> this on an issue, it is very likely we are already looking into this.
9+
Please see the [PIE Maintainers Handbook](docs/pie-maintainers-handbook.md) for our full contributing guide and
10+
documentation targeting contributors who want to work on PIE itself.

‎docs/pie-maintainers-handbook.md‎

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,35 @@ order: 3
44
---
55
# PIE Maintainers Handbook
66

7+
Thank you for taking an interest in contributing to PIE. This guide is written to help humans develop on PIE.
8+
9+
## Contributing to PIE
10+
11+
## Reporting bugs
12+
13+
If you think you have a bug, please [open an issue](https://github.com/php/pie/issues), and include:
14+
15+
- What platform and PIE version you're using
16+
- A summary of:
17+
- What are you trying to do?
18+
- What do you expect to happen?
19+
- What is actually happening?
20+
- The steps to reproduce the issue - please use the `-v` verbose flag (or higher)
21+
- Complete error messages and/or logs, including stack traces (hint: use `-v` for verbose), but please redact sensitive information
22+
23+
## Submitting PRs
24+
25+
Please **do not** just submit PRs for features or ideas without discussing them first. **Start by
26+
[creating a discussion](https://github.com/php/pie/discussions) first** if there is not already an open discussion or
27+
issue. If there is already an open discussion or issue, please comment and wait for feedback before starting any work.
28+
This is because the work may already be in progress or being investigated already. The PIE project is actively being
29+
developed, and many features are already in discussion or being developed, so if you do not discuss with us first, you
30+
may be duplicating work already in progress.
31+
32+
> [!TIP]
33+
> We try to stay on top of issues already being worked on with the `maintainer investigating` purple label. If you see
34+
> this on an issue, it is very likely we are already looking into this.
35+
736
## Branching strategy
837

938
Since 1.3.0, we operate a branch per minor release, with an `x` for the patch
@@ -22,6 +51,129 @@ Bugfixes/patches should be based on the oldest supported or desired patch
2251
version. For example, if a bug affects the 1.3 series, a PR should be made
2352
from the feature branch to the `1.3.x` branch.
2453

54+
## Developing on PIE
55+
56+
> [!IMPORTANT]
57+
> Please read the above section BEFORE doing work on PIE; PIE is in very active development by the PHP Foundation, and
58+
> many features/issues are already being worked on.
59+
60+
All the below tests are run in CI, on various versions of PHP, to ensure they do not get missed! However, when
61+
developing on PIE, you will likely need to run some, or all, of these tests by hand.
62+
63+
> [!TIP]
64+
> The development guide is primarily aimed at Linux systems, please adjust accordingly for your own platform.
65+
66+
### Unit & Integration Tests
67+
68+
> [!CAUTION]
69+
> The integration test suite interacts directly with the PHP installation that is used to run the tests. This may
70+
> result in the `asgrim/example-pie-extension` extension being added (sometimes in a broken state) to your PHP install.
71+
> Do not run this using a PHP installation that you care about!
72+
73+
We use PHPUnit, which is installed with Composer, and can be run with:
74+
75+
```shell
76+
sudo vendor/bin/phpunit
77+
```
78+
79+
If you have multiple versions of PHP, you may add that in, e.g.:
80+
81+
```shell
82+
sudo /usr/bin/php8.3 vendor/bin/phpunit
83+
```
84+
85+
> [!TIP]
86+
> `sudo` **is** required due to the way some of the tests interact with `sudo`
87+
88+
Coverage HTML report can be generated with:
89+
90+
```shell
91+
sudo XDEBUG_MODE=coverage vendor/bin/phpunit --coverage-html coverage
92+
```
93+
94+
### Behaviour Testing
95+
96+
The primary features are documented in `features/*.feature`, in [Gherkin](https://cucumber.io/docs/gherkin/) format.
97+
98+
Some of these features are implemented as behaviour tests, using [Behat](https://behat.org/). However, these tests
99+
should NOT be run on your machine directly. Instead, there is a predefined and controlled environment made with Docker.
100+
This is done because many of the tests require specific dependencies, and this helps provide a repeatable test harness.
101+
102+
This can be run in two steps; first build the Docker container:
103+
104+
```shell
105+
GITHUB_TOKEN=$(composer config --global --auth github-oauth.github.com) docker buildx build --file .github/pie-behaviour-tests/Dockerfile --secret id=GITHUB_TOKEN,env=GITHUB_TOKEN -t pie-behat-test .
106+
```
107+
108+
Optionally, pass the `--build-arg PHP_VERSION=8.3` parameter to the above command to change the version of PHP used in
109+
the test. Then you may run the tests with:
110+
111+
```shell
112+
docker run --volume .:/github/workspace -ti pie-behat-test
113+
```
114+
115+
### End-to-end Testing
116+
117+
We have a small number of end-to-end type tests, that are designed to ensure PIE runs in particular ways in different
118+
systems. The test runner is a shell script that builds a Docker target, and runs it; and fails if a non-zero exit
119+
occurs.
120+
121+
```shell
122+
test/end-to-end/dockerfile-e2e-test.sh
123+
```
124+
125+
This is particularly useful for testing different setups (e.g. Alpine, Ubuntu, Fedora, Brew, etc.) and ensuring build
126+
tools and system dependencies are automatically installed.
127+
128+
### Static Analysis
129+
130+
We use [PHPStan](https://phpstan.org/) and several plugins to run static analysis on the codebase. You can run this:
131+
132+
```shell
133+
vendor/bin/phpstan
134+
```
135+
136+
### Coding Standards
137+
138+
We use [PHP_CodeSniffer](https://github.com/PHPCSStandards/PHP_CodeSniffer/) using the
139+
[Doctrine Coding Standard](https://github.com/doctrine/coding-standard) to check and automatically fix CS issues.
140+
141+
To check the CS conformity:
142+
143+
```shell
144+
vendor/bin/phpcs
145+
```
146+
147+
To attempt to auto-apply CS fixes:
148+
149+
```shell
150+
vendor/bin/phpcbf
151+
```
152+
153+
### Mutation Testing
154+
155+
Install the `infection.phar` according to the [Infection PHP documentation](https://infection.github.io/guide/installation.html#Phar).
156+
157+
Tests can be run with:
158+
159+
```shell
160+
sudo infection --min-msi=68 --min-covered-msi=68 --threads=max
161+
```
162+
163+
> [!TIP]
164+
> As with the tests, `sudo` is required due to the way some of the tests interact with `sudo`
165+
166+
### Documentation
167+
168+
Documentation can be generated with:
169+
170+
```shell
171+
.github/docs/build-docs.sh
172+
```
173+
174+
This is not normally needed, unless you are changing the template, as CI will generate the documentation and publish to
175+
the GitHub Page.
176+
25177
## Release process
26178

27179
Make sure you have the latest version of the trunk to be released, for example,

‎infection.json5‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"$schema": "https://raw.githubusercontent.com/infection/infection/0.32.6/resources/schema.json",
3+
"source": {
4+
"directories": [
5+
"src"
6+
]
7+
},
8+
"mutators": {
9+
"@default": true
10+
}
11+
}

0 commit comments

Comments
 (0)