Skip to content

Commit 9b3ff16

Browse files
colinmurphycolinmurphy
committed
Refactored codeception tests not to pass if under 95% code coverage. Added new badges for the plugin. Updated testsing docs.
1 parent 8e09c54 commit 9b3ff16

6 files changed

Lines changed: 125 additions & 173 deletions

File tree

‎plugins/hwp-previews/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
![GitHub forks](https://img.shields.io/github/forks/wpengine/hwptoolkit?style=social)
1414
![GitHub stars](https://img.shields.io/github/stars/wpengine/hwptoolkit?style=social)
1515
[![Testing Integration](https://img.shields.io/github/check-runs/wpengine/hwptoolkit/main?checkName=hwp-previews%20codeception%20tests&label=Automated%20Tests)](https://github.com/wpengine/hwptoolkit/actions)
16+
[![Code Coverage](https://img.shields.io/badge/coverage-%3E95%25-brightgreen?label=Code%20Coverage)](https://github.com/wpengine/hwptoolkit/actions)
1617
[![Code Quality](https://img.shields.io/github/check-runs/wpengine/hwptoolkit/main?checkName=hwp-previews%20php%20code%20quality%20checks&label=Code%20Quality%20Checks)](https://github.com/wpengine/hwptoolkit/actions)
1718
[![End-to-End Tests](https://github.com/wpengine/hwptoolkit/workflows/End-to-End%20Tests/badge.svg)](https://github.com/wpengine/hwptoolkit/actions?query=workflow%3A%22End-to-End+Tests%22)
1819
-----

‎plugins/hwp-previews/TESTING.md‎

Lines changed: 70 additions & 121 deletions
Original file line numberDiff line numberDiff line change
@@ -1,155 +1,104 @@
11
# Testing HWP Previews
22

3-
This plugin uses [Codeception](https://codeception.com/) with [WPBrowser](https://wpbrowser.wptestkit.dev/) for automated testing.
4-
Tests are organized into suites for integration (wpunit), functional, and acceptance testing.
3+
## Table of Contents
54

6-
---
7-
8-
## Test Suites
9-
10-
- **wpunit**: Unit/integration tests with WordPress loaded.
11-
- **functional**: Simulates web requests, runs WordPress in a test environment.
12-
- **acceptance**: Browser-based tests (WPBrowser/WPWebDriver).
13-
14-
Configuration files for each suite are in the `tests/` directory (e.g., `wpunit.suite.dist.yml`).
15-
16-
---
17-
18-
## Local Test Environment
19-
20-
The plugin provides scripts to set up a local WordPress environment for testing, using Docker and environment variables defined in `.env.dist`.
21-
22-
### Prerequisites
23-
24-
- Docker (for local environment)
25-
- Composer
26-
- Node.js (for building assets, if needed)
27-
28-
---
29-
30-
## Setup
31-
32-
1. **Copy and configure environment variables:**
33-
34-
```bash
35-
@TODO
36-
cp .env.dist .env
37-
# Edit .env as needed for your local setup
38-
```
39-
40-
41-
2. **Set up the test WordPress environment:**
42-
43-
```bash
44-
bin/install-test-env.sh
45-
```
46-
47-
This script will:
48-
- Create the test database (unless `SKIP_DB_CREATE=true`)
49-
- Download and install WordPress in the directory specified by `WORDPRESS_ROOT_DIR`
50-
- Symlink the plugin into the WordPress plugins directory
51-
- Activate the plugin and set up test data
5+
- [Overview](#overview)
6+
- [Directory Structure](#directory-structure)
7+
- [Technologies](#technologies)
8+
- [Usage](#usage)
9+
- [Running Tests](#running-tests)
10+
- [GitHub Actions](#github-actions)
11+
- [Setup Tests Locally](#setup-tests-locally)
5212

5313
---
5414

55-
## Running Tests
56-
57-
Currently the plugin has the following suite of tests
15+
## Overview
5816

59-
1. WP Unit Tests - (Unit and Integration Tests)
60-
2. E2E Tests - Playright tests
17+
HWP Previews comes with automated tests for unit, integration, and acceptance (E2E) scenarios.
6118

62-
### WPUnit (WordPress-aware Unit/Integration) Tests
19+
## Directory Structure
6320

64-
You can also run WPUnit tests using Composer scripts:
21+
A list of related files and directories for testing:
6522

66-
```bash
67-
composer run test:unit
68-
```
23+
```text
24+
bin/
25+
├── install-test-env.sh # Set up test WP environment
26+
├── run-codeception.sh # Run Codeception tests
27+
├── run-e2e.sh # Run E2E (Playwright) tests
28+
├── run-coverage.sh # Generate coverage reports
29+
└── local/
30+
├── run-unit-tests.sh # Run unit tests in Docker with Codeception
31+
├── run-e2e-tests.sh # Run e2e tests in Docker with Playwright
32+
├── run-qa.sh # Run php code quality checks with PHPStan, Psalm and PHPCS
33+
├── run-wpunit.sh # Run WPUnit tests in Docker
34+
└── run-functional.sh # Run functional tests in Docker
6935
70-
To generate coverage reports:
36+
tests/
37+
├── _data/ # Test data (e.g. DB dumps)
38+
├── _envs/ # Environment configs
39+
├── _output/ # Test output (logs, coverage)
40+
├── _support/ # Helper classes, modules
41+
├── wpunit/ # WPUnit (WordPress-aware unit/integration) test cases
42+
├── wpunit.suite.dist.yml
43+
└── wpunit/
44+
└── bootstrap.php # Bootstrap for WPUnit tests
7145
72-
```bash
73-
composer run test:unit:coverage
46+
.env.dist # Example environment variables for testing
47+
codeception.dist.yml # Main Codeception config
7448
```
49+
---
7550

76-
To generate an HTML coverage report:
51+
## Technologies
7752

78-
```bash
79-
composer run test:unit:coverage-html
80-
```
81-
> [!NOTE]
82-
> HTML code coverage can be found here [tests/_output/coverage/index.html](tests/_output/coverage/index.html)
53+
We use the following technologies to run our tests:
8354

55+
- [Codeception](https://codeception.com/)
56+
- [WPBrowser](https://wpbrowser.wptestkit.dev/)
57+
- [WPUnit](https://github.com/lipemat/wp-unit)
58+
- [Docker](https://www.docker.com/)
59+
- [Composer](https://getcomposer.org/)
60+
- [Playwright](https://playwright.dev/)
61+
- [npm](https://www.npmjs.com/)
8462

85-
### E2WTests
63+
---
8664

87-
Run browser-based acceptance tests:
65+
## Usage
8866

89-
```bash
90-
sh bin/local/run-e2e-tests.sh coverage
91-
```
67+
Currently, the plugin has the following suite of tests:
9268

93-
### All Tests
69+
1. **WP Unit Tests** – Unit and Integration Tests
70+
2. **E2E Tests** – Acceptance tests
9471

95-
To run all suites:
72+
### Running Tests
9673

97-
```bash
98-
composer run test
99-
# or
100-
vendor/bin/codecept run
101-
```
74+
| Command | Description |
75+
|------------------------------------------|----------------------------------------------------------|
76+
| `composer run test:unit:coverage` | Run WPUnit (unit/integration) tests with coverage report |
77+
| `composer run test:unit:coverage-html` | Generate an HTML code coverage report |
78+
| `composer run test:e2e` | Run end-to-end (E2E) acceptance tests |
79+
| `composer run test` | Run all available test suites |
10280

10381
---
10482

105-
## Code Coverage
106-
107-
To generate code coverage reports (requires Xdebug or PCOV):
83+
## GitHub Actions
10884

109-
```bash
110-
# Example for wpunit suite
111-
SUITES=wpunit COVERAGE=1 bin/run-codeception.sh
112-
```
85+
We have a few checks which run for a new PR being merged to main
11386

114-
Coverage output will be in `tests/_output/` or as specified by `COVERAGE_OUTPUT`.
87+
| Workflow | Description | Link |
88+
|-------------------------|---------------------------------------------|----------------------------------------------------------------------|
89+
| Code Quality | Runs static analysis and linting checks | [View Workflow](../../actions/workflows/code-quality.yml) |
90+
| E2E Tests | Runs Playwright end-to-end acceptance tests | [View Workflow](../../actions/workflows/e2e.yml) |
91+
| Codeception (WPUnit) | Runs unit and integration tests | [View Workflow](../../actions/workflows/codeception.yml) |
11592

116-
---
11793

118-
## Useful Scripts
94+
> **INFO:**
95+
> All tests are automatically run on every pull request via GitHub Actions. You can review test results and logs directly in the "Checks" tab of your PR on GitHub.
11996
120-
- `bin/install-test-env.sh` — Sets up the WordPress test environment.
121-
- `bin/run-codeception.sh` — Runs Codeception tests inside the plugin directory.
122-
- `bin/local/run-unit-tests.sh` — Runs unit tests in Docker.
123-
- `bin/local/run-qa.sh` — Runs code quality checks (PHPStan, PHPCS, Psalm).
97+
> **IMPORTANT:**
98+
> Test coverage for WP Unit Tests is 95%. Any new code will require tests to be added in order to pass.
12499
125100
---
126101

127-
## Notes
128-
129-
- The test database will be reset during setup. **Do not use a database with important data.**
130-
- You can customize which suites to run by setting the `SUITES` environment variable.
131-
- See `.env.dist` for all available environment variables and their descriptions.
102+
## Setup Tests Locally
132103

133-
---
134-
135-
```text
136-
tests/
137-
├── _data/ # Test data (e.g. DB dumps)
138-
├── _envs/ # Environment configs
139-
├── _output/ # Test output (logs, coverage)
140-
├── _support/ # Helper classes, modules
141-
├── wpunit/ # WPUnit (WordPress-aware unit/integration) test cases
142-
├── wpunit.suite.dist.yml
143-
└── wpunit/
144-
└── bootstrap.php # Bootstrap for WPUnit tests
145-
146-
bin/
147-
├── install-test-env.sh # Script to set up test WP environment
148-
├── run-codeception.sh # Script to run Codeception tests
149-
└── local/
150-
├── run-unit-tests.sh # Run unit tests in Docker
151-
└── run-qa.sh # Run code quality checks
152-
153-
.env.dist # Example environment variables for testing
154-
codeception.dist.yml # Main Codeception config
155-
```
104+
@TODO

‎plugins/hwp-previews/bin/run-codeception.sh‎

Lines changed: 50 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -21,31 +21,6 @@ setup_before() {
2121
curl -L 'https://raw.github.com/Codeception/c3/2.0/c3.php' > "c3.php"
2222
fi
2323

24-
# Enable XDebug or PCOV for code coverage.
25-
if [[ "$COVERAGE" == '1' ]]; then
26-
if [[ "$USING_XDEBUG" == '1' ]]; then
27-
echo "Enabling XDebug 3"
28-
cp /usr/local/etc/php/conf.d/disabled/docker-php-ext-xdebug.ini /usr/local/etc/php/conf.d/
29-
echo "xdebug.mode=coverage" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
30-
else
31-
echo "Using pcov/clobber for code coverage"
32-
docker-php-ext-enable pcov
33-
echo "pcov.enabled=1" >> /usr/local/etc/php/conf.d/docker-php-ext-pcov.ini
34-
echo "pcov.directory=${PROJECT_DIR}" >> /usr/local/etc/php/conf.d/docker-php-ext-pcov.ini
35-
COMPOSER_MEMORY_LIMIT=-1 composer require pcov/clobber --dev
36-
vendor/bin/pcov clobber
37-
fi
38-
elif [[ -f /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini ]]; then
39-
echo "Disabling XDebug"
40-
rm /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
41-
fi
42-
43-
# Install the PHP dev-dependencies.
44-
if [ ! -d "vendor" ]; then
45-
echo "Running composer install"
46-
COMPOSER_MEMORY_LIMIT=-1 composer install
47-
fi
48-
4924
# Set output permission
5025
echo "Setting Codeception output directory permissions"
5126
chmod 777 -R tests/_output
@@ -59,12 +34,6 @@ run_tests() {
5934
local debug="--debug"
6035
fi
6136

62-
local suites=$1
63-
if [[ -z "$SUITES" ]]; then
64-
echo "No test suites specified. Must specify variable SUITES."
65-
exit 1
66-
fi
67-
6837
if [[ -n "$COVERAGE" ]]; then
6938
if [[ -n "$COVERAGE_OUTPUT" ]]; then
7039
local coverage="--coverage --coverage-xml $COVERAGE_OUTPUT"
@@ -79,25 +48,61 @@ run_tests() {
7948
wp maintenance-mode deactivate --allow-root
8049
fi
8150

82-
83-
# Suites is the comma separated list of suites/tests to run.
84-
echo "Running Test Suite $suites"
51+
echo "Running Unit and Integration tests"
8552
cd "$PROJECT_DIR"
8653

8754
# IMPORTANT: Build Codeception classes before running tests
88-
echo "Building Codeception test classes"
89-
vendor/bin/codecept build -c codeception.dist.yml
55+
echo "Building Codeception test classes"
56+
vendor/bin/codecept build -c codeception.dist.yml
9057

91-
if [ $? -ne 0 ]; then
92-
echo "Error: Codeception build failed"
93-
exit 1
94-
fi
58+
if [ $? -ne 0 ]; then
59+
echo "Error: Codeception build failed"
60+
exit 1
61+
fi
9562

96-
XDEBUG_MODE=coverage vendor/bin/codecept run -c codeception.dist.yml ${suites} ${coverage:-} ${debug:-} --no-exit
63+
XDEBUG_MODE=coverage vendor/bin/codecept run -c codeception.dist.yml ${suites} ${coverage:-} ${debug:-} ${debug:-}
9764
if [ $? -ne 0 ]; then
9865
echo "Error: Codeception tests failed with exit code $?"
9966
exit 1
10067
fi
68+
69+
# Check code coverage if coverage was requested
70+
if [[ -n "$COVERAGE" ]]; then
71+
72+
if [[ -n "$COVERAGE_OUTPUT" ]]; then
73+
coverage_percent=$(grep -oP '(\d+\.\d+)%' "tests/_output/coverage/index.html" | head -1 | tr -d '%')
74+
else
75+
coverage_percent=$(grep -oP 'line-rate="(\d+\.\d+)"' "tests/_output/coverage.xml" | head -1 | grep -oP '\d+\.\d+')
76+
# Convert to percent
77+
if [[ -n "$coverage_percent" ]]; then
78+
coverage_percent=$(awk "BEGIN { printf \"%.2f\", $coverage_percent * 100 }")
79+
fi
80+
fi
81+
if [[ -z "$coverage_percent" ]]; then
82+
echo "Warning: Could not determine code coverage percentage."
83+
exit 1
84+
fi
85+
86+
echo "Code coverage percentage found: $coverage_percent"
87+
88+
89+
required_coverage=$(grep 'min_coverage:' codeception.dist.yml | awk '{print $2}')
90+
91+
if [[ -z "$required_coverage" ]]; then
92+
echo "No min_coverage found in codeception.dist.yml. Defaulting to 80%"
93+
required_coverage=80
94+
fi
95+
96+
coverage_int=${coverage_percent%.*}
97+
if (( coverage_int < required_coverage )); then
98+
echo -e "\033[0;31mError: Code coverage is ${coverage_percent}%, which is below the required ${required_coverage}%.\033[0m"
99+
exit 1
100+
else
101+
echo -e "\033[0;32mCode coverage is ${coverage_percent}% (required: ${required_coverage}%)\033[0m"
102+
fi
103+
104+
fi
105+
101106
}
102107

103108
##
@@ -117,12 +122,11 @@ cleanup_after() {
117122
if [[ "$USING_XDEBUG" == '1' ]]; then
118123
echo "Disabling XDebug 3"
119124
rm /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
120-
else
125+
else98
121126
echo "Disabling pcov/clobber"
122127
docker-php-ext-disable pcov
123128
sed -i '/pcov.enabled=1/d' /usr/local/etc/php/conf.d/docker-php-ext-pcov.ini
124129
sed -i '/pcov.directory=${PROJECT_DIR}/d' /usr/local/etc/php/conf.d/docker-php-ext-pcov.ini
125-
COMPOSER_MEMORY_LIMIT=-1 composer remove pcov/clobber --dev
126130
fi
127131
fi
128132

@@ -135,18 +139,16 @@ cleanup_after() {
135139
echo "Setting up for Codeception tests"
136140
setup_before
137141

138-
139-
# Run the tests
140-
run_tests $SUITES
142+
run_tests
141143

142144
# Clean up after running tests.
143145
echo "Cleaning up after Codeception tests"
144146
cleanup_after
145147

146148
# Check results and exit accordingly.
147149
if [ -f "tests/_output/failed" ]; then
148-
echo "Uh oh, Codeception tests failed."
150+
echo "Codeception tests failed."
149151
exit 1
150152
else
151-
echo "Woohoo! Codeception tests completed succesfully!"
153+
echo "Codeception tests completed successfully!"
152154
fi

‎plugins/hwp-previews/codeception.dist.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,8 @@ coverage:
5050
- /tests/*
5151
- /vendor/*
5252
- /src/Templates/*
53-
show_only_summary: false
53+
show_only_summary: true
54+
min_coverage: 95
5455
modules:
5556
config:
5657
REST:

0 commit comments

Comments
 (0)