Skip to content

Commit 5f60d7c

Browse files
authored
[#2172] Added more information to docs about how we test Vortex. (#2262)
1 parent af79fd1 commit 5f60d7c

2 files changed

Lines changed: 109 additions & 18 deletions

File tree

.vortex/docs/content/architecture.mdx

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -185,7 +185,7 @@ the schedule as needed.
185185

186186
## Documentation & onboarding
187187

188-
**Vortex** includes centralized documentation (what youre reading now), as well as
188+
**Vortex** includes centralized documentation (what you're reading now), as well as
189189
a scaffold for adding project-specific docs within your own repository.
190190

191191
## Automation scripts
@@ -246,3 +246,18 @@ These variables are either defined in the `.env` file or set within
246246
CI/hosting environments as secrets.
247247

248248
➡️ See [Variables](./development/variables)
249+
250+
## Testing the template
251+
252+
**Vortex** itself is tested at multiple levels to ensure the template and its
253+
automation scripts work correctly:
254+
255+
- **Unit tests**: Shell scripts are tested in isolation using
256+
[Bats](https://github.com/bats-core/bats-core) with mocked commands
257+
- **End-to-end tests**: Full build pipelines are tested using
258+
[PHPUnit](https://phpunit.de/) in real Docker containers
259+
- **Real-world validation**: The
260+
[DrevOps website](https://github.com/drevops/website) serves as a production
261+
reference site that receives regular upstream updates
262+
263+
➡️ See [Contributing > Maintenance > Tests](./contributing/maintenance/tests)

.vortex/docs/content/contributing/maintenance/tests.mdx

Lines changed: 93 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,81 @@
1-
# Authoring tests
1+
# Tests
22

3-
There are 2 types of tests used in **Vortex**:
3+
## Testing strategy
44

5-
- [PHPUnit](https://phpunit.de/) for functional end-to-end testing of a site build pipeline.<br/>
6-
With help of [`phpunit-helpers`](https://github.com/AlexSkrypnyk/phpunit-helpers) testing library, we run tests
7-
for automations, including container builds, database imports, and Drupal
8-
operations as if they were run by a consumer site developer.
5+
**Vortex** is a project template, not a traditional application. A change to
6+
a script, configuration file, or workflow can affect every project that uses
7+
it. Because of this, **Vortex** uses a multi-layered testing approach to catch
8+
issues at different levels of integration before they reach consumers.
99

10-
- [Bats](https://github.com/bats-core/bats-core) for unit testing of scripts in `scripts/vortex`.<br/>
11-
With help of [`BATS Helpers`](https://github.com/drevops/bats-helpers) testing
12-
library, we run unit tests for shell scripts with coverage.
10+
### Unit testing
1311

14-
:::info
12+
[Bats](https://github.com/bats-core/bats-core) is used to unit test the
13+
shell scripts in `scripts/vortex/`. Each script is tested in isolation with
14+
external commands (like `drush`, `docker`, `composer`) replaced by mocks.
15+
This allows us to verify that individual scripts handle environment variables,
16+
flags, and edge cases correctly without needing a running Drupal site or
17+
Docker containers.
1518

16-
Heads up! Scripts are changing from Bash to PHP in `2.0` release.
17-
This will make the use of BATS obsolete. Track the progress
18-
in [this issue](https://github.com/drevops/vortex/issues/1192).
19+
The [`bats-helpers`](https://github.com/drevops/bats-helpers) library provides
20+
a step-based testing approach with built-in mocking and assertions, making it
21+
straightforward to define expected inputs and outputs for each script.
1922

20-
:::
23+
Unit tests execute in seconds, providing fast feedback during development.
24+
There are 25+ test files covering deployment, database operations,
25+
notifications, provisioning, and other automation scripts.
2126

22-
## Functional tests with PHPUnit
27+
:::info
28+
29+
Scripts are transitioning from Bash to PHP in **Vortex** `2.0`, which will
30+
make the use of BATS obsolete. Track the progress
31+
in [this issue](https://github.com/drevops/vortex/issues/1192).
32+
33+
:::
34+
35+
### End-to-end testing
36+
37+
[PHPUnit](https://phpunit.de/) is used for functional end-to-end testing that
38+
exercises the full site build pipeline in real Docker containers. These tests
39+
simulate what a consumer site developer would experience: installing the
40+
template, building containers, importing databases, running Drupal operations,
41+
linting code, and deploying artifacts.
42+
43+
The end-to-end tests cover several critical workflows:
44+
45+
- **Installer** - verifying that a fresh project can be scaffolded from the
46+
template and that custom files are preserved during updates.
47+
- **Docker Compose** - building the container stack, verifying environment
48+
variables, running linters, executing PHPUnit and Behat tests inside
49+
containers, and checking Solr integration.
50+
- **Ahoy workflows** - testing the full developer experience through Ahoy
51+
commands, including build, login, Drush operations, Composer, database
52+
import/export, and code quality checks.
53+
- **Deployment** - testing deployment pipelines for artifact and webhook-based
54+
deployment strategies.
55+
56+
The [`phpunit-helpers`](https://github.com/AlexSkrypnyk/phpunit-helpers)
57+
library provides test helpers, traits, and assertions purpose-built for running
58+
shell commands and validating their output within PHPUnit tests.
59+
60+
End-to-end tests take minutes to run because they operate on real containers
61+
and a real Drupal site, but they provide the highest confidence that
62+
everything works together correctly.
63+
64+
### Example site
65+
66+
The [DrevOps website](https://github.com/drevops/website) is a real-world
67+
production site built using **Vortex**. It serves as the ultimate validation
68+
ground: if **Vortex** works correctly, the website should continue to build,
69+
test, and deploy without issues after each upstream update.
70+
71+
The website repository receives regular upstream updates from **Vortex**,
72+
including CI configuration, testing workflows, and infrastructure changes.
73+
This ensures that updates are validated against a real, long-lived codebase
74+
rather than just in test environments.
75+
76+
## Running tests
77+
78+
### Functional tests with PHPUnit
2379

2480
```shell
2581
cd .vortex/tests
@@ -35,9 +91,9 @@ TEST_PACKAGE_TOKEN=<yourtoken> TEST_VORTEX_CONTAINER_REGISTRY_USER=<youruser> TE
3591
```
3692

3793
Functional tests rely on database fixtures - see the section below on
38-
[updating test](#updating-test-assets) assets.
94+
[updating test assets](#updating-test-assets).
3995

40-
## Unit tests with BATS
96+
### Unit tests with BATS
4197

4298
```shell
4399
cd .vortex/tests
@@ -60,6 +116,9 @@ There are *demo* and *test* database dumps captured as *files* and *container im
60116

61117
### Updating *demo* database dump *file*
62118

119+
<details>
120+
<summary>Show instructions</summary>
121+
63122
1. Run fresh build of **Vortex** locally:
64123
```shell
65124
rm .data/db.sql || true
@@ -90,8 +149,13 @@ ahoy export-db db.demo.sql
90149
```
91150
4. Upload `db.demo.sql` to the latest release as an asset and name it `db_d11.demo.sql`.
92151

152+
</details>
153+
93154
### Updating *demo* database *container image*
94155

156+
<details>
157+
<summary>Show instructions</summary>
158+
95159
1. Run fresh build of **Vortex** locally:
96160
```shell
97161
rm .data/db.sql || true
@@ -130,8 +194,13 @@ chmod +x seed.sh
130194
./seed.sh .data/db.demo_image.sql drevops/vortex-dev-mariadb-drupal-data-demo-11.x:latest
131195
```
132196

197+
</details>
198+
133199
### Updating *test* database dump *file*
134200

201+
<details>
202+
<summary>Show instructions</summary>
203+
135204
1. Run a fresh install of **Vortex** into a new directory and name the project `Star Wars`:
136205
```shell
137206
mkdir /tmp/star-wars
@@ -168,8 +237,13 @@ ahoy export-db db.test.sql
168237
```
169238
5. Upload `db.test.sql` to the latest release as an asset and name it `db_d11.test.sql`.
170239

240+
</details>
241+
171242
### Updating *test* database *container image*
172243

244+
<details>
245+
<summary>Show instructions</summary>
246+
173247
1. Run a fresh install of **Vortex** into a new directory and name the project `Star Wars`:
174248
```shell
175249
mkdir /tmp/star-wars
@@ -221,3 +295,5 @@ docker push drevops/vortex-dev-mariadb-drupal-data-demo-destination-11.x:vortex-
221295
docker tag drevops/vortex-dev-mariadb-drupal-data-demo-11.x:latest drevops/vortex-dev-mariadb-drupal-data-demo-destination-11.x:vortex-dev-didi-database-fi
222296
docker push drevops/vortex-dev-mariadb-drupal-data-demo-destination-11.x:vortex-dev-didi-database-fi
223297
```
298+
299+
</details>

0 commit comments

Comments
 (0)