diff --git a/.vortex/docs/content/architecture.mdx b/.vortex/docs/content/architecture.mdx index 5f5f0ebbc..76ded5832 100644 --- a/.vortex/docs/content/architecture.mdx +++ b/.vortex/docs/content/architecture.mdx @@ -260,4 +260,4 @@ automation scripts work correctly: [DrevOps website](https://github.com/drevops/website) serves as a production reference site that receives regular upstream updates -➡️ See [Contributing > Maintenance > Tests](./contributing/maintenance/tests) +➡️ See [Contributing > Maintenance > Template](./contributing/maintenance/template) diff --git a/.vortex/docs/content/contributing/maintenance/README.mdx b/.vortex/docs/content/contributing/maintenance/README.mdx index 3432c1c4a..479dcafc8 100644 --- a/.vortex/docs/content/contributing/maintenance/README.mdx +++ b/.vortex/docs/content/contributing/maintenance/README.mdx @@ -1,7 +1,17 @@ --- -sidebar_position: 3 +sidebar_label: Overview +sidebar_position: 1 --- # Maintenance -See the subsections for detailed documentation on specific maintenance tasks. +This section covers the processes and guidelines for maintaining the +**Vortex** project itself, including the template, installer, documentation, +and release workflows. + +| Topic | Description | +|-------|-------------| +| [Template](template) | Maintaining template scripts and testing | +| [Installer](installer) | Maintaining the template installer and testing | +| [Documentation](documentation) | Authoring and publishing Vortex documentation | +| [Release](release) | Versioning strategy and release process | diff --git a/.vortex/docs/content/contributing/maintenance/documentation.mdx b/.vortex/docs/content/contributing/maintenance/documentation.mdx index 0e3a776bb..d4bc9c5e7 100644 --- a/.vortex/docs/content/contributing/maintenance/documentation.mdx +++ b/.vortex/docs/content/contributing/maintenance/documentation.mdx @@ -1,4 +1,8 @@ -# Authoring documentation +--- +sidebar_position: 4 +--- + +# Documentation There are 2 types of the documentation that **Vortex** provides: diff --git a/.vortex/docs/content/contributing/maintenance/installer.mdx b/.vortex/docs/content/contributing/maintenance/installer.mdx new file mode 100644 index 000000000..61a12eecd --- /dev/null +++ b/.vortex/docs/content/contributing/maintenance/installer.mdx @@ -0,0 +1,212 @@ +--- +sidebar_position: 3 +--- + +import AsciinemaPlayer from '@site/src/components/AsciinemaPlayer'; + +# Installer + +The **Vortex** installer is a self-contained Symfony Console application that +customizes the template based on user selections. It lives inside the same +repository as the template at `.vortex/installer/`, which allows changes to the +template and installer to be combined within a single pull request — ensuring +that template modifications and the corresponding installer logic stay in sync. + + + +## Architecture + +The installer is a PHP application rather than a shell script because it +touches many different parts of the template and requires complex management of +different permutations of the template variants. Using PHP allows the installer +to provide a **terminal UI experience** via +[Laravel Prompts](https://github.com/laravel/prompts), where users are guided +through a series of questions to customize their project. At the same time, +the installer supports **non-interactive installs** (via the `--no-interaction` +flag) for both new and existing projects, as well as updates — making it +suitable for automated pipelines and CI environments. + +Prompt answers can be provided from multiple sources, resolved in the following +priority order: + +1. **Configuration file** — a JSON file passed via the `--config` option. +2. **Environment variables** — using the `VORTEX_INSTALLER_PROMPT_` + naming convention. +3. **Discovery** — auto-detection from existing project files (e.g., + `composer.json`, `.env`, hosting-specific files like `.lagoon.yml`). +4. **Handler defaults** — fallback values defined by each handler. + +This means the installer can be driven entirely by a config file or environment +variables without any user interaction, which is essential for reproducible +installations and automated testing. + +The core flow is: + +1. User downloads the installer PHAR from https://www.vortextemplate.com/install + and runs it with `php installer.php`. +2. `InstallCommand` orchestrates the installation process. +3. `PromptManager` collects user choices via interactive prompts. +4. Handlers queue file operations based on selections — each handler is + responsible for a specific aspect of the template (CI provider, hosting, + services, theme, deployment strategy, etc.). +5. File operations execute: conditional tokens are processed to include or + exclude content based on the selected features. +6. The output is a fully customized Drupal project. + +### Conditional token system + +In addition to string substitution for handling simple replacements and +additions, the installer uses a token system to conditionally exclude entire +blocks of content from template files. This simplifies the management of +optional features — rather than requiring complex logic to surgically remove +lines from configuration files, scripts, or documentation, maintainers simply +wrap the relevant content in token markers. When a feature is not selected +during installation, the installer removes everything between the markers. + +**Markdown**: + +```markdown +[//]: # (#;< TOKEN_NAME) +Content removed if feature not selected +[//]: # (#;> TOKEN_NAME) +``` + +**Shell/YAML**: + +```bash +#;< TOKEN_NAME +Content removed if feature not selected +#;> TOKEN_NAME +``` + +## Testing + +The installer uses a multi-layered testing approach with unit tests for +individual components and functional tests for full installation scenarios. + +### Unit testing + +Unit tests cover core utilities (`Yaml`, `File`, `Git`, `Validator`, +`Strings`), downloaders, runners, and handler discovery logic. Each component +is tested in isolation with mocks provided by +[Mockery](https://github.com/mockery/mockery). + +### Functional testing with snapshots + +For every test permutation, the installer *initiates a fresh project* from the +Vortex template with a specific combination of user selections and runs +assertions against the resulting files. Because a single template change can +affect a hundred plus installation permutations, snapshot testing makes it easy to +**review the impact of a change across all scenarios as diffs** — ensuring that +regressions are caught before they reach consumers. + +Each handler has a dedicated functional test class extending +`AbstractHandlerProcessTestCase`, covering every aspect of the installation +process: CI providers, hosting providers, services, themes, deployment +strategies, database sources, notification channels, and more. + +These tests use the [`alexskrypnyk/snapshot`](https://github.com/AlexSkrypnyk/snapshot) +library for snapshot-based testing. The snapshot system works on a +**baseline + diff** pattern: + +- `_baseline/` contains the *complete* reference installation — a full set of + template files as they would appear after a default installation. +- Each scenario directory (e.g., `services_no_clamav/`, `hosting_acquia/`, + `deploy_types_all_circleci/`) contains only the *delta* changes from the + baseline. + +When a test runs, the snapshot library applies the scenario's diffs on top of +the baseline and compares the result against the actual installer output. This +approach keeps the fixture files maintainable — instead of duplicating the +entire template for each scenario, only the differences are stored. + +The fixture directories in `.vortex/installer/tests/Fixtures/handler_process/` +cover scenarios like: + +- **CI providers**: GitHub Actions, CircleCI +- **Hosting**: Acquia, Lagoon +- **Services**: Solr, Redis, ClamAV (enabled/disabled combinations) +- **Deployment**: artifact, container image, webhook, Lagoon, all combined +- **Database sources**: Acquia, Lagoon, FTP, S3, URL, container registry +- And many more permutations + +### Running tests + +```shell +cd .vortex/installer + +# Install Composer dependencies. +composer install + +# Run all tests. +composer test + +# Run only handler functional tests. +./vendor/bin/phpunit --filter "Handlers\\\\" + +# Run a specific handler test. +./vendor/bin/phpunit --filter "ServicesHandlerProcessTest" +``` + +### Updating snapshots + +Fixture files should never be modified directly. When the template or installer +logic changes, update the snapshots: + +```shell +# From .vortex/ directory (recommended). +ahoy update-snapshots + +# Manual (for debugging specific scenarios). +cd .vortex/installer +UPDATE_SNAPSHOTS=1 ./vendor/bin/phpunit --filter "testHandlerProcess.*baseline" +``` + +When `UPDATE_SNAPSHOTS` is set, the installer *runs for every permutation*, +initiates a fresh project for each scenario, and automatically updates the +fixture files to match the current output. The resulting changes appear as +diffs in version control, making it straightforward to review exactly how a +template or installer change affects each scenario. + +## Releasing + +The installer is packaged as a PHAR archive using +[Box](https://github.com/box-project/box). It is released on each **GitHub +release** of the Vortex template and deployed to +https://www.vortextemplate.com/install. Additionally, the installer is +published for every branch containing `release-docs` or `release-installer` +in its name, which allows testing installer changes before a formal release. + +## Installer video + +The documentation includes an interactive recording of the installer captured +with [asciinema](https://asciinema.org/). The recording is produced by a script +at `.vortex/docs/.utils/update-installer-video.sh` which: + +1. Builds the installer PHAR from source. +2. Generates an [Expect](https://en.wikipedia.org/wiki/Expect) script that + automates the installer interaction, simulating human-like typing with + delays. +3. Records the session with `asciinema rec` into a JSON cast file. +4. Post-processes the recording (sanitizes paths, removes spawn lines). +5. Converts the cast file to SVG and PNG using a custom + `svg-term-render.js` script, and optionally to GIF using `agg`. + +The output files are stored in `.vortex/docs/static/img/` and embedded in the +documentation using a custom `AsciinemaPlayer` React component. + +To update the video: + +```shell +cd .vortex +ahoy update-installer-video +``` diff --git a/.vortex/docs/content/contributing/maintenance/release.mdx b/.vortex/docs/content/contributing/maintenance/release.mdx index 5db474592..af31cf6b7 100644 --- a/.vortex/docs/content/contributing/maintenance/release.mdx +++ b/.vortex/docs/content/contributing/maintenance/release.mdx @@ -1,3 +1,7 @@ +--- +sidebar_position: 5 +--- + # Release ## Versioning Strategy diff --git a/.vortex/docs/content/contributing/maintenance/scripts.mdx b/.vortex/docs/content/contributing/maintenance/scripts.mdx deleted file mode 100644 index 761fdae16..000000000 --- a/.vortex/docs/content/contributing/maintenance/scripts.mdx +++ /dev/null @@ -1,118 +0,0 @@ -# Authoring scripts - -:::info - -Heads up! Scripts are changing from Bash to PHP in `2.0` release. Track the progress -in [this issue](https://github.com/drevops/vortex/issues/1192). - -::: - -## Requirements - -:::note - - Please refer to [RFC2119](https://www.ietf.org/rfc/rfc2119.txt) for meaning of words `MUST`, `SHOULD` and `MAY`. - -::: - -1. MUST adhere to [POSIX standard](https://en.wikipedia.org/wiki/POSIX). -2. MUST pass Shellcheck code analysis scan -3. MUST start with: -```shell - #!/usr/bin/env bash - ## - # Action description that the script performs. - # - # More description and usage information with a last empty - # comment line. - # - - set -eu - [ "${VORTEX_DEBUG-}" = "1" ] && set -x -``` -4. MUST list all variables with their default values and descriptions. i.e.: -```shell -# Deployment reference, such as a git SHA. -VORTEX_NOTIFY_REF="${VORTEX_NOTIFY_REF:-}" -``` -5. MUST include a delimiter between variables and the script body preceded and - followed by an empty line (3 lines in total): -```shell -# ------------------------------------------------------------------------------ -``` -6. SHOULD include formatting helper functions: -```shell -# @formatter:off -note() { printf " %s\n" "${1}"; } -info() { [ "${TERM:-}" != "dumb" ] && tput colors >/dev/null 2>&1 && printf "\033[34m[INFO] %s\033[0m\n" "${1}" || printf "[INFO] %s\n" "${1}"; } -pass() { [ "${TERM:-}" != "dumb" ] && tput colors >/dev/null 2>&1 && printf "\033[32m[ OK ] %s\033[0m\n" "${1}" || printf "[ OK ] %s\n" "${1}"; } -fail() { [ "${TERM:-}" != "dumb" ] && tput colors >/dev/null 2>&1 && printf "\033[31m[FAIL] %s\033[0m\n" "${1}" || printf "[FAIL] %s\n" "${1}"; } -# @formatter:on -``` -7. SHOULD include variable values checks with errors and early exist, i.e.: -```shell -[ -z "${VORTEX_NOTIFY_REF}" ] && fail "Missing required value for VORTEX_NOTIFY_REF" && exit 1 -``` -8. SHOULD include binaries checks if the script relies on them, i.e.: -```shell -command -v curl > /dev/null || ( fail "curl command is not available." && exit 1 ) -``` -9. MUST contain an `info` message about the start of the script body, e.g.: -```shell -info "Started GitHub notification for operation ${VORTEX_NOTIFY_EVENT}" -``` -10. MUST contain an `pass` message about the finish of the script body, e.g.: -```shell -pass "Finished GitHub notification for operation ${VORTEX_NOTIFY_EVENT}" -``` -11. MUST use uppercase global variables -12. MUST use lowercase local variables. -13. MUST use long options instead of short options for readability. I.e., `drush cache:rebuild` instead of `drush cr`. -14. MUST use `VORTEX_` prefix for variables, unless it is a known 3-rd party - variable like `PACKAGE_TOKEN` or `COMPOSER`. -15. MUST use script-specific prefix. I.e., for `notify.sh`, the variable to skip - notifications should start with `VORTEX_NOTIFY_`. -16. MAY rely on variables from the external scripts (not prefixed with a - script-specific prefix), but MUST declare such variables in the header of - the file. -17. MAY call other **Vortex** scripts (discouraged), but MUST source them rather - than creating a sub-process. This is to allow passing environment variables - down the call stack. -18. SHOULD use `note` messages for informing about the script progress. -19. MUST use variables in the form of `${VAR}`. - - -## Variables - -Follow these guidelines when creating or updating **Vortex** variables. - -1. Local variables MUST be in lowercase, and global variables MUST be in - uppercase. - -2. All **Vortex** variables MUST start with `VORTEX_` to separate **Vortex** from - third-party variables. - -3. Global variables MAY be re-used as-is across scripts. For instance, the - `WEBROOT` variable is used in several scripts. - -4. **Vortex** action-specific script variables MUST be scoped within their own - script. For instance, the `VORTEX_PROVISION_OVERRIDE_DB` - variable in the `provision.sh`. - -5. Drupal-related variables SHOULD start with `DRUPAL_` and SHOULD have a module - name added as a second prefix. This is to separate **Vortex**, - third-party services variables, and Drupal variables. For instance, to set - a user for Drupal's Shield module configuration, use `DRUPAL_SHIELD_USER`. - -6. Variables SHOULD NOT be exported into the global scope unless absolutely - necessary. Thus, values in `.env` SHOULD have default values set, but SHOULD - be commented out to provide visibility and avoid exposure to the global scope. - -## Boilerplate script - -Use the boilerplate script below to kick-start your custom **Vortex** script. - -import CodeBlock from '@theme/CodeBlock'; -import MyComponentSource from '!!raw-loader!./script-boilerplate.sh'; - -{MyComponentSource} diff --git a/.vortex/docs/content/contributing/maintenance/tests.mdx b/.vortex/docs/content/contributing/maintenance/template.mdx similarity index 64% rename from .vortex/docs/content/contributing/maintenance/tests.mdx rename to .vortex/docs/content/contributing/maintenance/template.mdx index fc7147353..be25714e1 100644 --- a/.vortex/docs/content/contributing/maintenance/tests.mdx +++ b/.vortex/docs/content/contributing/maintenance/template.mdx @@ -1,6 +1,135 @@ -# Tests +--- +sidebar_position: 2 +--- -## Testing strategy +# Template + +This page covers maintaining the **Vortex** template: authoring scripts and +testing the template functionality. + +## Authoring scripts + +:::info + +Heads up! Scripts are changing from Bash to PHP in `2.0` release. Track the progress +in [this issue](https://github.com/drevops/vortex/issues/1192). + +::: + +### Requirements + +:::note + + Please refer to [RFC2119](https://www.ietf.org/rfc/rfc2119.txt) for meaning of words `MUST`, `SHOULD` and `MAY`. + +::: + +1. MUST adhere to [POSIX standard](https://en.wikipedia.org/wiki/POSIX). +2. MUST pass Shellcheck code analysis scan +3. MUST start with: + ```shell + #!/usr/bin/env bash + ## + # Action description that the script performs. + # + # More description and usage information with a last empty + # comment line. + # + + set -eu + [ "${VORTEX_DEBUG-}" = "1" ] && set -x + ``` +4. MUST list all variables with their default values and descriptions. i.e.: + ```shell + # Deployment reference, such as a git SHA. + VORTEX_NOTIFY_REF="${VORTEX_NOTIFY_REF:-}" + ``` +5. MUST include a delimiter between variables and the script body preceded and + followed by an empty line (3 lines in total): + ```shell + # ------------------------------------------------------------------------------ + ``` +6. SHOULD include formatting helper functions: + ```shell + # @formatter:off + note() { printf " %s\n" "${1}"; } + info() { [ "${TERM:-}" != "dumb" ] && tput colors >/dev/null 2>&1 && printf "\033[34m[INFO] %s\033[0m\n" "${1}" || printf "[INFO] %s\n" "${1}"; } + pass() { [ "${TERM:-}" != "dumb" ] && tput colors >/dev/null 2>&1 && printf "\033[32m[ OK ] %s\033[0m\n" "${1}" || printf "[ OK ] %s\n" "${1}"; } + fail() { [ "${TERM:-}" != "dumb" ] && tput colors >/dev/null 2>&1 && printf "\033[31m[FAIL] %s\033[0m\n" "${1}" || printf "[FAIL] %s\n" "${1}"; } + # @formatter:on + ``` +7. SHOULD include variable values checks with errors and early exist, i.e.: + ```shell + [ -z "${VORTEX_NOTIFY_REF}" ] && fail "Missing required value for VORTEX_NOTIFY_REF" && exit 1 + ``` +8. SHOULD include binaries checks if the script relies on them, i.e.: + ```shell + command -v curl > /dev/null || ( fail "curl command is not available." && exit 1 ) + ``` +9. MUST contain an `info` message about the start of the script body, e.g.: + ```shell + info "Started GitHub notification for operation ${VORTEX_NOTIFY_EVENT}" + ``` +10. MUST contain an `pass` message about the finish of the script body, e.g.: + ```shell + pass "Finished GitHub notification for operation ${VORTEX_NOTIFY_EVENT}" + ``` +11. MUST use uppercase global variables +12. MUST use lowercase local variables. +13. MUST use long options instead of short options for readability. I.e., `drush cache:rebuild` instead of `drush cr`. +14. MUST use `VORTEX_` prefix for variables, unless it is a known 3-rd party + variable like `PACKAGE_TOKEN` or `COMPOSER`. +15. MUST use script-specific prefix. I.e., for `notify.sh`, the variable to skip + notifications should start with `VORTEX_NOTIFY_`. +16. MAY rely on variables from the external scripts (not prefixed with a + script-specific prefix), but MUST declare such variables in the header of + the file. +17. MAY call other **Vortex** scripts (discouraged), but MUST source them rather + than creating a sub-process. This is to allow passing environment variables + down the call stack. +18. SHOULD use `note` messages for informing about the script progress. +19. MUST use variables in the form of `${VAR}`. + + +### Variables + +Follow these guidelines when creating or updating **Vortex** variables. + +1. Local variables MUST be in lowercase, and global variables MUST be in + uppercase. + +2. All **Vortex** variables MUST start with `VORTEX_` to separate **Vortex** from + third-party variables. + +3. Global variables MAY be re-used as-is across scripts. For instance, the + `WEBROOT` variable is used in several scripts. + +4. **Vortex** action-specific script variables MUST be scoped within their own + script. For instance, the `VORTEX_PROVISION_OVERRIDE_DB` + variable in the `provision.sh`. + +5. Drupal-related variables SHOULD start with `DRUPAL_` and SHOULD have a module + name added as a second prefix. This is to separate **Vortex**, + third-party services variables, and Drupal variables. For instance, to set + a user for Drupal's Shield module configuration, use `DRUPAL_SHIELD_USER`. + +6. Variables SHOULD NOT be exported into the global scope unless absolutely + necessary. Thus, values in `.env` SHOULD have default values set, but SHOULD + be commented out to provide visibility and avoid exposure to the global scope. + +### Boilerplate script + +import CodeBlock from '@theme/CodeBlock'; +import MyComponentSource from '!!raw-loader!./script-boilerplate.sh'; + +
+Expand to see the boilerplate script + +{MyComponentSource} + +
+ +## Testing **Vortex** is a project template, not a traditional application. A change to a script, configuration file, or workflow can affect every project that uses @@ -36,14 +165,12 @@ in [this issue](https://github.com/drevops/vortex/issues/1192). [PHPUnit](https://phpunit.de/) is used for functional end-to-end testing that exercises the full site build pipeline in real Docker containers. These tests -simulate what a consumer site developer would experience: installing the -template, building containers, importing databases, running Drupal operations, -linting code, and deploying artifacts. +simulate what a consumer site developer would experience: building containers, +importing databases, running Drupal operations, linting code, and deploying +artifacts. -The end-to-end tests cover several critical workflows: +The template end-to-end tests cover several critical workflows: -- **Installer** - verifying that a fresh project can be scaffolded from the - template and that custom files are preserved during updates. - **Docker Compose** - building the container stack, verifying environment variables, running linters, executing PHPUnit and Behat tests inside containers, and checking Solr integration. @@ -109,11 +236,6 @@ bats .vortex/tests/bats/deploy.bats There are *demo* and *test* database dumps captured as *files* and *container images*. -- Demo database dump file - *demonstration* of the database import capabilities from a *file*. -- Demo database container image - *demonstration* of the database import capabilities from a *container image*. -- Test database dump file - *test* of the database import capabilities from a *file*. -- Test database container image - *test* of the database import capabilities from a *container image*. - ### Updating *demo* database dump *file*
diff --git a/.vortex/docs/docusaurus.config.js b/.vortex/docs/docusaurus.config.js index a639daf4b..de55d2519 100644 --- a/.vortex/docs/docusaurus.config.js +++ b/.vortex/docs/docusaurus.config.js @@ -230,6 +230,14 @@ const config = { from: ['/contributing'], to: '/docs/contributing', }, + { + from: '/docs/contributing/maintenance/scripts', + to: '/docs/contributing/maintenance/template', + }, + { + from: '/docs/contributing/maintenance/tests', + to: '/docs/contributing/maintenance/template', + }, { from: ['/docs/getting-started'], to: '/docs', diff --git a/.vortex/docs/sidebars.js b/.vortex/docs/sidebars.js index f98a624f4..393a77cd0 100644 --- a/.vortex/docs/sidebars.js +++ b/.vortex/docs/sidebars.js @@ -10,10 +10,17 @@ const sidebars = { 'faqs', 'support', { - Contributing: [{ - type: 'autogenerated', - dirName: 'contributing', - }], + Contributing: [ + 'contributing/README', + 'contributing/roadmap', + 'contributing/code-of-conduct', + { + Maintenance: [{ + type: 'autogenerated', + dirName: 'contributing/maintenance', + }], + }, + ], }, { type: 'html',