Merge pull request #29 from os2display/feature/e2e-workflow

B29: E2E CI workflow on release branches

Author: turegjorup

.github/workflows/e2e.yaml

@@ -0,0 +1,124 @@
+### ### E2E
+###
+### End-to-end test of the localhost-cert install recipe, walked
+### through the supplied tasks the same way an operator would: bootstrap
+### env files via `task env:init`, configure traefik via
+### `task env:traefik`, generate a self-signed cert via `task dev:cert`,
+### bring the stack up via `task up`, run migrations + JWT + tenant +
+### admin user via `task console`, install templates via
+### `task templates:install`, and tear down via `task dev:teardown`.
+### Curls `/admin/` over HTTPS using the generated cert as the success
+### gate.
+###
+### Triggers on `pull_request` against `release/**` and pushes to
+### `release/**` only — too slow + too much docker churn to run on every
+### casual PR. Acts as the release-cut quality bar that the static
+### `markdown` / `yaml` / `sh` / `compose` / `tasks` workflows can't catch:
+### actual stack boot, Doctrine migrations, JWT keypair generation, and
+### Traefik routing.
+###
+### #### Known fragility points
+###
+### - **`APP_ENV=dev` upstream image bug**: the bundled `/app/.env` still
+###   defaults to `APP_ENV=dev` until a 3.0.0-rc3+ tag ships the fix
+###   from display-api-service#435. We sed it to `prod` after `task
+###   env:init`. Drop the workaround once a fixed image tag is in
+###   `OS2DISPLAY_VERSION_API`.
+### - **`task install` is interactive**: chained `task: <subtask>` calls
+###   don't reliably pipe stdin to the Symfony console for tenant:add
+###   and user:add (B23 caveat). We walk the install steps individually
+###   via `task console -- ARGS` (which is non-interactive when the
+###   bin/console subcommand accepts CLI args) instead of calling
+###   `task install` directly.
+### - **`scripts/install-secrets.sh` runs as the first step of `task
+###   install`**: not currently a standalone task, so we invoke the
+###   script directly. It's tracked under `scripts/` and shellcheck-
+###   validated; equivalent to running task install's secrets-fill
+###   step in isolation.
+### - **GHCR pulls without auth**: bot quota, can flake on heavy CI days.
+
+name: E2E
+
+on:
+  pull_request:
+    branches:
+      - "release/**"
+  push:
+    branches:
+      - "release/**"
+
+jobs:
+  install-localhost:
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Install Task
+        uses: arduino/setup-task@v2
+        with:
+          version: 3.x
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: task env:init (domain prompt piped)
+        run: printf 'os2display.localhost\n' | task env:init
+
+      - name: APP_ENV=prod workaround (upstream image bug — see header)
+        run: sed -i 's|^APP_ENV=dev|APP_ENV=prod|' .env.symfony
+
+      - name: task env:traefik (cert-file branch, prompts piped)
+        # Choice 2 = cert-file. Domain = traefik.localhost. LE email blank
+        # (cert-file branch ignores it). Cert filename defaults accepted
+        # via blank lines. Dashboard admin / password.
+        run: printf '2\ntraefik.localhost\n\n\nadmin\npassword\n' | task env:traefik
+
+      - name: task dev:cert
+        run: task dev:cert
+
+      - name: install-secrets (would be `task install`'s first step)
+        run: ./scripts/install-secrets.sh
+
+      - name: Pre-create bind-mount dirs with container-writable perms
+        # The os2display container writes JWT keys as UID 1042 (`deploy`)
+        # per README's deploy-host UID/GID contract, and uploads to
+        # ./media as the same user. CI runners aren't the operator's
+        # 1042-group host, so chown the bind-mount sources to match.
+        run: |
+          mkdir -p jwt media backup
+          sudo chown -R 1042:1042 jwt media backup
+
+      - name: task up
+        # `task up` uses `compose up -d --wait`, so this blocks until every
+        # service's healthcheck passes. Migrations on the next step can
+        # connect immediately.
+        run: task up
+
+      - name: task console -- app:update (Doctrine migrations)
+        run: task console -- --no-interaction app:update
+
+      - name: task console -- lexik:jwt:generate-keypair
+        run: task console -- lexik:jwt:generate-keypair --skip-if-exists
+
+      - name: task console -- app:tenant:add
+        run: task console -- app:tenant:add e2e 'E2E Tenant' CI
+
+      - name: task console -- app:user:add
+        run: task console -- app:user:add admin@os2display.localhost e2epassword 'E2E Admin' admin e2e
+
+      - name: task templates:install
+        run: task templates:install
+
+      - name: Smoke test — admin route over HTTPS
+        # --resolve maps os2display.localhost → 127.0.0.1; --cacert trusts
+        # the generated dev cert (covers os2display.localhost as a SAN).
+        # -f makes curl exit non-zero on >=400.
+        run: |
+          curl -fv \
+            --resolve os2display.localhost:443:127.0.0.1 \
+            --cacert traefik/ssl/dev.crt \
+            https://os2display.localhost/admin/ \
+            -o /dev/null
+
+      - name: task dev:teardown
+        if: always()
+        run: task -y dev:teardown

CHANGELOG.md

@@ -53,6 +53,27 @@ and aligned to the v3 image's env contract. **For 1.x → 3.x operators: see
   aggregate) — set-membership comparison of `task --list` against the README block. Section
   headings, descriptions, and alias notes in the README stay human-curated; only the SET of
   task names is checked. Caught the missing `dev:teardown` entry in the existing block.
+- New **`E2E`** workflow on release branches: bootstraps env files via `task env:init`,
+  generates a self-signed cert via `task dev:cert`, brings the full stack up, runs Doctrine
+  migrations, creates a tenant + admin user, and curls `/admin/` over HTTPS via the dev cert.
+  Catches install-path regressions the static checks can't (image boot, migration replay,
+  Traefik routing). Triggers on `pull_request` against `release/**` and pushes to
+  `release/**` — too slow (~5 min wall-clock) for every casual PR.
+- **`task up` / `install` / `update` now use `compose up --wait`** — blocks until every
+  service's healthcheck passes before returning, instead of returning immediately and
+  leaving the operator to poll. Matches the operator mental-model "after `task up` the
+  stack is up". Replaces `task install`'s previous `sleep 20` hack.
+- **`docker-compose.yml` `depends_on` graph filled in.** B6 added healthchecks to every
+  service but only wired one dependency (`nginx-api → os2display: service_healthy`).
+  Migrations on a fresh install raced the database — `task install` papered over with a
+  20-second sleep. Now:
+  - `os2display` waits for `mariadb` (`service_healthy`, `required: false` so the dep is
+    skipped when an operator runs against an external db / drops the `mariadb` profile)
+    and `redis` (`service_healthy`).
+  - `traefik` waits for `socket-proxy` (`service_healthy`) — traefik queries socket-proxy
+    for docker-label discovery, no point asking before the proxy is reachable.
+  Combined with `--wait` above, `task up` returns when the stack is genuinely ready, with
+  startup ordering enforced at the container level rather than via wall-clock guesswork.
 - `scripts/` directory: extracted helpers for the longer Taskfile bodies that the upstream
   [Taskfile style guide](https://taskfile.dev/styleguide/) recommends moving out
   ("Prefer using external scripts instead of multi-line commands"). `host-resources.sh`,

README.md

@@ -939,14 +939,17 @@ file-copy header at the top of each lints config preserves provenance for sync.
 
 ### CI workflows
 
-`.github/workflows/`, four files:
+`.github/workflows/`, five static checks plus a release-branch e2e:
 
 - **`markdown.yaml`** — runs markdownlint via `docker compose --profile dev run --rm
   markdownlint`. Catches doc rot.
 - **`yaml.yaml`** — runs prettier via the same pattern. Catches YAML drift.
 - **`sh.yaml`** — runs shellcheck against `scripts/*.sh` via the `shellcheck` dev-profile
   service. Catches shell footguns in the extracted helpers (unquoted vars, masked exit
   codes, etc.).
+- **`tasks.yaml`** — runs `scripts/check-tasks-readme.sh`, asserting the set of task
+  names in the README's "All tasks" reference block matches `task --list`. Section
+  labels, descriptions, and alias notes stay human-curated; only membership is checked.
 - **`compose.yaml`** — three jobs:
   1. `compose-config` — synthesises the stack with example env files + a stub `.env.symfony`.
      Catches typos and dangling `${VAR}` references.
@@ -955,8 +958,15 @@ file-copy header at the top of each lints config preserves provenance for sync.
   3. `env-coverage` — every bare `${VAR}` reference in `docker-compose.yml` is declared in
      `.env.example` or `.env.traefik.example`. Catches the silent-empty
      substitution case.
+- **`e2e.yaml`** — release-branch only (`pull_request` against `release/**`, push to
+  `release/**`). Bootstraps env files via `task env:init`, generates a self-signed cert,
+  brings the full stack up, runs migrations, creates a tenant + admin user, and curls
+  `/admin/` over HTTPS via the cert. Catches install-path regressions the static checks
+  can't (image boot, migration replay, Traefik routing). ~5 minutes wall-clock.
 
-All four trigger on `pull_request` and pushes to `main` / `develop` / `release/**`.
+All static checks trigger on `pull_request` and pushes to `main` / `develop` /
+`release/**`. The e2e workflow runs on release branches only — too slow + too much docker
+churn for the casual review cycle.
 
 ---
 
@@ -968,7 +978,7 @@ All four trigger on `pull_request` and pushes to `main` / `develop` / `release/*
 Lifecycle
   install              Install the project — first-time setup (interactive)
   update               Pull images, recreate containers, run app:update
-  up                   Start the stack without recreating containers
+  up                   Start the stack; blocks until healthchecks pass
   down                 Remove all containers (preserves named volumes)
   stop                 Stop all containers
   purge                Remove all containers AND named volumes  (prompts)

Taskfile.yml

@@ -31,9 +31,7 @@ tasks:
       - echo "Installing"
       - ./scripts/install-secrets.sh
       - "{{.COMPOSE}} pull"
-      - "{{.COMPOSE}} up --force-recreate --detach --remove-orphans"
-      - echo "Waiting for database to be ready"
-      - sleep 20
+      - "{{.COMPOSE}} up --force-recreate --detach --remove-orphans --wait"
       - echo "Run database migrations and required updates"
       - task: console
         vars: { CLI_ARGS: "app:update" }
@@ -55,18 +53,18 @@ tasks:
         msg: ".env.symfony missing — run 'task env:init' first."
     cmds:
       - "{{.COMPOSE}} pull"
-      - "{{.COMPOSE}} up --force-recreate --detach --remove-orphans"
+      - "{{.COMPOSE}} up --force-recreate --detach --remove-orphans --wait"
       - task: console
         vars: { EXEC_FLAGS: "--user deploy", CLI_ARGS: "app:update" }
       - echo "NB! Run 'task templates:install' if the new image added templates."
 
   up:
-    desc: Start the stack without recreating containers
+    desc: Start the stack without recreating containers; blocks until healthchecks pass
     preconditions:
       - sh: "[ -f .env.symfony ]"
         msg: ".env.symfony missing — run 'task env:init' first."
     cmds:
-      - "{{.COMPOSE}} up -d"
+      - "{{.COMPOSE}} up -d --wait"
 
   down:
     desc: Remove all containers (preserves named volumes)

docker-compose.yml

@@ -47,6 +47,14 @@ services:
         required: true
       - path: .env.php.local
         required: false
+    depends_on:
+      # mariadb is profile-gated (`mariadb`). `required: false` lets compose
+      # skip this dep when an operator runs against an external database.
+      mariadb:
+        condition: service_healthy
+        required: false
+      redis:
+        condition: service_healthy
     healthcheck:
       test: ["CMD", "php", "-r", "exit(@fsockopen('127.0.0.1', 9000) ? 0 : 1);"]
       interval: 30s
@@ -147,6 +155,12 @@ services:
         required: true
       - path: .env.traefik.local
         required: false
+    depends_on:
+      # Traefik queries the docker socket via socket-proxy for label
+      # discovery; wait for the proxy to pass its /version healthcheck
+      # before traefik starts asking it for routes.
+      socket-proxy:
+        condition: service_healthy
     ports:
       - "80:80"
       - "443:443"

scripts/install-secrets.sh

@@ -18,8 +18,19 @@ if [ ! -f .env.mariadb ] || [ ! -f .env.symfony ]; then
   exit 1
 fi
 
-# Sentinel detection via env vars (loaded by Taskfile's `dotenv:` directive).
-# If neither password is the CHANGE_ME placeholder, there's nothing to do.
+# Source .env.mariadb so $MARIADB_PASSWORD etc. are set whether the
+# script runs via `task install` (Taskfile's `dotenv:` already loaded it)
+# or stand-alone (CI workflow, manual invocation). Without this, an empty
+# env defeats the sentinel check below — both `${VAR:-}` evaluate to
+# empty (≠ "CHANGE_ME"), so the script silently exits 0 and mariadb
+# starts with the literal CHANGE_ME password from the file.
+set -a
+# shellcheck disable=SC1091
+. ./.env.mariadb
+set +a
+
+# Sentinel detection. If neither password is the CHANGE_ME placeholder,
+# there's nothing to do.
 if [ "${MARIADB_PASSWORD:-}" != "CHANGE_ME" ] && [ "${MARIADB_ROOT_PASSWORD:-}" != "CHANGE_ME" ]; then
   exit 0
 fi