e2e via tasks: --wait, depends_on graph, install-secrets fix

Three threads, all on the same install-time correctness theme:

(1) install-secrets.sh standalone-invocation fix.

When invoked via `task install`, the script reads
$MARIADB_PASSWORD from Taskfile's `dotenv:`-loaded env. When invoked
directly (CI workflow, manual operator use), the env is empty —
"${VAR:-}" != "CHANGE_ME" matches because both sides are empty
strings, the script silently exits 0, and mariadb starts on the
literal CHANGE_ME password while DATABASE_URL has the would-be-random
hex. Surfaced by PR #29's first e2e run with Doctrine reporting access
denied. Fix: source .env.mariadb at the top of the script.

(2) `compose up --wait` + `depends_on` graph.

Post-install migrations need a connectable database. Previously
`task install` fired `compose up -d` then `sleep 20` to paper over a
race; `task up` returned immediately with no wait at all. B6 wired
healthchecks on every service but only one `depends_on`
(nginx-api → os2display: service_healthy). Now:

- task up / install / update use `compose up -d --wait` — blocks until
  every healthcheck passes. install drops its sleep 20 hack.
- os2display.depends_on adds mariadb (service_healthy, required: false
  so the dep skips cleanly when an operator runs external db / drops
  the mariadb profile) and redis (service_healthy).
- traefik.depends_on adds socket-proxy (service_healthy) — traefik
  queries it for docker-label discovery; no point asking before it's
  reachable.

(3) E2E workflow rewritten to drive everything through tasks.

Previously the workflow bypassed Taskfile for the install steps and
called docker compose exec ... bin/console directly, because
`task install`'s nested `task: <subtask>` invocations don't pipe
stdin reliably to interactive Symfony prompts (B23 caveat). Now:

- task env:init / env:traefik / dev:cert (was: same)
- ./scripts/install-secrets.sh (script, not a standalone task)
- task up (waits for healthchecks via above)
- task console -- --no-interaction app:update
- task console -- lexik:jwt:generate-keypair --skip-if-exists
- task console -- app:tenant:add e2e 'E2E Tenant' CI
- task console -- app:user:add admin@os2display.localhost ...
- task templates:install
- curl smoke test against the dev cert
- task -y dev:teardown

`task console -- ARGS` proxies bin/console with explicit args, which
sidesteps the chained-subtask stdin issue for tenant:add / user:add.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: turegjorup

.github/workflows/e2e.yaml

@@ -1,9 +1,14 @@
 ### ### E2E
 ###
-### End-to-end test of the localhost-cert install recipe: bootstrap env
-### files via `task env:init`, generate the self-signed cert via
-### `task dev:cert`, bring the stack up, create a tenant + admin user,
-### and curl the admin route over HTTPS using the generated cert.
+### End-to-end test of the localhost-cert install recipe, walked
+### through the supplied tasks the same way an operator would: bootstrap
+### env files via `task env:init`, configure traefik via
+### `task env:traefik`, generate a self-signed cert via `task dev:cert`,
+### bring the stack up via `task up`, run migrations + JWT + tenant +
+### admin user via `task console`, install templates via
+### `task templates:install`, and tear down via `task dev:teardown`.
+### Curls `/admin/` over HTTPS using the generated cert as the success
+### gate.
 ###
 ### Triggers on `pull_request` against `release/**` and pushes to
 ### `release/**` only — too slow + too much docker churn to run on every
@@ -14,16 +19,23 @@
 ###
 ### #### Known fragility points
 ###
-### - `APP_ENV=dev` workaround: the upstream display-api-service image's
-###   bundled .env still defaults to `APP_ENV=dev` until a 3.0.0-rc3+ tag
-###   ships the fix from display-api-service#435. We sed it to `prod`
-###   before bringing the stack up. Drop the override once a fixed tag
-###   is in `OS2DISPLAY_VERSION_API`.
-### - `task install` chains `task: <subtask>` calls that don't reliably
-###   pipe stdin to the Symfony console (B23 caveat). The workflow
-###   sidesteps by running the install steps directly via
-###   `docker compose exec`.
-### - GHCR pulls without auth: bot quota, can flake on heavy CI days.
+### - **`APP_ENV=dev` upstream image bug**: the bundled `/app/.env` still
+###   defaults to `APP_ENV=dev` until a 3.0.0-rc3+ tag ships the fix
+###   from display-api-service#435. We sed it to `prod` after `task
+###   env:init`. Drop the workaround once a fixed image tag is in
+###   `OS2DISPLAY_VERSION_API`.
+### - **`task install` is interactive**: chained `task: <subtask>` calls
+###   don't reliably pipe stdin to the Symfony console for tenant:add
+###   and user:add (B23 caveat). We walk the install steps individually
+###   via `task console -- ARGS` (which is non-interactive when the
+###   bin/console subcommand accepts CLI args) instead of calling
+###   `task install` directly.
+### - **`scripts/install-secrets.sh` runs as the first step of `task
+###   install`**: not currently a standalone task, so we invoke the
+###   script directly. It's tracked under `scripts/` and shellcheck-
+###   validated; equivalent to running task install's secrets-fill
+###   step in isolation.
+### - **GHCR pulls without auth**: bot quota, can flake on heavy CI days.
 
 name: E2E
 
@@ -48,57 +60,44 @@ jobs:
           version: 3.x
           repo-token: ${{ secrets.GITHUB_TOKEN }}
 
-      - name: Bootstrap env files (domain prompt piped)
-        # `task env:init` prompts for the public domain; pipe the localhost
-        # default. The script also extracts .env.symfony from the API image,
-        # auto-generates APP_SECRET + JWT_PASSPHRASE, and bumps serverVersion.
+      - name: task env:init (domain prompt piped)
         run: printf 'os2display.localhost\n' | task env:init
 
-      - name: APP_ENV=prod workaround (upstream image bug)
-        # The image's bundled .env defaults to APP_ENV=dev but the image is
-        # built --no-dev, so dev-mode boot fails on a missing WebProfilerBundle.
-        # Drop this step once the fix from display-api-service#435 ships
-        # in a 3.0.0-rc3+ tag.
+      - name: APP_ENV=prod workaround (upstream image bug — see header)
         run: sed -i 's|^APP_ENV=dev|APP_ENV=prod|' .env.symfony
 
-      - name: Configure traefik (cert-file branch, prompts piped)
-        # Choice 2 = cert-file. Domain = traefik.localhost. LE email
-        # skipped via blank line (cert-file branch ignores it). Cert
-        # filename defaults accepted via blank lines. Dashboard
-        # admin / password.
-        run: |
-          printf '2\ntraefik.localhost\n\n\nadmin\npassword\n' | task env:traefik
+      - name: task env:traefik (cert-file branch, prompts piped)
+        # Choice 2 = cert-file. Domain = traefik.localhost. LE email blank
+        # (cert-file branch ignores it). Cert filename defaults accepted
+        # via blank lines. Dashboard admin / password.
+        run: printf '2\ntraefik.localhost\n\n\nadmin\npassword\n' | task env:traefik
 
-      - name: Generate self-signed cert
+      - name: task dev:cert
         run: task dev:cert
 
-      - name: Bring stack up
-        # `task install`'s chained `task: <subtask>` calls don't pipe
-        # stdin reliably for tenant:add / user:add (B23 caveat), so
-        # we step through manually below. install-secrets.sh
-        # (auto-fills MARIADB_*=CHANGE_ME sentinels) gets called first
-        # to match the real `task install` order.
-        run: |
-          ./scripts/install-secrets.sh
-          docker compose --env-file .env --env-file .env.traefik pull
-          docker compose --env-file .env --env-file .env.traefik up --detach --wait
+      - name: install-secrets (would be `task install`'s first step)
+        run: ./scripts/install-secrets.sh
 
-      - name: Run database migrations
-        run: |
-          docker compose --env-file .env --env-file .env.traefik exec -T os2display \
-            bin/console --no-interaction app:update
+      - name: task up
+        # `task up` uses `compose up -d --wait`, so this blocks until every
+        # service's healthcheck passes. Migrations on the next step can
+        # connect immediately.
+        run: task up
 
-      - name: Generate JWT keypair
-        run: |
-          docker compose --env-file .env --env-file .env.traefik exec -T os2display \
-            bin/console lexik:jwt:generate-keypair --skip-if-exists
+      - name: task console -- app:update (Doctrine migrations)
+        run: task console -- --no-interaction app:update
 
-      - name: Create tenant + admin user
-        run: |
-          docker compose --env-file .env --env-file .env.traefik exec -T os2display \
-            bin/console app:tenant:add e2e 'E2E Tenant' 'CI'
-          docker compose --env-file .env --env-file .env.traefik exec -T os2display \
-            bin/console app:user:add admin@os2display.localhost e2epassword 'E2E Admin' admin e2e
+      - name: task console -- lexik:jwt:generate-keypair
+        run: task console -- lexik:jwt:generate-keypair --skip-if-exists
+
+      - name: task console -- app:tenant:add
+        run: task console -- app:tenant:add e2e 'E2E Tenant' CI
+
+      - name: task console -- app:user:add
+        run: task console -- app:user:add admin@os2display.localhost e2epassword 'E2E Admin' admin e2e
+
+      - name: task templates:install
+        run: task templates:install
 
       - name: Smoke test — admin route over HTTPS
         # --resolve maps os2display.localhost → 127.0.0.1; --cacert trusts
@@ -111,7 +110,6 @@ jobs:
             https://os2display.localhost/admin/ \
             -o /dev/null
 
-      - name: Tear down
+      - name: task dev:teardown
         if: always()
-        run: |
-          docker compose --env-file .env --env-file .env.traefik down --volumes --remove-orphans
+        run: task -y dev:teardown

CHANGELOG.md

@@ -59,6 +59,21 @@ and aligned to the v3 image's env contract. **For 1.x → 3.x operators: see
   Catches install-path regressions the static checks can't (image boot, migration replay,
   Traefik routing). Triggers on `pull_request` against `release/**` and pushes to
   `release/**` — too slow (~5 min wall-clock) for every casual PR.
+- **`task up` / `install` / `update` now use `compose up --wait`** — blocks until every
+  service's healthcheck passes before returning, instead of returning immediately and
+  leaving the operator to poll. Matches the operator mental-model "after `task up` the
+  stack is up". Replaces `task install`'s previous `sleep 20` hack.
+- **`docker-compose.yml` `depends_on` graph filled in.** B6 added healthchecks to every
+  service but only wired one dependency (`nginx-api → os2display: service_healthy`).
+  Migrations on a fresh install raced the database — `task install` papered over with a
+  20-second sleep. Now:
+  - `os2display` waits for `mariadb` (`service_healthy`, `required: false` so the dep is
+    skipped when an operator runs against an external db / drops the `mariadb` profile)
+    and `redis` (`service_healthy`).
+  - `traefik` waits for `socket-proxy` (`service_healthy`) — traefik queries socket-proxy
+    for docker-label discovery, no point asking before the proxy is reachable.
+  Combined with `--wait` above, `task up` returns when the stack is genuinely ready, with
+  startup ordering enforced at the container level rather than via wall-clock guesswork.
 - `scripts/` directory: extracted helpers for the longer Taskfile bodies that the upstream
   [Taskfile style guide](https://taskfile.dev/styleguide/) recommends moving out
   ("Prefer using external scripts instead of multi-line commands"). `host-resources.sh`,

README.md

@@ -978,7 +978,7 @@ churn for the casual review cycle.
 Lifecycle
   install              Install the project — first-time setup (interactive)
   update               Pull images, recreate containers, run app:update
-  up                   Start the stack without recreating containers
+  up                   Start the stack; blocks until healthchecks pass
   down                 Remove all containers (preserves named volumes)
   stop                 Stop all containers
   purge                Remove all containers AND named volumes  (prompts)

Taskfile.yml

@@ -31,9 +31,7 @@ tasks:
       - echo "Installing"
       - ./scripts/install-secrets.sh
       - "{{.COMPOSE}} pull"
-      - "{{.COMPOSE}} up --force-recreate --detach --remove-orphans"
-      - echo "Waiting for database to be ready"
-      - sleep 20
+      - "{{.COMPOSE}} up --force-recreate --detach --remove-orphans --wait"
       - echo "Run database migrations and required updates"
       - task: console
         vars: { CLI_ARGS: "app:update" }
@@ -55,18 +53,18 @@ tasks:
         msg: ".env.symfony missing — run 'task env:init' first."
     cmds:
       - "{{.COMPOSE}} pull"
-      - "{{.COMPOSE}} up --force-recreate --detach --remove-orphans"
+      - "{{.COMPOSE}} up --force-recreate --detach --remove-orphans --wait"
       - task: console
         vars: { EXEC_FLAGS: "--user deploy", CLI_ARGS: "app:update" }
       - echo "NB! Run 'task templates:install' if the new image added templates."
 
   up:
-    desc: Start the stack without recreating containers
+    desc: Start the stack without recreating containers; blocks until healthchecks pass
     preconditions:
       - sh: "[ -f .env.symfony ]"
         msg: ".env.symfony missing — run 'task env:init' first."
     cmds:
-      - "{{.COMPOSE}} up -d"
+      - "{{.COMPOSE}} up -d --wait"
 
   down:
     desc: Remove all containers (preserves named volumes)

docker-compose.yml

@@ -47,6 +47,14 @@ services:
         required: true
       - path: .env.php.local
         required: false
+    depends_on:
+      # mariadb is profile-gated (`mariadb`). `required: false` lets compose
+      # skip this dep when an operator runs against an external database.
+      mariadb:
+        condition: service_healthy
+        required: false
+      redis:
+        condition: service_healthy
     healthcheck:
       test: ["CMD", "php", "-r", "exit(@fsockopen('127.0.0.1', 9000) ? 0 : 1);"]
       interval: 30s
@@ -147,6 +155,12 @@ services:
         required: true
       - path: .env.traefik.local
         required: false
+    depends_on:
+      # Traefik queries the docker socket via socket-proxy for label
+      # discovery; wait for the proxy to pass its /version healthcheck
+      # before traefik starts asking it for routes.
+      socket-proxy:
+        condition: service_healthy
     ports:
       - "80:80"
       - "443:443"