Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,9 @@ See the [v2.x.x — Skipped](#v2xx---skipped) entry below for why this major ski
templates.
- `task env:diff` compares your `.env.symfony` against the example shipped in the
currently-pinned API image.
- `task env:migrate` stages a v3 `.env.symfony` from `env.3x` (the export the 1.x
`task upgrade_prep` produces via the 2.8 API's `app:utils:convert-env-to-3x`) when present —
splitting off the infrastructure advisory — and falls back to the sed rewrite of a 1.x
`.env.docker.local` otherwise.
- `task env:migrate` refines the `.env.symfony.migrated` that the 1.x `task env_migrate`
produces via the 2.8 API's `app:utils:convert-env-to-3x` — splitting off the infrastructure
advisory — and falls back to the sed rewrite of a 1.x `.env.docker.local` otherwise.
- Per-service `.env.<svc>.local` override layer for site-specific tuning; gitignored.
- Compose profiles (`COMPOSE_PROFILES=mariadb,traefik`) gate built-in services — drop
`mariadb` for an external DB, drop `traefik` for an external proxy.
Expand All @@ -48,7 +47,7 @@ See the [v2.x.x — Skipped](#v2xx---skipped) entry below for why this major ski
- The mariadb service sets `stop_grace_period: 1m`, so the engine gets room for a clean shutdown
instead of being SIGKILLed mid-flush by the 10s default — important right before a major bump.
- `UPGRADE.md` restructured around the 2.8 converter: a pre-upgrade checklist that exports the
running 1.x configuration (`task upgrade_prep`) *before* the stack is stopped, converter-first
running 1.x configuration (`task env_migrate`) *before* the stack is stopped, converter-first
env migration with the sed rename demoted to a fallback, MariaDB auto-upgrade and data-volume
continuity notes, and the `rm docker-compose.yml` step needed before `git checkout`.
- New MariaDB diagnostics: `task db:metrics`, `db:processes`, `db:errors`.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1250,7 +1250,7 @@ Lifecycle
Bootstrap and env-file tooling
env:init Bootstrap .env.symfony from the API image
env:diff Compare .env.symfony against the image's shipped example
env:migrate Stage .env.symfony.migrated from env.3x (preferred) or a 1.x .env.docker.local
env:migrate Refine the 1.x 'task env_migrate' output, or convert a 1.x .env.docker.local
env:traefik Interactive .env.traefik setup (alias: traefik_env)

Operations
Expand Down
60 changes: 31 additions & 29 deletions Taskfile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -285,33 +285,35 @@ tasks:
rm -f "$tmp"

env:migrate:
desc: Stage a 3.x-shaped .env.symfony.migrated — from env.3x (preferred) or a 1.x .env.docker.local
desc: Stage a 3.x-shaped .env.symfony.migrated — refine the one from 1.x 'task env_migrate', or convert a 1.x .env.docker.local
cmds:
- |
OUT=.env.symfony.migrated

# Preferred path: env.3x produced by `task upgrade_prep` on the 1.x
# stack (the API's app:utils:convert-env-to-3x command). It already
# carries the loaded values under their 3.x names AND the admin/client
# config.json conversion the sed path below can't see — so when it's
# present we just split off the trailing infrastructure-advisory
# comment block (COMPOSE_*/PHP_*/NGINX_*/MARIADB_* notes) into a
# reminder, and the rest becomes .env.symfony directly.
if [ -f env.3x ]; then
echo "Found env.3x from the 1.x upgrade prep — using it as the source."
# env.3x ends with a trailing advisory block the converter
# introduces with a fixed marker line; everything above it is the
# application env. Split at the marker: app env to .env.symfony,
# the advisory to its own file (surfaced below, not buried in the
# app env).
ADVISORY=.env.symfony.infra-advisory
ADVISORY=.env.symfony.infra-advisory

# Preferred path: .env.symfony.migrated produced on the 1.x stack by
# `task env_migrate` (the API's app:utils:convert-env-to-3x command),
# carried across the branch switch (it is gitignored on both branches).
# It already carries the loaded values under their 3.x names AND the
# admin/client config.json conversion the sed path below can't see — so
# all that's left is to split the trailing infrastructure-advisory
# comment block (COMPOSE_*/PHP_*/NGINX_*/MARIADB_* notes) out of it.
# Done in place: input and output are the same file.
if [ -f "$OUT" ]; then
echo "Found $OUT from the 1.x 'task env_migrate' — splitting the infra advisory."
# The advisory is a trailing block the converter introduces with a
# fixed marker line; everything above it is the application env. Split
# at the marker: app env stays in $OUT, the advisory goes to its own
# file (surfaced below, not buried in the app env). Re-running is a
# no-op once the advisory has already been removed.
rm -f "$ADVISORY"
TMP=$(mktemp "${OUT}.XXXXXX")
awk -v adv="$ADVISORY" '
/^# The following loaded variables are NOT read/ { infra = 1 }
infra { print > adv; next }
{ print }
' env.3x > "$OUT"
echo "Wrote $OUT (from env.3x)."
' "$OUT" > "$TMP"
mv "$TMP" "$OUT"
if [ -s "$ADVISORY" ]; then
echo ""
echo "Infrastructure variables were split into $ADVISORY."
Expand All @@ -321,28 +323,28 @@ tasks:
echo " NGINX_* -> .env.nginx MARIADB_* -> .env.mariadb"
fi
echo ""
echo "Review with: diff -u env.3x $OUT"
echo "Apply with: mv $OUT .env.symfony"
echo "Review $OUT, then apply with: mv $OUT .env.symfony"
echo ""
echo "Still set by hand: DATABASE_URL serverVersion=11.4.10-MariaDB"
echo "(after the MariaDB upgrade — see UPGRADE.md step 6)."
echo "(after the MariaDB upgrade — see UPGRADE.md step 4)."
exit 0
fi

# Fallback path: no env.3x (older 1.x without the converter command, or
# the stack is already down). Convert a 1.x .env.docker.local by sed —
# an APP_ strip plus the per-site renames the converter would do.
# Fallback path: no .env.symfony.migrated (the 1.x install predated the
# converter, or the stack was already down). Convert a 1.x
# .env.docker.local by sed — an APP_ strip plus the per-site renames the
# converter would do.
if [ -f .env.docker.local ]; then
SRC=.env.docker.local
elif [ -f .env.local ]; then
SRC=.env.local
else
echo "Error: no env.3x, .env.docker.local or .env.local found."
echo "On the 1.x stack, prefer 'task upgrade_prep' to produce env.3x."
echo "Error: no .env.symfony.migrated, .env.docker.local or .env.local found."
echo "On the 1.x stack, prefer 'task env_migrate' to produce .env.symfony.migrated."
exit 1
fi
echo "No env.3x found — falling back to the sed conversion of $SRC."
echo "(The env.3x path from 'task upgrade_prep' on 1.x is preferred:"
echo "No .env.symfony.migrated found — falling back to the sed conversion of $SRC."
echo "(Producing .env.symfony.migrated with 'task env_migrate' on 1.x is preferred:"
echo " it also converts the admin/client config.json, which sed can't.)"
echo ""
# Strip APP_ prefix from every key EXCEPT the framework-defined trio
Expand Down
57 changes: 29 additions & 28 deletions UPGRADE.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,8 @@ The full breaking-change list is in [CHANGELOG.md](CHANGELOG.md) under
- **Symfony env vars lose the `APP_` prefix** (except `APP_ENV` and `APP_SECRET`, which Symfony
defines). `APP_DATABASE_URL` becomes `DATABASE_URL`, `APP_JWT_PASSPHRASE` becomes
`JWT_PASSPHRASE`, etc. The 2.8 API ships `app:utils:convert-env-to-3x`, which does the rename
for you (and converts the admin/client `config.json`) — this repo's `task upgrade_prep` runs
it; see the pre-upgrade checklist below.
for you (and converts the admin/client `config.json`) — this repo's `task env_migrate` (on the
1.x branch) runs it; see the pre-upgrade checklist below.
- **One env file per service.** The single `.env.docker.local` is gone, replaced by
`.env.symfony` + `.env.php` + `.env.nginx` + `.env.mariadb` + `.env.traefik` + the
compose-orchestration `.env`. See [README § Configuration files](README.md#configuration-files).
Expand All @@ -63,7 +63,7 @@ The full breaking-change list is in [CHANGELOG.md](CHANGELOG.md) under
### Pre-upgrade checklist (while still on 1.x)

Everything in this section runs on the **1.x** checkout, with the stack **still up**. The
configuration export (`task upgrade_prep`) reads the running application and fetches the live
configuration export (`task env_migrate`) reads the running application and fetches the live
admin/client `config.json` — it cannot run once the stack is stopped. Confirm prerequisites
first:

Expand All @@ -84,29 +84,30 @@ Then, on the 1.x stack:
- [ ] **Be on the final 1.x release running 2.8 API images.** The export command ships with
`os2display-api-service` 2.8.0. Set `COMPOSE_VERSION_API=2.8.0` (or a later 2.x) in
`.env.docker.local` and `task install` to pull and recreate. The last 1.x release of this repo
(v1.2.0) provides `task upgrade_check` / `task upgrade_prep`; upgrade to it first if you are on
(v1.2.0) provides `task upgrade_check` / `task env_migrate`; upgrade to it first if you are on
an earlier 1.x.
- [ ] **Run the pre-flight check:** `task upgrade_check`. It confirms the api image provides
`app:utils:convert-env-to-3x` and prints the bundled MariaDB volume name
(`<COMPOSE_PROJECT_NAME>_mariadb`). **Record that volume name** — the 3.x stack reuses it in
place, and only does so when the 3.x `.env` keeps the same `COMPOSE_PROJECT_NAME`. A different
project name silently boots an *empty* database (see step 4).
- [ ] **Back up the database:** `task backup_db` (1.x), and keep the dump somewhere off-host.
- [ ] **Export the configuration in 3.x shape:** `task upgrade_prep`. It writes `env.3x` — the
loaded env converted to 3.x names *plus* the admin/client `config.json` conversion. The
trailing advisory block lists infrastructure variables (`COMPOSE_*`, `PHP_*`, `NGINX_*`,
`MARIADB_*`) that move to per-service files in 3.x, never into the application env.
- [ ] **Export the configuration in 3.x shape:** `task env_migrate` (1.x). It writes
`.env.symfony.migrated` — the loaded env converted to 3.x names *plus* the admin/client
`config.json` conversion. A trailing advisory block lists infrastructure variables
(`COMPOSE_*`, `PHP_*`, `NGINX_*`, `MARIADB_*`) that move to per-service files in 3.x, never
into the application env.
- [ ] **Copy env files + JWT keys aside** in case a rollback is needed:

```bash
mkdir -p /tmp/os2display-1x-backup
cp .env .env.docker.local .env.local env.3x /tmp/os2display-1x-backup/ # whatever subset you have
cp .env .env.docker.local .env.local .env.symfony.migrated /tmp/os2display-1x-backup/ # whatever subset you have
cp -r jwt/ /tmp/os2display-1x-backup/
```

`env.3x` contains every application secret (`APP_SECRET`, database and OIDC credentials, ...). It
is gitignored on both branches; treat it like a credentials file and it will survive the branch
switch in step 2 untouched.
`.env.symfony.migrated` contains every application secret (`APP_SECRET`, database and OIDC
credentials, ...). It is gitignored on both branches; treat it like a credentials file and it
will survive the branch switch in step 2 untouched.

You do **not** need to copy `./media/` aside for the rollback. It's a host bind-mount, not a
docker named volume — `task purge`, `task down --volumes`, and `docker compose down --volumes`
Expand Down Expand Up @@ -169,15 +170,15 @@ per-service `env_file:` directives:
| `.env.mariadb` | MariaDB credentials. |
| `.env.traefik` | Traefik dashboard auth, Let's Encrypt email, cert provider. |

Build `.env.symfony` from the `env.3x` you exported in the pre-upgrade checklist, then bootstrap
the rest:
Build `.env.symfony` from the `.env.symfony.migrated` you exported in the pre-upgrade checklist
(it survives the branch switch — gitignored on both branches), then bootstrap the rest:

```bash
# 1. Application config from env.3x. task env:migrate detects env.3x, splits off the
# infrastructure advisory (into .env.symfony.infra-advisory) and writes the rest to
# .env.symfony.migrated for review.
# 1. Application config. task env:migrate finds the .env.symfony.migrated produced by 'task
# env_migrate' on 1.x and splits its trailing infrastructure advisory into
# .env.symfony.infra-advisory, leaving the clean application env in .env.symfony.migrated.
task env:migrate
diff -u env.3x .env.symfony.migrated # sanity check
$EDITOR .env.symfony.migrated # sanity check
mv .env.symfony.migrated .env.symfony

# 2. Per-service files + .env. task env:init creates .env (prompts for the domain) and the
Expand All @@ -194,8 +195,8 @@ $EDITOR .env
# 4. Finish .env.symfony.
$EDITOR .env.symfony
# - Set DATABASE_URL serverVersion to "11.4.10-MariaDB" (post-MariaDB upgrade — see step 4).
# env.3x carries the old 10.x serverVersion through verbatim; this is the one value env:migrate
# leaves for you.
# The converter carries the old 10.x serverVersion through verbatim; this is the one value
# env:migrate leaves for you.
# - Distribute the keys from .env.symfony.infra-advisory: COMPOSE_* -> .env, PHP_* -> .env.php,
# NGINX_* -> .env.nginx, MARIADB_* -> .env.mariadb.

Expand All @@ -215,17 +216,17 @@ task env:diff
```

<details>
<summary>Manual fallback (no `env.3x` — pre-2.8 images, or stack already stopped)</summary>
<summary>Manual fallback (no `.env.symfony.migrated` — pre-2.8 images, or stack already stopped)</summary>

If you never produced `env.3x` (the 1.x install predates the converter, or the stack is already
down), `task env:migrate` falls back to a sed conversion of `.env.docker.local`: it strips the
`APP_` prefix from every key except the framework-defined trio (`APP_ENV` / `APP_SECRET` /
`APP_DEBUG`) and writes `.env.symfony.migrated`. This path **cannot** see the admin/client
`config.json`, so you also convert those by hand and apply the per-site renames the converter
would have done:
If you never produced `.env.symfony.migrated` (the 1.x install predates the converter, or the
stack was already down), `task env:migrate` falls back to a sed conversion of `.env.docker.local`:
it strips the `APP_` prefix from every key except the framework-defined trio (`APP_ENV` /
`APP_SECRET` / `APP_DEBUG`) and writes `.env.symfony.migrated`. This path **cannot** see the
admin/client `config.json`, so you also convert those by hand and apply the per-site renames the
converter would have done:

```bash
task env:migrate # sed path when env.3x is absent
task env:migrate # sed path when .env.symfony.migrated is absent
diff -u .env.docker.local .env.symfony.migrated
$EDITOR .env.symfony.migrated
```
Expand Down
Loading