Add configurable TLS support to frontend-proxy

The frontend-proxy (Caddy) now supports three TLS modes controlled by
tls.mode in settings.yaml:
- off: plain HTTP on a single port (previous behavior)
- internal: self-signed certificate with HTTPS + plain HTTP side-by-side
- acme: Let's Encrypt certificate via ACME with automatic renewal

The Caddyfile is rendered at container start from a Jinja2 template
(Caddyfile.prod.j2) by render_caddyfile.py based on environment
variables. Shared routing directives live in Caddyfile.routes, imported
by all prod configurations.

New settings: tls.mode, tls.domain (required when TLS is enabled),
tls.redirect_http_to_https, and ports.https_host_port. New installs
default to internal mode on ports 8080/8443. Existing configs are
backfilled to off mode preserving current port assignments.

A caddy-data Docker volume persists certificate state across restarts.

Author: Nils Bars

README.md

@@ -4,6 +4,59 @@ The framework consists of multiple components that have to be built the first ti
 
 The following describes how to build REF, how to run it, and how to upgrade it. To learn more about creating new exercises, head to [exercises.md](./EXERCISES.md).
 
+### Configuration
+
+All configuration lives in `settings.yaml`, the single source of truth. On first use, `./ctrl.sh build` auto-runs `./prepare.py` which generates `settings.yaml` with cryptographically random secrets and renders two downstream artifacts from it:
+
+* **`settings.env`** — environment variables consumed by docker-compose.
+* **`docker-compose.yml`** — rendered from `docker-compose.template.yml`.
+
+To inspect or edit `settings.yaml` before the first build, run `./prepare.py` manually. After editing an existing `settings.yaml`, re-run `./prepare.py` to propagate changes to the downstream files. The script backfills new fields and prunes obsolete ones automatically, so it is safe to re-run on upgrades. Use `./prepare.py --fresh` to regenerate everything from scratch (the old `settings.yaml` is backed up first).
+
+#### Settings reference
+
+| Section | Key | Default | Description |
+|---------|-----|---------|-------------|
+| `ports` | `ssh_host_port` | 2222 | SSH reverse-proxy listen port |
+| | `http_host_port` | 8080 | HTTP port (plain or redirect) |
+| | `https_host_port` | 8443 | HTTPS port |
+| `tls` | `mode` | `internal` | `off` (plain HTTP), `internal` (self-signed), or `acme` (Let's Encrypt) |
+| | `domain` | — | Required for `internal` and `acme` modes |
+| | `redirect_http_to_https` | `false` | Redirect HTTP port to HTTPS (`internal` and `acme` modes only) |
+| `paths` | `data` | `./data` | Persistent data directory on the host |
+| | `exercises` | `./exercises` | Exercise definitions directory |
+| `runtime` | `binfmt_support` | `false` | Enable multi-architecture container support |
+| `admin` | `password` | *(random)* | Admin password (username is `0`); printed on first run |
+| | `ssh_key` | — | Optional SSH public key for the admin account |
+
+Secrets (`secrets` section) are auto-generated and should not normally be edited. They include the Flask session key, the HMAC key shared between the SSH proxy and the web API, and the PostgreSQL password.
+
+#### TLS modes
+
+The `tls.mode` setting controls how the frontend-proxy serves traffic:
+
+| Mode | Ports | Behavior |
+|------|-------|----------|
+| `off` | `http_host_port` only | Plain HTTP on a single port. No TLS. |
+| `internal` | `http_host_port` + `https_host_port` | Self-signed TLS certificate (generated by Caddy). HTTPS on `https_host_port`, plain HTTP on `http_host_port`. Both serve the full site independently by default. Set `redirect_http_to_https: true` to redirect HTTP to HTTPS instead. Accessible by domain and by IP. |
+| `acme` | `http_host_port` + `https_host_port` | Let's Encrypt certificate via ACME. HTTP automatically redirects to HTTPS. Requires `domain` to resolve to the server and host ports 80 + 443 to be publicly reachable. |
+
+After changing `tls.mode` or `tls.domain`, run `./prepare.py && ./ctrl.sh build && ./ctrl.sh restart`.
+
+#### Web entry points
+
+All HTTP traffic is served through a single Caddy reverse proxy
+(`frontend-proxy/`). Students reach `/spa/register`; admins reach
+`/admin/` (redirects to the exercise view). SSH connections go through
+the SSH reverse proxy on the configured SSH port.
+
+In production the Vue SPA is baked into the `frontend-proxy` image as a
+static bundle — rebuild with `./ctrl.sh build` after any SPA change.
+
+#### Hot reloading (development only)
+
+`./ctrl.sh up --hot-reloading` starts an extra Vite dev server for the SPA and enables Flask auto-reload. Do **not** use this on a publicly reachable host — Vite's dev server is not hardened (see `docs/ARCHITECTURE.md` for details).
+
 ### Building REF
 The build process is split into two parts. While the first part is mandatory and entails building the framework itself, the second part is only required if you plan to host exercises where ASLR is disabled for setuid binaries.
 
@@ -16,19 +69,11 @@ git clone git@github.com:remote-exercise-framework/ref.git
 cd ref
 git submodule update --init --recursive
 
-# Generate configuration (settings.yaml + settings.env), docker-compose.yml, and
-# container SSH keys. This MUST be run once before the first ./ctrl.sh build.
-# All secrets are generated with a cryptographically secure RNG. The generated
-# settings.yaml file is the canonical configuration; settings.env is rendered
-# from it for docker-compose. The admin password is printed at the end of the
-# run — note it down, it can also be found in settings.yaml.
-#
-# prepare.py refuses to run if settings.yaml already exists, so existing
-# secrets are never silently overwritten.
-./prepare.py
-
-# Build all images. This command will check if your system meets the requirements
-# and will print error messages in case something is not working as expected.
+# Optional: run prepare.py manually to inspect or edit settings.yaml
+# before building. If skipped, ctrl.sh build auto-runs it on first use
+# with secure random defaults.
+# ./prepare.py
+
 ./ctrl.sh build
 ```
 
@@ -44,39 +89,6 @@ After successfully building REF, the database has to be initialized:
 ./ctrl.sh down
 ```
 
-### Single-port web interface
-
-The Flask web app (`/`, `/admin/*`, `/api/*`), the Vue student SPA
-(`/spa/*`), and Flask's static assets (`/static/*`) are all fronted by
-a single Caddy reverse proxy (`frontend-proxy/`) on host port 8000.
-Students go to `http://<host>:8000/spa/register` (or whichever landing
-page is configured); admins go to `http://<host>:8000/admin/` (which
-redirects to the exercise view). Student SSH connections go through the
-SSH reverse proxy on port 2222.
-
-In production, `frontend-proxy` serves the Vue SPA as a static bundle
-that is baked into the image at `docker build` time via a multi-stage
-Dockerfile — rebuild with `./ctrl.sh build` after any change under
-`spa-frontend/`.
-
-### Hot reloading (local development only)
-
-Use `./ctrl.sh up --hot-reloading` during local development to get
-Flask auto-reload **and** Vite HMR for the SPA. This flag starts an
-extra `spa-frontend` container (running `vite dev`) which is gated
-behind the `dev` compose profile; Caddy then reverse-proxies `/spa/*`
-(including the HMR websocket) to that container.
-
-> ⚠️ **Do not run `--hot-reloading` on a publicly reachable host.**
-> Vite's dev server is not designed for hostile clients — it serves
-> raw source, exposes `/@fs/`, and has had several path-traversal CVEs
-> in recent releases. While `--hot-reloading` is active, the SPA
-> renders a yellow/black hazard-striped warning strip at the top of
-> every page as a visible reminder; the whole warning is tree-shaken
-> out of the production `vite build`, so nothing about it leaks into
-> the prod bundle. See `docs/ARCHITECTURE.md` for the full model.
-
-
 #### Build the custom Linux kernel
 Building the custom Linux kernel is only required if you need the `no-randomize` attribute for some exercises. This attribute allows you to disable ASLR for a specific binary, even if it is a setuid binary. This is not allowed for unmodified kernels. The following assumes that your system is based on Debian and uses GRUB as a bootloader. For other systems or bootloaders, the instructions have to be adapted accordingly.
 
@@ -183,18 +195,18 @@ In case the update fails, remove the `data` directory and move the backup to `da
 After starting the application, the following services are running on the host:
 
 #### SSH Entry-Server
-This services is the entry server for all SSH connections to the exercises. Based on the clients user name and the public key, incoming SSH connection are forwarded to a container of the respective exercise.
+The entry server for all SSH connections to the exercises. Based on the client's username and public key, incoming SSH connections are forwarded to a container of the respective exercise.
 
 ```
 Hostname: ssh-reverse-proxy
-Port: 2222
+Port: See settings.yaml (ports.ssh_host_port, default 2222)
 ```
 
 #### Web Interface
-The web interface for managing exercises and users. Students also use it to register. All HTTP traffic is served through the `frontend-proxy` (Caddy) on port 8000, which reverse-proxies to the `web` (Flask) service internally.
+The web interface for managing exercises and users. Students also use it to register. All HTTP traffic is served through the `frontend-proxy` (Caddy), which reverse-proxies to the `web` (Flask) service internally.
 ```
 Hostname: frontend-proxy (host-facing), web (internal Flask app)
-Port: 8000
+Port: See settings.yaml (ports.http_host_port / ports.https_host_port)
 User: 0
 Password: See settings.yaml (admin.password)
 ```

docker-compose.template.yml

@@ -1,6 +1,7 @@
 version: "3.7"
-{% if testing %}
 volumes:
+    caddy-data:
+{% if testing %}
     coverage-data:
         name: "{{ prefix }}_coverage_data"
 {% endif %}
@@ -125,21 +126,43 @@ services:
             dockerfile: frontend-proxy/Dockerfile
         environment:
             - HOT_RELOADING=${HOT_RELOADING:-false}
+            - TLS_MODE=${TLS_MODE:-off}
+            - DOMAIN=${TLS_DOMAIN:-}
+            - HTTPS_HOST_PORT=${HTTPS_HOST_PORT:-8443}
+            - REDIRECT_HTTP_TO_HTTPS=${TLS_REDIRECT_HTTP:-false}
         volumes:
             # Serve Flask's static assets directly from Caddy, skipping
             # uWSGI. Read-only to keep the proxy sandboxed.
             - ./webapp/ref/static:/srv/flask-static:ro
-    {% if not testing %}
+            # Persist Caddy's TLS certificates and ACME state across
+            # container restarts. Essential for acme mode (avoids
+            # hitting Let's Encrypt rate limits).
+            - caddy-data:/data
+    {%- if not testing %}
         ports:
+    {%- if tls_mode == 'off' %}
             - "${HTTP_HOST_PORT}:8000"
-    {% endif %}
+    {%- elif tls_mode == 'internal' %}
+            - "${HTTP_HOST_PORT}:8080"
+            - "${HTTPS_HOST_PORT}:8443"
+    {%- elif tls_mode == 'acme' %}
+            - "${HTTP_HOST_PORT}:80"
+            - "${HTTPS_HOST_PORT}:443"
+    {%- endif %}
+    {%- endif %}
         networks:
             - web-host
         depends_on:
             - web
         cgroup_parent: "{{ cgroup_parent }}-core.slice"
         healthcheck:
+    {%- if tls_mode == 'off' %}
             test: ["CMD", "wget", "-q", "--spider", "http://localhost:8000/static/favicon.ico"]
+    {%- elif tls_mode == 'internal' %}
+            test: ["CMD", "wget", "-q", "--spider", "--no-check-certificate", "https://localhost:8443/static/favicon.ico"]
+    {%- elif tls_mode == 'acme' %}
+            test: ["CMD", "wget", "-q", "--spider", "--no-check-certificate", "https://localhost:443/static/favicon.ico"]
+    {%- endif %}
             interval: 10s
             timeout: 3s
             retries: 5

docs/ARCHITECTURE.md

@@ -8,15 +8,15 @@ Remote Exercise Framework - A platform for hosting programming exercises with is
 ┌──────────────────────────────────────────────────────────────────────┐
 │                            HOST SYSTEM                               │
 ├──────────────────────────────────────────────────────────────────────┤
-│  Port 2222 ──> ssh-reverse-proxy (Rust) ──> Instance (SSH)           │
-│  Port 8000 ──> frontend-proxy (Caddy) ──┬─> web (Flask)              │
-│                                         ├─> spa-frontend (vite dev)  │
-│                                         └─> baked SPA dist/ (prod)   │
+│  ssh_host_port ──> ssh-reverse-proxy (Rust) ──> Instance (SSH)       │
+│  http(s)_host_port ──> frontend-proxy (Caddy) ──┬─> web (Flask)      │
+│                                                 ├─> spa-frontend     │
+│                                                 └─> baked SPA dist/  │
 └──────────────────────────────────────────────────────────────────────┘
 ```
 
-The `frontend-proxy` Caddy container terminates host port 8000 and routes
-traffic by URL prefix:
+The `frontend-proxy` Caddy container serves HTTP and/or HTTPS (depending
+on `tls.mode` in `settings.yaml`) and routes traffic by URL prefix:
 
 - `/spa/*` — the Vue SPA. In dev (`--hot-reloading`) proxied to the
   `spa-frontend` container running `vite dev` with HMR; in prod served as
@@ -28,7 +28,7 @@ traffic by URL prefix:
 - Everything else (`/`, `/admin/*`, `/api/*`, `/student/*`) — reverse-proxied
   to the Flask `web` container on the internal `web-host` network.
 
-The `ssh-reverse-proxy` still calls `http://web:8000` over the internal
+The `ssh-reverse-proxy` calls `http://web:8000` over the internal
 `web-and-ssh` network and does **not** go through Caddy.
 
 ## Components
@@ -116,22 +116,39 @@ Isolated Docker container per student/exercise based on Ubuntu 24.04.
 
 ### 2b. Frontend Proxy (`frontend-proxy/`)
 
-Caddy-based reverse proxy container that terminates host port 8000 and fans
-out to the Flask webapp and the Vue SPA. Built from a multi-stage
-Dockerfile that compiles the SPA bundle (stage 1: `node:22-alpine`,
-`npm run build`) and copies it into a `caddy:2-alpine` runtime image
-(stage 2). At container start, `entrypoint.sh` picks `Caddyfile.dev`
-(reverse-proxies `/spa/*` to `spa-frontend:5173` for HMR) or
-`Caddyfile.prod` (serves the baked `/srv/spa-dist` with SPA history-mode
-fallback) based on `$HOT_RELOADING`.
+Caddy-based reverse proxy container that serves the web interface over
+HTTP and/or HTTPS depending on `tls.mode` in `settings.yaml`. Built from
+a multi-stage Dockerfile that compiles the SPA bundle (stage 1:
+`node:22-alpine`, `npm run build`) and copies it into a
+`caddy:2.8-alpine` runtime image (stage 2) along with Python 3 and
+Jinja2.
 
-**Stack:** Caddy 2 + multi-stage Node builder
+At container start, `entrypoint.sh` either uses `Caddyfile.dev` (for
+`--hot-reloading`) or calls `render_caddyfile.py` which renders
+`Caddyfile.prod.j2` into a Caddyfile based on `TLS_MODE`, `DOMAIN`, and
+`HTTPS_HOST_PORT` environment variables passed from docker-compose.
+
+**TLS modes** (set via `tls.mode` in `settings.yaml`):
+
+| Mode | Container ports | Description |
+|------|-----------------|-------------|
+| `off` | `:8000` (HTTP) | Plain HTTP, no TLS. |
+| `internal` | `:8443` (HTTPS) + `:8080` (HTTP) | Self-signed certificate generated by Caddy. Both ports serve the full site independently; the HTTP port does not redirect to HTTPS by default. Accessible by domain name and by IP address. |
+| `acme` | `:443` (HTTPS) + `:80` (HTTP) | Let's Encrypt certificate via ACME. Caddy handles provisioning and renewal automatically. HTTP redirects to HTTPS. |
+
+A `caddy-data` Docker volume persists Caddy's certificate storage across
+container restarts (essential for `acme` mode to avoid hitting Let's
+Encrypt rate limits).
+
+**Stack:** Caddy 2 + Python 3 / Jinja2 + multi-stage Node builder
 
 **Key files:**
-- `Dockerfile` — multi-stage SPA build + Caddy runtime
+- `Dockerfile` — multi-stage SPA build + Caddy runtime with Python/Jinja2
 - `Caddyfile.dev` — dev routing (proxies `/spa/*` to vite dev)
-- `Caddyfile.prod` — prod routing (serves baked dist with cache headers)
-- `entrypoint.sh` — selects config based on `HOT_RELOADING`
+- `Caddyfile.prod.j2` — Jinja2 template rendered at container start per TLS mode
+- `Caddyfile.routes` — shared routing directives imported by all prod configs
+- `render_caddyfile.py` — renders the Jinja2 template from environment variables
+- `entrypoint.sh` — selects dev config or renders prod config, then starts Caddy
 
 **Notes:**
 - The Flask rate limiter reads `X-Tinyproxy` to key on the real client IP;
@@ -278,8 +295,7 @@ profile-gated services can still be built and cleaned up.
 > serving raw source over `/spa/*` (including the `/@fs/` endpoint,
 > which has had repeated path-traversal CVEs). The SPA itself renders
 > a hazard-striped warning strip at the top of every page in this mode
-> as a visible reminder. Intended use is local development against
-> `http://localhost:8000` only.
+> as a visible reminder. Intended for local development only.
 
 ## Test Structure