feat: v1.6.0 — Multi-Factor Authentication (Issue #18)

Adds opt-in TOTP-based Multi-Factor Authentication that is fully
backwards compatible with existing logins. Operators choose to enable
MFA per account; nothing changes for users who do not opt in.

Highlights
==========

* RFC 6238 TOTP (6 digits, 30s period, SHA1) with ±30s skew tolerance,
  compatible with Microsoft / Google Authenticator, Authy, Duo, 1Password.
* Per-step replay protection (`mfa_last_used_totp_step`) so a captured
  code cannot be reused inside the same window.
* Fernet-encrypted TOTP secrets at rest, key resolution via
  `MFA_ENCRYPTION_KEY` env (HKDF-derived from `SECRET_KEY` as fallback).
* 10 single-use, bcrypt-hashed backup codes per user, formatted
  `XXXX-YYYY` from a confusion-free alphabet (no 0/O/1/I/L).
* Two-step login flow: `POST /api/auth/login` returns `mfa_required`
  + `mfa_token`, then `POST /api/auth/login/mfa-verify` accepts a TOTP
  code OR a backup code. JWT is minted only after MFA succeeds.
* Self-service: users enable / disable MFA from their own row in the
  Users page; admins reset (single user or bulk) but never enable on
  behalf of someone else (matches AWS IAM / GitHub / Google Workspace).
* Bulk emergency reset CLI: `scripts/admin-mfa-reset-all.sh`.

Security hardening
==================

* Atomic transactions with `SELECT … FOR UPDATE` on `mfa_pending_logins`
  and `users` rows so concurrent verify / enroll calls cannot race.
* `/api/mfa/enroll/start` refuses re-enrollment when MFA is already on
  (prevents silent secret rotation via a stolen JWT).
* Pydantic `ValidationError` messages are sanitized before reaching the
  audit log so request bodies (TOTP / backup codes in flight) never
  appear in plaintext.
* Slowapi rate limits are per-USER, not per-IP, with a trusted-proxy
  XFF strategy so a single ingress address cannot exhaust the bucket
  for thousands of operators (`MFA_TRUSTED_PROXY_CIDRS`,
  `MFA_RATE_LIMIT_*` env-overridable).
* Login query now scopes to `is_active = TRUE` so a soft-deleted row
  with the same username can no longer occlude the active user
  (also closes a small account-enumeration side channel).

Database
========

Additive migrations (idempotent `ADD COLUMN IF NOT EXISTS`,
`CREATE TABLE IF NOT EXISTS`):

  - users: mfa_enabled, mfa_method, mfa_secret_encrypted,
    mfa_enrolled_at, mfa_last_used_at, mfa_last_used_totp_step
  - mfa_backup_codes (user_id ON DELETE CASCADE)
  - mfa_pending_logins (user_id ON DELETE CASCADE, challenge_token,
    attempts, expires_at)
  - mfa_pending_enrollments (user_id ON DELETE CASCADE)

Frontend
========

* Login page becomes a 3-phase state machine
  (credentials → MFA → submitting); legacy single-step login is
  preserved for users who haven't enrolled.
* New MFAEnrollModal (3-step wizard: QR + secret → verify → backup
  codes) using `qrcode.react`.
* Users page shows MFA column + per-row enable/disable/reset actions.
  Admins viewing other users with MFA off see a non-actionable info
  icon explaining that only the user themselves can enable MFA.

Deployment
==========

* `MFA_ENCRYPTION_KEY` is added to `k8s/manifests/03-secrets.yaml` as
  a placeholder; `SECRET_KEY` is also placeholder-ized so both are
  injected by the existing pipeline pattern (sed-replace + apply).
* No new build-time env vars are required for the frontend. The SPA
  uses `window.location.host` for `/api/*` and is routed by the
  existing nginx ingress configuration.
* `frontend/.dockerignore` ensures host `.env*` files cannot bleed
  into the production bundle.

Tests
=====

* New unit suites:
  - `test_mfa_service.py` (TOTP, encryption, backup codes)
  - `test_mfa_backwards_compat.py` (regression — non-MFA flow unchanged)
  - `test_mfa_rate_limits.py` (env override + dataclass immutability)
  - `test_mfa_rate_limit_key.py` (JWT key, trusted-proxy XFF, fallbacks)
* All existing 1000+ unit tests continue to pass.

Documentation
=============

* README MFA section (overview, day-to-day operations, emergency
  reset CLI, env variables, rate-limit tuning).
* `scripts/README.md` documents the bulk reset script.

Issue: #18

Author: taylanbakircioglu

.gitignore

@@ -39,9 +39,6 @@ venv.bak/
 *.sqlite
 *.sqlite3
 
-# Docker
-.dockerignore
-
 # Build-time staged version.json (CI `cp version.json backend/`).
 # The canonical file lives at repo root; this path is a transient
 # copy for the backend Docker build context.

README.md

@@ -1065,6 +1065,115 @@ The IP Inventory page provides a unified view of all IP addresses across every c
 - **API Keys**: User API key generation and management
 - **Role Assignment**: Dynamic role assignment and permission updates
 
+#### Multi-Factor Authentication (MFA) — v1.6.0 (Issue #18)
+
+MFA is **optional per account** and **default OFF**. Existing users keep their
+single-factor (username/password) login unless they choose to enable it. The
+feature is fully additive: nothing changes for accounts that don't opt in.
+
+**For end-users**
+
+- Open **Users** → find your own row → click **Enable MFA**.
+- A wizard opens with three steps:
+  1. **Set up** — scan the QR code with Google Authenticator / Authy /
+     1Password / Microsoft Authenticator, or paste the displayed secret manually.
+  2. **Verify** — enter the current 6-digit code from your app.
+  3. **Backup codes** — save the 10 single-use recovery codes (format
+     `XXXX-YYYY`). They are shown only once. Use the **Copy all** /
+     **Download .txt** buttons.
+- After enrollment your sign-in becomes two-step: username/password →
+  6-digit TOTP (or a backup code).
+- To turn MFA off again, open your row's **Disable MFA** action and enter a
+  current TOTP or backup code.
+
+**For admins**
+
+- On any user with MFA enabled, the **Reset MFA** action wipes the user's
+  TOTP secret, backup codes, and pending challenges. A required *reason* is
+  written to the audit log. After reset the user logs in with their password
+  and may re-enroll.
+
+**Emergency: reset MFA for every user**
+
+Use the CLI helper when an authenticator outage / mass key loss happens.
+Double confirmation is required; the action is irreversible.
+
+```bash
+API_URL=https://hap.example.com ADMIN_TOKEN=eyJ... \
+  ./scripts/admin-mfa-reset-all.sh
+# Prompts ask for: 'yes' → 'RESET ALL MFA' → reason
+# Audit log:      action='mfa.disabled.admin_bulk_reset'
+```
+
+**Configuration**
+
+- Backend env var `MFA_ENCRYPTION_KEY` — a 44-char Fernet key used to encrypt
+  TOTP secrets at rest. Generate with:
+  ```bash
+  python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
+  ```
+  On Kubernetes the key lives in the `backend-secret` Secret
+  (`k8s/manifests/03-secrets.yaml`). The shipped manifest uses the placeholder
+  `mfa_encryption_key_replace_me`; replace it via your CI/CD pipeline (e.g.
+  `sed` step) before `kubectl apply`.
+- Optional `MFA_ACCOUNT_LABEL_DOMAIN` — overrides the per-user otpauth label
+  domain so QR codes show e.g. `alice@hap.example.com` instead of the request
+  hostname.
+
+**API endpoints (all additive)**
+
+| Method | Path | Notes |
+|---|---|---|
+| `POST` | `/api/auth/login` | Returns `mfa_required:true`+`mfa_token` for MFA users; legacy shape otherwise |
+| `POST` | `/api/auth/login/mfa-verify` | Submits TOTP or backup code; returns JWT |
+| `GET`  | `/api/mfa/status` | Self status (enabled / method / backup codes remaining) |
+| `POST` | `/api/mfa/enroll/start` | Begin TOTP enrollment |
+| `POST` | `/api/mfa/enroll/confirm` | Confirm enrollment, returns 10 backup codes once |
+| `POST` | `/api/mfa/disable` | Self-disable (TOTP or backup required) |
+| `POST` | `/api/mfa/backup-codes/regenerate` | Issue 10 fresh backup codes (TOTP only) |
+| `GET`  | `/api/mfa/admin/status/{id}` | Admin: any user's MFA status |
+| `POST` | `/api/mfa/admin-reset/{id}` | Admin: reset a single user's MFA |
+| `POST` | `/api/mfa/admin-reset-all` | Admin: emergency reset for all users |
+
+**Rate limits (per-user, ingress-aware, operationally tunable)**
+
+MFA endpoints are rate-limited via slowapi+Redis with a **user-aware key
+function** (`backend/middleware/mfa_rate_limit_key.py`):
+
+1. If the request carries a valid Bearer JWT → bucket is `user:<id>`.
+   Each operator gets an isolated bucket; an org-wide MFA rollout is no
+   longer bottlenecked by the shared ingress IP.
+2. Else, if the TCP peer is in `MFA_TRUSTED_PROXY_CIDRS` → bucket is the
+   first `X-Forwarded-For` hop (real client IP behind the ingress).
+3. Else → bucket is the TCP peer (slowapi default).
+
+Defaults live in `backend/middleware/mfa_rate_limits.py` and are sized for
+**enterprise-scale** rollouts (thousands of operators). Each one is
+overridable via env var; an invalid string logs a `WARNING` and falls back
+to the default without crashing.
+
+| Endpoint | Env var | Default | Bucket |
+|---|---|---|---|
+| `POST /api/mfa/enroll/start` | `MFA_RATE_LIMIT_ENROLL_START` | `10/minute` | per user |
+| `POST /api/mfa/enroll/confirm` | `MFA_RATE_LIMIT_ENROLL_CONFIRM` | `10/minute` | per user |
+| `POST /api/mfa/disable` | `MFA_RATE_LIMIT_DISABLE` | `10/minute` | per user |
+| `POST /api/mfa/backup-codes/regenerate` | `MFA_RATE_LIMIT_REGENERATE_BACKUP_CODES` | `5/hour` | per user |
+| `POST /api/mfa/admin-reset/{id}` | `MFA_RATE_LIMIT_ADMIN_RESET` | `60/hour` | per admin |
+| `POST /api/mfa/admin-reset-all` | `MFA_RATE_LIMIT_ADMIN_RESET_ALL` | `1/day` | per admin |
+
+Limit string format follows slowapi: `<count>/<second|minute|hour|day>`.
+
+**Trusted-proxy configuration**
+`MFA_TRUSTED_PROXY_CIDRS` — comma-separated CIDR list, e.g.
+`10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`. Empty (default) disables XFF
+parsing — XFF from any peer is then ignored, which is the safe choice when
+the topology is unknown. Set this when your backend sits behind a known
+ingress / load balancer so anonymous flows still get per-real-IP buckets.
+
+The pre-existing `/api/auth/login` rate-limiting policy is unchanged. On
+Kubernetes, see commented overrides in
+`k8s/manifests/07-configmaps.yaml::backend-config`.
+
 ### Settings - System Configuration
 - **Theme Settings**: Light/dark mode toggle and UI customization
 - **ACME / SSL Automation**: Configure ACME provider, directory URL, staging mode, auto-renewal, EAB credentials, and test CA connectivity
@@ -1524,12 +1633,25 @@ AGENT_CONFIG_SYNC_INTERVAL_SECONDS=30
 ```
 
 #### Frontend Configuration
+
+The frontend is a Create-React-App single-page app served as a static bundle
+(`serve -s build`). It uses **same-origin** (`window.location.host`) for all
+`/api/*` calls — no env vars are needed in production. Routing is handled
+entirely by the nginx reverse proxy in front of the frontend pod (Kubernetes
+ingress + `nginx-config` ConfigMap, or `nginx/nginx.conf` for Docker Compose).
+
+> ⚠️ **Do NOT set `REACT_APP_API_URL` in your CI/CD pipeline.** CRA inlines
+> `REACT_APP_*` values into the bundle at **build time**, so any value baked
+> in there overrides the runtime same-origin detection and breaks every
+> deployment whose URL does not match the inlined string. Leave the variable
+> unset; the bundle will resolve to whatever host the user is browsing.
+
 ```bash
-# API Endpoint (auto-detected if empty)
-# Leave empty in production to use same-origin (window.location)
-REACT_APP_API_URL=""  # For development: "http://localhost:8000"
+# Optional, only when you intentionally need a cross-origin API
+# (then CORS_ORIGINS on the backend must include the SPA's origin):
+# REACT_APP_API_URL="https://api.example.com"
 
-# Environment
+# Build settings
 NODE_ENV="production"
 GENERATE_SOURCEMAP="false"
 ```

backend/database/migrations.py

@@ -1700,8 +1700,87 @@ async def run_all_migrations():
     # (cluster_id, bind_address, bind_port) WHERE is_active.
     await ensure_frontends_bind_unique_constraint()
 
+    # Issue #18 — TOTP MFA (v1.6.0): additive columns + 3 new tables
+    await ensure_mfa_columns()
+
     logger.info("Database migrations completed successfully.")
 
+
+async def ensure_mfa_columns():
+    """Issue #18 — TOTP MFA (v1.6.0): additive columns on users + 3 new tables.
+
+    All operations are idempotent (ADD COLUMN IF NOT EXISTS, CREATE TABLE IF NOT EXISTS).
+    Default behavior preserved: every existing user gets mfa_enabled=FALSE, so login
+    flow is byte-identical for accounts that don't opt in.
+    """
+    conn = None
+    try:
+        conn = await get_database_connection()
+
+        await conn.execute("""
+            ALTER TABLE users
+                ADD COLUMN IF NOT EXISTS mfa_enabled BOOLEAN DEFAULT FALSE NOT NULL,
+                ADD COLUMN IF NOT EXISTS mfa_method VARCHAR(20),
+                ADD COLUMN IF NOT EXISTS mfa_secret_encrypted TEXT,
+                ADD COLUMN IF NOT EXISTS mfa_enrolled_at TIMESTAMP,
+                ADD COLUMN IF NOT EXISTS mfa_last_used_at TIMESTAMP,
+                ADD COLUMN IF NOT EXISTS mfa_last_used_totp_step BIGINT;
+        """)
+
+        await conn.execute("""
+            CREATE TABLE IF NOT EXISTS mfa_backup_codes (
+                id SERIAL PRIMARY KEY,
+                user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
+                code_hash VARCHAR(255) NOT NULL,
+                used_at TIMESTAMP,
+                created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+            );
+        """)
+        await conn.execute("""
+            CREATE INDEX IF NOT EXISTS idx_mfa_backup_codes_user
+              ON mfa_backup_codes(user_id);
+        """)
+
+        await conn.execute("""
+            CREATE TABLE IF NOT EXISTS mfa_pending_logins (
+                id SERIAL PRIMARY KEY,
+                user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
+                challenge_token VARCHAR(64) UNIQUE NOT NULL,
+                attempts INTEGER DEFAULT 0 NOT NULL,
+                expires_at TIMESTAMP NOT NULL,
+                used_at TIMESTAMP,
+                created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
+                ip_address INET
+            );
+        """)
+        await conn.execute(
+            "CREATE INDEX IF NOT EXISTS idx_mfa_pending_token ON mfa_pending_logins(challenge_token);"
+        )
+        await conn.execute(
+            "CREATE INDEX IF NOT EXISTS idx_mfa_pending_expires ON mfa_pending_logins(expires_at);"
+        )
+
+        await conn.execute("""
+            CREATE TABLE IF NOT EXISTS mfa_pending_enrollments (
+                user_id INTEGER PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE,
+                secret_encrypted TEXT NOT NULL,
+                attempts INTEGER DEFAULT 0 NOT NULL,
+                expires_at TIMESTAMP NOT NULL,
+                created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+            );
+        """)
+        await conn.execute(
+            "CREATE INDEX IF NOT EXISTS idx_mfa_pending_enroll_expires ON mfa_pending_enrollments(expires_at);"
+        )
+
+        logger.info("✅ MFA migration completed (Issue #18 — Phase 1)")
+    except Exception as e:
+        logger.error(f"Failed to ensure MFA columns: {e}")
+        # Don't raise — follow the same defensive pattern as ensure_user_activity_logs_table
+    finally:
+        if conn:
+            await close_database_connection(conn)
+
 async def add_ssl_certificate_id_to_backend_servers():
     """Add ssl_certificate_id column to backend_servers table for SSL certificate management"""
     conn = None

backend/main.py

@@ -8,7 +8,7 @@
 import asyncio
 from datetime import datetime, timedelta
 
-_version_info = {"version": "1.5.2", "releaseName": "ACME Diagnostics Panel Hardening", "releaseDate": "2026-05-13"}
+_version_info = {"version": "1.6.0", "releaseName": "Multi-Factor Authentication (MFA)", "releaseDate": "2026-05-18"}
 for _vpath in ["/app/version.json", os.path.join(os.path.dirname(__file__), "..", "version.json")]:
     try:
         with open(_vpath) as _vf:
@@ -40,6 +40,7 @@
 from routers.letsencrypt import router as letsencrypt_router
 from routers.acme_diagnostics import router as acme_diagnostics_router
 from routers.site_wizard import router as site_wizard_router
+from routers.mfa import router as mfa_router
 
 # Production logging configuration
 from utils.logging_config import setup_production_logging
@@ -820,6 +821,7 @@ async def global_exception_handler(request: Request, exc: Exception):
 app.include_router(maintenance_router, prefix="/api", tags=["maintenance"])  # Database cleanup & maintenance
 app.include_router(auth_router)
 app.include_router(user_router)
+app.include_router(mfa_router)
 app.include_router(frontend_router)
 app.include_router(backend_router)
 app.include_router(cluster_router)

backend/middleware/error_handler.py

@@ -126,10 +126,26 @@ async def handle_validation_error(request: Request, exc: Exception) -> JSONRespo
             except Exception as body_error:
                 logger.debug(f"Could not extract raw body for debugging: {body_error}")
 
-        # Enhanced log message for agent heartbeats
-        log_message = f"Validation error: {str(exc)}"
+        # Build a sanitized log summary. The raw `str(exc)` from Pydantic
+        # contains the user-supplied `input` value for each failed field —
+        # which leaks secrets like TOTP codes, backup codes, mfa_token, and
+        # passwords to plaintext logs. Use only field NAMES + types here;
+        # `validation_details` (already sanitized to {field, message, type})
+        # is attached separately for downstream structured logging.
+        _field_names = [
+            err.get("field", "unknown")
+            for err in error_details.get("validation_errors", [])
+        ]
+        _err_count = len(error_details.get("validation_errors", []))
+        log_message = (
+            f"Validation error: {_err_count} field(s) failed validation: "
+            f"[{', '.join(_field_names)}]"
+        )
         if agent_name != "unknown":
-            log_message = f"Agent '{agent_name}' heartbeat validation error: {str(exc)}"
+            log_message = (
+                f"Agent '{agent_name}' heartbeat validation error: "
+                f"{_err_count} field(s) failed: [{', '.join(_field_names)}]"
+            )
 
         # Log validation error with enhanced details
         log_with_correlation(