feat(seeder): expand db-seed with demo files + multi-type comments (sbpp#1484)

rumblefrog · web-flow · commit 424d6d478a60 · 2026-05-28T00:03:35.000Z
`./sbpp.sh db-seed` previously stopped at empty `:prefix_demos` rows and ban-only comments. Two visible-but-empty surfaces on a freshly-seeded dev DB: 1. **"Review Demo" link on the banlist** rendered as a dead anchor — `:prefix_demos` was untouched by the seeder, so the page handler never emitted the affordance and the surface was unreachable for manual exploration / acceptance audits / screenshot work without hand-uploading a demo first. 2. **Moderation queues** (admin submissions + protests) and the public banlist's per-row comment disclosure rendered with empty comment threads — `:prefix_comments` only ever got type `'B'` (ban-attached) rows, so the C / S / P columns of the table were structurally untestable. This PR closes both gaps with one expansion of `Sbpp\Tests\Synthesizer`: **Demo seeding.** `insertDemos()` writes ~80 rows at medium scale (64 ban-attached, 16 submission-attached — 80/20 split that mirrors the panel's own bias toward bans-with-evidence) and lands a paired ~1 KiB opaque text payload on disk under `SB_DEMOS` (`web/demos/` on the dev bind-mount). Filenames are `md5(seed|demtype|demid)` so re-seeds round-trip identically and the truncate-time wipe can find them by re-reading the table. Picks use `array_rand` for exact unique counts — rejection-sampling-with-collisions undershoots the budget by ~6% which made the screenshot-flow rendering jittery across runs. The synth payload is text (not a Source-engine binary), so the "Review Demo" link round-trips end-to-end through `getdemo.php` with `Content-Type: application/octet-stream` and a download lands in the browser, but the SDK demoviewer won't replay them — that would need a real demo binary and is out of scope for a panel-chrome seeder. **Multi-type comments.** `insertComments()` now fans the comment budget across all four `:prefix_comments.type` values: - `B` (ban) — 60% — `comments.bid` → `bans.bid` - `C` (comm block) — 20% — `comments.bid` → `comms.bid` - `S` (submission) — 10% — `comments.bid` → `submissions.subid` - `P` (protest) — 10% — `comments.bid` → `protests.pid` Medium scale → 200 comments → 120/40/20/20. The split matches the overload contract in `api_bans_add_comment`'s `match($ctype)` arms documented in AGENTS.md → "Add a comment-delete trash icon". The parent-ID tracking arrays (`commsCids`, `submissionSubids`, `protestPids`) populate during their respective `insert*()` passes via `lastInsertId()` so the comment-insert pass has the FK pool ready. **Idempotency contract.** The pre-existing truncate-then-reseed flow got two refinements: - **Snapshot filenames BEFORE truncate, unlink AFTER.** Previous pre-fix shape was wipe-files-then-truncate, which left a transient window where files were gone but DB rows still pointed at them (any panel page query during that window 404'd the "Review Demo" link). New order: snapshot → truncate → unlink, so rows + files drop visibly together from the panel's POV. - **Wipe scope is the snapshotted set, not the directory.** Manual panel uploads (anyone who uploaded a demo against the dev stack) and `.gitkeep` survive untouched. The `db-reset` companion (see below) is the only thing that strips by-name across the whole directory, scoped to the MD5-basename shape. **Paired surface changes:** - `.gitignore` — `web/demos/*` + `!.gitkeep` exception. Pre-fix a fresh `db-seed` polluted `git status` with 80 untracked MD5-named files; the demo store is also where the panel's `UploadHandler` writes real ban evidence on production installs, so checking those in would leak operator evidence into the source tree. - `sbpp.sh db-reset` — strips MD5-basename files from `web/demos/` after the DB volume drop. The `_demos` rows that pointed at them vanish with the volume, so the next `db-seed`'s referenced-only wipe reads zero rows from the fresh DB and skips them; without the reset-time strip those files would accumulate as permanent orphans across worktree lifetimes. `find … -regex '.*/[0-9a-f]{32}$'` matches both shapes the write path produces (synth + the panel's own `UploadHandler::handle()` with `renameToHash: true`). - `AGENTS.md` + `docker/README.md` — exact per-scale counts, demo payload caveat (not playable in the demoviewer), the new wipe ordering contract, gitignore + `db-reset` cleanup behavior. **Verification.** Default-seed run produces exactly 80 demos + exactly 200 comments split B=120/C=40/S=20/P=20; FK joins return zero orphans across all six pairs (`demos.B` → bans, `demos.S` → submissions, `comments.B/C/S/P` → bans/comms/submissions/protests); two consecutive seeds with the default seed produce identical filename sets (`md5sum` of `ls` output matches byte-for-byte); a manually-touched decoy file in `web/demos/` survives every seed. PHPStan clean; the one PHPUnit test that touches `:prefix_demos` / `:prefix_comments` (`ExportBundleWriterTest`, 426 assertions) passes unchanged. Follow-up: sbpp#1483 tracks the pre-existing tech debt where the seeder writes bare letter codes (`'U'` / `'D'` / `'m'` / `'w'` / `'e'`) into the column-typed string fields covered by `BanRemoval` / `LogType` instead of routing through the enum's `->value` per AGENTS.md "Backed enums for column-typed fields". Out of scope for this PR.
diff --git a/.gitignore b/.gitignore
@@ -53,6 +53,16 @@ Temporary Items
 /web/cache/sessions/sess_*
 *.smx
 
+# Uploaded demo evidence (production) + synthetic demo files written by
+# `./sbpp.sh db-seed` (dev). `.gitkeep` is the structural marker so the
+# directory exists for fresh tarball installs and tests that probe
+# `is_dir(SB_DEMOS)`. Both upload paths land MD5-hash basenames here;
+# checking them in would (a) blow up the working tree on every db-seed
+# and (b) leak real ban evidence into the source tree on a self-hoster
+# install.
+/web/demos/*
+!/web/demos/.gitkeep
+
 # Local stash file used by the install-screenshot capture flow
 # (`docs/README.md` → "Capturing screenshots"). Gitignored so an
 # interrupted run never leaks the panel's DB credentials into a
diff --git a/AGENTS.md b/AGENTS.md
@@ -618,24 +618,61 @@ two idiomatic shapes (#1273):
 
 `./sbpp.sh db-seed` populates the dev DB (`sourcebans`) with a deterministic,
 realistic synthetic dataset across bans, comms, servers, admins, groups,
-submissions, protests, comments, notes, banlog, and the audit log. Use it
-when you need the panel surfaces (banlist, dashboard, drawer, moderation
-queues, audit log) to render with real-looking data instead of empty
-states. Acceptance audits and screenshot work both depend on it.
+submissions, protests, comments (per-type: B / C / S / P), notes, banlog,
+the audit log, AND demo evidence files (rows + on-disk files under
+`SB_DEMOS`). Use it when you need the panel surfaces (banlist, dashboard,
+drawer, moderation queues, audit log, "Review Demo" download flow) to
+render with real-looking data instead of empty states. Acceptance audits
+and screenshot work both depend on it.
 
 - Lives at `web/tests/Synthesizer.php` (`Sbpp\Tests\Synthesizer`); CLI
   driver at `web/tests/scripts/seed-dev-db.php`. Both are dev-only —
   the synthesizer refuses any `DB_NAME` other than `sourcebans`, so
   `sourcebans_test` / `sourcebans_e2e` stay untouched and the E2E
   suite (which builds its own rows per spec) is unaffected.
-- Idempotent: every run truncates the synth-owned tables first
-  (preserving `sb_settings` / `sb_mods` from `data.sql`), re-seeds the
-  CONSOLE + `admin/admin` rows, then inserts the synthetic dataset.
+- Idempotent: every run snapshots the filenames currently referenced
+  by `:prefix_demos` BEFORE the TRUNCATE wipes the table, executes
+  the truncates (preserving `sb_settings` / `sb_mods` from `data.sql`),
+  unlinks the snapshotted files AFTER (so rows + files drop visibly
+  together from the panel's POV — closes a transient pre-fix window
+  where rows pointed at files that had just been removed), re-seeds
+  the CONSOLE + `admin/admin` rows, then inserts the new synthetic
+  dataset (rows AND on-disk demo payloads). The wipe scope is the
+  set of filenames currently in `:prefix_demos`; anything else in
+  `SB_DEMOS` (the tracked `.gitkeep`, manual panel uploads, a dev's
+  SDK demoviewer working copy) survives untouched. `./sbpp.sh
+  db-reset` carries a sibling cleanup that strips MD5-named
+  basenames from `web/demos/` (those rows vanished with the DB
+  volume so they'd otherwise be permanent orphans).
 - Deterministic given a fixed `--seed` — `mt_srand($seed)` pins PHP's
-  RNG so two devs hit the same names/reasons/timestamps. Default seed
-  is `Synthesizer::DEFAULT_SEED` (pinned in code).
-- Three tiers: `--scale=small` (~30 bans, fast iteration), `medium`
-  (default, ~200 bans), `large` (~2000 bans for pagination / perf).
+  RNG so two devs hit the same names/reasons/timestamps AND so
+  `array_rand` picks the same parent IDs for demo attachment across
+  runs. Demo filenames ride `md5(seed|demtype|demid)` so re-seeds
+  round-trip identically and the next run's cleanup pass finds +
+  unlinks the prior set cleanly (filenames stay 32-char lowercase
+  hex, indistinguishable from `UploadHandler::handle()`'s
+  `renameToHash` output). Default seed is `Synthesizer::DEFAULT_SEED`
+  (pinned in code).
+- Three tiers: `--scale=small` (30 bans + 15 demos, fast iteration),
+  `medium` (default, 200 bans + 80 demos), `large` (2000 bans + 800
+  demos for pagination / perf). Comment budget per scale fans
+  exactly 60% Ban (`B`) / 20% Comm (`C`) / 10% Submission (`S`) /
+  10% Protest (`P`) (medium → 120/40/20/20); demo budget fans
+  exactly 80% ban-attached (`B`) / 20% submission-attached (`S`)
+  (medium → 64/16). Counts are exact (no rejection-sampling
+  shortfall) — `insertDemos()` uses `array_rand` for unique picks
+  upfront. The splits give each surface enough rows to render
+  without leaving the moderation-queue detail cards empty.
+- Demo payloads are opaque ~1 KiB text blobs carrying the synth
+  marker + parent ID, served verbatim by `getdemo.php` with
+  `Content-Type: application/octet-stream`. They won't replay in the
+  SDK demoviewer (would need a real Source-engine demo binary);
+  they DO exercise the panel's "Review Demo" download chrome end-to-end
+  (link → 200 → file download with `Content-Disposition`). On-disk
+  files land in `web/demos/`, which is gitignored alongside the
+  production demo-upload directory (the panel's `UploadHandler` writes
+  there too with `renameToHash: true`) so synthetic and real-production
+  uploads share one storage shape.
 - Do NOT reach for this from `Fixture::truncateAndReseed()` (the e2e
   hot path) — those two diverge by design. The synthesizer is for
   manual dev / screenshot work; the e2e fixture stays minimal so each
diff --git a/docker/README.md b/docker/README.md
@@ -86,27 +86,48 @@ matching docs source file on disk.
 ./sbpp.sh e2e                     # Playwright E2E gate (lazy chromium install) against the running stack
 ./sbpp.sh db-dump backup.sql      # mysqldump to host file
 ./sbpp.sh db-load fixtures.sql    # pipe a SQL file into the DB
-./sbpp.sh db-reset                # drop just the DB volume and re-seed
-./sbpp.sh db-seed                 # populate the dev DB with realistic synthetic data
+./sbpp.sh db-reset                # drop the DB volume + wipe MD5-named demo files from web/demos/
+./sbpp.sh db-seed                 # populate the dev DB with realistic synthetic data + write demo files to web/demos/
 ./sbpp.sh rebuild                 # `--no-cache` rebuild of the web image
 ```
 
 `db-seed` lights up every data-driven panel surface — banlist + commslist
 beyond a single page, dashboard "Latest …" cards, the drawer's history /
-comments / notes panes, admin moderation queues (submissions and
-protests), admin audit log, multiple groups/admins/servers — without
-touching `data.sql` / `struc.sql` (the install path stays minimal).
-Idempotent: every run truncates the synthesizer-owned tables first and
-re-seeds. Deterministic given `--seed=<int>` so two devs see the same
-data; pinned default seed in code. Three scale tiers:
+comments / notes panes (B/C/S/P comment types covered, so the moderation
+queues and protest threads paint with real text), admin moderation
+queues, admin audit log, multiple groups/admins/servers, and demo files
+on disk under `web/demos/` (so the banlist's "Review Demo" link returns
+an actual download body) — without touching `data.sql` / `struc.sql`
+(the install path stays minimal). Idempotent: every run truncates the
+synthesizer-owned tables first and re-seeds. Deterministic given
+`--seed=<int>` so two devs see the same data; pinned default seed in
+code. Three scale tiers:
 
 ```sh
-./sbpp.sh db-seed                 # default scale (~200 bans, ~100 comms)
-./sbpp.sh db-seed --scale=small   # ~30 bans, fast iteration
-./sbpp.sh db-seed --scale=large   # ~2000 bans, pagination / perf
+./sbpp.sh db-seed                 # default scale (~200 bans, ~100 comms, ~80 demos)
+./sbpp.sh db-seed --scale=small   # ~30 bans, ~15 demos, fast iteration
+./sbpp.sh db-seed --scale=large   # ~2000 bans, ~800 demos, pagination / perf
 ./sbpp.sh db-seed --seed=42       # alternate RNG seed
 ```
 
+Demo files: each `:prefix_demos` row lands a paired ~1 KiB opaque file
+in `web/demos/` (host bind-mount; gitignored alongside production
+upload evidence). Filenames are MD5(seed|demtype|demid) so re-runs with
+the same seed produce stable basenames and the truncate-time wipe can
+clean them up by re-reading the table. The synth payload is text, not a
+playable Source-engine demo — the panel chrome and `getdemo.php` stream
+the bytes verbatim with `Content-Type: application/octet-stream`, which
+is enough for the download affordance to work end-to-end, but the SDK
+demoviewer won't replay them. Manual uploads (panel-side or anything
+else not referenced by `:prefix_demos`) are never touched by the wipe.
+
+`db-reset` drops the entire DB volume AND wipes MD5-named files from
+`web/demos/` (orphans after the DB drop — the `_demos` rows that
+pointed at them vanished with the volume). The wipe is scoped to
+`[0-9a-f]{32}` basenames so `.gitkeep` stays put; non-MD5 manual
+uploads (the icon / mapimg flows preserve original filenames) survive
+too, though those don't normally land in `web/demos/`.
+
 Refusal guard: the seeder strictly refuses any `DB_NAME` other than
 `sourcebans`, so the PHPUnit DB (`sourcebans_test`) and Playwright DB
 (`sourcebans_e2e`) cannot be wiped by accident. The Synthesizer source
diff --git a/sbpp.sh b/sbpp.sh
@@ -33,7 +33,8 @@ Run things inside containers:
 Database:
   db-dump [file]  Dump the DB to file (default: dump-YYYYMMDD-HHMMSS.sql).
   db-load <file>  Load a SQL dump into the dev DB.
-  db-reset        Drop the DB volume and re-seed (faster than full reset).
+  db-reset        Drop the DB volume + wipe MD5-named files from web/demos/
+                  (orphans after the DB drop). Faster than full reset.
   db-seed [args]  Populate the dev DB with realistic synthetic data.
                   Accepts --scale=small|medium|large (default medium) and
                   --seed=<int> (default pinned in code). Idempotent: every
@@ -228,6 +229,22 @@ case "$cmd" in
     db-reset)
         dc rm -fsv db
         docker volume rm "$(basename "$PWD")_dbdata" 2>/dev/null || true
+        # `db-reset` wipes the DB but `web/demos/` is bind-mounted from
+        # the host worktree (not a named volume) so demo files written
+        # by a previous `db-seed` (or by any panel-side upload that
+        # happened against the dev stack) would otherwise survive as
+        # permanent orphans — the `_demos` rows that pointed at them
+        # vanish with the DB volume, and the next `db-seed`'s
+        # referenced-only wipe reads zero rows from the fresh DB and
+        # skips them. Strict MD5-name regex matches both shapes the
+        # demo write path produces (`Sbpp\Tests\Synthesizer::deterministicDemoFilename`
+        # for synth files, `Sbpp\Upload\UploadHandler::handle()` with
+        # `renameToHash: true` for panel uploads) while leaving the
+        # `.gitkeep` structural marker in place. Demos are dev-only
+        # ban evidence here; production installs never run `sbpp.sh`.
+        if [[ -d web/demos ]]; then
+            find web/demos -maxdepth 1 -type f -regextype posix-extended -regex '.*/[0-9a-f]{32}$' -delete 2>/dev/null || true
+        fi
         dc up -d db
         ;;
     db-seed)
diff --git a/web/tests/Synthesizer.php b/web/tests/Synthesizer.php
