|
| 1 | +# Switch hub-client e2e to `vite preview` instead of `vite dev` |
| 2 | + |
| 3 | +**Date:** 2026-05-11 |
| 4 | +**Branch:** `chore/e2e-ci` (PR #172) — stay on this branch, validate by pushing to the open PR |
| 5 | +**Worktree:** `.worktrees/e2e-ci` |
| 6 | +**Status:** Planned — not started |
| 7 | + |
| 8 | +## Overview |
| 9 | + |
| 10 | +Goal: cut wall-clock and reduce flakiness in the `Hub-Client E2E Tests` |
| 11 | +workflow by serving the hub-client to Playwright from a prebuilt bundle |
| 12 | +instead of vite's dev server. |
| 13 | + |
| 14 | +### Motivation |
| 15 | + |
| 16 | +Each Playwright test creates a fresh browser context, which downloads |
| 17 | +roughly: |
| 18 | + |
| 19 | +| Asset | Size (uncompressed) | |
| 20 | +|---|---| |
| 21 | +| `wasm_quarto_hub_client_bg.wasm` | **32 MB** | |
| 22 | +| `automerge_wasm_bg.wasm` | 1.8 MB | |
| 23 | +| `web-tree-sitter.wasm` | 192 KB | |
| 24 | +| dart-sass dynamic-import bundle | ~5 MB | |
| 25 | +| Monaco editor chunks | ~3 MB | |
| 26 | +| Hundreds of small TS/JSX modules | ~5 MB total | |
| 27 | + |
| 28 | +Per fresh context, **~50 MB of bytes through `vite dev`**, served by a |
| 29 | +single-threaded dev server that also has to run its plugin pipeline on |
| 30 | +every TS/JSX module on demand. With 2 Playwright workers contending for |
| 31 | +one dev server on a 2-core runner, the cold-context page load can hold |
| 32 | +up another worker's test for tens of seconds. |
| 33 | + |
| 34 | +Working theory: the "preview iframe didn't render in 45s" timeouts we |
| 35 | +keep hitting (and the 5-10 "flaky" retries per run) are mostly **page |
| 36 | +loads** blocked on vite dev's serialized module pipeline + uncompressed |
| 37 | +binary serving, not actual render time. A static prebuilt bundle served |
| 38 | +via `vite preview` should remove this whole class of contention: |
| 39 | + |
| 40 | +- Gzip/brotli compression for binary assets (32 MB → ~8-12 MB on the wire) |
| 41 | +- No transform pipeline → no per-request serialization point |
| 42 | +- ~10 bundled JS chunks vs. ~500 separate dev-mode module requests |
| 43 | +- HTTP cache reuse across same-worker tests is more predictable |
| 44 | + |
| 45 | +### Target outcome |
| 46 | + |
| 47 | +| Metric | Current | Target | |
| 48 | +|---|---|---| |
| 49 | +| Workflow total | ~16 min | sub-12 min | |
| 50 | +| `Run E2E tests` step | ~7 min | sub-5 min | |
| 51 | +| Flaky tests | 5-10 / run | ≤ 2 / run | |
| 52 | +| Hard failures | 0-1 / run | 0 across ≥ 3 runs | |
| 53 | + |
| 54 | +## Approach |
| 55 | + |
| 56 | +Stay on `chore/e2e-ci`. Each iteration: edit, build locally, run smoke-all |
| 57 | +locally with 2 workers, then push to PR #172 to validate on CI. The |
| 58 | +existing 75s preview-render timeout (commit `81cc5264`) is a safety net |
| 59 | +during the switch; once preview is stable, drop it back to 45s in the |
| 60 | +same series. |
| 61 | + |
| 62 | +## Work items |
| 63 | + |
| 64 | +### Phase 0 — Baseline measurement (so we know we improved) |
| 65 | + |
| 66 | +- [ ] Record current local timing: `cd hub-client && CI=1 npx playwright test --grep smoke-all --workers=2 --reporter=line --timeout=90000` (with current `vite dev`). Capture: total wall, flaky count, slowest 5 tests. |
| 67 | +- [ ] Record current CI timing from the most recent green run (`25647967388`): total job, `Run E2E tests`, `Build TypeScript packages` durations. |
| 68 | + |
| 69 | +### Phase 1 — Vite preview config |
| 70 | + |
| 71 | +- [ ] `hub-client/vite.config.ts`: lift the proxy block into a `preview.proxy` mirror (vite preview ignores `server.proxy`). Same target/target-handling/rewrite as the existing `server.proxy`. The shared constant `hubTarget` is fine to reuse. |
| 72 | +- [ ] Confirm `vite build` emits the WASM file as a static asset under `dist/assets/` (the local `dist/` already shows `wasm_quarto_hub_client_bg-<hash>.wasm` and `automerge_wasm_bg-<hash>.wasm`, so this works today). |
| 73 | +- [ ] Verify HTTP `Content-Encoding: gzip` is sent for the WASM by `vite preview` by hitting it manually: `cd hub-client && npm run build && npm run preview -- --port 5173 &; curl -sI http://localhost:5173/assets/wasm_quarto_hub_client*.wasm -H 'Accept-Encoding: gzip'`. If preview doesn't gzip by default, decide whether to: (a) accept the uncompressed 32 MB and rely on the no-transform-pipeline win alone, (b) front it with `compression` middleware via a small Express wrapper, or (c) live with current size and focus on the other wins. |
| 74 | + |
| 75 | +### Phase 2 — Playwright wiring |
| 76 | + |
| 77 | +- [ ] `hub-client/playwright.config.ts`: change `webServer.command` from `npm run dev` to `npm run preview -- --port 5173`. Keep `url: 'http://localhost:5173'`. Keep the `reuseExistingServer` setting. Bump `timeout` if 120s isn't enough to cover the preview-server startup (it should be near-instant). |
| 78 | +- [ ] Decide where `VITE_HUB_SERVER` is set: |
| 79 | + - Option A: bake it into the build (set on the workflow's "Build TypeScript packages" step env). Cleanest. The hub URL would be hard-coded into the built JS, which is fine for CI but not for shareable artifacts. |
| 80 | + - Option B: read it at runtime via vite preview's `preview.proxy` config. Vite reads `process.env.*` at config-eval time when starting the preview server, so passing it via the `webServer.env` in playwright.config.ts should work. |
| 81 | + - Pick Option B if it works — keeps the build artifact reusable; only the preview-server invocation needs the env. |
| 82 | +- [ ] Local run: same command as Phase 0 baseline. Confirm tests pass, capture timings. |
| 83 | + |
| 84 | +### Phase 3 — CI integration |
| 85 | + |
| 86 | +- [ ] If we used Option B above: no workflow change beyond what's already there. |
| 87 | +- [ ] If we used Option A: add `VITE_HUB_SERVER: http://localhost:3030` to the "Build TypeScript packages" step env in `.github/workflows/hub-client-e2e.yml`. |
| 88 | +- [ ] Push to `chore/e2e-ci`. Watch the new PR-triggered run. Compare against the Phase 0 CI baseline. |
| 89 | + |
| 90 | +### Phase 4 — Iterate / validate |
| 91 | + |
| 92 | +- [ ] If first run is green with good timings: trigger 2 more runs via `gh workflow run hub-client-e2e.yml --ref chore/e2e-ci` to check stability across consecutive runs. |
| 93 | +- [ ] If first run has new failures (production-build code-path differences, missing assets, broken proxy): diagnose, fix, push again. |
| 94 | +- [ ] If the preview switch alone clears flakes: drop the 75s preview timeout back to 45s in a follow-up commit to confirm it's the preview mode (not the timeout headroom) that's doing the work. |
| 95 | + |
| 96 | +### Phase 5 — Cleanup |
| 97 | + |
| 98 | +- [ ] If preview migration sticks, consider squashing the timeout-bump commit (`81cc5264`) into the preview commit via `git revise` — or keep them separate so the bisect history tells the story. |
| 99 | +- [ ] Update the squashed CI commit's message (or add a fresh commit) describing the dev→preview switch. |
| 100 | +- [ ] Update the PR description / comment with before/after timing numbers. |
| 101 | +- [ ] If a measurable speed-up landed, add a sentence to `hub-client/playwright.config.ts` explaining why it's `preview` (so a future agent doesn't "helpfully" revert it to `dev` for HMR). |
| 102 | + |
| 103 | +## Risks / unknowns |
| 104 | + |
| 105 | +- **`preview.proxy` may not honor every option of `server.proxy`**. The shared proxy logic should be straightforward (forward `/auth/*` and the ws upgrade to `http://localhost:3030`), but worth checking against vite docs and an actual local run before pushing. |
| 106 | +- **Production-only code paths in hub-client**. If anything reads `import.meta.env.PROD` and behaves differently in built mode, e2e would suddenly hit it. Could surface real bugs (good!) or break tests for non-bug reasons (need fixing). The hub-client doesn't seem to have many of these from a quick scan, but full local-run validation in Phase 1 catches it. |
| 107 | +- **Source maps and debugging**. Built mode has external source maps; failure investigation needs `playwright show-trace` or careful artifact-spelunking, same as today. Likely no regression. |
| 108 | +- **The `optimizeDeps.exclude: ['wasm-quarto-hub-client']` line is dev-only**; vite build handles the alias via `resolve.alias` regardless. The local `dist/` already proves this works. |
| 109 | +- **Cache busting**. Vite build produces hashed filenames. If anything in test setup or `e2e/helpers/` hard-codes a JS asset path (it shouldn't — everything goes through the HTML entry), it'd break. |
| 110 | +- **HMR is gone in preview** — fine for CI, not a regression because we never used HMR in tests. |
| 111 | +- **First-test-in-worker is still slower than subsequent** (browser cache cold), but the absolute number should be much smaller. Acceptable. |
| 112 | + |
| 113 | +## Success criteria |
| 114 | + |
| 115 | +The PR's next push triggers a CI run that: |
| 116 | + |
| 117 | +1. Completes the `Run E2E tests` step in **under 5 minutes**. |
| 118 | +2. Reports **0 hard failures**. |
| 119 | +3. Reports **≤ 2 flaky tests**. |
| 120 | +4. Total workflow under **12 minutes**. |
| 121 | + |
| 122 | +Then run the workflow 2 more times via `workflow_dispatch` and confirm the |
| 123 | +numbers are stable, not lucky-once. |
| 124 | + |
| 125 | +## Estimated effort |
| 126 | + |
| 127 | +30-90 minutes including local validation. The risk is that the |
| 128 | +production build surfaces a hub-client bug that doesn't exist in dev |
| 129 | +mode, in which case scope expands. |
0 commit comments