|
| 1 | +# Manual Dev Environment Testing |
| 2 | + |
| 3 | +Automated tests can pass while the development environment is completely broken. CI starts services explicitly and runs in a controlled environment — it does not exercise `bin/dev`, Procfile orchestration, or the actual browser experience. This guide ensures agents verify the dev environment works end-to-end before submitting a PR. |
| 4 | + |
| 5 | +**Related:** [PR Testing Guide](pr-testing-guide.md), [Testing Build Scripts](testing-build-scripts.md) |
| 6 | + |
| 7 | +## The Rule |
| 8 | + |
| 9 | +> Automated tests passing is necessary but not sufficient. Any PR that changes how the app starts, builds, or serves must include manual dev environment verification. |
| 10 | +
|
| 11 | +## When Manual Testing Is Required |
| 12 | + |
| 13 | +If your PR touches **any** of these, you must run through the applicable checklist phases below: |
| 14 | + |
| 15 | +- `Procfile.dev` or process manager config |
| 16 | +- Webpack, Rspack, or Shakapacker configuration |
| 17 | +- `Gemfile` or `package.json` (dependency changes) |
| 18 | +- Rails initializers or environment config |
| 19 | +- SSR configuration, Node renderer, or server bundle setup |
| 20 | +- Routes that add new pages |
| 21 | +- Environment variables used at startup |
| 22 | + |
| 23 | +If your PR only touches test files, docs, or CI workflows, manual dev testing is optional (but still recommended). |
| 24 | + |
| 25 | +## Checklist Phases |
| 26 | + |
| 27 | +### Phase 1: Dev Server Startup (BLOCKING) |
| 28 | + |
| 29 | +Nothing else matters if the dev server won't start. |
| 30 | + |
| 31 | +```bash |
| 32 | +# 1. Install dependencies |
| 33 | +bundle install |
| 34 | +pnpm install # or your project's required JS package manager |
| 35 | + |
| 36 | +# 2. Start the dev server |
| 37 | +bin/dev |
| 38 | +``` |
| 39 | + |
| 40 | +Verify: |
| 41 | + |
| 42 | +- [ ] `bin/dev` does not exit immediately with an error |
| 43 | +- [ ] All processes defined in `Procfile.dev` start successfully |
| 44 | +- [ ] No process crashes within the first 30 seconds |
| 45 | +- [ ] Asset compilation completes (look for "compiled successfully" in output) |
| 46 | +- [ ] The app responds at its configured port (typically `http://localhost:3000`) |
| 47 | +- [ ] No "can't find X in manifest.json" or similar asset resolution errors |
| 48 | + |
| 49 | +### Phase 2: Page Smoke Test |
| 50 | + |
| 51 | +Visit the primary routes and every route touched by your PR. For each page: |
| 52 | + |
| 53 | +- [ ] Page loads without 500/404 errors |
| 54 | +- [ ] React components are visible (not empty containers waiting for JS) |
| 55 | +- [ ] Browser console has no JavaScript errors |
| 56 | +- [ ] Navigation links work |
| 57 | + |
| 58 | +If your PR adds a new route, verify it appears in navigation and renders correctly. |
| 59 | + |
| 60 | +### Phase 3: SSR Verification |
| 61 | + |
| 62 | +If the app uses server-side rendering, verify it actually works: |
| 63 | + |
| 64 | +- [ ] View page source shows rendered component markup (not empty `<div>` containers) |
| 65 | +- [ ] Disabling JavaScript in the browser still shows meaningful content |
| 66 | +- [ ] No timeout errors in the Rails server logs (e.g., `Net::ReadTimeout`) |
| 67 | +- [ ] No SSR error stack traces in server output |
| 68 | + |
| 69 | +### Phase 4: Interactive Functionality |
| 70 | + |
| 71 | +- [ ] Forms submit and produce expected results |
| 72 | +- [ ] Client-side navigation works without full page reloads (for SPA routes) |
| 73 | +- [ ] Interactive elements respond (toggles, buttons, modals) |
| 74 | +- [ ] Hot reload works: edit a component file and see the change in the browser without manual refresh |
| 75 | + |
| 76 | +### Phase 5: Process Health |
| 77 | + |
| 78 | +After running for 1-2 minutes: |
| 79 | + |
| 80 | +- [ ] No process has crashed and restarted |
| 81 | +- [ ] `Ctrl+C` cleanly stops all processes |
| 82 | +- [ ] Starting `bin/dev` a second time works (no stale port locks or zombie processes) |
| 83 | + |
| 84 | +## Which Phases to Run |
| 85 | + |
| 86 | +| Change type | Required phases | |
| 87 | +| ------------------------------------------ | -------------------------- | |
| 88 | +| Process/Procfile changes | 1, 2, 5 | |
| 89 | +| Webpack/bundler config | 1, 2, 3 | |
| 90 | +| Dependency changes (Gemfile, package.json) | 1, 2 | |
| 91 | +| SSR / Node renderer changes | 1, 2, 3 | |
| 92 | +| Rails initializer or env config | 1, 2, 3 | |
| 93 | +| Environment variables used at startup | 1, 2, 3 | |
| 94 | +| New routes or pages | 1, 2, 3, 4 | |
| 95 | +| React component changes | 1, 2, 4 | |
| 96 | +| CI workflow only | None (Phase 1 recommended) | |
| 97 | + |
| 98 | +## Common Failure Modes |
| 99 | + |
| 100 | +These are real failures that have shipped because agents relied solely on automated tests: |
| 101 | + |
| 102 | +**`bin/dev` won't start** — A new service was added to `Procfile.dev` but its startup config was broken. Rspec passed because CI starts services independently, not through `bin/dev`. |
| 103 | + |
| 104 | +**Tests pass, app doesn't render** — Webpack config changes produce bundles that work in test mode but fail in development mode, or `manifest.json` becomes stale between runs. |
| 105 | + |
| 106 | +**SSR silently disabled** — The SSR service isn't running or its config guard excludes the current environment. Pages load via client rendering only, which looks fine in a quick glance but breaks the expected user experience. |
| 107 | + |
| 108 | +**Port conflicts** — A previous `bin/dev` instance is still running, or a new service was hardcoded to a port already in use. |
| 109 | + |
| 110 | +## Reporting Results |
| 111 | + |
| 112 | +Include a dev environment verification section in your PR description: |
| 113 | + |
| 114 | +```markdown |
| 115 | +## Dev Environment Verification |
| 116 | + |
| 117 | +- [x] `bin/dev` starts all processes without errors |
| 118 | +- [x] Asset compilation completes successfully |
| 119 | +- [x] All routes load without errors |
| 120 | +- [x] SSR verified via view-source |
| 121 | +- [x] No browser console errors |
| 122 | +- [x] Interactive features work |
| 123 | +- [x] Clean shutdown with Ctrl+C |
| 124 | +``` |
| 125 | + |
| 126 | +If you **cannot** run `bin/dev` (missing database, credentials, services), say so explicitly: |
| 127 | + |
| 128 | +```markdown |
| 129 | +⚠️ **DEV ENVIRONMENT UNTESTED** — Could not run `bin/dev` because [reason]. |
| 130 | +Automated tests pass but manual dev server verification is required before merge. |
| 131 | +``` |
| 132 | + |
| 133 | +Include this section for every PR: if manual dev testing was required or run, provide checklist results; if skipped (for optional cases), state that it was skipped and why. |
0 commit comments