|
| 1 | +# E2E Testing |
| 2 | + |
| 3 | +Complete guide for running, monitoring, and debugging e2e tests. |
| 4 | + |
| 5 | +## Key Concept |
| 6 | + |
| 7 | +E2E tests run against pushed code in AWS CodeBuild, not local changes. Always commit and push before triggering. |
| 8 | + |
| 9 | +## When to Run E2E Tests |
| 10 | + |
| 11 | +- User explicitly requests e2e tests |
| 12 | +- User approves e2e testing as part of a task |
| 13 | +- Implied when user says "fix and test ..." or "add feature ... and test" |
| 14 | + |
| 15 | +## Cloud E2E Workflow |
| 16 | + |
| 17 | +```sh |
| 18 | +# 1. Commit and push all changes first |
| 19 | +git push |
| 20 | + |
| 21 | +# 2. Trigger e2e suite |
| 22 | +yarn cloud-e2e |
| 23 | + |
| 24 | +# 3. Monitor (auto-retries failed builds, polls every 5 min) |
| 25 | +yarn e2e-monitor {batchId} |
| 26 | +``` |
| 27 | + |
| 28 | +Batch ID format: `amplify-category-api-e2e-workflow:{UUID}` — always use the full ID. |
| 29 | + |
| 30 | +### Other Commands |
| 31 | + |
| 32 | +```sh |
| 33 | +yarn e2e-status {batchId} # Check status once |
| 34 | +yarn e2e-retry {batchId} # Retry failed builds |
| 35 | +yarn e2e-list [limit] # List recent batches |
| 36 | +yarn e2e-failed {batchId} # Show failed builds |
| 37 | +yarn e2e-logs {buildId} # View build logs |
| 38 | +``` |
| 39 | + |
| 40 | +### Monitor Behavior |
| 41 | + |
| 42 | +The monitor auto-retries failed builds up to 10 times by default. It skips retrying `build_linux`, `build_windows`, `test`, and `lint` because failures in those are typically code-related and require fixes, not retries. |
| 43 | + |
| 44 | +If errors persist after multiple retries or multiply as fixes are applied, ask the user for guidance. |
| 45 | + |
| 46 | +## Running Individual E2E Tests Locally |
| 47 | + |
| 48 | +```sh |
| 49 | +cd packages/amplify-e2e-tests |
| 50 | +yarn e2e src/__tests__/api_1.test.ts |
| 51 | +``` |
| 52 | + |
| 53 | +Local e2e tests create real AWS resources. You need valid credentials. |
| 54 | + |
| 55 | +### Authentication |
| 56 | + |
| 57 | +The repo scripts use `ada` (Amazon's credential management tool), called automatically. Setup: |
| 58 | + |
| 59 | +1. Ensure `ada` and `mwinit` are installed |
| 60 | +2. Create `scripts/.env`: |
| 61 | + ```bash |
| 62 | + E2E_ACCOUNT_PROD=<account-id> |
| 63 | + E2E_ACCOUNT_BETA=<account-id> |
| 64 | + ``` |
| 65 | +3. If you see auth errors, run `mwinit` |
| 66 | + |
| 67 | +For personal AWS profiles, export credentials before running: |
| 68 | + |
| 69 | +```sh |
| 70 | +export AWS_PROFILE=your-profile-name |
| 71 | +``` |
| 72 | + |
| 73 | +## Debugging E2E Failures |
| 74 | + |
| 75 | +### 1. Identify the Failing Build |
| 76 | + |
| 77 | +```sh |
| 78 | +yarn e2e-failed <batch-id> |
| 79 | +``` |
| 80 | + |
| 81 | +### 2. Get Build Logs |
| 82 | + |
| 83 | +```sh |
| 84 | +yarn e2e-logs <build-id> |
| 85 | +``` |
| 86 | + |
| 87 | +### 3. Simulate Locally |
| 88 | + |
| 89 | +Run the equivalent local command to reproduce. See [Development Commands](./DEVELOPMENT.md) for build step equivalents. |
| 90 | + |
| 91 | +### 4. Check for Pre-existing Issues |
| 92 | + |
| 93 | +```sh |
| 94 | +git stash |
| 95 | +git checkout main |
| 96 | +yarn <failing-command> |
| 97 | +git checkout <your-branch> |
| 98 | +git stash pop |
| 99 | +``` |
| 100 | + |
| 101 | +## Common Failure Patterns |
| 102 | + |
| 103 | +| Pattern | Symptoms | Action | |
| 104 | +| ------------------------ | -------------------------------------------------- | ------------------------------------------------- | |
| 105 | +| Transient infrastructure | Timeouts, credential expiration, quota errors | Retry the build | |
| 106 | +| Code-related | Test failures, build errors, coverage threshold | Fix the code, don't retry | |
| 107 | +| Dependency-related | Module not found, version conflicts, breaking APIs | Check for major version changes, consider pinning | |
| 108 | + |
| 109 | +## Build Job Types |
| 110 | + |
| 111 | +- `build_linux` / `build_windows` — Platform builds (not auto-retried) |
| 112 | +- `test` — Unit tests (not auto-retried) |
| 113 | +- `lint` — Linting (not auto-retried) |
| 114 | +- `verify_dependency_licenses_extract` — License verification |
| 115 | +- `verify_api_extract` — API surface verification |
| 116 | +- `verify_yarn_lock` — yarn.lock consistency |
| 117 | +- `publish_to_local_registry` — Verdaccio publish |
| 118 | +- `graphql_e2e_tests_*` — GraphQL e2e test suites (171 jobs) |
| 119 | + |
| 120 | +## Resource Cleanup |
| 121 | + |
| 122 | +E2e tests create real AWS resources. Run periodically: |
| 123 | + |
| 124 | +```sh |
| 125 | +yarn cleanup-stale-resources |
| 126 | +``` |
| 127 | + |
| 128 | +## Troubleshooting |
| 129 | + |
| 130 | +- **"Command failed with exit code 1"** — Generic error. Read the full output for the actual message. |
| 131 | +- **"Cannot read properties of undefined"** — Usually a dependency version mismatch or breaking API change. |
| 132 | +- **"Coverage threshold not met"** — Check if pre-existing by testing on the base branch. |
| 133 | +- **"License change detected"** — Expected after dependency updates. Run `yarn extract-dependency-licenses` and commit. |
0 commit comments