Playwright test reliability#32
Merged
Merged
Conversation
- Increase timeouts in playwright.config.ts (test: 60s, expect: 15s, action: 15s, navigation: 30s) - Add screenshot/video capture on failure/first retry for debugging - Add global setup to clean fixtures directory before test runs - Add retries (3 in CI, 1 locally) for flaky test recovery - Run tests in series in CI for better database isolation - Increase webServer startup timeout to 120s - Add retryDbOperation utility for transient database errors - Add fixture cleanup for phone numbers in test fixtures - Increase waitFor default timeout to 10s with better polling intervals - Add waitForPageStable and navigateAndWait helpers - Improve mock utils with increased retry attempts and timeouts - Add deleteText and deleteFixture utilities for cleanup - Add explicit waitFor visibility checks before interactions - Add waitForLoadState after navigation - Increase assertion timeouts - Clean up stale fixtures at test start and end - Better error messages for waitForText calls Co-authored-by: me <me@kentcdodds.com>
- Fix await in non-async callback by pre-computing password hash - Add PRISMA_USER_CONSENT_FOR_DANGEROUS_AI_ACTION env var for CI database setup Co-authored-by: me <me@kentcdodds.com>
|
Cursor Agent can help with this pull request. Just |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
- Fix cleanupFixturesForPhone to use filenamify() consistently with fixture creation - Remove unused waitForPageStable and navigateAndWait helper functions
|
Bugbot Autofix resolved both of the 2 bugs found in the latest run.
|
Use separate final variables instead of nullish coalescing assignment to properly narrow types for phoneNumber and username Co-authored-by: me <me@kentcdodds.com>
…tFor, remove unused import
|
Bugbot Autofix resolved all 3 of the 3 bugs found in the latest run.
|
|
Bugbot Autofix resolved the bug found in the latest run.
|
| } catch { | ||
| // Ignore errors if file doesn't exist | ||
| } | ||
| } |
There was a problem hiding this comment.
Exported function deleteFixture is never imported externally
Low Severity
The deleteFixture function is exported but never imported anywhere in the codebase. It's only used internally by deleteText within the same file. This function doesn't need to be exported and could be a private helper function instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR significantly improves the reliability of Playwright E2E tests by:
retryDbOperationfor transient database errors, improvedwaitForutility with longer defaults, and added explicit fixture cleanup functions.waitForvisibility checks,waitForLoadStateafter navigation, increased assertion timeouts, and comprehensive fixture cleanup at test start/end.These changes aim to reduce flakiness, provide better debugging information for failures, and ensure test isolation.
Test Plan
All 14 Playwright E2E tests passed locally with
CI=true npx playwright test.Checklist
Screenshots
N/A
Note
Low Risk
Changes are confined to test configuration/utilities and E2E specs, with no production runtime impact; main risk is longer CI runtimes or masking legitimate timing issues via higher timeouts.
Overview
Improves Playwright E2E reliability by loosening timeouts/retries and adding better CI diagnostics: higher global/expect/action/navigation timeouts, CI-only serial execution with 1 worker, more retries, CI
githubreporter, and a longerwebServerstartup timeout.Adds fixture/database isolation utilities and setup: a new Playwright
globalSetupclearstests/fixturesand runs DB setup; mocks gaindeleteFixture/deleteText, longer retry defaults, and more informativewaitForText/waitFortimeouts. Test helpers now retry transient Prisma ops (retryDbOperation) and automatically clean text fixtures aroundlogin/insertNewUser, while multiple E2E specs add explicitdomcontentloaded/visibility waits and increased assertion timeouts (including a much longer scheduled-send test).Written by Cursor Bugbot for commit 20d457e. This will update automatically on new commits. Configure here.