playwright(fix): test reliability — ES indexing, locator strict-mode, waitForResponse races#29585
playwright(fix): test reliability — ES indexing, locator strict-mode, waitForResponse races#29585chirag-madlani wants to merge 9 commits into
Conversation
… waitForResponse races
Bundle of test-side bug fixes for races and locator issues in existing
Playwright E2E specs. No config, no test-file moves, no behavior change
to the system under test — just makes flaky assertions deterministic.
ES indexing races (entities created via API, then read by UI before
search index updates):
- BulkEditEntity.spec.ts: wait for glossary_term_search_index before
opening bulk-edit table (otherwise the term shows as "Entity created"
instead of "Entity updated")
- BulkEditOperationBadges.spec.ts: same pattern for the file's beforeAll
(term + table search indexes)
- BulkImport.spec.ts: wait for database_service / database_schema /
table search indexes; pre-wait .rdg-cell-details rendering before
asserting text (was racing the async grid populate)
- TestCaseImportExportE2eFlow.spec.ts: wait for test_case_search_index
so the just-created test case appears in the export's CSV
waitForResponse races (listener registered after the action, or matching
too-broad URL pattern that resolves on a stale earlier response):
- utils/searchRBAC.ts searchForEntityShouldWork[ShowNoResult]: register
before press('Enter')
- SearchIndexNestedColumns.spec.ts: predicate-match the URL that
contains the encoded search term (the empty-query suggestion fetch
was resolving the listener first)
Locator-stability / timeout fixes:
- Roles.spec.ts (×7 sites): replace strict-mode-violating
expect(loader).not.toBeVisible() with waitForAllLoadersToDisappear
(handles N loaders correctly)
- ExploreBrowse.spec.ts: explicit toBeVisible before clicking
explore-tree-title-mysql so the tree settles after expandTreeNode
- Table.spec.ts: wait for sort + next-page responses, replace
count()===15 with toHaveCount(15) auto-retry
Backend-propagation timeouts (the UI fetches lag the API write):
- utils/searchRBAC.ts exploreTreeCategories: wrap visibility/absence
checks in expect.toPass({ timeout: 20s }) — tree counts come from
multiple aggregation queries, single search-query wait is not enough
- utils/searchRBAC.ts exploreShouldShowEntity: bump negative-assertion
to 45s — RBAC filtering against newly-assigned roles lags the patch
by several seconds
- utils/taskWorkflow.ts openTaskDetails: 15s → 45s — activity-feed
refresh after API task create lags past default under load
Removed 7 unannotated `waitForTimeout` calls that were leaking past
ESLint and replaced with proper waits where applicable (Tasks/, Pages/
TaskComments + TasksUIFlow, ActivityStream, ActivityFeed). The 4 in
DomainUIInteractions / DataProductAndSubdomains were migrated to the
existing `waitForSearchIndexed` polling helper.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
❌ PR checklist incompleteThis PR cannot be merged until the following are addressed on its linked issue:
The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically. Maintainers can bypass this check by adding the |
github-actions
Bot
added
safe to test
![gitar-bot[bot]](https://avatars.githubusercontent.com/in/827041?s=60&v=4){kind=small}
🟡 Playwright Results — all passed (9 flaky)✅ 807 passed · ❌ 0 failed · 🟡 9 flaky · ⏭️ 9 skipped
🟡 9 flaky test(s) (passed on retry)
How to debug locally# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip # view trace |
Code Review 👍 Approved with suggestions 1 resolved / 4 findingsStabilizes flaky Playwright E2E tests by addressing ES indexing race conditions, correcting 💡 Edge Case: Empty-FQN fallback silently no-ops waitForSearchIndexed📄 openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/BulkImport.spec.ts:808 📄 openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/DataQuality/TestCaseImportExportE2eFlow.spec.ts:79 📄 openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/DataQuality/TestCaseImportExportE2eFlow.spec.ts:150 Several new call sites pass a
This only triggers if the FQN is actually undefined, but in that case failing fast with a clear message is far more debuggable than a silent bypass. Consider asserting the FQN is present (or have Assert the FQN is present before waiting, so a missing FQN fails fast with a clear message instead of a match-all no-op.Make the helper itself reject an empty query so the bypass can never happen silently.💡 Quality: Closed-tab verification can silently no-op after reload📄 openmetadata-ui/src/main/resources/ui/playwright/e2e/Pages/TasksUIFlow.spec.ts:458-472 The 'Resolve task and verify it moves to Closed' step previously called Re-navigate to the tasks tab after reload and assert the Closed tab is present rather than conditionally skipping verification.💡 Edge Case: openEntityTasksTab drops the button-variant Tasks tab fallback📄 openmetadata-ui/src/main/resources/ui/playwright/utils/taskWorkflow.ts:357-371 The refactor of Preserve support for both the menuitem and button Tasks-tab variants.✅ 1 resolved✅ Quality: Unused
|
| Compact |
|
Was this helpful? React with 👍 / 👎 | Gitar
An empty FQN encoded into the q= parameter becomes a match-all query, which lets the helper return on the first poll against any non-empty index — silently bypassing the very indexing race it exists to close. Throw a clear error at the source instead, and drop the misleading `?? ''` fallback at call sites so the type contract reflects reality. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Code Review 👍 Approved with suggestions 2 resolved / 4 findingsHardens E2E tests against ES indexing races and locator timeouts by implementing robust polling and deterministic response handling. Address the remaining 💡 Quality: Closed-tab verification can silently no-op after reload📄 openmetadata-ui/src/main/resources/ui/playwright/e2e/Pages/TasksUIFlow.spec.ts:458-472 The 'Resolve task and verify it moves to Closed' step previously called Re-navigate to the tasks tab after reload and assert the Closed tab is present rather than conditionally skipping verification.💡 Edge Case: openEntityTasksTab drops the button-variant Tasks tab fallback📄 openmetadata-ui/src/main/resources/ui/playwright/utils/taskWorkflow.ts:357-371 The refactor of Preserve support for both the menuitem and button Tasks-tab variants.✅ 2 resolved✅ Quality: Unused
|
| Compact |
|
Was this helpful? React with 👍 / 👎 | Gitar
2 participants
Describe your changes:
PR-A of a 4-way split of #29435. This PR contains only the test-side bug fixes — no config changes, no spec-file moves, no sharding shift. Each fix addresses a real race or locator issue in an existing Playwright E2E spec that was previously surviving on retries:2 but became visible under retry pressure.
The fixes fall into 4 categories:
ES indexing races — entity created via API, then read by UI before search index updates:
BulkEditEntity.spec.ts: wait forglossary_term_search_indexBulkEditOperationBadges.spec.ts: same pattern inbeforeAllBulkImport.spec.ts: wait fordatabase_service/database_schema/tableindexes; pre-wait.rdg-cell-detailscount before asserting textTestCaseImportExportE2eFlow.spec.ts: wait fortest_case_search_indexso the just-created test case is in the export CSVwaitForResponseraces — listener registered after the action, or matching a too-broad URL that resolves on a stale earlier response:utils/searchRBAC.tssearchForEntity helpers: register beforepress('Enter')SearchIndexNestedColumns.spec.ts: predicate-match the URL containing the encoded search termLocator stability / timeout fixes:
Roles.spec.ts(×7 sites): replace strict-mode-violatingexpect(loader).not.toBeVisible()withwaitForAllLoadersToDisappear(handles N loaders correctly)ExploreBrowse.spec.ts: explicittoBeVisiblebefore clicking tree title so the tree settles after expandTreeNodeTable.spec.ts: wait for sort + next-page responses, replacecount()===15withtoHaveCount(15)auto-retryBackend-propagation timeouts — UI fetches lag the API write:
utils/searchRBAC.ts exploreTreeCategories: wrap visibility/absence checks inexpect.toPass({ timeout: 20s })— tree counts come from multiple aggregation queriesutils/searchRBAC.ts exploreShouldShowEntity: bump negative-assertion to 45s — RBAC filtering against newly-assigned roles lags the patchutils/taskWorkflow.ts openTaskDetails: 15s → 45s — activity-feed refresh after API task create lags past default under loadAlso removed 7 unannotated
waitForTimeoutcalls leaking past ESLint, replaced with proper waits. The 4 in DomainUIInteractions / DataProductAndSubdomains migrated to the existingwaitForSearchIndexedpolling helper.Type of change:
High-level design:
Each fix is independent and self-contained. No shared helpers introduced; just makes existing tests' assertions deterministic against the system they're already testing. The PR is a clean 16-file diff (+214 / -49) with no file moves, deletes, or new tests.
Tests:
Pure test-side fixes. No new tests needed — each change makes an existing flaky assertion deterministic. Verified locally that prettier + organize-imports pass and that
tsc --noEmitintroduces no new errors (pre-existing TS errors on main are unaffected).Playwright (UI) tests
UI screen recording / screenshots:
Not applicable — no UI changes.
Checklist:
Split off from #29435 — this is the first of four progressively-merging PRs (test fixes → single-spec caps → config + dedup → PR/stress project split). No associated GitHub issue — this is internal test-tooling work.
🤖 Generated with Claude Code
Greptile Summary
This PR addresses several categories of Playwright E2E test flakiness: ES indexing races (entities created via API are read by the UI before search indexes update),
waitForResponselistener ordering issues, locator strict-mode violations, and backend-propagation timeouts.waitForSearchIndexedcalls added inbeforeAll/beforeEachblocks forBulkEditEntity,BulkEditOperationBadges,BulkImport, andTestCaseImportExportE2eFlow;polling.tsgains a guard that throws on an empty FQN to prevent silent match-all queries.waitForResponsefixes:searchForEntityShouldWorkandsearchForEntityShouldWorkShowNoResultnow register the listener afterfilland beforepress('Enter');SearchIndexNestedColumnsnarrows the predicate to the encoded search term;ExploreBrowsewraps tree-node clicks with targeted response waits.openTaskDetailstimeout bumped to 45 s,exploreTreeCategorieswrapped inexpect.toPass,Table.spec.tsusestoHaveCountinstead of synchronouscount(), and sevenwaitForTimeoutcalls replaced with proper event-driven waits.Confidence Score: 4/5
Safe to merge with one known remaining race in exploreShouldShowEntity that was explicitly fixed elsewhere in this same PR but not at this call site.
The PR correctly fixes the waitForResponse ordering in searchForEntityShouldWork and searchForEntityShouldWorkShowNoResult, but exploreShouldShowEntity on line 48 still registers its broad listener before fill(), meaning an autocomplete response triggered by typing can resolve the listener prematurely and let the resultCard assertion run against stale UI. This is the exact race the PR set out to close, and it remains open in this function.
openmetadata-ui/src/main/resources/ui/playwright/utils/searchRBAC.ts — the exploreShouldShowEntity function still has the waitForResponse listener registered before fill().
Important Files Changed
Sequence Diagram
%%{init: {'theme': 'neutral'}}%% sequenceDiagram participant T as Test participant P as Playwright Page participant API as Search API participant ES as Elasticsearch Note over T,ES: Fixed pattern (searchForEntityShouldWork) T->>P: fill(fqn) P-->>API: autocomplete query (may fire) T->>P: register waitForResponse BEFORE press T->>P: press('Enter') P-->>API: search results query API-->>ES: query index ES-->>API: hits API-->>P: response P-->>T: waitForResponse resolves correctly Note over T,ES: Remaining race (exploreShouldShowEntity - NOT fixed) T->>P: register waitForResponse (line 48) T->>P: fill(fqn) P-->>API: autocomplete query fires API-->>P: autocomplete response resolves listener prematurely T->>P: press('Enter') P-->>API: actual search results query T->>P: assertion runs against stale UI%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% sequenceDiagram participant T as Test participant P as Playwright Page participant API as Search API participant ES as Elasticsearch Note over T,ES: Fixed pattern (searchForEntityShouldWork) T->>P: fill(fqn) P-->>API: autocomplete query (may fire) T->>P: register waitForResponse BEFORE press T->>P: press('Enter') P-->>API: search results query API-->>ES: query index ES-->>API: hits API-->>P: response P-->>T: waitForResponse resolves correctly Note over T,ES: Remaining race (exploreShouldShowEntity - NOT fixed) T->>P: register waitForResponse (line 48) T->>P: fill(fqn) P-->>API: autocomplete query fires API-->>P: autocomplete response resolves listener prematurely T->>P: press('Enter') P-->>API: actual search results query T->>P: assertion runs against stale UIComments Outside Diff (5)
openmetadata-ui/src/main/resources/ui/playwright/utils/searchRBAC.ts, line 48-51 (link)waitForResponsestill registered beforefill()inexploreShouldShowEntitysearchForEntityShouldWorkandsearchForEntityShouldWorkShowNoResultwere fixed in this PR to register the listener betweenfillandpress('Enter')specifically becausefillcan trigger an autocomplete/suggestion query that resolves the broadwaitForResponse('/api/v1/search/query?*')listener prematurely.exploreShouldShowEntitystill registerssearchResbeforefill()on line 48, leaving it exposed to the same race: if the search box fires a suggestion fetch while the user is typing,searchResresolves on that response andpress('Enter')'s actual result page is never waited for, so theresultCardassertion can run against stale UI.openmetadata-ui/src/main/resources/ui/playwright/utils/searchRBAC.ts, line 48-51 (link)waitForResponsestill registered beforefill()inexploreShouldShowEntitysearchResis registered beforefill(fqn)on line 48. If the search box fires an autocomplete/suggestion query while the user is typing,searchResresolves on that early response and thepress('Enter')result page is never awaited — so theresultCardassertion runs against stale UI. This is the exact race that was deliberately fixed insearchForEntityShouldWork(register afterfill, beforepress) but the same pattern was left unremedied here.openmetadata-ui/src/main/resources/ui/playwright/utils/searchRBAC.ts, line 48-51 (link)searchResis registered beforefill(fqn)(line 49), so if the search box fires an autocomplete/suggestion query while the user types, that early response resolves the listener and thepress('Enter')result-page response is never awaited — theresultCardassertion then runs against stale UI. This is exactly the race fixed insearchForEntityShouldWorkandsearchForEntityShouldWorkShowNoResultin this same PR: register the listener afterfill, beforepress.openmetadata-ui/src/main/resources/ui/playwright/utils/searchRBAC.ts, line 48-51 (link)searchResis registered beforefill(fqn)on line 48. Thefill()call on line 49 can immediately trigger an autocomplete/suggestion query whose URL matches'/api/v1/search/query?*', resolving the listener beforepress('Enter')fires the actual results-page query. The assertion then runs against stale autocomplete results rather than the real search page. This is the exact race that was deliberately fixed insearchForEntityShouldWorkandsearchForEntityShouldWorkShowNoResultin this same PR — register afterfill, beforepress.openmetadata-ui/src/main/resources/ui/playwright/utils/searchRBAC.ts, line 48-51 (link)waitForResponseregistered beforefill()— same race left unfixedsearchRes(line 48) is created beforefill(fqn)executes on line 49. Typing into the search box triggers autocomplete queries whose URL matches'/api/v1/search/query?*', which can resolvesearchReson that autocomplete response beforepress('Enter')fires the actual search-page query. When that happens thewaitForAllLoadersToDisappearand theresultCardassertion run against the autocomplete-populated UI, not the search-results page — sotoHaveCount(0, { timeout: 45_000 })for the negative path may pass or fail spuriously.searchForEntityShouldWorkandsearchForEntityShouldWorkShowNoResultwere fixed in this very PR to move the listener registration to afterfilland beforepress('Enter')for exactly this reason. The same fix is needed here.Reviews (7): Last reviewed commit: "playwright(fix): guard waitForSearchInde..." | Re-trigger Greptile