Skip to content

e2e-flakiness-detector #56

e2e-flakiness-detector

e2e-flakiness-detector #56

Triggered via schedule September 4, 2025 14:16
Status Failure
Total duration 1h 47m 40s
Artifacts 2
Matrix: test-e2e
Fit to window
Zoom out
Zoom in

Annotations

12 errors, 2 warnings, and 3 notices
command-commit.test.ts:23:3 › use terminal output as context: vscode\test\e2e\helpers.ts#L407
6) command-commit.test.ts:23:3 › use terminal output as context ────────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscenJTsut after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vscenJTsut\logs\20250904T143011\window1\exthost\exthost.log' at helpers.ts:407 405 | } catch (error) { 406 | if (attempts === 1) { > 407 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 408 | } 409 | 410 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:407:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:288:13)
code-actions.test.ts:86:3 › code action: fix: vscode\test\e2e\helpers.ts#L407
5) code-actions.test.ts:86:3 › code action: fix ────────────────────────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsce7jB1s4 after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsce7jB1s4\Crashpad\reports\ac4ddf9f-00fb-463f-ac7d-c176e69f671b.dmp' at helpers.ts:407 405 | } catch (error) { 406 | if (attempts === 1) { > 407 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 408 | } 409 | 410 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:407:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:288:13)
chat-input.test.ts:198:3 › chat model selector: vscode\test\e2e\chat-input.test.ts#L218
4) chat-input.test.ts:198:3 › chat model selector ──────────────────────────────────────────────── Error: locator.click: Target page, context or browser has been closed Call log: - waiting for frameLocator('iframe.webview:first-child:last-child').frameLocator('iframe').getByRole('listbox', { name: 'Suggestions' }).getByRole('option', { name: 'Claude 3 Haiku' }) 216 | await modelSelect.click() 217 | const modelChoices = chatFrame.getByRole('listbox', { name: 'Suggestions' }) > 218 | await modelChoices.getByRole('option', { name: 'Claude 3 Haiku' }).click() | ^ 219 | const lastChatInput = getChatInputs(chatFrame).last() 220 | await expect(lastChatInput).toBeFocused() 221 | await expect(modelSelect).toHaveText(/^Claude 3 Haiku/) at D:\a\cody\cody\vscode\test\e2e\chat-input.test.ts:218:72
chat-input.test.ts:198:3 › chat model selector: vscode\test\e2e\chat-input.test.ts#L0
4) chat-input.test.ts:198:3 › chat model selector ──────────────────────────────────────────────── Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\chat_model_selector\temp-videos\8f1d70cff9c98ed2425622f76bc2342a.webm' -> 'D:\a\cody\cody\playwright\chat_model_selector\videos\run_3_retry_0_failure.webm'
chat-input.test.ts:198:3 › chat model selector: vscode\test\e2e\chat-input.test.ts#L0
4) chat-input.test.ts:198:3 › chat model selector ──────────────────────────────────────────────── Test timeout of 30000ms exceeded.
chat-input.test.ts:198:3 › chat model selector: vscode\test\e2e\chat-input.test.ts#L221
3) chat-input.test.ts:198:3 › chat model selector ──────────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected) Locator: frameLocator('iframe.webview:first-child:last-child').frameLocator('iframe').getByRole('combobox', { name: 'Select a model' }).last() Expected pattern: /^Claude 3 Haiku/ Received string: "Claude 3.5 Sonnet" Call log: - expect.toHaveText with timeout 5000ms - waiting for frameLocator('iframe.webview:first-child:last-child').frameLocator('iframe').getByRole('combobox', { name: 'Select a model' }).last() - locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:r1g:" data-testid="chat-model-selector" aria-label="Select a model or an agent" class="tw-border-none tw-flex tw-items-center focus-visible:tw-outline-none tw-justify-between tw-mr-1 _button_1wufw_5 _button--secondary_1wufw_60 _button--no-icon-start_1wufw_51">…</button> - unexpected value "Claude 3.5 Sonnet" - locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:r1g:" data-testid="chat-model-selector" aria-label="Select a model or an agent" class="tw-border-none tw-flex tw-items-center focus-visible:tw-outline-none tw-justify-between tw-mr-1 _button_1wufw_5 _button--secondary_1wufw_60 _button--no-icon-start_1wufw_51">…</button> - unexpected value "Claude 3.5 Sonnet" - locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:r1g:" data-testid="chat-model-selector" aria-label="Select a model or an agent" class="tw-border-none tw-flex tw-items-center focus-visible:tw-outline-none tw-justify-between tw-mr-1 _button_1wufw_5 _button--secondary_1wufw_60 _button--no-icon-start_1wufw_51">…</button> - unexpected value "Claude 3.5 Sonnet" - locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:r1g:" data-testid="chat-model-selector" aria-label="Select a model or an agent" class="tw-border-none tw-flex tw-items-center focus-visible:tw-outline-none tw-justify-between tw-mr-1 _button_1wufw_5 _button--secondary_1wufw_60 _button--no-icon-start_1wufw_51">…</button> - unexpected value "Claude 3.5 Sonnet" - locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:r1g:" data-testid="chat-model-selector" aria-label="Select a model or an agent" class="tw-border-none tw-flex tw-items-center focus-visible:tw-outline-none tw-justify-between tw-mr-1 _button_1wufw_5 _button--secondary_1wufw_60 _button--no-icon-start_1wufw_51">…</button> - unexpected value "Claude 3.5 Sonnet" - locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:r1g:" data-testid="chat-model-selector" aria-label="Select a model or an agent" class="tw-border-none tw-flex tw-items-center focus-visible:tw-outline-none tw-justify-between tw-mr-1 _button_1wufw_5 _button--secondary_1wufw_60 _button--no-icon-start_1wufw_51">…</button> - unexpected value "Claude 3.5 Sonnet" - locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:r1g:" data-testid="chat-model-selector" aria-label="Select a model or an agent" class="tw-border-none tw-flex tw-items-center focus-visible:tw-outline-none tw-justify-between tw-mr-1 _button_1wufw_5 _button--secondary_1wufw_60 _button--no-icon-start_1wufw_51">…</button> - unexpected value "Claude 3.5 Sonnet" - locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:r1g:" da
chat-input.test.ts:198:3 › chat model selector: vscode\test\e2e\chat-input.test.ts#L218
2) chat-input.test.ts:198:3 › chat model selector ──────────────────────────────────────────────── Error: locator.click: Target page, context or browser has been closed Call log: - waiting for frameLocator('iframe.webview:first-child:last-child').frameLocator('iframe').getByRole('listbox', { name: 'Suggestions' }).getByRole('option', { name: 'Claude 3 Haiku' }) - locator resolved to <div id=":r13:" cmdk-item="" role="option" aria-disabled="false" aria-selected="false" data-disabled="false" data-selected="false" title="Claude 3 Haiku by Anthropic" data-testid="chat-model-popover-option" data-value="anthropic::2023-06-01::claude-3-haiku-20240307" class="tw-relative tw-flex tw-cursor-pointer tw-select-none tw-items-center tw-py-3 tw-px-3 tw-text-md tw-outline-none aria-selected:tw-bg-accent aria-selected:tw-text-accent-foreground hover:tw-bg-accent hover:tw-text-accent-foreground…>…</div> - attempting click action - waiting for element to be visible, enabled and stable - element is visible, enabled and stable - scrolling into view if needed - done scrolling - element was detached from the DOM, retrying 216 | await modelSelect.click() 217 | const modelChoices = chatFrame.getByRole('listbox', { name: 'Suggestions' }) > 218 | await modelChoices.getByRole('option', { name: 'Claude 3 Haiku' }).click() | ^ 219 | const lastChatInput = getChatInputs(chatFrame).last() 220 | await expect(lastChatInput).toBeFocused() 221 | await expect(modelSelect).toHaveText(/^Claude 3 Haiku/) at D:\a\cody\cody\vscode\test\e2e\chat-input.test.ts:218:72
chat-input.test.ts:198:3 › chat model selector: vscode\test\e2e\chat-input.test.ts#L0
2) chat-input.test.ts:198:3 › chat model selector ──────────────────────────────────────────────── Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\chat_model_selector\temp-videos\02d63250174b4a947e06e74695034371.webm' -> 'D:\a\cody\cody\playwright\chat_model_selector\videos\run_1_retry_0_failure.webm'
chat-input.test.ts:198:3 › chat model selector: vscode\test\e2e\chat-input.test.ts#L0
2) chat-input.test.ts:198:3 › chat model selector ──────────────────────────────────────────────── Test timeout of 30000ms exceeded.
chat-atFile.test.ts:375:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L0
1) chat-atFile.test.ts:375:3 › Add Selection to Cody Chat ──────────────────────────────────────── Test timeout of 30000ms exceeded.
test-e2e (ubuntu)
Process completed with exit code 1.
chat-atFile.test.ts:49:7 › @-mention file in chat: vscode/test/e2e/chat-atFile.test.ts#L167
1) chat-atFile.test.ts:49:7 › @-mention file in chat ───────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected) Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last() Expected string: "Main.java and Main.java " Received string: "Main.java and @Main.ja" Call log: - expect.toHaveText with timeout 5000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last() - locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" aria-controls="typeahead-menu" aria-activedescendant="typeahead-item-0" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div> - unexpected value "Main.java and @Main.ja" - locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" aria-controls="typeahead-menu" aria-activedescendant="typeahead-item-0" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div> - unexpected value "Main.java and @Main.ja" - locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" aria-controls="typeahead-menu" aria-activedescendant="typeahead-item-0" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div> - unexpected value "Main.java and @Main.ja" - locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" aria-controls="typeahead-menu" aria-activedescendant="typeahead-item-0" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div> - unexpected value "Main.java and @Main.ja" - locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" aria-controls="typeahead-menu" aria-activedescendant="typeahead-item-0" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div> - unexpected value "Main.java and @Main.ja" - locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" aria-controls="typeahead-menu" aria-activedescendant="typeahead-item-0" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div> - unexpected value "Main.java and @Main.ja" - locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" aria-controls="typeahead-menu" aria-activedescendant="typeahead-item-0" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div> - unexpected value "Main.java and @Main.ja" - locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" aria-controls="typeahead-menu" aria-activedescendant="typeahead-item-0" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div> - unexpected value "Main.java and @Main.ja" - locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" aria-controls="typeahead-menu" aria-activedescendant="typeahead-item-0" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div> - unexpected value "Main.java and @Main.ja" 165 | await chatInput.pressSequentially('and @Main.ja', { delay: 10 }) 166 | await chatInput.press('Tab') > 167 | await expect(chatInput).toHaveText('Main.java and Main.java ') | ^ 168 | await expect(chatInputMentions(chatInput)).toHaveText(['Main.java', 'Main.java'])
test-e2e (windows)
The process 'C:\Program Files\Git\bin\git.exe' failed with exit code 128
test-e2e (ubuntu)
The process '/usr/bin/git' failed with exit code 128
🎭 Playwright Run Summary
11 failed chat-atFile.test.ts:375:3 › Add Selection to Cody Chat ───────────────────────────────────────── chat-input.test.ts:198:3 › chat model selector ───────────────────────────────────────────────── chat-input.test.ts:198:3 › chat model selector ───────────────────────────────────────────────── chat-input.test.ts:198:3 › chat model selector ───────────────────────────────────────────────── code-actions.test.ts:86:3 › code action: fix ─────────────────────────────────────────────────── command-commit.test.ts:23:3 › use terminal output as context ─────────────────────────────────── command-edit.test.ts:121:1 › edit (fixup) input - range selection ────────────────────────────── command-edit.test.ts:121:1 › edit (fixup) input - range selection ────────────────────────────── command-edit.test.ts:121:1 › edit (fixup) input - range selection ────────────────────────────── command-menu.test.ts:24:3 › Start a new chat from Cody Command Menu ──────────────────────────── initial-context.test.ts:46:18 › initial context - file ───────────────────────────────────────── 60 skipped 153 passed (52.6m)
test-e2e (windows)
The windows-latest label will migrate from Windows Server 2022 to Windows Server 2025 beginning September 2, 2025. For more information see https://github.com/actions/runner-images/issues/12677
🎭 Playwright Run Summary
1 failed chat-atFile.test.ts:49:7 › @-mention file in chat ────────────────────────────────────────────── 150 skipped 409 passed (1.7h)

Artifacts

Produced during runtime
Name Size Digest
playwright-recordings ubuntu Expired
699 KB
sha256:9063f3c1fa80de993ccc31de9e1b380512357d225c09ec0b4b325ee20a5451fb
playwright-recordings windows Expired
3.68 MB
sha256:ea4c3a1f7c7ec41d0789f1bd3cc19d30cb87a7036ab71f87ec05a26d2500709b