e2e-flakiness-detector #54
Annotations
14 errors, 2 warnings, and 3 notices
|
command-commit.test.ts:23:3 › use terminal output as context:
vscode\test\e2e\helpers.ts#L407
5) command-commit.test.ts:23:3 › use terminal output as context ──────────────────────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceiuV0ju after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceiuV0ju\logs\20250902T143007\window1\exthost\exthost.log'
at helpers.ts:407
405 | } catch (error) {
406 | if (attempts === 1) {
> 407 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
408 | }
409 |
410 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:407:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:288:13)
|
|
chat-input.test.ts:198:3 › chat model selector:
vscode\test\e2e\chat-input.test.ts#L216
4) chat-input.test.ts:198:3 › chat model selector ────────────────────────────────────────────────
Error: locator.click: Target page, context or browser has been closed
Call log:
- waiting for frameLocator('iframe.webview:first-child:last-child').frameLocator('iframe').getByRole('combobox', { name: 'Select a model' }).last()
- locator resolved to <button type="button" role="combobox" data-state="closed" aria-expanded="false" aria-haspopup="dialog" aria-controls="radix-:rf:" data-testid="chat-model-selector" aria-label="Select a model or an agent" class="tw-border-none tw-flex tw-items-center focus-visible:tw-outline-none tw-justify-between tw-mr-1 _button_1wufw_5 _button--secondary_1wufw_60 _button--no-icon-start_1wufw_51">…</button>
- attempting click action
- waiting for element to be visible, enabled and stable
- element is not stable
- retrying click action, attempt #1
- waiting for element to be visible, enabled and stable
- element is visible, enabled and stable
- scrolling into view if needed
- done scrolling
- <p dir="ltr" class="_theme-paragraph_s4gjm_29">…</p> from <div class="_editor_pujow_6 _editor_1eiqs_17 _seamless_pujow_16">…</div> subtree intercepts pointer events
- retrying click action, attempt #2
- waiting 20ms
- waiting for element to be visible, enabled and stable
- element is visible, enabled and stable
- scrolling into view if needed
- done scrolling
- <div class="tw-flex tw-flex-col tw-justify-center tw-w-full tw-gap-2 tw-py-1">…</div> intercepts pointer events
- retrying click action, attempt #3
- waiting 100ms
- waiting for element to be visible, enabled and stable
- element is visible, enabled and stable
- scrolling into view if needed
- done scrolling
- <header class="tw-flex tw-gap-4 tw-items-center [&_>_*]:tw-flex-shrink-0">…</header> from <div class="tw-flex tw-flex-col tw-justify-center tw-w-full tw-gap-2 tw-py-1">…</div> subtree intercepts pointer events
- retrying click action, attempt #4
- waiting 100ms
- waiting for element to be visible, enabled and stable
- element is visible, enabled and stable
- scrolling into view if needed
- done scrolling
- <header class="tw-flex tw-gap-4 tw-items-center [&_>_*]:tw-flex-shrink-0">…</header> from <div class="tw-flex tw-flex-col tw-justify-center tw-w-full tw-gap-2 tw-py-1">…</div> subtree intercepts pointer events
- retrying click action, attempt #5
- waiting 500ms
- waiting for element to be visible, enabled and stable
- element is visible, enabled and stable
- scrolling into view if needed
- done scrolling
- <p dir="ltr" class="_theme-paragraph_s4gjm_29">…</p> from <div class="_editor_pujow_6 _editor_1eiqs_17 _seamless_pujow_16">…</div> subtree intercepts pointer events
- retrying click action, attempt #6
- waiting 500ms
- waiting for element to be visible, enabled and stable
- element is visible, enabled and stable
- scrolling into view if needed
- done scrolling
- <div class="tw-flex tw-flex-col tw-justify-center tw-w-full tw-gap-2 tw-py-1">…</div> intercepts pointer events
- retrying click action, attempt #7
- waiting 500ms
- waiting for element to be visible, enabled and stable
- element is visible, enabled and stable
- scrolling into view if needed
- done scrolling
- <header class="tw-flex tw-gap-4 tw-items-center [&_>_*]:tw-flex-shrink-0">…</header> from <div class="tw-flex tw-flex-col tw-justify-center tw-w-full tw-gap-2 tw-py-1">…</div> subtree intercepts pointer events
- retrying click action, attempt #8
- waiting 500ms
- waiting for element to be visible, enabled and stable
- element is visible, enabled and stable
- scrolling into view if needed
- done scrolling
- <header c
|
|
chat-input.test.ts:198:3 › chat model selector:
vscode\test\e2e\chat-input.test.ts#L0
4) chat-input.test.ts:198:3 › chat model selector ────────────────────────────────────────────────
Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\chat_model_selector\temp-videos\0b35e9c3a88f782a0698a1ea720a3ae1.webm' -> 'D:\a\cody\cody\playwright\chat_model_selector\videos\run_3_retry_0_failure.webm'
|
|
chat-input.test.ts:198:3 › chat model selector:
vscode\test\e2e\chat-input.test.ts#L0
4) chat-input.test.ts:198:3 › chat model selector ────────────────────────────────────────────────
Test timeout of 30000ms exceeded.
|
|
chat-input.test.ts:198:3 › chat model selector:
vscode\test\e2e\chat-input.test.ts#L218
3) chat-input.test.ts:198:3 › chat model selector ────────────────────────────────────────────────
Error: locator.click: Target page, context or browser has been closed
Call log:
- waiting for frameLocator('iframe.webview:first-child:last-child').frameLocator('iframe').getByRole('listbox', { name: 'Suggestions' }).getByRole('option', { name: 'Claude 3 Haiku' })
216 | await modelSelect.click()
217 | const modelChoices = chatFrame.getByRole('listbox', { name: 'Suggestions' })
> 218 | await modelChoices.getByRole('option', { name: 'Claude 3 Haiku' }).click()
| ^
219 | const lastChatInput = getChatInputs(chatFrame).last()
220 | await expect(lastChatInput).toBeFocused()
221 | await expect(modelSelect).toHaveText(/^Claude 3 Haiku/)
at D:\a\cody\cody\vscode\test\e2e\chat-input.test.ts:218:72
|
|
chat-input.test.ts:198:3 › chat model selector:
vscode\test\e2e\chat-input.test.ts#L0
3) chat-input.test.ts:198:3 › chat model selector ────────────────────────────────────────────────
Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\chat_model_selector\temp-videos\6d7f7e3986a552bb1d2bc62307a4de8c.webm' -> 'D:\a\cody\cody\playwright\chat_model_selector\videos\run_0_retry_0_failure.webm'
|
|
chat-input.test.ts:198:3 › chat model selector:
vscode\test\e2e\chat-input.test.ts#L0
3) chat-input.test.ts:198:3 › chat model selector ────────────────────────────────────────────────
Test timeout of 30000ms exceeded.
|
|
chat-atFile.test.ts:375:3 › Add Selection to Cody Chat:
vscode\test\e2e\helpers.ts#L407
2) chat-atFile.test.ts:375:3 › Add Selection to Cody Chat ────────────────────────────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsce6IPTTO after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsce6IPTTO\Crashpad\reports\a0c38749-177e-4b8d-9add-ee24da44281f.dmp'
at helpers.ts:407
405 | } catch (error) {
406 | if (attempts === 1) {
> 407 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
408 | }
409 |
410 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:407:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:288:13)
|
|
chat-atFile.test.ts:316:3 › @-mention symbol in chat:
vscode\test\e2e\chat-atFile.test.ts#L0
1) chat-atFile.test.ts:316:3 › @-mention symbol in chat ──────────────────────────────────────────
Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\__mention_symbol_in_chat\temp-videos\54ddab1ee36836df577d4fdedabc8c01.webm' -> 'D:\a\cody\cody\playwright\__mention_symbol_in_chat\videos\run_0_retry_0_failure.webm'
|
|
chat-atFile.test.ts:316:3 › @-mention symbol in chat:
vscode\test\e2e\chat-atFile.test.ts#L351
1) chat-atFile.test.ts:316:3 › @-mention symbol in chat ──────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected)
Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last()
Expected pattern: /buzz.ts fizzbuzz\(\) /
Received string: "fizzbuzz() "
Call log:
- expect.toHaveText with timeout 5000ms
- waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last()
- locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div>
- unexpected value "fizzbuzz() "
- locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div>
- unexpected value "fizzbuzz() "
- locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div>
- unexpected value "fizzbuzz() "
- locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div>
- unexpected value "fizzbuzz() "
- locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div>
- unexpected value "fizzbuzz() "
- locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div>
- unexpected value "fizzbuzz() "
- locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div>
- unexpected value "fizzbuzz() "
- locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div>
- unexpected value "fizzbuzz() "
- locator resolved to <div role="textbox" spellcheck="true" contenteditable="true" aria-label="Chat message" data-lexical-editor="true" class="_content-editable_s4gjm_6 _editor-content-editable_1eiqs_30">…</div>
- unexpected value "fizzbuzz() "
349 | await expect(chatPanelFrame.getByRole('option', { name: 'fizzbuzz()' })).toBeVisible()
350 | await chatPanelFrame.getByRole('option', { name: 'fizzbuzz()' }).click()
> 351 | await expect(chatInput).toHaveText(/buzz.ts fizzbuzz\(\) /)
| ^
352 | await expect(chatInputMentions(chatInput)).toContainText(['buzz.ts', 'fizzbuzz()'])
353 |
354 | // Submit the message
at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:351:29
|
|
test-e2e (ubuntu)
Process completed with exit code 1.
|
|
command-edit.test.ts:33:3 › edit (fixup) task:
vscode/test/e2e/command-edit.test.ts#L53
3) command-edit.test.ts:33:3 › edit (fixup) task ─────────────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toBeVisible()
Locator: getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a')
Expected: visible
Received: hidden
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a')
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
51 | const modelItem = page.getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a')
52 | await nap()
> 53 | expect(modelItem).toBeVisible()
| ^
54 |
55 | // Open the model input and check it has the correct item selected
56 | await modelItem.click()
at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:53:23
|
|
command-edit.test.ts:33:3 › edit (fixup) task:
vscode/test/e2e/command-edit.test.ts#L53
2) command-edit.test.ts:33:3 › edit (fixup) task ─────────────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toBeVisible()
Locator: getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a')
Expected: visible
Received: hidden
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a')
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
- locator resolved to <a class="label-name">…</a>
- unexpected value "hidden"
51 | const modelItem = page.getByLabel('$(anthropic-logo) Claude 3.5 Sonnet').locator('a')
52 | await nap()
> 53 | expect(modelItem).toBeVisible()
| ^
54 |
55 | // Open the model input and check it has the correct item selected
56 | await modelItem.click()
at /home/runner/work/cody/cody/vscode/test/e2e/command-edit.test.ts:53:23
|
|
command-custom.test.ts:45:3 › create a new user command via the custom commands menu:
vscode/test/e2e/command-custom.test.ts#L92
1) command-custom.test.ts:45:3 › create a new user command via the custom commands menu ──────────
Error: Timed out 5000ms waiting for expect(locator).toBeVisible()
Locator: getByText('New Custom Cody Command: Prompt')
Expected: visible
Received: <element(s) not found>
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByText('New Custom Cody Command: Prompt')
90 | // Enter prompt
91 | const promptInputTitle = page.getByText('New Custom Cody Command: Prompt')
> 92 | await expect(promptInputTitle).toBeVisible()
| ^
93 | const promptInputBox = page.getByPlaceholder(
94 | 'e.g. Create five different test cases for the selected code'
95 | )
at /home/runner/work/cody/cody/vscode/test/e2e/command-custom.test.ts:92:36
|
|
test-e2e (windows)
The process 'C:\Program Files\Git\bin\git.exe' failed with exit code 128
|
|
test-e2e (ubuntu)
The process '/usr/bin/git' failed with exit code 128
|
|
🎭 Playwright Run Summary
9 failed
chat-atFile.test.ts:316:3 › @-mention symbol in chat ───────────────────────────────────────────
chat-atFile.test.ts:375:3 › Add Selection to Cody Chat ─────────────────────────────────────────
chat-input.test.ts:198:3 › chat model selector ─────────────────────────────────────────────────
chat-input.test.ts:198:3 › chat model selector ─────────────────────────────────────────────────
command-commit.test.ts:23:3 › use terminal output as context ───────────────────────────────────
command-edit.test.ts:121:1 › edit (fixup) input - range selection ──────────────────────────────
command-edit.test.ts:33:3 › edit (fixup) task ──────────────────────────────────────────────────
initial-context.test.ts:46:18 › initial context - file ─────────────────────────────────────────
initial-context.test.ts:46:18 › initial context - file ─────────────────────────────────────────
60 skipped
155 passed (50.1m)
|
|
test-e2e (windows)
The windows-latest label will migrate from Windows Server 2022 to Windows Server 2025 beginning September 2, 2025. For more information see https://github.com/actions/runner-images/issues/12677
|
|
🎭 Playwright Run Summary
3 failed
command-custom.test.ts:45:3 › create a new user command via the custom commands menu ───────────
command-edit.test.ts:33:3 › edit (fixup) task ──────────────────────────────────────────────────
command-edit.test.ts:33:3 › edit (fixup) task ──────────────────────────────────────────────────
150 skipped
407 passed (1.7h)
|
Artifacts
Produced during runtime
| Name | Size | Digest | |
|---|---|---|---|
|
playwright-recordings ubuntu
Expired
|
901 KB |
sha256:fc09c2729eed33f9d2d6178161a1ce06909d17c4707134a676a7addd8dc8b7a7
|
|
|
playwright-recordings windows
Expired
|
874 KB |
sha256:9ee85f6c33d5e77e9629572d969283d95ce7cae278af17bcbcaec420146fd5a2
|
|