Skip to content

Deep Cody (#5687) #22306

Deep Cody (#5687)

Deep Cody (#5687) #22306

Triggered via push October 4, 2024 05:23
Status Failure
Total duration 17m 14s
Artifacts 1

ci.yml

on: push
fast_tests_matrix_prep
0s
fast_tests_matrix_prep
slow_tests_matrix_prep
0s
slow_tests_matrix_prep
Matrix: test-integration
Matrix: test-unit
Matrix: test-e2e
Fit to window
Zoom out
Zoom in

Annotations

25 errors, 19 warnings, and 3 notices
test-unit (ubuntu, 20)
The v0 series of google-github-actions/auth is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions: https://github.com/google-github-actions/auth
test-unit (ubuntu, 20)
The v0 series of google-github-actions/setup-gcloud is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions: https://github.com/google-github-actions/setup-gcloud
test-unit (ubuntu, 18)
The v0 series of google-github-actions/setup-gcloud is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions: https://github.com/google-github-actions/setup-gcloud
test-unit (ubuntu, 18)
The v0 series of google-github-actions/auth is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions: https://github.com/google-github-actions/auth
test-unit (macos, 20)
The v0 series of google-github-actions/auth is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions: https://github.com/google-github-actions/auth
test-unit (macos, 20)
The v0 series of google-github-actions/setup-gcloud is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions: https://github.com/google-github-actions/setup-gcloud
test-unit (windows, 20)
The v0 series of google-github-actions/auth is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions: https://github.com/google-github-actions/auth
test-unit (windows, 20)
The v0 series of google-github-actions/setup-gcloud is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions: https://github.com/google-github-actions/setup-gcloud
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode/test/e2e/chat-atFile.test.ts#L369
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected) Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - Expected - 2 + Received + 0 Array [ - "buzz.ts", - "buzz.ts:2-5", "buzz.ts:7-10", ] Call log: - expect.toHaveText with timeout 5000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element 367 | await selectLineRangeInEditorTab(page, 7, 10) 368 | await executeCommandInPalette(page, 'Cody: Add Selection to Cody Chat') > 369 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5', 'buzz.ts:7-10']) | ^ 370 | }) 371 | at /Users/runner/work/cody/cody/vscode/test/e2e/chat-atFile.test.ts:369:52
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode/test/e2e/chat-atFile.test.ts#L369
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected) Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - Expected - 2 + Received + 0 Array [ - "buzz.ts", - "buzz.ts:2-5", "buzz.ts:7-10", ] Call log: - expect.toHaveText with timeout 5000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element 367 | await selectLineRangeInEditorTab(page, 7, 10) 368 | await executeCommandInPalette(page, 'Cody: Add Selection to Cody Chat') > 369 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5', 'buzz.ts:7-10']) | ^ 370 | }) 371 | at /Users/runner/work/cody/cody/vscode/test/e2e/chat-atFile.test.ts:369:52
enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled: vscode/test/e2e/enterprise-server-sent-models.test.ts#L1
2) enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled Test timeout of 30000ms exceeded.
enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled: vscode/test/e2e/enterprise-server-sent-models.test.ts#L26
2) enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled Error: locator.click: Target page, context or browser has been closed Call log: - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('listbox', { name: 'Suggestions' }).getByRole('option', { name: 'Titan' }) 24 | await modelSelect.click() 25 | const modelChoices = chatFrame.getByRole('listbox', { name: 'Suggestions' }) > 26 | await modelChoices.getByRole('option', { name: 'Titan' }).click() | ^ 27 | await expect(modelSelect).toHaveText(/^Titan/) 28 | 29 | // Close chat window and create a new one and assert the default model is preserved at /Users/runner/work/cody/cody/vscode/test/e2e/enterprise-server-sent-models.test.ts:26:63
enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled: vscode/test/e2e/enterprise-server-sent-models.test.ts#L1
2) enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Test timeout of 30000ms exceeded.
enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled: vscode/test/e2e/enterprise-server-sent-models.test.ts#L26
2) enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: locator.click: Target page, context or browser has been closed Call log: - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('listbox', { name: 'Suggestions' }).getByRole('option', { name: 'Titan' }) 24 | await modelSelect.click() 25 | const modelChoices = chatFrame.getByRole('listbox', { name: 'Suggestions' }) > 26 | await modelChoices.getByRole('option', { name: 'Titan' }).click() | ^ 27 | await expect(modelSelect).toHaveText(/^Titan/) 28 | 29 | // Close chat window and create a new one and assert the default model is preserved at /Users/runner/work/cody/cody/vscode/test/e2e/enterprise-server-sent-models.test.ts:26:63
test-e2e (macos)
Process completed with exit code 1.
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L369
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected) Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - Expected - 2 + Received + 0 Array [ - "buzz.ts", - "buzz.ts:2-5", "buzz.ts:7-10", ] Call log: - expect.toHaveText with timeout 5000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element 367 | await selectLineRangeInEditorTab(page, 7, 10) 368 | await executeCommandInPalette(page, 'Cody: Add Selection to Cody Chat') > 369 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5', 'buzz.ts:7-10']) | ^ 370 | }) 371 | at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:369:52
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L1
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\temp-videos\45b8567cb4f33a37569d6dd1b1d85b73.webm' -> 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\videos\run_0_retry_0_failure.webm'
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L369
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected) Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - Expected - 2 + Received + 0 Array [ - "buzz.ts", - "buzz.ts:2-5", "buzz.ts:7-10", ] Call log: - expect.toHaveText with timeout 5000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - locator resolved to 0 elements - locator resolved to 0 elements - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element 367 | await selectLineRangeInEditorTab(page, 7, 10) 368 | await executeCommandInPalette(page, 'Cody: Add Selection to Cody Chat') > 369 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5', 'buzz.ts:7-10']) | ^ 370 | }) 371 | at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:369:52
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L1
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\temp-videos\28c8f4037034369a342fcd5bead3323c.webm' -> 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\videos\run_0_retry_1_failure.webm'
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L369
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Retry #2 ─────────────────────────────────────────────────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected) Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - Expected - 2 + Received + 0 Array [ - "buzz.ts", - "buzz.ts:2-5", "buzz.ts:7-10", ] Call log: - expect.toHaveText with timeout 5000ms - waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node') - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element - locator resolved to 1 element 367 | await selectLineRangeInEditorTab(page, 7, 10) 368 | await executeCommandInPalette(page, 'Cody: Add Selection to Cody Chat') > 369 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5', 'buzz.ts:7-10']) | ^ 370 | }) 371 | at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:369:52
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat: vscode\test\e2e\chat-atFile.test.ts#L1
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ──────────────────────────────────────── Retry #2 ─────────────────────────────────────────────────────────────────────────────────────── Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\temp-videos\063fbdfdb5cd2e062024f4960e8ca9b7.webm' -> 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\videos\run_0_retry_2_failure.webm'
chat-rateLimit.test.ts:80:3 › shows standard rate limit message for non-dotCom users: vscode\test\e2e\helpers.ts#L332
2) chat-rateLimit.test.ts:80:3 › shows standard rate limit message for non-dotCom users ────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsce0AICub after 5 attempts: Error: EPERM: operation not permitted, lstat '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsce0AICub\User\globalStorage\sourcegraph.cody-ai\symf\symf-v0.0.16-x86_64-windows' at helpers.ts:332 330 | } catch (error) { 331 | if (attempts === 1) { > 332 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 333 | } 334 | 335 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:332:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:213:13)
chat-rateLimit.test.ts:80:3 › shows standard rate limit message for non-dotCom users: vscode\test\e2e\helpers.ts#L332
2) chat-rateLimit.test.ts:80:3 › shows standard rate limit message for non-dotCom users ────────── Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceTqOwL5 after 5 attempts: Error: EPERM: operation not permitted, lstat '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceTqOwL5\User\globalStorage\sourcegraph.cody-ai\symf\symf-v0.0.16-x86_64-windows' at helpers.ts:332 330 | } catch (error) { 331 | if (attempts === 1) { > 332 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 333 | } 334 | 335 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:332:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:213:13)
command-core.test.ts:29:3 › Generate Unit Test Command (Edit): vscode\test\e2e\helpers.ts#L332
3) command-core.test.ts:29:3 › Generate Unit Test Command (Edit) ───────────────────────────────── Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceVxutie after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceVxutie\Crashpad\reports\4f0f5aab-8f58-4df1-9a38-4aada5f64ab6.dmp' at helpers.ts:332 330 | } catch (error) { 331 | if (attempts === 1) { > 332 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`) | ^ 333 | } 334 | 335 | await new Promise(resolve => setTimeout(resolve, 100)) at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:332:23) at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:213:13)
command-core.test.ts:29:3 › Generate Unit Test Command (Edit): vscode\test\e2e\command-core.test.ts#L47
3) command-core.test.ts:29:3 › Generate Unit Test Command (Edit) ───────────────────────────────── Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: getByRole('button', { name: 'Accept' }) Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByRole('button', { name: 'Accept' }) 45 | 46 | // Code lens should be visible > 47 | await expect(page.getByRole('button', { name: 'Accept' })).toBeVisible() | ^ 48 | await expect(page.getByRole('button', { name: 'Reject' })).toBeVisible() 49 | }) 50 | at D:\a\cody\cody\vscode\test\e2e\command-core.test.ts:47:64
test-unit (ubuntu, 20)
The following actions use a deprecated Node.js version and will be forced to run on node20: google-github-actions/auth@v0, google-github-actions/setup-gcloud@v0. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
test-unit (ubuntu, 18)
The following actions use a deprecated Node.js version and will be forced to run on node20: google-github-actions/auth@v0, google-github-actions/setup-gcloud@v0. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
test-unit (macos, 20)
The following actions use a deprecated Node.js version and will be forced to run on node20: google-github-actions/auth@v0, google-github-actions/setup-gcloud@v0. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
test-unit (windows, 20)
The following actions use a deprecated Node.js version and will be forced to run on node20: google-github-actions/auth@v0, google-github-actions/setup-gcloud@v0. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
Slow Test: vscode/chat-atFile.test.ts#L1
vscode/chat-atFile.test.ts took 1.1m
Slow Test: vscode/command-custom.test.ts#L1
vscode/command-custom.test.ts took 56.4s
Slow Test: vscode/chat-input.test.ts#L1
vscode/chat-input.test.ts took 46.6s
Slow Test: vscode/initial-context.test.ts#L1
vscode/initial-context.test.ts took 39.5s
Slow Test: vscode/command-core.test.ts#L1
vscode/command-core.test.ts took 35.6s
Slow Test: vscode/command-custom.test.ts#L1
vscode/command-custom.test.ts took 58.4s
Slow Test: vscode/chat-input.test.ts#L1
vscode/chat-input.test.ts took 45.7s
Slow Test: vscode/initial-context.test.ts#L1
vscode/initial-context.test.ts took 34.6s
Slow Test: vscode/code-actions.test.ts#L1
vscode/code-actions.test.ts took 34.3s
Slow Test: vscode/command-core.test.ts#L1
vscode/command-core.test.ts took 33.9s
Slow Test: vscode\chat-input.test.ts#L1
vscode\chat-input.test.ts took 51.5s
Slow Test: vscode\code-actions.test.ts#L1
vscode\code-actions.test.ts took 34.3s
Slow Test: vscode\chat-keyboard-shortcuts.test.ts#L1
vscode\chat-keyboard-shortcuts.test.ts took 26.3s
Slow Test: vscode\attribution.test.ts#L1
vscode\attribution.test.ts took 21.1s
Slow Test: vscode\enterprise-cody-ignore.test.ts#L1
vscode\enterprise-cody-ignore.test.ts took 20.5s
🎭 Playwright Run Summary
3 skipped 43 passed (8.2m)
🎭 Playwright Run Summary
2 failed chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ───────────────────────────────────────── enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled 3 skipped 41 passed (9.6m)
🎭 Playwright Run Summary
7 flaky chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ───────────────────────────────────────── chat-rateLimit.test.ts:80:3 › shows standard rate limit message for non-dotCom users ─────────── command-core.test.ts:29:3 › Generate Unit Test Command (Edit) ────────────────────────────────── command-custom.test.ts:141:3 › execute custom commands with context defined in cody.json ─────── command-edit.test.ts:95:1 › edit (fixup) input - range selection ─────────────────────────────── initial-context.test.ts:28:18 › initial context - enterprise repo ────────────────────────────── initial-context.test.ts:83:82 › chat try-again actions ───────────────────────────────────────── 3 skipped 36 passed (13.2m)

Artifacts

Produced during runtime
Name Size
playwright-recordings macos
3.56 MB