Deep Cody (#5687) #22306
ci.yml
on: push
fast_tests_matrix_prep
0s
slow_tests_matrix_prep
0s
build
2m 13s
Matrix: test-integration
Matrix: test-unit
Matrix: test-e2e
Annotations
25 errors, 19 warnings, and 3 notices
test-unit (ubuntu, 20)
The v0 series of google-github-actions/auth is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions:
https://github.com/google-github-actions/auth
|
test-unit (ubuntu, 20)
The v0 series of google-github-actions/setup-gcloud is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions:
https://github.com/google-github-actions/setup-gcloud
|
test-unit (ubuntu, 18)
The v0 series of google-github-actions/setup-gcloud is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions:
https://github.com/google-github-actions/setup-gcloud
|
test-unit (ubuntu, 18)
The v0 series of google-github-actions/auth is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions:
https://github.com/google-github-actions/auth
|
test-unit (macos, 20)
The v0 series of google-github-actions/auth is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions:
https://github.com/google-github-actions/auth
|
test-unit (macos, 20)
The v0 series of google-github-actions/setup-gcloud is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions:
https://github.com/google-github-actions/setup-gcloud
|
test-unit (windows, 20)
The v0 series of google-github-actions/auth is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions:
https://github.com/google-github-actions/auth
|
test-unit (windows, 20)
The v0 series of google-github-actions/setup-gcloud is no longer maintained. It will not receive updates, improvements, or security patches. Please upgrade to the latest supported versions:
https://github.com/google-github-actions/setup-gcloud
|
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat:
vscode/test/e2e/chat-atFile.test.ts#L369
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected)
Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- Expected - 2
+ Received + 0
Array [
- "buzz.ts",
- "buzz.ts:2-5",
"buzz.ts:7-10",
]
Call log:
- expect.toHaveText with timeout 5000ms
- waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
367 | await selectLineRangeInEditorTab(page, 7, 10)
368 | await executeCommandInPalette(page, 'Cody: Add Selection to Cody Chat')
> 369 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5', 'buzz.ts:7-10'])
| ^
370 | })
371 |
at /Users/runner/work/cody/cody/vscode/test/e2e/chat-atFile.test.ts:369:52
|
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat:
vscode/test/e2e/chat-atFile.test.ts#L369
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ────────────────────────────────────────
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected)
Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- Expected - 2
+ Received + 0
Array [
- "buzz.ts",
- "buzz.ts:2-5",
"buzz.ts:7-10",
]
Call log:
- expect.toHaveText with timeout 5000ms
- waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
367 | await selectLineRangeInEditorTab(page, 7, 10)
368 | await executeCommandInPalette(page, 'Cody: Add Selection to Cody Chat')
> 369 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5', 'buzz.ts:7-10'])
| ^
370 | })
371 |
at /Users/runner/work/cody/cody/vscode/test/e2e/chat-atFile.test.ts:369:52
|
enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled:
vscode/test/e2e/enterprise-server-sent-models.test.ts#L1
2) enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled
Test timeout of 30000ms exceeded.
|
enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled:
vscode/test/e2e/enterprise-server-sent-models.test.ts#L26
2) enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled
Error: locator.click: Target page, context or browser has been closed
Call log:
- waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('listbox', { name: 'Suggestions' }).getByRole('option', { name: 'Titan' })
24 | await modelSelect.click()
25 | const modelChoices = chatFrame.getByRole('listbox', { name: 'Suggestions' })
> 26 | await modelChoices.getByRole('option', { name: 'Titan' }).click()
| ^
27 | await expect(modelSelect).toHaveText(/^Titan/)
28 |
29 | // Close chat window and create a new one and assert the default model is preserved
at /Users/runner/work/cody/cody/vscode/test/e2e/enterprise-server-sent-models.test.ts:26:63
|
enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled:
vscode/test/e2e/enterprise-server-sent-models.test.ts#L1
2) enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Test timeout of 30000ms exceeded.
|
enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled:
vscode/test/e2e/enterprise-server-sent-models.test.ts#L26
2) enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: locator.click: Target page, context or browser has been closed
Call log:
- waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('listbox', { name: 'Suggestions' }).getByRole('option', { name: 'Titan' })
24 | await modelSelect.click()
25 | const modelChoices = chatFrame.getByRole('listbox', { name: 'Suggestions' })
> 26 | await modelChoices.getByRole('option', { name: 'Titan' }).click()
| ^
27 | await expect(modelSelect).toHaveText(/^Titan/)
28 |
29 | // Close chat window and create a new one and assert the default model is preserved
at /Users/runner/work/cody/cody/vscode/test/e2e/enterprise-server-sent-models.test.ts:26:63
|
test-e2e (macos)
Process completed with exit code 1.
|
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat:
vscode\test\e2e\chat-atFile.test.ts#L369
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected)
Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- Expected - 2
+ Received + 0
Array [
- "buzz.ts",
- "buzz.ts:2-5",
"buzz.ts:7-10",
]
Call log:
- expect.toHaveText with timeout 5000ms
- waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- locator resolved to 0 elements
- locator resolved to 0 elements
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
367 | await selectLineRangeInEditorTab(page, 7, 10)
368 | await executeCommandInPalette(page, 'Cody: Add Selection to Cody Chat')
> 369 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5', 'buzz.ts:7-10'])
| ^
370 | })
371 |
at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:369:52
|
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat:
vscode\test\e2e\chat-atFile.test.ts#L1
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ────────────────────────────────────────
Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\temp-videos\45b8567cb4f33a37569d6dd1b1d85b73.webm' -> 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\videos\run_0_retry_0_failure.webm'
|
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat:
vscode\test\e2e\chat-atFile.test.ts#L369
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ────────────────────────────────────────
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected)
Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- Expected - 2
+ Received + 0
Array [
- "buzz.ts",
- "buzz.ts:2-5",
"buzz.ts:7-10",
]
Call log:
- expect.toHaveText with timeout 5000ms
- waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- locator resolved to 0 elements
- locator resolved to 0 elements
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
367 | await selectLineRangeInEditorTab(page, 7, 10)
368 | await executeCommandInPalette(page, 'Cody: Add Selection to Cody Chat')
> 369 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5', 'buzz.ts:7-10'])
| ^
370 | })
371 |
at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:369:52
|
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat:
vscode\test\e2e\chat-atFile.test.ts#L1
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ────────────────────────────────────────
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\temp-videos\28c8f4037034369a342fcd5bead3323c.webm' -> 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\videos\run_0_retry_1_failure.webm'
|
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat:
vscode\test\e2e\chat-atFile.test.ts#L369
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ────────────────────────────────────────
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected)
Locator: frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- Expected - 2
+ Received + 0
Array [
- "buzz.ts",
- "buzz.ts:2-5",
"buzz.ts:7-10",
]
Call log:
- expect.toHaveText with timeout 5000ms
- waiting for frameLocator('.simple-find-part-wrapper + iframe.webview').last().frameLocator('iframe').getByRole('textbox', { name: 'Chat message' }).last().locator('.context-item-mention-node')
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
- locator resolved to 1 element
367 | await selectLineRangeInEditorTab(page, 7, 10)
368 | await executeCommandInPalette(page, 'Cody: Add Selection to Cody Chat')
> 369 | await expect(chatInputMentions(lastChatInput)).toHaveText(['buzz.ts', 'buzz.ts:2-5', 'buzz.ts:7-10'])
| ^
370 | })
371 |
at D:\a\cody\cody\vscode\test\e2e\chat-atFile.test.ts:369:52
|
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat:
vscode\test\e2e\chat-atFile.test.ts#L1
1) chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ────────────────────────────────────────
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: EBUSY: resource busy or locked, rename 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\temp-videos\063fbdfdb5cd2e062024f4960e8ca9b7.webm' -> 'D:\a\cody\cody\playwright\Add_Selection_to_Cody_Chat\videos\run_0_retry_2_failure.webm'
|
chat-rateLimit.test.ts:80:3 › shows standard rate limit message for non-dotCom users:
vscode\test\e2e\helpers.ts#L332
2) chat-rateLimit.test.ts:80:3 › shows standard rate limit message for non-dotCom users ──────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsce0AICub after 5 attempts: Error: EPERM: operation not permitted, lstat '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsce0AICub\User\globalStorage\sourcegraph.cody-ai\symf\symf-v0.0.16-x86_64-windows'
at helpers.ts:332
330 | } catch (error) {
331 | if (attempts === 1) {
> 332 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
333 | }
334 |
335 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:332:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:213:13)
|
chat-rateLimit.test.ts:80:3 › shows standard rate limit message for non-dotCom users:
vscode\test\e2e\helpers.ts#L332
2) chat-rateLimit.test.ts:80:3 › shows standard rate limit message for non-dotCom users ──────────
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceTqOwL5 after 5 attempts: Error: EPERM: operation not permitted, lstat '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceTqOwL5\User\globalStorage\sourcegraph.cody-ai\symf\symf-v0.0.16-x86_64-windows'
at helpers.ts:332
330 | } catch (error) {
331 | if (attempts === 1) {
> 332 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
333 | }
334 |
335 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:332:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:213:13)
|
command-core.test.ts:29:3 › Generate Unit Test Command (Edit):
vscode\test\e2e\helpers.ts#L332
3) command-core.test.ts:29:3 › Generate Unit Test Command (Edit) ─────────────────────────────────
Error: Failed to rmSync C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceVxutie after 5 attempts: Error: EBUSY: resource busy or locked, unlink '\\?\C:\Users\RUNNER~1\AppData\Local\Temp\cody-vsceVxutie\Crashpad\reports\4f0f5aab-8f58-4df1-9a38-4aada5f64ab6.dmp'
at helpers.ts:332
330 | } catch (error) {
331 | if (attempts === 1) {
> 332 | throw new Error(`Failed to rmSync ${path} after ${maxAttempts} attempts: ${error}`)
| ^
333 | }
334 |
335 | await new Promise(resolve => setTimeout(resolve, 100))
at rmSyncWithRetries (D:\a\cody\cody\vscode\test\e2e\helpers.ts:332:23)
at Object.app (D:\a\cody\cody\vscode\test\e2e\helpers.ts:213:13)
|
command-core.test.ts:29:3 › Generate Unit Test Command (Edit):
vscode\test\e2e\command-core.test.ts#L47
3) command-core.test.ts:29:3 › Generate Unit Test Command (Edit) ─────────────────────────────────
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toBeVisible()
Locator: getByRole('button', { name: 'Accept' })
Expected: visible
Received: <element(s) not found>
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByRole('button', { name: 'Accept' })
45 |
46 | // Code lens should be visible
> 47 | await expect(page.getByRole('button', { name: 'Accept' })).toBeVisible()
| ^
48 | await expect(page.getByRole('button', { name: 'Reject' })).toBeVisible()
49 | })
50 |
at D:\a\cody\cody\vscode\test\e2e\command-core.test.ts:47:64
|
test-unit (ubuntu, 20)
The following actions use a deprecated Node.js version and will be forced to run on node20: google-github-actions/auth@v0, google-github-actions/setup-gcloud@v0. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
|
test-unit (ubuntu, 18)
The following actions use a deprecated Node.js version and will be forced to run on node20: google-github-actions/auth@v0, google-github-actions/setup-gcloud@v0. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
|
test-unit (macos, 20)
The following actions use a deprecated Node.js version and will be forced to run on node20: google-github-actions/auth@v0, google-github-actions/setup-gcloud@v0. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
|
test-unit (windows, 20)
The following actions use a deprecated Node.js version and will be forced to run on node20: google-github-actions/auth@v0, google-github-actions/setup-gcloud@v0. For more info: https://github.blog/changelog/2024-03-07-github-actions-all-actions-will-run-on-node20-instead-of-node16-by-default/
|
Slow Test:
vscode/chat-atFile.test.ts#L1
vscode/chat-atFile.test.ts took 1.1m
|
Slow Test:
vscode/command-custom.test.ts#L1
vscode/command-custom.test.ts took 56.4s
|
Slow Test:
vscode/chat-input.test.ts#L1
vscode/chat-input.test.ts took 46.6s
|
Slow Test:
vscode/initial-context.test.ts#L1
vscode/initial-context.test.ts took 39.5s
|
Slow Test:
vscode/command-core.test.ts#L1
vscode/command-core.test.ts took 35.6s
|
Slow Test:
vscode/command-custom.test.ts#L1
vscode/command-custom.test.ts took 58.4s
|
Slow Test:
vscode/chat-input.test.ts#L1
vscode/chat-input.test.ts took 45.7s
|
Slow Test:
vscode/initial-context.test.ts#L1
vscode/initial-context.test.ts took 34.6s
|
Slow Test:
vscode/code-actions.test.ts#L1
vscode/code-actions.test.ts took 34.3s
|
Slow Test:
vscode/command-core.test.ts#L1
vscode/command-core.test.ts took 33.9s
|
Slow Test:
vscode\chat-input.test.ts#L1
vscode\chat-input.test.ts took 51.5s
|
Slow Test:
vscode\code-actions.test.ts#L1
vscode\code-actions.test.ts took 34.3s
|
Slow Test:
vscode\chat-keyboard-shortcuts.test.ts#L1
vscode\chat-keyboard-shortcuts.test.ts took 26.3s
|
Slow Test:
vscode\attribution.test.ts#L1
vscode\attribution.test.ts took 21.1s
|
Slow Test:
vscode\enterprise-cody-ignore.test.ts#L1
vscode\enterprise-cody-ignore.test.ts took 20.5s
|
🎭 Playwright Run Summary
3 skipped
43 passed (8.2m)
|
🎭 Playwright Run Summary
2 failed
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ─────────────────────────────────────────
enterprise-server-sent-models.test.ts:6:5 › allows multiple enterprise models when server-sent models is enabled
3 skipped
41 passed (9.6m)
|
🎭 Playwright Run Summary
7 flaky
chat-atFile.test.ts:358:3 › Add Selection to Cody Chat ─────────────────────────────────────────
chat-rateLimit.test.ts:80:3 › shows standard rate limit message for non-dotCom users ───────────
command-core.test.ts:29:3 › Generate Unit Test Command (Edit) ──────────────────────────────────
command-custom.test.ts:141:3 › execute custom commands with context defined in cody.json ───────
command-edit.test.ts:95:1 › edit (fixup) input - range selection ───────────────────────────────
initial-context.test.ts:28:18 › initial context - enterprise repo ──────────────────────────────
initial-context.test.ts:83:82 › chat try-again actions ─────────────────────────────────────────
3 skipped
36 passed (13.2m)
|
Artifacts
Produced during runtime
Name | Size | |
---|---|---|
playwright-recordings macos
|
3.56 MB |
|