Fix flaky playwright tests #4100
Annotations
13 errors, 26 warnings, and 6 notices
[Chrome] › crypto/invisible-crypto.spec.ts:20:9 › Invisible cryptography › Messages fail to decrypt when sender is previously verified:
playwright/e2e/crypto/invisible-crypto.spec.ts#L1
1) [Chrome] › crypto/invisible-crypto.spec.ts:20:9 › Invisible cryptography › Messages fail to decrypt when sender is previously verified
Test timeout of 30000ms exceeded.
|
[Chrome] › crypto/invisible-crypto.spec.ts:20:9 › Invisible cryptography › Messages fail to decrypt when sender is previously verified:
playwright/e2e/crypto/utils.ts#L346
1) [Chrome] › crypto/invisible-crypto.spec.ts:20:9 › Invisible cryptography › Messages fail to decrypt when sender is previously verified
Error: locator.click: Test timeout of 30000ms exceeded.
Call log:
- waiting for locator('.mx_RightPanel').getByRole('button', { name: 'Got it' })
- locator resolved to <div tabindex="0" role="button" class="mx_AccessibleButton mx_UserInfo_wideButton mx_AccessibleButton_hasKind mx_AccessibleButton_kind_primary">Got it</div>
- attempting click action
- waiting for element to be visible, enabled and stable
- element is visible, enabled and stable
- scrolling into view if needed
- done scrolling
at crypto/utils.ts:346
344 | await roomInfo.getByRole("button", { name: "They match" }).click();
345 | await expect(roomInfo.getByText("You've successfully verified Bob!")).toBeVisible();
> 346 | await roomInfo.getByRole("button", { name: "Got it" }).click();
| ^
347 | };
348 |
349 | /**
at verify (/home/runner/work/element-web/element-web/playwright/e2e/crypto/utils.ts:346:60)
at /home/runner/work/element-web/element-web/playwright/e2e/crypto/invisible-crypto.spec.ts:45:9
|
[Chrome] › knock/create-knock-room.spec.ts:61:9 › Create Knock Room › should create a public knock room:
playwright/e2e/knock/create-knock-room.spec.ts#L82
2) [Chrome] › knock/create-knock-room.spec.ts:61:9 › Create Knock Room › should create a public knock room
Error: Timed out 5000ms waiting for expect(locator).toContainText(expected)
Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option').first()
Expected string: "Cybersecurity"
Received string: "TTestRoom!OvqIDFrsWXJVfqtlxC:localhost2 Members↵Join"
Call log:
- expect.toContainText with timeout 5000ms
- waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option').first()
6 × locator resolved to <li tabindex="-1" role="option" aria-selected="true" id="mx_SpotlightDialog_button_result_!OvqIDFrsWXJVfqtlxC:localhost" aria-labelledby="mx_SpotlightDialog_button_result_!OvqIDFrsWXJVfqtlxC:localhost_name" aria-details="mx_SpotlightDialog_button_result_!OvqIDFrsWXJVfqtlxC:localhost_details" aria-describedby="mx_SpotlightDialog_button_result_!OvqIDFrsWXJVfqtlxC:localhost_alias" class="mx_AccessibleButton mx_SpotlightDialog_result_multiline mx_SpotlightDialog_option">…</li>
- unexpected value "TTestRoom!OvqIDFrsWXJVfqtlxC:localhost2 Members↵Join"
80 | const spotlightDialog = await app.openSpotlight();
81 | await spotlightDialog.filter(Filter.PublicRooms);
> 82 | await expect(spotlightDialog.results.nth(0)).toContainText("Cybersecurity");
| ^
83 | });
84 | });
85 |
at /home/runner/work/element-web/element-web/playwright/e2e/knock/create-knock-room.spec.ts:82:54
|
[Chrome] › audio-player/audio-player.spec.ts:243:9 › Audio player › should support creating a reply chain with multiple audio files @no-firefox @no-webkit @screenshot:
playwright/e2e/audio-player/audio-player.spec.ts#L1
1) [Chrome] › audio-player/audio-player.spec.ts:243:9 › Audio player › should support creating a reply chain with multiple audio files @no-firefox @no-webkit @screenshot
Test timeout of 30000ms exceeded.
|
[Chrome] › audio-player/audio-player.spec.ts:243:9 › Audio player › should support creating a reply chain with multiple audio files @no-firefox @no-webkit @screenshot:
playwright/e2e/audio-player/audio-player.spec.ts#L257
1) [Chrome] › audio-player/audio-player.spec.ts:243:9 › Audio player › should support creating a reply chain with multiple audio files @no-firefox @no-webkit @screenshot
Error: locator.click: Test timeout of 30000ms exceeded.
Call log:
- waiting for locator('.mx_EventTile_last').getByRole('button', { name: 'Reply', exact: true })
255 | const clickButtonReply = async () => {
256 | await tile.hover();
> 257 | await tile.getByRole("button", { name: "Reply", exact: true }).click();
| ^
258 | };
259 |
260 | await uploadFile(page, "playwright/sample-files/upload-first.ogg");
at clickButtonReply (/home/runner/work/element-web/element-web/playwright/e2e/audio-player/audio-player.spec.ts:257:80)
at /home/runner/work/element-web/element-web/playwright/e2e/audio-player/audio-player.spec.ts:273:13
|
[Chrome] › spotlight/spotlight.spec.ts:132:9 › Spotlight › should find known public rooms:
playwright/e2e/spotlight/spotlight.spec.ts#L138
1) [Chrome] › spotlight/spotlight.spec.ts:132:9 › Spotlight › should find known public rooms ─────
Error: Timed out 5000ms waiting for expect(locator).toHaveCount(expected)
Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option')
Expected: 1
Received: 3
Call log:
- expect.toHaveCount with timeout 5000ms
- waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option')
2 × locator resolved to 0 elements
- unexpected value "0"
7 × locator resolved to 3 elements
- unexpected value "3"
136 | await spotlight.search(room1Name);
137 | const resultLocator = spotlight.results;
> 138 | await expect(resultLocator).toHaveCount(1);
| ^
139 | await expect(resultLocator.first()).toContainText(room1Name);
140 | await expect(resultLocator.first()).toContainText("View");
141 | await resultLocator.first().click();
at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:138:37
|
[Chrome] › spotlight/spotlight.spec.ts:146:9 › Spotlight › should find unknown public rooms:
playwright/e2e/spotlight/spotlight.spec.ts#L152
2) [Chrome] › spotlight/spotlight.spec.ts:146:9 › Spotlight › should find unknown public rooms ───
Error: Timed out 5000ms waiting for expect(locator).toHaveCount(expected)
Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option')
Expected: 1
Received: 2
Call log:
- expect.toHaveCount with timeout 5000ms
- waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option')
2 × locator resolved to 0 elements
- unexpected value "0"
7 × locator resolved to 2 elements
- unexpected value "2"
150 | await spotlight.search(room2Name);
151 | const resultLocator = spotlight.results;
> 152 | await expect(resultLocator).toHaveCount(1);
| ^
153 | await expect(resultLocator.first()).toContainText(room2Name);
154 | await expect(resultLocator.first()).toContainText("Join");
155 | await resultLocator.first().click();
at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:152:37
|
[Chrome] › spotlight/spotlight.spec.ts:161:9 › Spotlight › should find unknown public world readable rooms:
playwright/e2e/spotlight/spotlight.spec.ts#L167
3) [Chrome] › spotlight/spotlight.spec.ts:161:9 › Spotlight › should find unknown public world readable rooms
Error: Timed out 5000ms waiting for expect(locator).toHaveCount(expected)
Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option')
Expected: 1
Received: 2
Call log:
- expect.toHaveCount with timeout 5000ms
- waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option')
2 × locator resolved to 0 elements
- unexpected value "0"
7 × locator resolved to 2 elements
- unexpected value "2"
165 | await spotlight.search(room3Name);
166 | const resultLocator = spotlight.results;
> 167 | await expect(resultLocator).toHaveCount(1);
| ^
168 | await expect(resultLocator.first()).toContainText(room3Name);
169 | await expect(resultLocator.first()).toContainText("View");
170 | await resultLocator.first().click();
at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:167:37
|
[Chrome] › spotlight/spotlight.spec.ts:220:9 › Spotlight › should find unknown people:
playwright/e2e/spotlight/spotlight.spec.ts#L226
4) [Chrome] › spotlight/spotlight.spec.ts:220:9 › Spotlight › should find unknown people ─────────
Error: Timed out 5000ms waiting for expect(locator).toHaveCount(expected)
Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option')
Expected: 1
Received: 3
Call log:
- expect.toHaveCount with timeout 5000ms
- waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option')
2 × locator resolved to 0 elements
- unexpected value "0"
7 × locator resolved to 3 elements
- unexpected value "3"
224 | await spotlight.search(bot2Name);
225 | const resultLocator = spotlight.results;
> 226 | await expect(resultLocator).toHaveCount(1);
| ^
227 | await expect(resultLocator.first()).toContainText(bot2Name);
228 | await resultLocator.first().click();
229 | await expect(roomHeaderName(page)).toHaveText(bot2Name);
at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:226:37
|
[Chrome] › spotlight/spotlight.spec.ts:321:9 › Spotlight › should allow opening group chat dialog:
playwright/e2e/spotlight/spotlight.spec.ts#L329
5) [Chrome] › spotlight/spotlight.spec.ts:321:9 › Spotlight › should allow opening group chat dialog
Error: Timed out 5000ms waiting for expect(locator).toHaveCount(expected)
Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option')
Expected: 1
Received: 4
Call log:
- expect.toHaveCount with timeout 5000ms
- waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option')
9 × locator resolved to 4 elements
- unexpected value "4"
327 |
328 | const resultLocator = spotlight.results;
> 329 | await expect(resultLocator).toHaveCount(1);
| ^
330 | await expect(resultLocator.first()).toContainText(bot2Name);
331 |
332 | await expect(spotlight.dialog.locator(".mx_SpotlightDialog_startGroupChat")).toContainText(
at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:329:37
|
[Chrome] › spotlight/spotlight.spec.ts:339:9 › Spotlight › should close spotlight after starting a DM:
playwright/e2e/spotlight/spotlight.spec.ts#L27
6) [Chrome] › spotlight/spotlight.spec.ts:339:9 › Spotlight › should close spotlight after starting a DM
Error: Timed out 5000ms waiting for expect(locator).toHaveCount(expected)
Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option')
Expected: 1
Received: 2
Call log:
- expect.toHaveCount with timeout 5000ms
- waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option')
9 × locator resolved to 2 elements
- unexpected value "2"
25 | await expect(spotlight.dialog.locator(".mx_Spinner")).not.toBeAttached();
26 | const result = spotlight.results;
> 27 | await expect(result).toHaveCount(1);
| ^
28 | await expect(result.first()).toContainText(name);
29 | await result.first().click();
30 |
at startDM (/home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:27:26)
at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:340:9
|
[Chrome] › spotlight/spotlight.spec.ts:344:9 › Spotlight › should show the same user only once:
playwright/e2e/spotlight/spotlight.spec.ts#L27
7) [Chrome] › spotlight/spotlight.spec.ts:344:9 › Spotlight › should show the same user only once
Error: Timed out 5000ms waiting for expect(locator).toHaveCount(expected)
Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option')
Expected: 1
Received: 2
Call log:
- expect.toHaveCount with timeout 5000ms
- waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option')
9 × locator resolved to 2 elements
- unexpected value "2"
25 | await expect(spotlight.dialog.locator(".mx_Spinner")).not.toBeAttached();
26 | const result = spotlight.results;
> 27 | await expect(result).toHaveCount(1);
| ^
28 | await expect(result.first()).toContainText(name);
29 | await result.first().click();
30 |
at startDM (/home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:27:26)
at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:345:9
|
[Chrome] › spotlight/spotlight.spec.ts:357:9 › Spotlight › should be able to navigate results via keyboard:
playwright/e2e/spotlight/spotlight.spec.ts#L364
8) [Chrome] › spotlight/spotlight.spec.ts:357:9 › Spotlight › should be able to navigate results via keyboard
Error: Timed out 5000ms waiting for expect(locator).toHaveCount(expected)
Locator: locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option')
Expected: 2
Received: 4
Call log:
- expect.toHaveCount with timeout 5000ms
- waiting for locator('[role=dialog][aria-label="Search Dialog"]').locator('.mx_SpotlightDialog_section.mx_SpotlightDialog_results .mx_SpotlightDialog_option')
2 × locator resolved to 1 element
- unexpected value "1"
7 × locator resolved to 4 elements
- unexpected value "4"
362 |
363 | let resultLocator = spotlight.results;
> 364 | await expect(resultLocator).toHaveCount(2);
| ^
365 | await expect(resultLocator.first()).toHaveAttribute("aria-selected", "true");
366 | await expect(resultLocator.last()).toHaveAttribute("aria-selected", "false");
367 |
at /home/runner/work/element-web/element-web/playwright/e2e/spotlight/spotlight.spec.ts:364:37
|
Slow Test:
[Chrome] › pinned-messages/pinned-messages.spec.ts#L1
[Chrome] › pinned-messages/pinned-messages.spec.ts took 58.9s
|
Slow Test:
[Chrome] › right-panel/right-panel.spec.ts#L1
[Chrome] › right-panel/right-panel.spec.ts took 24.3s
|
Slow Test:
[Chrome] › one-to-one-chat/one-to-one-chat.spec.ts#L1
[Chrome] › one-to-one-chat/one-to-one-chat.spec.ts took 23.9s
|
Slow Test:
[Chrome] › right-panel/file-panel.spec.ts#L1
[Chrome] › right-panel/file-panel.spec.ts took 19.3s
|
Slow Test:
[Chrome] › polls/polls.spec.ts#L1
[Chrome] › polls/polls.spec.ts took 18.8s
|
Slow Test:
[Chrome] › sliding-sync/sliding-sync.spec.ts#L1
[Chrome] › sliding-sync/sliding-sync.spec.ts took 54.3s
|
Slow Test:
[Chrome] › room/room-header.spec.ts#L1
[Chrome] › room/room-header.spec.ts took 32.9s
|
Slow Test:
[Chrome] › settings/account-user-settings-tab.spec.ts#L1
[Chrome] › settings/account-user-settings-tab.spec.ts took 17.1s
|
Slow Test:
[Chrome] › login/login-consent.spec.ts#L1
[Chrome] › login/login-consent.spec.ts took 1.1m
|
Slow Test:
[Chrome] › crypto/backups-mas.spec.ts#L1
[Chrome] › crypto/backups-mas.spec.ts took 26.8s
|
Slow Test:
[Chrome] › app-loading/guest-registration.spec.ts#L1
[Chrome] › app-loading/guest-registration.spec.ts took 25.5s
|
Slow Test:
[Chrome] › user-onboarding/user-onboarding-new.spec.ts#L1
[Chrome] › user-onboarding/user-onboarding-new.spec.ts took 24.1s
|
Slow Test:
[Chrome] › register/register.spec.ts#L1
[Chrome] › register/register.spec.ts took 16.6s
|
Slow Test:
[Chrome] › messages/messages.spec.ts#L1
[Chrome] › messages/messages.spec.ts took 58.3s
|
Slow Test:
[Chrome] › integration-manager/kick.spec.ts#L1
[Chrome] › integration-manager/kick.spec.ts took 23.8s
|
Slow Test:
[Chrome] › knock/knock-into-room.spec.ts#L1
[Chrome] › knock/knock-into-room.spec.ts took 18.9s
|
Slow Test:
[Chrome] › editing/editing.spec.ts#L1
[Chrome] › editing/editing.spec.ts took 15.8s
|
Slow Test:
[Chrome] › crypto/event-shields.spec.ts#L1
[Chrome] › crypto/event-shields.spec.ts took 54.1s
|
Slow Test:
[Chrome] › composer/RTE.spec.ts#L1
[Chrome] › composer/RTE.spec.ts took 36.1s
|
Slow Test:
[Chrome] › crypto/crypto.spec.ts#L1
[Chrome] › crypto/crypto.spec.ts took 34.1s
|
Slow Test:
[Chrome] › accessibility/keyboard-navigation.spec.ts#L1
[Chrome] › accessibility/keyboard-navigation.spec.ts took 32.2s
|
Slow Test:
[Chrome] › crypto/decryption-failure-messages.spec.ts#L1
[Chrome] › crypto/decryption-failure-messages.spec.ts took 28.2s
|
Slow Test:
[Chrome] › timeline/timeline.spec.ts#L1
[Chrome] › timeline/timeline.spec.ts took 1.3m
|
Slow Test:
[Chrome] › spaces/spaces.spec.ts#L1
[Chrome] › spaces/spaces.spec.ts took 50.4s
|
Slow Test:
[Chrome] › spaces/threads-activity-centre/threadsActivityCentre.spec.ts#L1
[Chrome] › spaces/threads-activity-centre/threadsActivityCentre.spec.ts took 48.0s
|
Slow Test:
[Chrome] › threads/threads.spec.ts#L1
[Chrome] › threads/threads.spec.ts took 26.6s
|
🎭 Playwright Run Summary
1 skipped
35 passed (2.9m)
|
🎭 Playwright Run Summary
1 skipped
49 passed (3.4m)
|
🎭 Playwright Run Summary
44 passed (5.3m)
|
🎭 Playwright Run Summary
2 flaky
[Chrome] › crypto/invisible-crypto.spec.ts:20:9 › Invisible cryptography › Messages fail to decrypt when sender is previously verified
[Chrome] › knock/create-knock-room.spec.ts:61:9 › Create Knock Room › should create a public knock room
1 skipped
63 passed (5.2m)
|
🎭 Playwright Run Summary
1 flaky
[Chrome] › audio-player/audio-player.spec.ts:243:9 › Audio player › should support creating a reply chain with multiple audio files @no-firefox @no-webkit @screenshot
55 passed (5.6m)
|
🎭 Playwright Run Summary
8 flaky
[Chrome] › spotlight/spotlight.spec.ts:132:9 › Spotlight › should find known public rooms ──────
[Chrome] › spotlight/spotlight.spec.ts:146:9 › Spotlight › should find unknown public rooms ────
[Chrome] › spotlight/spotlight.spec.ts:161:9 › Spotlight › should find unknown public world readable rooms
[Chrome] › spotlight/spotlight.spec.ts:220:9 › Spotlight › should find unknown people ──────────
[Chrome] › spotlight/spotlight.spec.ts:321:9 › Spotlight › should allow opening group chat dialog
[Chrome] › spotlight/spotlight.spec.ts:339:9 › Spotlight › should close spotlight after starting a DM
[Chrome] › spotlight/spotlight.spec.ts:344:9 › Spotlight › should show the same user only once ─
[Chrome] › spotlight/spotlight.spec.ts:357:9 › Spotlight › should be able to navigate results via keyboard
1 skipped
51 passed (7.3m)
|
Artifacts
Produced during runtime
Name | Size | |
---|---|---|
all-blob-reports-Chrome-1
Expired
|
5.28 MB |
|
all-blob-reports-Chrome-2
Expired
|
12.3 MB |
|
all-blob-reports-Chrome-3
Expired
|
122 KB |
|
all-blob-reports-Chrome-4
Expired
|
139 KB |
|
all-blob-reports-Chrome-5
Expired
|
36 MB |
|
all-blob-reports-Chrome-6
Expired
|
464 KB |
|
html-report
Expired
|
47.7 MB |
|
webapp
Expired
|
28.7 MB |
|