Skip to content

Playwright: fix (hopefully) flaky shields test #2996

Playwright: fix (hopefully) flaky shields test

Playwright: fix (hopefully) flaky shields test #2996

Triggered via pull request December 4, 2024 21:37
Status Success
Total duration 16m 17s
Artifacts 8

end-to-end-tests.yaml

on: pull_request
Build Element-Web
1m 41s
Build Element-Web
Matrix: playwright
end-to-end-tests
35s
end-to-end-tests
Fit to window
Zoom out
Zoom in

Annotations

10 errors, 30 warnings, and 6 notices
[Chrome] › right-panel/right-panel.spec.ts:41:13 › RightPanel › in rooms › should handle long room address and long room name @screenshot: playwright/e2e/right-panel/right-panel.spec.ts#L51
1) [Chrome] › right-panel/right-panel.spec.ts:41:13 › RightPanel › in rooms › should handle long room address and long room name @screenshot Error: Timed out 5000ms waiting for expect(locator).toHaveClass(expected) Locator: locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#loremIpsumDolorSitAmetConsecteturAdipisicingElitSedDoEiusmodTemporIncididuntUtLaboreEtDoloreMagnaAliqua:localhost') Expected string: "mx_EditableItem_item" Received: <element(s) not found> Call log: - expect.toHaveClass with timeout 5000ms - waiting for locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#loremIpsumDolorSitAmetConsecteturAdipisicingElitSedDoEiusmodTemporIncididuntUtLaboreEtDoloreMagnaAliqua:localhost') 49 | await localAddresses.getByRole("textbox").fill(ROOM_ADDRESS_LONG); 50 | await localAddresses.getByRole("button", { name: "Add" }).click(); > 51 | await expect(localAddresses.getByText(`#${ROOM_ADDRESS_LONG}:localhost`)).toHaveClass( | ^ 52 | "mx_EditableItem_item", 53 | ); 54 | at /home/runner/work/element-web/element-web/playwright/e2e/right-panel/right-panel.spec.ts:51:87
[Chrome] › read-receipts/redactions-in-threads.spec.ts:89:17 › Read receipts › redactions › in threads › Reading an unread thread after a redaction of the latest message makes it read after restart: eval at evaluate (:234:30)#L1
1) [Chrome] › read-receipts/redactions-in-threads.spec.ts:89:17 › Read receipts › redactions › in threads › Reading an unread thread after a redaction of the latest message makes it read after restart Error: jsHandle.evaluate: TypeError: Cannot read properties of undefined (reading 'roomId') at eval (eval at evaluate (:234:30), <anonymous>:1:14) at UtilityScript.evaluate (<anonymous>:236:17) at UtilityScript.<anonymous> (<anonymous>:1:44) at eval (/home/runner/work/element-web/element-web/eval at evaluate (:234:30), <anonymous>:1:14) at UtilityScript.evaluate (/home/runner/work/element-web/element-web/<anonymous>:236:17) at UtilityScript.<anonymous> (/home/runner/work/element-web/element-web/<anonymous>:1:44) at Helpers.sendMessageAsClient (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:338:35) at Helpers.receiveMessages (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:582:9) at /home/runner/work/element-web/element-web/playwright/e2e/read-receipts/redactions-in-threads.spec.ts:97:17
[Chrome] › read-receipts/redactions-in-threads.spec.ts:182:17 › Read receipts › redactions › in threads › Sending and redacting a message after marking the thread as read leaves it read: eval at evaluate (:234:30)#L1
2) [Chrome] › read-receipts/redactions-in-threads.spec.ts:182:17 › Read receipts › redactions › in threads › Sending and redacting a message after marking the thread as read leaves it read Error: jsHandle.evaluate: TypeError: Cannot read properties of undefined (reading 'roomId') at eval (eval at evaluate (:234:30), <anonymous>:1:14) at UtilityScript.evaluate (<anonymous>:236:17) at UtilityScript.<anonymous> (<anonymous>:1:44) at eval (/home/runner/work/element-web/element-web/eval at evaluate (:234:30), <anonymous>:1:14) at UtilityScript.evaluate (/home/runner/work/element-web/element-web/<anonymous>:236:17) at UtilityScript.<anonymous> (/home/runner/work/element-web/element-web/<anonymous>:1:44) at Helpers.sendMessageAsClient (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:338:35) at Helpers.receiveMessages (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:582:9) at /home/runner/work/element-web/element-web/playwright/e2e/read-receipts/redactions-in-threads.spec.ts:190:17
[Chrome] › pinned-messages/pinned-messages.spec.ts:46:9 › Pinned messages › should pin messages and show them in the room info panel: playwright/e2e/pinned-messages/index.ts#L157
1) [Chrome] › pinned-messages/pinned-messages.spec.ts:46:9 › Pinned messages › should pin messages and show them in the room info panel Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected) Locator: getByRole('menuitem', { name: 'Pinned messages' }) Expected string: "Pinned messages3" Received string: "Pinned messages2" Call log: - expect.toHaveText with timeout 5000ms - waiting for getByRole('menuitem', { name: 'Pinned messages' }) 9 × locator resolved to <button role="menuitem" data-kind="primary" class="_item_8j2l6_17 _interactive_8j2l6_35">…</button> - unexpected value "Pinned messages2" at pinned-messages/index.ts:157 155 | */ 156 | async assertPinnedCountInRoomInfo(count: number) { > 157 | await expect(this.page.getByRole("menuitem", { name: "Pinned messages" })).toHaveText( | ^ 158 | `Pinned messages${count}`, 159 | ); 160 | } at Helpers.assertPinnedCountInRoomInfo (/home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/index.ts:157:84) at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:52:20
[Chrome] › pinned-messages/pinned-messages.spec.ts:55:9 › Pinned messages › should pin messages and show them in the pinned message panel: playwright/e2e/pinned-messages/index.ts#L182
2) [Chrome] › pinned-messages/pinned-messages.spec.ts:55:9 › Pinned messages › should pin messages and show them in the pinned message panel Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected) Locator: locator('#mx_RightPanel').getByRole('heading', { name: 'Pinned messages' }) Expected string: "3 Pinned messages" Received string: "2 Pinned messages" Call log: - expect.toHaveText with timeout 5000ms - waiting for locator('#mx_RightPanel').getByRole('heading', { name: 'Pinned messages' }) 9 × locator resolved to <p role="heading" class="_typography_yh5dq_162 _font-body-md-medium_yh5dq_69 mx_BaseCard_header_title_heading">2 Pinned messages</p> - unexpected value "2 Pinned messages" at pinned-messages/index.ts:182 180 | async assertPinnedMessagesList(messages: string[]) { 181 | const rightPanel = this.getRightPanel(); > 182 | await expect(rightPanel.getByRole("heading", { name: "Pinned messages" })).toHaveText( | ^ 183 | `${messages.length} Pinned messages`, 184 | ); 185 | at Helpers.assertPinnedMessagesList (/home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/index.ts:182:84) at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:63:20
[Chrome] › pinned-messages/pinned-messages.spec.ts:66:9 › Pinned messages › should unpin one message: playwright/e2e/pinned-messages/index.ts#L182
3) [Chrome] › pinned-messages/pinned-messages.spec.ts:66:9 › Pinned messages › should unpin one message Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected) Locator: locator('#mx_RightPanel').getByRole('heading', { name: 'Pinned messages' }) Expected string: "2 Pinned messages" Received: <element(s) not found> Call log: - expect.toHaveText with timeout 5000ms - waiting for locator('#mx_RightPanel').getByRole('heading', { name: 'Pinned messages' }) at pinned-messages/index.ts:182 180 | async assertPinnedMessagesList(messages: string[]) { 181 | const rightPanel = this.getRightPanel(); > 182 | await expect(rightPanel.getByRole("heading", { name: "Pinned messages" })).toHaveText( | ^ 183 | `${messages.length} Pinned messages`, 184 | ); 185 | at Helpers.assertPinnedMessagesList (/home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/index.ts:182:84) at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:74:20
[Chrome] › pinned-messages/pinned-messages.spec.ts:112:9 › Pinned messages › should display 2 messages in the banner @screenshot: playwright/e2e/pinned-messages/pinned-messages.spec.ts#L118
4) [Chrome] › pinned-messages/pinned-messages.spec.ts:112:9 › Pinned messages › should display 2 messages in the banner @screenshot Error: expect(locator).toHaveScreenshot(expected) 857 pixels (ratio 0.02 of all image pixels) are different. Expected: /home/runner/work/element-web/element-web/playwright/snapshots/pinned-messages/pinned-messages.spec.ts/pinned-message-banner-2-Msg2-linux.png Received: /home/runner/work/element-web/element-web/playwright/test-results/pinned-messages-pinned-mes-4b2e4-ay-2-messages-in-the-banner-Chrome/pinned-message-banner-2-Msg2-actual.png Diff: /home/runner/work/element-web/element-web/playwright/test-results/pinned-messages-pinned-mes-4b2e4-ay-2-messages-in-the-banner-Chrome/pinned-message-banner-2-Msg2-diff.png Call log: - expect.toHaveScreenshot(pinned-message-banner-2-Msg2.png) with timeout 5000ms - verifying given screenshot expectation - waiting for getByTestId('pinned-message-banner') - locator resolved to <div data-single-message="true" class="mx_PinnedMessageBanner" data-testid="pinned-message-banner" aria-label="This room has pinned messages. Click to view them.">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 857 pixels (ratio 0.02 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for getByTestId('pinned-message-banner') - locator resolved to <div data-single-message="true" class="mx_PinnedMessageBanner" data-testid="pinned-message-banner" aria-label="This room has pinned messages. Click to view them.">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 857 pixels (ratio 0.02 of all image pixels) are different. 116 | 117 | await util.assertMessageInBanner("Msg2"); > 118 | await expect(util.getBanner()).toMatchScreenshot("pinned-message-banner-2-Msg2.png"); | ^ 119 | 120 | await util.getBanner().click(); 121 | await util.assertMessageInBanner("Msg1"); at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:118:40
[Chrome] › pinned-messages/pinned-messages.spec.ts:141:9 › Pinned messages › should open the pinned messages list from the banner: playwright/e2e/pinned-messages/pinned-messages.spec.ts#L1
5) [Chrome] › pinned-messages/pinned-messages.spec.ts:141:9 › Pinned messages › should open the pinned messages list from the banner Test timeout of 30000ms exceeded.
[Chrome] › pinned-messages/pinned-messages.spec.ts:141:9 › Pinned messages › should open the pinned messages list from the banner: playwright/e2e/pinned-messages/pinned-messages.spec.ts#L146
5) [Chrome] › pinned-messages/pinned-messages.spec.ts:141:9 › Pinned messages › should open the pinned messages list from the banner Error: locator.click: Test timeout of 30000ms exceeded. Call log: - waiting for getByRole('button', { name: 'View all' }) 144 | await util.pinMessages(["Msg1", "Msg2"]); 145 | > 146 | await util.getViewAllButton().click(); | ^ 147 | await util.assertPinnedMessagesList(["Msg1", "Msg2"]); 148 | 149 | await expect(util.getCloseListButton()).toBeVisible(); at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:146:39
[Chrome] › pinned-messages/pinned-messages.spec.ts:152:9 › Pinned messages › banner should listen to pinned message list: playwright/e2e/pinned-messages/pinned-messages.spec.ts#L157
6) [Chrome] › pinned-messages/pinned-messages.spec.ts:152:9 › Pinned messages › banner should listen to pinned message list Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: getByRole('button', { name: 'View all' }) Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByRole('button', { name: 'View all' }) 155 | await util.pinMessages(["Msg1", "Msg2"]); 156 | > 157 | await expect(util.getViewAllButton()).toBeVisible(); | ^ 158 | 159 | await util.openRoomInfo(); 160 | await util.openPinnedMessagesList(); at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:157:47
Slow Test: [Chrome] › spaces/spaces.spec.ts#L1
[Chrome] › spaces/spaces.spec.ts took 1.1m
Slow Test: [Chrome] › settings/account-user-settings-tab.spec.ts#L1
[Chrome] › settings/account-user-settings-tab.spec.ts took 46.6s
Slow Test: [Chrome] › right-panel/file-panel.spec.ts#L1
[Chrome] › right-panel/file-panel.spec.ts took 40.4s
Slow Test: [Chrome] › read-receipts/room-list-order.spec.ts#L1
[Chrome] › read-receipts/room-list-order.spec.ts took 33.5s
Slow Test: [Chrome] › settings/appearance-user-settings-tab/theme-choice-panel.spec.ts#L1
[Chrome] › settings/appearance-user-settings-tab/theme-choice-panel.spec.ts took 33.3s
Slow Test: [Chrome] › read-receipts/high-level.spec.ts#L1
[Chrome] › read-receipts/high-level.spec.ts took 3.3m
Slow Test: [Chrome] › read-receipts/new-messages-in-threads.spec.ts#L1
[Chrome] › read-receipts/new-messages-in-threads.spec.ts took 1.8m
Slow Test: [Chrome] › read-receipts/editing-messages-in-threads.spec.ts#L1
[Chrome] › read-receipts/editing-messages-in-threads.spec.ts took 1.3m
Slow Test: [Chrome] › read-receipts/new-messages-main-timeline.spec.ts#L1
[Chrome] › read-receipts/new-messages-main-timeline.spec.ts took 1.2m
Slow Test: [Chrome] › read-receipts/editing-messages-main-timeline.spec.ts#L1
[Chrome] › read-receipts/editing-messages-main-timeline.spec.ts took 1.2m
Slow Test: [Chrome] › timeline/timeline.spec.ts#L1
[Chrome] › timeline/timeline.spec.ts took 2.8m
Slow Test: [Chrome] › spotlight/spotlight.spec.ts#L1
[Chrome] › spotlight/spotlight.spec.ts took 2.3m
Slow Test: [Chrome] › spaces/threads-activity-centre/threadsActivityCentre.spec.ts#L1
[Chrome] › spaces/threads-activity-centre/threadsActivityCentre.spec.ts took 1.8m
Slow Test: [Chrome] › user-onboarding/user-onboarding-new.spec.ts#L1
[Chrome] › user-onboarding/user-onboarding-new.spec.ts took 35.9s
Slow Test: [Chrome] › threads/threads.spec.ts#L1
[Chrome] › threads/threads.spec.ts took 35.2s
Slow Test: [Chrome] › read-receipts/redactions-main-timeline.spec.ts#L1
[Chrome] › read-receipts/redactions-main-timeline.spec.ts took 2.1m
Slow Test: [Chrome] › read-receipts/read-receipts.spec.ts#L1
[Chrome] › read-receipts/read-receipts.spec.ts took 1.1m
Slow Test: [Chrome] › read-receipts/reactions-in-threads.spec.ts#L1
[Chrome] › read-receipts/reactions-in-threads.spec.ts took 59.9s
Slow Test: [Chrome] › read-receipts/redactions-thread-roots.spec.ts#L1
[Chrome] › read-receipts/redactions-thread-roots.spec.ts took 55.9s
Slow Test: [Chrome] › read-receipts/new-messages-thread-roots.spec.ts#L1
[Chrome] › read-receipts/new-messages-thread-roots.spec.ts took 53.5s
Slow Test: [Chrome] › composer/RTE.spec.ts#L1
[Chrome] › composer/RTE.spec.ts took 1.3m
Slow Test: [Chrome] › audio-player/audio-player.spec.ts#L1
[Chrome] › audio-player/audio-player.spec.ts took 1.2m
Slow Test: [Chrome] › crypto/event-shields.spec.ts#L1
[Chrome] › crypto/event-shields.spec.ts took 1.2m
Slow Test: [Chrome] › crypto/crypto.spec.ts#L1
[Chrome] › crypto/crypto.spec.ts took 1.1m
Slow Test: [Chrome] › crypto/decryption-failure-messages.spec.ts#L1
[Chrome] › crypto/decryption-failure-messages.spec.ts took 54.2s
Slow Test: [Chrome] › messages/messages.spec.ts#L1
[Chrome] › messages/messages.spec.ts took 2.4m
Slow Test: [Chrome] › login/login.spec.ts#L1
[Chrome] › login/login.spec.ts took 1.1m
Slow Test: [Chrome] › knock/knock-into-room.spec.ts#L1
[Chrome] › knock/knock-into-room.spec.ts took 40.4s
Slow Test: [Chrome] › integration-manager/read_events.spec.ts#L1
[Chrome] › integration-manager/read_events.spec.ts took 37.1s
Slow Test: [Chrome] › polls/polls.spec.ts#L1
[Chrome] › polls/polls.spec.ts took 36.7s
🎭 Playwright Run Summary
1 flaky [Chrome] › right-panel/right-panel.spec.ts:41:13 › RightPanel › in rooms › should handle long room address and long room name @screenshot 9 skipped 67 passed (9.7m)
🎭 Playwright Run Summary
32 skipped 50 passed (9.9m)
🎭 Playwright Run Summary
3 skipped 69 passed (10.2m)
🎭 Playwright Run Summary
2 flaky [Chrome] › read-receipts/redactions-in-threads.spec.ts:89:17 › Read receipts › redactions › in threads › Reading an unread thread after a redaction of the latest message makes it read after restart [Chrome] › read-receipts/redactions-in-threads.spec.ts:182:17 › Read receipts › redactions › in threads › Sending and redacting a message after marking the thread as read leaves it read 15 skipped 58 passed (10.3m)
🎭 Playwright Run Summary
79 passed (11.9m)
🎭 Playwright Run Summary
6 flaky [Chrome] › pinned-messages/pinned-messages.spec.ts:46:9 › Pinned messages › should pin messages and show them in the room info panel [Chrome] › pinned-messages/pinned-messages.spec.ts:55:9 › Pinned messages › should pin messages and show them in the pinned message panel [Chrome] › pinned-messages/pinned-messages.spec.ts:66:9 › Pinned messages › should unpin one message [Chrome] › pinned-messages/pinned-messages.spec.ts:112:9 › Pinned messages › should display 2 messages in the banner @screenshot [Chrome] › pinned-messages/pinned-messages.spec.ts:141:9 › Pinned messages › should open the pinned messages list from the banner [Chrome] › pinned-messages/pinned-messages.spec.ts:152:9 › Pinned messages › banner should listen to pinned message list 1 skipped 72 passed (13.1m)

Artifacts

Produced during runtime
Name Size
all-blob-reports-1 Expired
674 KB
all-blob-reports-2 Expired
12.4 MB
all-blob-reports-3 Expired
308 KB
all-blob-reports-4 Expired
4.01 MB
all-blob-reports-5 Expired
3.9 MB
all-blob-reports-6 Expired
773 KB
html-report Expired
21.4 MB
webapp Expired
34.2 MB