Playwright: fix (hopefully) flaky shields test #2996
Annotations
10 errors, 30 warnings, and 6 notices
[Chrome] › right-panel/right-panel.spec.ts:41:13 › RightPanel › in rooms › should handle long room address and long room name @screenshot:
playwright/e2e/right-panel/right-panel.spec.ts#L51
1) [Chrome] › right-panel/right-panel.spec.ts:41:13 › RightPanel › in rooms › should handle long room address and long room name @screenshot
Error: Timed out 5000ms waiting for expect(locator).toHaveClass(expected)
Locator: locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#loremIpsumDolorSitAmetConsecteturAdipisicingElitSedDoEiusmodTemporIncididuntUtLaboreEtDoloreMagnaAliqua:localhost')
Expected string: "mx_EditableItem_item"
Received: <element(s) not found>
Call log:
- expect.toHaveClass with timeout 5000ms
- waiting for locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#loremIpsumDolorSitAmetConsecteturAdipisicingElitSedDoEiusmodTemporIncididuntUtLaboreEtDoloreMagnaAliqua:localhost')
49 | await localAddresses.getByRole("textbox").fill(ROOM_ADDRESS_LONG);
50 | await localAddresses.getByRole("button", { name: "Add" }).click();
> 51 | await expect(localAddresses.getByText(`#${ROOM_ADDRESS_LONG}:localhost`)).toHaveClass(
| ^
52 | "mx_EditableItem_item",
53 | );
54 |
at /home/runner/work/element-web/element-web/playwright/e2e/right-panel/right-panel.spec.ts:51:87
|
[Chrome] › read-receipts/redactions-in-threads.spec.ts:89:17 › Read receipts › redactions › in threads › Reading an unread thread after a redaction of the latest message makes it read after restart:
eval at evaluate (:234:30)#L1
1) [Chrome] › read-receipts/redactions-in-threads.spec.ts:89:17 › Read receipts › redactions › in threads › Reading an unread thread after a redaction of the latest message makes it read after restart
Error: jsHandle.evaluate: TypeError: Cannot read properties of undefined (reading 'roomId')
at eval (eval at evaluate (:234:30), <anonymous>:1:14)
at UtilityScript.evaluate (<anonymous>:236:17)
at UtilityScript.<anonymous> (<anonymous>:1:44)
at eval (/home/runner/work/element-web/element-web/eval at evaluate (:234:30), <anonymous>:1:14)
at UtilityScript.evaluate (/home/runner/work/element-web/element-web/<anonymous>:236:17)
at UtilityScript.<anonymous> (/home/runner/work/element-web/element-web/<anonymous>:1:44)
at Helpers.sendMessageAsClient (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:338:35)
at Helpers.receiveMessages (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:582:9)
at /home/runner/work/element-web/element-web/playwright/e2e/read-receipts/redactions-in-threads.spec.ts:97:17
|
[Chrome] › read-receipts/redactions-in-threads.spec.ts:182:17 › Read receipts › redactions › in threads › Sending and redacting a message after marking the thread as read leaves it read:
eval at evaluate (:234:30)#L1
2) [Chrome] › read-receipts/redactions-in-threads.spec.ts:182:17 › Read receipts › redactions › in threads › Sending and redacting a message after marking the thread as read leaves it read
Error: jsHandle.evaluate: TypeError: Cannot read properties of undefined (reading 'roomId')
at eval (eval at evaluate (:234:30), <anonymous>:1:14)
at UtilityScript.evaluate (<anonymous>:236:17)
at UtilityScript.<anonymous> (<anonymous>:1:44)
at eval (/home/runner/work/element-web/element-web/eval at evaluate (:234:30), <anonymous>:1:14)
at UtilityScript.evaluate (/home/runner/work/element-web/element-web/<anonymous>:236:17)
at UtilityScript.<anonymous> (/home/runner/work/element-web/element-web/<anonymous>:1:44)
at Helpers.sendMessageAsClient (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:338:35)
at Helpers.receiveMessages (/home/runner/work/element-web/element-web/playwright/e2e/read-receipts/index.ts:582:9)
at /home/runner/work/element-web/element-web/playwright/e2e/read-receipts/redactions-in-threads.spec.ts:190:17
|
[Chrome] › pinned-messages/pinned-messages.spec.ts:46:9 › Pinned messages › should pin messages and show them in the room info panel:
playwright/e2e/pinned-messages/index.ts#L157
1) [Chrome] › pinned-messages/pinned-messages.spec.ts:46:9 › Pinned messages › should pin messages and show them in the room info panel
Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected)
Locator: getByRole('menuitem', { name: 'Pinned messages' })
Expected string: "Pinned messages3"
Received string: "Pinned messages2"
Call log:
- expect.toHaveText with timeout 5000ms
- waiting for getByRole('menuitem', { name: 'Pinned messages' })
9 × locator resolved to <button role="menuitem" data-kind="primary" class="_item_8j2l6_17 _interactive_8j2l6_35">…</button>
- unexpected value "Pinned messages2"
at pinned-messages/index.ts:157
155 | */
156 | async assertPinnedCountInRoomInfo(count: number) {
> 157 | await expect(this.page.getByRole("menuitem", { name: "Pinned messages" })).toHaveText(
| ^
158 | `Pinned messages${count}`,
159 | );
160 | }
at Helpers.assertPinnedCountInRoomInfo (/home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/index.ts:157:84)
at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:52:20
|
[Chrome] › pinned-messages/pinned-messages.spec.ts:55:9 › Pinned messages › should pin messages and show them in the pinned message panel:
playwright/e2e/pinned-messages/index.ts#L182
2) [Chrome] › pinned-messages/pinned-messages.spec.ts:55:9 › Pinned messages › should pin messages and show them in the pinned message panel
Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected)
Locator: locator('#mx_RightPanel').getByRole('heading', { name: 'Pinned messages' })
Expected string: "3 Pinned messages"
Received string: "2 Pinned messages"
Call log:
- expect.toHaveText with timeout 5000ms
- waiting for locator('#mx_RightPanel').getByRole('heading', { name: 'Pinned messages' })
9 × locator resolved to <p role="heading" class="_typography_yh5dq_162 _font-body-md-medium_yh5dq_69 mx_BaseCard_header_title_heading">2 Pinned messages</p>
- unexpected value "2 Pinned messages"
at pinned-messages/index.ts:182
180 | async assertPinnedMessagesList(messages: string[]) {
181 | const rightPanel = this.getRightPanel();
> 182 | await expect(rightPanel.getByRole("heading", { name: "Pinned messages" })).toHaveText(
| ^
183 | `${messages.length} Pinned messages`,
184 | );
185 |
at Helpers.assertPinnedMessagesList (/home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/index.ts:182:84)
at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:63:20
|
[Chrome] › pinned-messages/pinned-messages.spec.ts:66:9 › Pinned messages › should unpin one message:
playwright/e2e/pinned-messages/index.ts#L182
3) [Chrome] › pinned-messages/pinned-messages.spec.ts:66:9 › Pinned messages › should unpin one message
Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected)
Locator: locator('#mx_RightPanel').getByRole('heading', { name: 'Pinned messages' })
Expected string: "2 Pinned messages"
Received: <element(s) not found>
Call log:
- expect.toHaveText with timeout 5000ms
- waiting for locator('#mx_RightPanel').getByRole('heading', { name: 'Pinned messages' })
at pinned-messages/index.ts:182
180 | async assertPinnedMessagesList(messages: string[]) {
181 | const rightPanel = this.getRightPanel();
> 182 | await expect(rightPanel.getByRole("heading", { name: "Pinned messages" })).toHaveText(
| ^
183 | `${messages.length} Pinned messages`,
184 | );
185 |
at Helpers.assertPinnedMessagesList (/home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/index.ts:182:84)
at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:74:20
|
[Chrome] › pinned-messages/pinned-messages.spec.ts:112:9 › Pinned messages › should display 2 messages in the banner @screenshot:
playwright/e2e/pinned-messages/pinned-messages.spec.ts#L118
4) [Chrome] › pinned-messages/pinned-messages.spec.ts:112:9 › Pinned messages › should display 2 messages in the banner @screenshot
Error: expect(locator).toHaveScreenshot(expected)
857 pixels (ratio 0.02 of all image pixels) are different.
Expected: /home/runner/work/element-web/element-web/playwright/snapshots/pinned-messages/pinned-messages.spec.ts/pinned-message-banner-2-Msg2-linux.png
Received: /home/runner/work/element-web/element-web/playwright/test-results/pinned-messages-pinned-mes-4b2e4-ay-2-messages-in-the-banner-Chrome/pinned-message-banner-2-Msg2-actual.png
Diff: /home/runner/work/element-web/element-web/playwright/test-results/pinned-messages-pinned-mes-4b2e4-ay-2-messages-in-the-banner-Chrome/pinned-message-banner-2-Msg2-diff.png
Call log:
- expect.toHaveScreenshot(pinned-message-banner-2-Msg2.png) with timeout 5000ms
- verifying given screenshot expectation
- waiting for getByTestId('pinned-message-banner')
- locator resolved to <div data-single-message="true" class="mx_PinnedMessageBanner" data-testid="pinned-message-banner" aria-label="This room has pinned messages. Click to view them.">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- 857 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for getByTestId('pinned-message-banner')
- locator resolved to <div data-single-message="true" class="mx_PinnedMessageBanner" data-testid="pinned-message-banner" aria-label="This room has pinned messages. Click to view them.">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 857 pixels (ratio 0.02 of all image pixels) are different.
116 |
117 | await util.assertMessageInBanner("Msg2");
> 118 | await expect(util.getBanner()).toMatchScreenshot("pinned-message-banner-2-Msg2.png");
| ^
119 |
120 | await util.getBanner().click();
121 | await util.assertMessageInBanner("Msg1");
at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:118:40
|
[Chrome] › pinned-messages/pinned-messages.spec.ts:141:9 › Pinned messages › should open the pinned messages list from the banner:
playwright/e2e/pinned-messages/pinned-messages.spec.ts#L1
5) [Chrome] › pinned-messages/pinned-messages.spec.ts:141:9 › Pinned messages › should open the pinned messages list from the banner
Test timeout of 30000ms exceeded.
|
[Chrome] › pinned-messages/pinned-messages.spec.ts:141:9 › Pinned messages › should open the pinned messages list from the banner:
playwright/e2e/pinned-messages/pinned-messages.spec.ts#L146
5) [Chrome] › pinned-messages/pinned-messages.spec.ts:141:9 › Pinned messages › should open the pinned messages list from the banner
Error: locator.click: Test timeout of 30000ms exceeded.
Call log:
- waiting for getByRole('button', { name: 'View all' })
144 | await util.pinMessages(["Msg1", "Msg2"]);
145 |
> 146 | await util.getViewAllButton().click();
| ^
147 | await util.assertPinnedMessagesList(["Msg1", "Msg2"]);
148 |
149 | await expect(util.getCloseListButton()).toBeVisible();
at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:146:39
|
[Chrome] › pinned-messages/pinned-messages.spec.ts:152:9 › Pinned messages › banner should listen to pinned message list:
playwright/e2e/pinned-messages/pinned-messages.spec.ts#L157
6) [Chrome] › pinned-messages/pinned-messages.spec.ts:152:9 › Pinned messages › banner should listen to pinned message list
Error: Timed out 5000ms waiting for expect(locator).toBeVisible()
Locator: getByRole('button', { name: 'View all' })
Expected: visible
Received: <element(s) not found>
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByRole('button', { name: 'View all' })
155 | await util.pinMessages(["Msg1", "Msg2"]);
156 |
> 157 | await expect(util.getViewAllButton()).toBeVisible();
| ^
158 |
159 | await util.openRoomInfo();
160 | await util.openPinnedMessagesList();
at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:157:47
|
Slow Test:
[Chrome] › spaces/spaces.spec.ts#L1
[Chrome] › spaces/spaces.spec.ts took 1.1m
|
Slow Test:
[Chrome] › settings/account-user-settings-tab.spec.ts#L1
[Chrome] › settings/account-user-settings-tab.spec.ts took 46.6s
|
Slow Test:
[Chrome] › right-panel/file-panel.spec.ts#L1
[Chrome] › right-panel/file-panel.spec.ts took 40.4s
|
Slow Test:
[Chrome] › read-receipts/room-list-order.spec.ts#L1
[Chrome] › read-receipts/room-list-order.spec.ts took 33.5s
|
Slow Test:
[Chrome] › settings/appearance-user-settings-tab/theme-choice-panel.spec.ts#L1
[Chrome] › settings/appearance-user-settings-tab/theme-choice-panel.spec.ts took 33.3s
|
Slow Test:
[Chrome] › read-receipts/high-level.spec.ts#L1
[Chrome] › read-receipts/high-level.spec.ts took 3.3m
|
Slow Test:
[Chrome] › read-receipts/new-messages-in-threads.spec.ts#L1
[Chrome] › read-receipts/new-messages-in-threads.spec.ts took 1.8m
|
Slow Test:
[Chrome] › read-receipts/editing-messages-in-threads.spec.ts#L1
[Chrome] › read-receipts/editing-messages-in-threads.spec.ts took 1.3m
|
Slow Test:
[Chrome] › read-receipts/new-messages-main-timeline.spec.ts#L1
[Chrome] › read-receipts/new-messages-main-timeline.spec.ts took 1.2m
|
Slow Test:
[Chrome] › read-receipts/editing-messages-main-timeline.spec.ts#L1
[Chrome] › read-receipts/editing-messages-main-timeline.spec.ts took 1.2m
|
Slow Test:
[Chrome] › timeline/timeline.spec.ts#L1
[Chrome] › timeline/timeline.spec.ts took 2.8m
|
Slow Test:
[Chrome] › spotlight/spotlight.spec.ts#L1
[Chrome] › spotlight/spotlight.spec.ts took 2.3m
|
Slow Test:
[Chrome] › spaces/threads-activity-centre/threadsActivityCentre.spec.ts#L1
[Chrome] › spaces/threads-activity-centre/threadsActivityCentre.spec.ts took 1.8m
|
Slow Test:
[Chrome] › user-onboarding/user-onboarding-new.spec.ts#L1
[Chrome] › user-onboarding/user-onboarding-new.spec.ts took 35.9s
|
Slow Test:
[Chrome] › threads/threads.spec.ts#L1
[Chrome] › threads/threads.spec.ts took 35.2s
|
Slow Test:
[Chrome] › read-receipts/redactions-main-timeline.spec.ts#L1
[Chrome] › read-receipts/redactions-main-timeline.spec.ts took 2.1m
|
Slow Test:
[Chrome] › read-receipts/read-receipts.spec.ts#L1
[Chrome] › read-receipts/read-receipts.spec.ts took 1.1m
|
Slow Test:
[Chrome] › read-receipts/reactions-in-threads.spec.ts#L1
[Chrome] › read-receipts/reactions-in-threads.spec.ts took 59.9s
|
Slow Test:
[Chrome] › read-receipts/redactions-thread-roots.spec.ts#L1
[Chrome] › read-receipts/redactions-thread-roots.spec.ts took 55.9s
|
Slow Test:
[Chrome] › read-receipts/new-messages-thread-roots.spec.ts#L1
[Chrome] › read-receipts/new-messages-thread-roots.spec.ts took 53.5s
|
Slow Test:
[Chrome] › composer/RTE.spec.ts#L1
[Chrome] › composer/RTE.spec.ts took 1.3m
|
Slow Test:
[Chrome] › audio-player/audio-player.spec.ts#L1
[Chrome] › audio-player/audio-player.spec.ts took 1.2m
|
Slow Test:
[Chrome] › crypto/event-shields.spec.ts#L1
[Chrome] › crypto/event-shields.spec.ts took 1.2m
|
Slow Test:
[Chrome] › crypto/crypto.spec.ts#L1
[Chrome] › crypto/crypto.spec.ts took 1.1m
|
Slow Test:
[Chrome] › crypto/decryption-failure-messages.spec.ts#L1
[Chrome] › crypto/decryption-failure-messages.spec.ts took 54.2s
|
Slow Test:
[Chrome] › messages/messages.spec.ts#L1
[Chrome] › messages/messages.spec.ts took 2.4m
|
Slow Test:
[Chrome] › login/login.spec.ts#L1
[Chrome] › login/login.spec.ts took 1.1m
|
Slow Test:
[Chrome] › knock/knock-into-room.spec.ts#L1
[Chrome] › knock/knock-into-room.spec.ts took 40.4s
|
Slow Test:
[Chrome] › integration-manager/read_events.spec.ts#L1
[Chrome] › integration-manager/read_events.spec.ts took 37.1s
|
Slow Test:
[Chrome] › polls/polls.spec.ts#L1
[Chrome] › polls/polls.spec.ts took 36.7s
|
🎭 Playwright Run Summary
1 flaky
[Chrome] › right-panel/right-panel.spec.ts:41:13 › RightPanel › in rooms › should handle long room address and long room name @screenshot
9 skipped
67 passed (9.7m)
|
🎭 Playwright Run Summary
32 skipped
50 passed (9.9m)
|
🎭 Playwright Run Summary
3 skipped
69 passed (10.2m)
|
🎭 Playwright Run Summary
2 flaky
[Chrome] › read-receipts/redactions-in-threads.spec.ts:89:17 › Read receipts › redactions › in threads › Reading an unread thread after a redaction of the latest message makes it read after restart
[Chrome] › read-receipts/redactions-in-threads.spec.ts:182:17 › Read receipts › redactions › in threads › Sending and redacting a message after marking the thread as read leaves it read
15 skipped
58 passed (10.3m)
|
🎭 Playwright Run Summary
79 passed (11.9m)
|
🎭 Playwright Run Summary
6 flaky
[Chrome] › pinned-messages/pinned-messages.spec.ts:46:9 › Pinned messages › should pin messages and show them in the room info panel
[Chrome] › pinned-messages/pinned-messages.spec.ts:55:9 › Pinned messages › should pin messages and show them in the pinned message panel
[Chrome] › pinned-messages/pinned-messages.spec.ts:66:9 › Pinned messages › should unpin one message
[Chrome] › pinned-messages/pinned-messages.spec.ts:112:9 › Pinned messages › should display 2 messages in the banner @screenshot
[Chrome] › pinned-messages/pinned-messages.spec.ts:141:9 › Pinned messages › should open the pinned messages list from the banner
[Chrome] › pinned-messages/pinned-messages.spec.ts:152:9 › Pinned messages › banner should listen to pinned message list
1 skipped
72 passed (13.1m)
|
Artifacts
Produced during runtime
Name | Size | |
---|---|---|
all-blob-reports-1
Expired
|
674 KB |
|
all-blob-reports-2
Expired
|
12.4 MB |
|
all-blob-reports-3
Expired
|
308 KB |
|
all-blob-reports-4
Expired
|
4.01 MB |
|
all-blob-reports-5
Expired
|
3.9 MB |
|
all-blob-reports-6
Expired
|
773 KB |
|
html-report
Expired
|
21.4 MB |
|
webapp
Expired
|
34.2 MB |
|