Playwright: fix (hopefully) flaky shields test #2996
Annotations
7 errors, 5 warnings, and 1 notice
Run Playwright tests:
playwright/e2e/pinned-messages/index.ts#L157
1) [Chrome] › pinned-messages/pinned-messages.spec.ts:46:9 › Pinned messages › should pin messages and show them in the room info panel
Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected)
Locator: getByRole('menuitem', { name: 'Pinned messages' })
Expected string: "Pinned messages3"
Received string: "Pinned messages2"
Call log:
- expect.toHaveText with timeout 5000ms
- waiting for getByRole('menuitem', { name: 'Pinned messages' })
9 × locator resolved to <button role="menuitem" data-kind="primary" class="_item_8j2l6_17 _interactive_8j2l6_35">…</button>
- unexpected value "Pinned messages2"
at pinned-messages/index.ts:157
155 | */
156 | async assertPinnedCountInRoomInfo(count: number) {
> 157 | await expect(this.page.getByRole("menuitem", { name: "Pinned messages" })).toHaveText(
| ^
158 | `Pinned messages${count}`,
159 | );
160 | }
at Helpers.assertPinnedCountInRoomInfo (/home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/index.ts:157:84)
at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:52:20
|
Run Playwright tests:
playwright/e2e/pinned-messages/index.ts#L182
2) [Chrome] › pinned-messages/pinned-messages.spec.ts:55:9 › Pinned messages › should pin messages and show them in the pinned message panel
Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected)
Locator: locator('#mx_RightPanel').getByRole('heading', { name: 'Pinned messages' })
Expected string: "3 Pinned messages"
Received string: "2 Pinned messages"
Call log:
- expect.toHaveText with timeout 5000ms
- waiting for locator('#mx_RightPanel').getByRole('heading', { name: 'Pinned messages' })
9 × locator resolved to <p role="heading" class="_typography_yh5dq_162 _font-body-md-medium_yh5dq_69 mx_BaseCard_header_title_heading">2 Pinned messages</p>
- unexpected value "2 Pinned messages"
at pinned-messages/index.ts:182
180 | async assertPinnedMessagesList(messages: string[]) {
181 | const rightPanel = this.getRightPanel();
> 182 | await expect(rightPanel.getByRole("heading", { name: "Pinned messages" })).toHaveText(
| ^
183 | `${messages.length} Pinned messages`,
184 | );
185 |
at Helpers.assertPinnedMessagesList (/home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/index.ts:182:84)
at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:63:20
|
Run Playwright tests:
playwright/e2e/pinned-messages/index.ts#L182
3) [Chrome] › pinned-messages/pinned-messages.spec.ts:66:9 › Pinned messages › should unpin one message
Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected)
Locator: locator('#mx_RightPanel').getByRole('heading', { name: 'Pinned messages' })
Expected string: "2 Pinned messages"
Received: <element(s) not found>
Call log:
- expect.toHaveText with timeout 5000ms
- waiting for locator('#mx_RightPanel').getByRole('heading', { name: 'Pinned messages' })
at pinned-messages/index.ts:182
180 | async assertPinnedMessagesList(messages: string[]) {
181 | const rightPanel = this.getRightPanel();
> 182 | await expect(rightPanel.getByRole("heading", { name: "Pinned messages" })).toHaveText(
| ^
183 | `${messages.length} Pinned messages`,
184 | );
185 |
at Helpers.assertPinnedMessagesList (/home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/index.ts:182:84)
at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:74:20
|
Run Playwright tests:
playwright/e2e/pinned-messages/pinned-messages.spec.ts#L118
4) [Chrome] › pinned-messages/pinned-messages.spec.ts:112:9 › Pinned messages › should display 2 messages in the banner @screenshot
Error: expect(locator).toHaveScreenshot(expected)
857 pixels (ratio 0.02 of all image pixels) are different.
Expected: /home/runner/work/element-web/element-web/playwright/snapshots/pinned-messages/pinned-messages.spec.ts/pinned-message-banner-2-Msg2-linux.png
Received: /home/runner/work/element-web/element-web/playwright/test-results/pinned-messages-pinned-mes-4b2e4-ay-2-messages-in-the-banner-Chrome/pinned-message-banner-2-Msg2-actual.png
Diff: /home/runner/work/element-web/element-web/playwright/test-results/pinned-messages-pinned-mes-4b2e4-ay-2-messages-in-the-banner-Chrome/pinned-message-banner-2-Msg2-diff.png
Call log:
- expect.toHaveScreenshot(pinned-message-banner-2-Msg2.png) with timeout 5000ms
- verifying given screenshot expectation
- waiting for getByTestId('pinned-message-banner')
- locator resolved to <div data-single-message="true" class="mx_PinnedMessageBanner" data-testid="pinned-message-banner" aria-label="This room has pinned messages. Click to view them.">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- 857 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for getByTestId('pinned-message-banner')
- locator resolved to <div data-single-message="true" class="mx_PinnedMessageBanner" data-testid="pinned-message-banner" aria-label="This room has pinned messages. Click to view them.">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 857 pixels (ratio 0.02 of all image pixels) are different.
116 |
117 | await util.assertMessageInBanner("Msg2");
> 118 | await expect(util.getBanner()).toMatchScreenshot("pinned-message-banner-2-Msg2.png");
| ^
119 |
120 | await util.getBanner().click();
121 | await util.assertMessageInBanner("Msg1");
at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:118:40
|
Run Playwright tests:
playwright/e2e/pinned-messages/pinned-messages.spec.ts#L1
5) [Chrome] › pinned-messages/pinned-messages.spec.ts:141:9 › Pinned messages › should open the pinned messages list from the banner
Test timeout of 30000ms exceeded.
|
Run Playwright tests:
playwright/e2e/pinned-messages/pinned-messages.spec.ts#L146
5) [Chrome] › pinned-messages/pinned-messages.spec.ts:141:9 › Pinned messages › should open the pinned messages list from the banner
Error: locator.click: Test timeout of 30000ms exceeded.
Call log:
- waiting for getByRole('button', { name: 'View all' })
144 | await util.pinMessages(["Msg1", "Msg2"]);
145 |
> 146 | await util.getViewAllButton().click();
| ^
147 | await util.assertPinnedMessagesList(["Msg1", "Msg2"]);
148 |
149 | await expect(util.getCloseListButton()).toBeVisible();
at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:146:39
|
Run Playwright tests:
playwright/e2e/pinned-messages/pinned-messages.spec.ts#L157
6) [Chrome] › pinned-messages/pinned-messages.spec.ts:152:9 › Pinned messages › banner should listen to pinned message list
Error: Timed out 5000ms waiting for expect(locator).toBeVisible()
Locator: getByRole('button', { name: 'View all' })
Expected: visible
Received: <element(s) not found>
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByRole('button', { name: 'View all' })
155 | await util.pinMessages(["Msg1", "Msg2"]);
156 |
> 157 | await expect(util.getViewAllButton()).toBeVisible();
| ^
158 |
159 | await util.openRoomInfo();
160 | await util.openPinnedMessagesList();
at /home/runner/work/element-web/element-web/playwright/e2e/pinned-messages/pinned-messages.spec.ts:157:47
|
Run Playwright tests:
[Chrome] › messages/messages.spec.ts#L1
[Chrome] › messages/messages.spec.ts took 2.4m
|
Run Playwright tests:
[Chrome] › login/login.spec.ts#L1
[Chrome] › login/login.spec.ts took 1.1m
|
Run Playwright tests:
[Chrome] › knock/knock-into-room.spec.ts#L1
[Chrome] › knock/knock-into-room.spec.ts took 40.4s
|
Run Playwright tests:
[Chrome] › integration-manager/read_events.spec.ts#L1
[Chrome] › integration-manager/read_events.spec.ts took 37.1s
|
Run Playwright tests:
[Chrome] › polls/polls.spec.ts#L1
[Chrome] › polls/polls.spec.ts took 36.7s
|
Run Playwright tests
6 flaky
[Chrome] › pinned-messages/pinned-messages.spec.ts:46:9 › Pinned messages › should pin messages and show them in the room info panel
[Chrome] › pinned-messages/pinned-messages.spec.ts:55:9 › Pinned messages › should pin messages and show them in the pinned message panel
[Chrome] › pinned-messages/pinned-messages.spec.ts:66:9 › Pinned messages › should unpin one message
[Chrome] › pinned-messages/pinned-messages.spec.ts:112:9 › Pinned messages › should display 2 messages in the banner @screenshot
[Chrome] › pinned-messages/pinned-messages.spec.ts:141:9 › Pinned messages › should open the pinned messages list from the banner
[Chrome] › pinned-messages/pinned-messages.spec.ts:152:9 › Pinned messages › banner should listen to pinned message list
1 skipped
72 passed (13.1m)
|
Loading