Skip to content

End to End Tests

End to End Tests #6

Triggered via workflow run January 9, 2024 15:56
@t3chguyt3chguy
completed 922f168
Status Success
Total duration 19m 11s
Artifacts 2

end-to-end-tests.yaml

on: workflow_run
Playwright  /  Prepare
2s
Playwright / Prepare
mark_skipped
0s
mark_skipped
Matrix: Playwright / tests
Playwright  /  Report results
58s
Playwright / Report results
Fit to window
Zoom out
Zoom in

Annotations

8 errors, 5 warnings, and 1 notice
[Legacy Crypto] › crypto/crypto.spec.ts:149:17 › Cryptography › setting up secure key backup should work isDeviceVerified=false › by recovery code: matrix-react-sdk/playwright/e2e/crypto/crypto.spec.ts#L166
1) [Legacy Crypto] › crypto/crypto.spec.ts:149:17 › Cryptography › setting up secure key backup should work isDeviceVerified=false › by recovery code Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: locator('.mx_InteractiveAuthDialog .mx_Dialog_title').getByText('Setting up keys') Expected: visible Received: hidden Call log: - expect.toBeVisible with timeout 5000ms - waiting for locator('.mx_InteractiveAuthDialog .mx_Dialog_title').getByText('Setting up keys') 164 | if (!isDeviceVerified) { 165 | const uiaDialogTitle = page.locator(".mx_InteractiveAuthDialog .mx_Dialog_title"); > 166 | await expect(uiaDialogTitle.getByText("Setting up keys")).toBeVisible(); | ^ 167 | await expect(uiaDialogTitle.getByText("Setting up keys")).not.toBeVisible(); 168 | } 169 | at /home/runner/work/element-web/element-web/matrix-react-sdk/playwright/e2e/crypto/crypto.spec.ts:166:79
[Legacy Crypto] › read-receipts/redactions.spec.ts:691:17 › Read receipts › redactions › in threads › A thread with a redacted unread is still read after restart: matrix-react-sdk/playwright/e2e/read-receipts/index.ts#L462
2) [Legacy Crypto] › read-receipts/redactions.spec.ts:691:17 › Read receipts › redactions › in threads › A thread with a redacted unread is still read after restart Error: Timed out 5000ms waiting for expect(locator).not.toBeVisible() Locator: getByRole('treeitem', { name: /^Room Beta/ }).locator('.mx_NotificationBadge_dot') Expected: not visible Received: visible Call log: - expect.not.toBeVisible with timeout 5000ms - waiting for getByRole('treeitem', { name: /^Room Beta/ }).locator('.mx_NotificationBadge_dot') - locator resolved to <div class="mx_NotificationBadge mx_NotificationBadge…>…</div> - unexpected value "visible" - locator resolved to <div class="mx_NotificationBadge mx_NotificationBadge…>…</div> - unexpected value "visible" - locator resolved to <div class="mx_NotificationBadge mx_NotificationBadge…>…</div> - unexpected value "visible" - locator resolved to <div class="mx_NotificationBadge mx_NotificationBadge…>…</div> - unexpected value "visible" - locator resolved to <div class="mx_NotificationBadge mx_NotificationBadge…>…</div> - unexpected value "visible" - locator resolved to <div class="mx_NotificationBadge mx_NotificationBadge…>…</div> - unexpected value "visible" - locator resolved to <div class="mx_NotificationBadge mx_NotificationBadge…>…</div> - unexpected value "visible" - locator resolved to <div class="mx_NotificationBadge mx_NotificationBadge…>…</div> - unexpected value "visible" - locator resolved to <div class="mx_NotificationBadge mx_NotificationBadge…>…</div> - unexpected value "visible" at read-receipts/index.ts:462 460 | async assertRead(room: string | { name: string }) { 461 | const tile = this.getRoomListTile(room); > 462 | await expect(tile.locator(".mx_NotificationBadge_dot")).not.toBeVisible(); | ^ 463 | await expect(tile.locator(".mx_NotificationBadge_count")).not.toBeVisible(); 464 | } 465 | at Helpers.assertRead (/home/runner/work/element-web/element-web/matrix-react-sdk/playwright/e2e/read-receipts/index.ts:462:69) at /home/runner/work/element-web/element-web/matrix-react-sdk/playwright/e2e/read-receipts/redactions.spec.ts:721:28
[Legacy Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A: matrix-react-sdk/playwright/e2e/room/room.spec.ts#L100
3) [Legacy Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A Error: Timed out 5000ms waiting for expect(locator).toBeInViewport() Locator: locator('.mx_MTextBody .mx_EventTile_body').getByText('test0') Expected: in viewport Received: outside viewport Call log: - expect.toBeInViewport with timeout 5000ms - waiting for locator('.mx_MTextBody .mx_EventTile_body').getByText('test0') 98 | // The timeline should display the first message 99 | // The previous position before switching to Room B should be remembered > 100 | await expect(page.locator(".mx_MTextBody .mx_EventTile_body").getByText("test0")).toBeInViewport(); | ^ 101 | }); 102 | }); 103 | at /home/runner/work/element-web/element-web/matrix-react-sdk/playwright/e2e/room/room.spec.ts:100:91
[Rust Crypto] › read-receipts/high-level.spec.ts:295:13 › Read receipts › Paging up › Looking in thread view to find old threads that were never read makes the room unread: matrix-react-sdk/playwright/e2e/read-receipts/high-level.spec.ts#L1
4) [Rust Crypto] › read-receipts/high-level.spec.ts:295:13 › Read receipts › Paging up › Looking in thread view to find old threads that were never read makes the room unread Test finished within timeout of 30000ms, but tearing down "homeserver" ran out of time. Please allow more time for the test, since teardown is attributed towards the test timeout budget.
[Rust Crypto] › read-receipts/high-level.spec.ts:295:13 › Read receipts › Paging up › Looking in thread view to find old threads that were never read makes the room unread: matrix-react-sdk/playwright/e2e/read-receipts/high-level.spec.ts#L1
4) [Rust Crypto] › read-receipts/high-level.spec.ts:295:13 › Read receipts › Paging up › Looking in thread view to find old threads that were never read makes the room unread Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Test timeout of 30000ms exceeded.
[Rust Crypto] › read-receipts/high-level.spec.ts:295:13 › Read receipts › Paging up › Looking in thread view to find old threads that were never read makes the room unread: matrix-react-sdk/playwright/e2e/read-receipts/index.ts#L532
4) [Rust Crypto] › read-receipts/high-level.spec.ts:295:13 › Read receipts › Paging up › Looking in thread view to find old threads that were never read makes the room unread Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: page.waitForTimeout: Target page, context or browser has been closed at read-receipts/index.ts:532 530 | // If we've just entered the room, the threads panel takes a while to decide 531 | // whether it's open or not - wait here to give it a chance to settle. > 532 | await this.page.waitForTimeout(200); | ^ 533 | 534 | const ariaCurrent = await this.page.getByTestId("threadsButton").getAttribute("aria-current"); 535 | if (ariaCurrent !== "true") { at Helpers.openThreadList (/home/runner/work/element-web/element-web/matrix-react-sdk/playwright/e2e/read-receipts/index.ts:532:25) at /home/runner/work/element-web/element-web/matrix-react-sdk/playwright/e2e/read-receipts/high-level.spec.ts:329:24
[Rust Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A: matrix-react-sdk/playwright/e2e/room/room.spec.ts#L100
5) [Rust Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A Error: Timed out 5000ms waiting for expect(locator).toBeInViewport() Locator: locator('.mx_MTextBody .mx_EventTile_body').getByText('test0') Expected: in viewport Received: outside viewport Call log: - expect.toBeInViewport with timeout 5000ms - waiting for locator('.mx_MTextBody .mx_EventTile_body').getByText('test0') 98 | // The timeline should display the first message 99 | // The previous position before switching to Room B should be remembered > 100 | await expect(page.locator(".mx_MTextBody .mx_EventTile_body").getByText("test0")).toBeInViewport(); | ^ 101 | }); 102 | }); 103 | at /home/runner/work/element-web/element-web/matrix-react-sdk/playwright/e2e/room/room.spec.ts:100:91
[Rust Crypto] › timeline/timeline.spec.ts:555:13 › Timeline › message displaying › should set inline start padding to a hidden event line: matrix-react-sdk/playwright/e2e/timeline/timeline.spec.ts#L587
6) [Rust Crypto] › timeline/timeline.spec.ts:555:13 › Timeline › message displaying › should set inline start padding to a hidden event line Error: Screenshot comparison failed: 28 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/element-web/element-web/matrix-react-sdk/playwright/snapshots/timeline/timeline.spec.ts/hidden-event-line-zero-padding-irc-layout-linux.png Received: /home/runner/work/element-web/element-web/matrix-react-sdk/playwright/test-results/timeline-timeline-Timeline-message-displaying-should-set-inline-start-padding-to-a-hidden-event-line-Rust-Crypto/hidden-event-line-zero-padding-irc-layout-actual.png Diff: /home/runner/work/element-web/element-web/matrix-react-sdk/playwright/test-results/timeline-timeline-Timeline-message-displaying-should-set-inline-start-padding-to-a-hidden-event-line-Rust-Crypto/hidden-event-line-zero-padding-irc-layout-diff.png Call log: - page._expectScreenshot with timeout 5000ms - verifying given screenshot expectation - waiting for locator('.mx_MainSplit') - locator resolved to <div class="mx_MainSplit">…</div> - taking element screenshot - disabled all CSS animations - waiting for element to be visible and stable - element is visible and stable - 28 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for locator('.mx_MainSplit') - locator resolved to <div class="mx_MainSplit">…</div> - taking element screenshot - disabled all CSS animations - waiting for element to be visible and stable - element is visible and stable - captured a stable screenshot - 28 pixels (ratio 0.01 of all image pixels) are different. 585 | }; 586 | > 587 | await expect(page.locator(".mx_MainSplit")).toMatchScreenshot( | ^ 588 | "hidden-event-line-zero-padding-irc-layout.png", 589 | screenshotOptions, 590 | ); at /home/runner/work/element-web/element-web/matrix-react-sdk/playwright/e2e/timeline/timeline.spec.ts:587:57
Slow Test: [Rust Crypto] › read-receipts/redactions.spec.ts#L1
[Rust Crypto] › read-receipts/redactions.spec.ts took 5.8m
Slow Test: [Legacy Crypto] › read-receipts/redactions.spec.ts#L1
[Legacy Crypto] › read-receipts/redactions.spec.ts took 5.6m
Slow Test: [Rust Crypto] › read-receipts/high-level.spec.ts#L1
[Rust Crypto] › read-receipts/high-level.spec.ts took 5.1m
Slow Test: [Rust Crypto] › read-receipts/new-messages.spec.ts#L1
[Rust Crypto] › read-receipts/new-messages.spec.ts took 3.9m
Slow Test: [Legacy Crypto] › read-receipts/high-level.spec.ts#L1
[Legacy Crypto] › read-receipts/high-level.spec.ts took 3.6m
🎭 Playwright Run Summary
6 flaky [Legacy Crypto] › crypto/crypto.spec.ts:149:17 › Cryptography › setting up secure key backup should work isDeviceVerified=false › by recovery code [Legacy Crypto] › read-receipts/redactions.spec.ts:691:17 › Read receipts › redactions › in threads › A thread with a redacted unread is still read after restart [Legacy Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A [Rust Crypto] › read-receipts/high-level.spec.ts:295:13 › Read receipts › Paging up › Looking in thread view to find old threads that were never read makes the room unread [Rust Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A [Rust Crypto] › timeline/timeline.spec.ts:555:13 › Timeline › message displaying › should set inline start padding to a hidden event line 114 skipped 626 passed (16.5m)

Artifacts

Produced during runtime
Name Size
all-blob-reports Expired
41.8 MB
html-report--attempt-1 Expired
69.2 MB