Improve test flakiness #275

Re-run triggered December 10, 2024 05:59

waliid

234ff6f

fix-tests

Status Failure

Total duration 39m 39s

Artifacts –

ci.yml

on: push

🔎 Check quality

26s

Matrix: 📦 Archives

Matrix: 🧪 Tests

Annotations

60 errors

ProgressTrackerTimeTests.testEntirePlayback(): Tests/PlayerTests/ProgressTracker/ProgressTrackerTimeTests.swift#L1

XCTAssertTrue failed - Expected: [__C.CMTime(value: 0, timescale: 0, flags: __C.CMTimeFlags(rawValue: 0), epoch: 0), __C.CMTime(value: 0, timescale: 1, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0), __C.CMTime(value: 1, timescale: 4, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0), __C.CMTime(value: 1, timescale: 2, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0), __C.CMTime(value: 3, timescale: 4, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0), __C.CMTime(value: 1, timescale: 1, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0), __C.CMTime(value: 0, timescale: 0, flags: __C.CMTimeFlags(rawValue: 0), epoch: 0)], actual: [__C.CMTime(value: 0, timescale: 0, flags: __C.CMTimeFlags(rawValue: 0), epoch: 0), __C.CMTime(value: 0, timescale: 1000, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0)] (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/ProgressTracker/ProgressTrackerTimeTests.swift#EndingLineNumber=52&StartingLineNumber=52)

🧪 Tests (tvos)

testEntirePlayback on 'Clone 1 of Apple TV - xctest (17098)' (2.794 seconds)

🧪 Tests (tvos)

testEntirePlayback on 'Clone 1 of Apple TV - xctest (17098)' (2.024 seconds)

🧪 Tests (tvos)

testEntirePlayback on 'Clone 1 of Apple TV - xctest (17098)' (2.014 seconds)

🧪 Tests (tvos)

testEntirePlayback on 'Clone 1 of Apple TV - xctest (17098)' (2.016 seconds)

🧪 Tests (tvos)

testMandatoryTracker on 'Clone 1 of Apple TV - xctest (13869)' (1.030 seconds)

🧪 Tests (tvos)

Process completed with exit code 2.

🧪 Tests (tvos)

❌ Tests reported 1 failures

PlayerItemTrackerUpdateTests.testMetadata(): Tests/PlayerTests/Tracking/PlayerItemTrackerUpdateTests.swift#L1

Asynchronous wait failed: Exceeded timeout of 20 seconds, with unfulfilled expectations: "Waiting for publisher output". (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Sources/Circumspect/Publishers.swift#EndingLineNumber=48&StartingLineNumber=48)

PlayerItemTrackerUpdateTests.testMetadataUpdate(): Tests/PlayerTests/Tracking/PlayerItemTrackerUpdateTests.swift#L1

ProgressTrackerProgressAvailabilityTests.testEntirePlayback(): Tests/PlayerTests/ProgressTracker/ProgressTrackerProgressAvailabilityTests.swift#L1

MetadataPublisherTests.testEntirePlayback(): Tests/PlayerTests/Publishers/MetadataPublisherTests.swift#L1

SpeedTests.testPlaylistEnd(): Tests/PlayerTests/Player/SpeedTests.swift#L1

expected to eventually be nil, got <0FE9EED7-343E-4F38-B198-DD01787D2CDD> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/Player/SpeedTests.swift#EndingLineNumber=128&StartingLineNumber=128)

SpeedTests.testSpeedUpdateWhenApproachingLiveEdge(): Tests/PlayerTests/Player/SpeedTests.swift#L1

expected to eventually equal <1>, got <2> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/Player/SpeedTests.swift#EndingLineNumber=120&StartingLineNumber=120)

ReplayChecksTests.testWithManyGoodItems(): Tests/PlayerTests/Player/ReplayChecksTests.swift#L1

expected to eventually be true, got <false> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/Player/ReplayChecksTests.swift#EndingLineNumber=46&StartingLineNumber=46)

ReplayChecksTests.testWithOneGoodItemAndOneBadItem(): Tests/PlayerTests/Player/ReplayChecksTests.swift#L1

ReplayChecksTests.testWithOneGoodItemPlayedEntirely(): Tests/PlayerTests/Player/ReplayChecksTests.swift#L1

PlayerPublisherTests.testPresentationSize(): Tests/PlayerTests/Publishers/PlayerPublisherTests.swift#L1

PlayerPublisherTests.testDuration(): Tests/PlayerTests/Publishers/PlayerPublisherTests.swift#L1

PlayerPublisherTests.testSeekableTimeRangeLifeCycle(): Tests/PlayerTests/Publishers/PlayerPublisherTests.swift#L1

PlayerPublisherTests.testPausedItemStatusLifeCycle(): Tests/PlayerTests/Publishers/PlayerPublisherTests.swift#L1

PlayerPublisherTests.testConsumedItemStatusLifeCycle(): Tests/PlayerTests/Publishers/PlayerPublisherTests.swift#L1

AVPlayerPeriodicTimePublisherTests.testPlayback(): Tests/PlayerTests/Publishers/AVPlayerPeriodicTimePublisherTests.swift#L1

QueueTests.testBetweenPlayableItems(): Tests/PlayerTests/Player/QueueTests.swift#L1

expected to eventually equal <[http://localhost:8123/simple/on_demand/master.m3u8]>, got <[http://localhost:8123/simple/on_demand_short/master.m3u8, http://localhost:8123/simple/on_demand/master.m3u8]> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/Player/QueueTests.swift#EndingLineNumber=72&StartingLineNumber=72)

QueueTests.testFailingItemUnavailableBetweenPlayableItems(): Tests/PlayerTests/Player/QueueTests.swift#L1

expected to eventually be empty, got <[http://localhost:8123/simple/on_demand_short/master.m3u8, http://localhost:8123/unavailable/master.m3u8]> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/Player/QueueTests.swift#EndingLineNumber=113&StartingLineNumber=113)

QueueTests.testEntirePlayback(): Tests/PlayerTests/Player/QueueTests.swift#L1

expected to eventually be empty, got <[http://localhost:8123/simple/on_demand_short/master.m3u8]> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/Player/QueueTests.swift#EndingLineNumber=30&StartingLineNumber=30)

QueueTests.testFailingMp3ItemBetweenPlayableItems(): Tests/PlayerTests/Player/QueueTests.swift#L1

expected to eventually be empty, got <[http://localhost:8123/simple/on_demand_short/master.m3u8, http://localhost:8123/unavailable.mp3]> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/Player/QueueTests.swift#EndingLineNumber=123&StartingLineNumber=123)

CurrentItemTests.testSetCurrentItemToSameItem(): Tests/PlayerTests/Playlist/CurrentItemTests.swift#L1

expected to eventually be greater than <1>, got <0> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/Playlist/CurrentItemTests.swift#EndingLineNumber=184&StartingLineNumber=184)

CurrentItemTests.testCurrentItemAfterPlayerEnded(): Tests/PlayerTests/Playlist/CurrentItemTests.swift#L1

CurrentItemTests.testCurrentItemWithMiddleFailedItem(): Tests/PlayerTests/Playlist/CurrentItemTests.swift#L1

CurrentItemTests.testSlowFirstCurrentItem(): Tests/PlayerTests/Playlist/CurrentItemTests.swift#L1

CurrentItemTests.testCurrentItem(): Tests/PlayerTests/Playlist/CurrentItemTests.swift#L1

CurrentItemTests.testCurrentItemWithLastFailedItem(): Tests/PlayerTests/Playlist/CurrentItemTests.swift#L1

PeriodicMetricsPublisherTests.testPlaylist(): Tests/PlayerTests/Publishers/PeriodicMetricsPublisherTests.swift#L1

PeriodicMetricsPublisherTests.testLimit(): Tests/PlayerTests/Publishers/PeriodicMetricsPublisherTests.swift#L1

PeriodicMetricsPublisherTests.testPlayback(): Tests/PlayerTests/Publishers/PeriodicMetricsPublisherTests.swift#L1

MediaSelectionTests.testCharacteristicsAndOptionsWhenExhausted(): Tests/PlayerTests/MediaSelection/MediaSelectionTests.swift#L1

expected to eventually be empty, got <Set([__C.AVMediaCharacteristic(_rawValue: AVMediaCharacteristicAudible), __C.AVMediaCharacteristic(_rawValue: AVMediaCharacteristicLegible)])> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/MediaSelection/MediaSelectionTests.swift#EndingLineNumber=42&StartingLineNumber=42)

RepeatModeTests.testRepeatModeUpdateDoesNotReplay(): Tests/PlayerTests/Playlist/RepeatModeTests.swift#L1

expected to eventually be nil, got <172DE99B-B00B-4C39-B8E1-6B76C7C11AA3> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/Playlist/RepeatModeTests.swift#EndingLineNumber=48&StartingLineNumber=48)

RepeatModeTests.testRepeatOne(): Tests/PlayerTests/Playlist/RepeatModeTests.swift#L1

expected to eventually equal <8B1E1327-43A9-4268-B5FB-8F9A7051531C>, got <EA08609D-AD2E-4E5F-A3FD-6477E855D1E3> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/Playlist/RepeatModeTests.swift#EndingLineNumber=20&StartingLineNumber=20)

RepeatModeTests.testRepeatAll(): Tests/PlayerTests/Playlist/RepeatModeTests.swift#L1

expected to eventually equal <392BB405-C642-4B38-8F31-DA2A96F2D7EB>, got <453AA410-8F16-454B-8104-CE2D3A35B8BD> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/Playlist/RepeatModeTests.swift#EndingLineNumber=30&StartingLineNumber=30)

MetricsCollectorTests.testPlayback(): Tests/PlayerTests/Metrics/MetricsCollectorTests.swift#L1

MetricsCollectorTests.testPlayerSetToNil(): Tests/PlayerTests/Metrics/MetricsCollectorTests.swift#L1

expected to eventually not be empty, got <[]> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/Metrics/MetricsCollectorTests.swift#EndingLineNumber=72&StartingLineNumber=72)

PlayerTrackingTests.testTrackingEnabledDuringPlayback(): Tests/PlayerTests/Tracking/PlayerTrackingTests.swift#L1

PlayerItemTrackerLifeCycleTests.testNetworkLoadedItemEntirePlayback(): Tests/PlayerTests/Tracking/PlayerItemTrackerLifeCycleTests.swift#L1

PlayerItemTrackerLifeCycleTests.testItemEntirePlayback(): Tests/PlayerTests/Tracking/PlayerItemTrackerLifeCycleTests.swift#L1

ReplayTests.testWithOneGoodItemAndOneBadItem(): Tests/PlayerTests/Player/ReplayTests.swift#L1

expected to eventually equal <57187F85-49B6-42EB-BC12-D13F28823FA1>, got <F9E646AF-A5D9-4866-BF34-68DB29B817D2> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/Player/ReplayTests.swift#EndingLineNumber=61&StartingLineNumber=61)

ReplayTests.testWithManyGoodItems(): Tests/PlayerTests/Player/ReplayTests.swift#L1

expected to eventually equal <2A2594D3-5D32-4BC2-B1D9-61CB7640D0DB>, got <F2ED9CE2-19C7-4DE9-889C-A6899BF01E4A> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/Player/ReplayTests.swift#EndingLineNumber=41&StartingLineNumber=41)

ReplayTests.testResumePlaybackIfNeeded(): Tests/PlayerTests/Player/ReplayTests.swift#L1

expected to eventually be nil, got <FD995E0F-B23F-4A7A-AEF1-A1653F018768> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/Player/ReplayTests.swift#EndingLineNumber=70&StartingLineNumber=70)

ReplayTests.testWithOneGoodItemPlayedEntirely(): Tests/PlayerTests/Player/ReplayTests.swift#L1

expected to eventually be nil, got <21E07A3F-4D55-43A5-9CEB-E86B0EB71F05> (/Users/runner/work/pillarbox-apple-github-runner-poc/pillarbox-apple-github-runner-poc/Tests/PlayerTests/Player/ReplayTests.swift#EndingLineNumber=23&StartingLineNumber=23)

🧪 Tests (ios)

❌ Tests reported 41 failures

🧪 Tests (ios)

testMetadata on 'Clone 1 of iPhone 16 - xctest (13514)' (21.739 seconds)

🧪 Tests (ios)

testMetadata on 'Clone 1 of iPhone 16 - xctest (13514)' (20.059 seconds)

🧪 Tests (ios)

testEntirePlayback on 'Clone 1 of iPhone 16 - xctest (16948)' (20.108 seconds)

🧪 Tests (ios)

testMetadata on 'Clone 1 of iPhone 16 - xctest (13514)' (20.039 seconds)

🧪 Tests (ios)

testEntirePlayback on 'Clone 1 of iPhone 16 - xctest (16948)' (20.069 seconds)