Test Apple Major OS releases 2023

[philipphofmann]

Description

iOS 17, tvOS 17, watchOS 10, and macOS 14 are about to GA in September 2023. We should test whether our SDK works appropriately on these new OS versions.

Other tasks

Must-have

Beta Give feedback

1. New required reason API #3192
   Platform: Cocoa Type: Feature Request +2
2. Provide Privacy/Data use manifest (see Apple docs) #3259 #3206
3. Test plan manual test (ex tvOS) (see below)
4. Test plan manual test tvOS (see below)
5. Test plan check with customer data (see below)
6. Test watchOs support

Optional / TBD

Beta Give feedback

1. Add SwiftData operations to iOS-Swift Sample #3105
   Platform: Cocoa +1
2. Add sample app using mergeable libraries #3103
   Platform: Cocoa +1
   
3. Add viewIsAppearing to UIViewController transactions #3100
   Platform: Cocoa Type: Enhancement +2
4. Swift Macro for Instrumenting SwiftUI #3104
   Platform: Cocoa +1
5. Add Vision OS Sample #3101
   Platform: Cocoa +1

Test plan manual test

Feature	iOS	macOS	tvOS	MacCatalyst
Crashes with symbolication	✅	✅	✅	✅
Error events with symbolication	✅	✅	✅	✅
OOMs	✅	╳	✅	╳
App hangs	✅	╳	✅	✅
Release health
View hierarchy	✅	╳	✅	✅
Screenshots	✅	╳	✅	✅
Auto-generated breadcrumbs	✅	✅	✅	✅
Attachments	✅	✅	✅	✅
App start	✅	╳	☑️	✅
Slow & frozen frames	✅	╳	✅	✅
UIViewController tracking	✅	╳	✅	✅
TTID/TTFD	✅	╳	✅	✅
Network tracking	✅	╳	✅	✅
File I/O tracking	✅	╳	✅	✅
Core data tracking	✅	╳	☑️	✅
User interaction tracking	✅	╳	✅	✅
Swift UI tracking	✅	✅	✅	✅
MetricKit	✅		☑️	✅
Profiling	✅	╳	☑️	✅

╳ = not available
☑️ = works on another platform. Not needed to put in the effort to test on this platform, because we are going to validate the feature with customer data.

Test plan check with customer data

Check features with customer data once iOS 17 is released. We only check iOS with customer data as things breaking there would have the most significant negative impact because we have, by far, the most projects on iOS. For the other platforms, the manual test above is sufficient.

Feature	iOS
Crashes with symbolication
Error events with symbolication
OOMs
App hangs
Release health
View hierarchy
Screenshots
Auto-generated breadcrumbs
Attachments
App start
Slow & frozen frames
UIViewController tracking
TTID/TTFD
Network tracking
File I/O tracking
Core data tracking
User interaction tracking
Swift UI tracking
MetricKit
Profiling

---

Similar to #2116

[brustolin]

• Test watchOs support

Tested with #3204

[philipphofmann]

@brustolin, and @kahest, what's the current state of this issue? If I'm not mistaken we only miss validating the features with customer data?

[kahest]

@philipphofmann most of the tvOS tests are still pending, we deprioritized them a bit

[brustolin]

@philipphofmann can you check customer data?

[philipphofmann]

@brustolin, what about we do it together and I show you how I do it so that you can do it as well next time?

[philipphofmann]

Everything done. We can close this.