macCatalyst: Snapshots generated differ a lot between local computer and a MacStadium M1 mac mini

[sergiocampama]

I think I'm using snapshot testing in a very niche environment, but I'm having some issues with snapshot testing that I'm not sure how to work around.

We build a Mac Catalyst app, targeting macOS directly (and not through the iPad adaptation). We also have a CI system built on top of Github actions, with MacStadium machines, specifially M2 Mac Minis (2 machines). We haven't been able to get snapshot tests to match our local development environment (Xcode 15, M2 Max MacBook pros).

Attached is one sample of a reference and a failure image. It seems on MacStadium it renders the images in way lower resolution for some reason.

Some additional information on the MacStadium machines: They don't have a monitor, they have a headless adapter that allows us to connect using screen sharing, and they have a specific color profile for that headless mode (not sure if color profile affects snapshot testing).

Are there any settings we need to configure on the CI machines that we might not know of to get rendering parity between development and CI machines?

Reference:

Failure:

[tdrhq]

Most larger companies do the "recording" step on CI, that completely avoids this issue, and also improves the developer workflow. Google, Meta, AirBnb etc. using an external service to manage the screenshots (this also avoids the operational overhead of maintaining things like GitLFS).

If this is something you're interested in, you could look at an open-source service like Screenshotbot or Vizzy. (Full disclosure, I built Screenshotbot, so I'm biased.)