Compile error in 1.17.4 with Xcode 16.1.0 Beta: Missing required module 'Testing'

[JRR-OSU]

Hi all,

We love your library, but currently it doesn't seem to be usable with Xcode 16.1. When importing the library, an error is observed that prevents compilation of the UI test target. I included a minimal sample project down below. This is pretty urgent since we are a week away from a new Xcode RC.

Describe the bug
When importing SnapshotTesting in a project, the following error occurs:

Missing required module 'Testing'

To Reproduce
swift-snapshot-testing.zip

import SnapshotTesting

final class SnapshotTestSampleUITests: XCTestCase {

Expected behavior
The project should compile without errors when importing SnapshotTesting in test targets, and the SnapshotTesting library should function as expected without the Testing module error.

Screenshots

Environment

• swift-snapshot-testing version 1.17.4
• Xcode 16.1.0 Beta
• Swift 5
• OS: macOS 14.5 / iOS 18.1 Simulator

@stephencelis Am happy to help fork and fix this if I can get some more information on the Testing framework and the wizardry happening with the imports here. Thanks!

[JRR-OSU]

FWIW, I cannot get the sample to compile with Xcode 16.0 Beta 5 or Beta 6 either. Both of those are running in Swift 5 mode.

[mbrandonw]

Hi @JRR-OSU, the project you have provided compiles just fine in Xcode 16.1 beta for me, and I also created a fresh project, added SnapshotTesting, and that too compiles just fine.

Can you please provide a project that demonstrates the problem? And since it is not clear this is an issue with the library, I am going to convert this to a discussion. Please feel free to continue the conversation over there.

[kondratk]

I can confirm the same issue happens in my project, within UI Tests target. This target has a dependency on a Swift Package (technical- with general test tools), that has a dependency on swift-package-testing. To me it looks like @JRR-OSU also had this setup- swift-snapshot-testing being imported within UI tests target.

[JRR-OSU]

@mbrandonw With respect, it was significantly unhelpful to move this out of the issue tracker without any follow up questions or details being requested.

Since I provided direct proof of the compile error from import of the lib including a screenshot and a basic sample project that does indeed demonstrate the problem, I would ask if you can provide relevant information as I did about what version of macOS you tested this on. From the looks of it, you either did not clear DerivedData or simply did not attempt to compile and run the test target (which is where the problem occurs.) The actual iOS target itself will still compile.

I have tested on two separate machines and the issue occurs.

Since @kondratk has kindly confirmed this is an issue that prevents previously working tests targets from compiling, I request moving this discussion back to the issue tracker so we can work on a PR. Thanks.

[mbrandonw]

@mbrandonw With respect, it was significantly unhelpful to move this out of the issue tracker without any follow up questions or details being requested.

Hi @JRR-OSU, I believe I did request more details in my reply: "Can you please provide a project that demonstrates the problem?" The project you provided does not reproduce the problem for me.

Since I provided direct proof of the compile error from import of the lib including a screenshot and a basic sample project that does indeed demonstrate the problem, I would ask if you can provide relevant information as I did about what version of macOS and Xcode you tested this on. From the looks of it, you either did not clear DerivedData or simply did not attempt to compile and run the test target (which is where the problem occurs.) The actual iOS target itself will still compile.

I said before that I used Xcode 16.1 beta. I am also on macOS 14.5. Here is a screenshot to show that when I open your project I can build and run tests just fine:

I don't know why I am seeing different behavior than you, but I can only work with the information I am seeing right in front of me. Both you and @kondratk mention something about this possibly being a problem with UI test targets, but your sample does not have such a target. Is your project perhaps missing something?

Since @kondratk has kindly confirmed this is an issue that prevents previously working tests targets from compiling, I request moving this discussion back to the issue tracker so we can work on a PR. Thanks.

If you (or @kondratk) can provide a project that demonstrates the problem, I will happily move this back to an issue. But, having an issue should not prevent you from working on a PR. We will happily accept a PR to fix any problem at any time.

I hope you can understand that this is a popular open source library and it takes a lot of time to maintain it. When I have the time to look into an issue, if it is 100% clear it is an issue, I convert it to a discussion. And a discussion is a perfectly fine place to provide more details and figure out what is going on. In fact, the vast majority of issues in this repo should really be discussions, and someday when we have time to triage them we will convert them.

Further, Xcode betas make this process even more untenable. Betas often introduce problems that have nothing to do with the library or how Swift/Xcode will work in the future. They are just temporary, ephemeral bugs, and we can't chase down each one of them for every beta. This is another reason why this conversation is more appropriate as a discussion.

In the end, I thank you for bringing up this potential problem, and we would greatly appreciate a project that reproduces the problem or a PR that fixes the problem.

[JRR-OSU]

If you (or @kondratk) can provide a project that demonstrates the problem, I will happily move this back to an issue. But, having an issue should not prevent you from working on a PR. We will happily accept a PR to fix any problem at any time.

The included project does demonstrate the problem, but I can see from your screenshot that you have it set to a Mac target. Here's a better sample below that restricts the test target to iOS only. Apologies if that was the reason for the mixup here. I thought this was clear from my screenshot.
SnapshotTestingBug.zip

In the above sample please make sure you click the following file with the same sim and run that UI test class.

Further, Xcode betas make this process even more untenable. Betas often introduce problems that have nothing to do with the library or how Swift/Xcode will work in the future. They are just temporary, ephemeral bugs, and we can't chase down each one of them for every beta. This is another reason why this conversation is more appropriate as a discussion.

This is true generally but this issue has been around for the last few betas. It's possible it goes away when the RC drops but that's unlikely. Since I discovered it works as a Mac target, I'm worried it may not work any longer for iOS targets, but my hope is that this issue might be caused by 81bbb32.

I am more than willing to work on a PR regardless of if this is filed as an issue or not, I am just making the point that it makes way more sense to track it as one, since there's a proven error that prevents any existing iOS project's test target from working when that org updates Xcode and imports this lib. It also helps for SEO and when others look for the solution, etc.

[mbrandonw]

Me converting an issue to a discussion is not an act of aggression or a judgement on you. It is a process that we apply across all of our repos. And I didn't assume anything. I can only work with the information I have in front of me, which was a project that compiled fine, and therefore I felt it was OK to convert to a discussion until more information was provided.

Discussions are not a lesser communication channel than issues in our minds, and one doesn't get more attention from us than the other. Honestly, there is a chance that someday we will unilaterally turn off issues in all of our repos and require people to open discussions instead, as issues are too often opened when they should be discussions.

I really do not think we need to continue debating the manner in which we manage our library. We field dozens of issues/discussions a day (truly), and so we do what is necessary to allow us to maintain this and our other libraries for many years to come.

[kondratk]

@mbrandonw Thank you for the quick fix! Do you have any plans for creating a release soon? :)

[mbrandonw]

Hi @kondratk, we will get a release out soon, but there have been some new developments since merging that PR.

Turns out the @_implementationOnly trick we employed is largely deprecated and has been replaced with private import. However, private import does not do everything @_implementationOnly does and so cannot be used. So, while we will leave it in snapshot-testing for now, the day it becomes a problem we will be removing it immediately, and thus you will have a problem with your tests again.

The root of the problem is that Swift's new native Testing framework does not work in UI test targets. Apple has confirmed this. They just aren't compatible. And because our library provides Testing-related tools, it cannot be used in a UI test target. So really we feel that this issue lies with Apple, and we recommend filing a feedback with them to let them know how this affects you.

But for the time being, we would warn that snapshot-testing in UI tests is "use at your own risk."

[kondratk]

Makes sense. Thank you @mbrandonw for the explanation and great work with maintaining this library.

[mbrandonw]

We just released 1.7.5 with this fix! Thanks for your patience :)