You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I maintain the Quick testing framework. This issue comes as a result of a bug reported to Quick. Quick provides a DSL for organizing tests that, amongst other things, allows you to use any arbitrary string as the name of a test. For example, the following bit from Quick's DSL:
classSomeSpec:QuickSpec{overrideclassfunc spec(){it("can include any character you want, including commas"){// whatever test you want it to be.}}}
This code sample will generate a test, using xcodebuild's test filtering syntax, named "SomeSpec"/"can include any character you want, including commas" (double quotes added to make it easier to differentiate).
On Darwin platforms, this takes advantage of the fact that objc selectors can be any UTF-8 string. On non-Darwin platforms, this uses the allTests infrastructure to name tests. That is to say, this is not a bug in Quick's XCTest integration, you can easily reproduce this in pure XCTest (in SPM) with the following code:
classSomeTest:XCTestCase{staticvarallTests:[(String,(SomeTest)->()throws->Void)]{[("XCTest totally works with commas, right?", someTestMethod)]}func someTestMethod()}
The bug is that, using swift test --filter, you can not select tests with commas in their names. That is, in the first example, swift test --filter "SomeSpec"/"can include any character you want, including commas or /s" will not find any tests to run. Similarly, with the second example, `swift test --filter "SomeTest/"XCTest totally can use /s or commas, right?" will not run any tests.
As @natinusala reported to us, this looks to be caused by XCTest's argument parser not considering that test names might contain commas. This causes XCTest to behave as if you wanted to filter tests named "SomeTest"/"XCTest totally works with commas" and " right?", which will almost always result in no tests found that match those filters (unless you have an XCTestCase subclass that shares the same name as the characters after the comma).
FWIW, this issue also exists when you use xcodebuild on Darwin, only the character that xcodebuild doesn't like filtering on is a slash.
The text was updated successfully, but these errors were encountered:
younata
changed the title
Cannot filter test names with commas.
Cannot filter test names with commas using swift test --filterJun 30, 2023
Hello!
I maintain the Quick testing framework. This issue comes as a result of a bug reported to Quick. Quick provides a DSL for organizing tests that, amongst other things, allows you to use any arbitrary string as the name of a test. For example, the following bit from Quick's DSL:
This code sample will generate a test, using xcodebuild's test filtering syntax, named
"SomeSpec"/"can include any character you want, including commas"
(double quotes added to make it easier to differentiate).On Darwin platforms, this takes advantage of the fact that objc selectors can be any UTF-8 string. On non-Darwin platforms, this uses the
allTests
infrastructure to name tests. That is to say, this is not a bug in Quick's XCTest integration, you can easily reproduce this in pure XCTest (in SPM) with the following code:The bug is that, using
swift test --filter
, you can not select tests with commas in their names. That is, in the first example,swift test --filter "SomeSpec"/"can include any character you want, including commas or /s"
will not find any tests to run. Similarly, with the second example, `swift test --filter "SomeTest/"XCTest totally can use /s or commas, right?" will not run any tests.As @natinusala reported to us, this looks to be caused by XCTest's argument parser not considering that test names might contain commas. This causes XCTest to behave as if you wanted to filter tests named
"SomeTest"/"XCTest totally works with commas"
and" right?"
, which will almost always result in no tests found that match those filters (unless you have an XCTestCase subclass that shares the same name as the characters after the comma).FWIW, this issue also exists when you use xcodebuild on Darwin, only the character that xcodebuild doesn't like filtering on is a slash.
The text was updated successfully, but these errors were encountered: