Swift Testing related cherry-picks #1443
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
brentleyjones
requested review from
BalestraPatrick,
keith,
luispadron,
mattrobmattrob and
thii
as code owners
brentleyjones
referenced
this pull request
Oct 23, 2024
… observer. This is work done in preparation for supporting the swift-testing framework. PiperOrigin-RevId: 662617920
auto-merge was automatically disabled
October 23, 2024 21:48
Pull request was converted to draft
|
I'm going to batch a couple changes, to allow it to compile correctly. |
… observer This is work done in preparation for supporting the swift-testing framework. PiperOrigin-RevId: 662617920 (cherry picked from commit 5aa34d4) Signed-off-by: Brentley Jones <[email protected]>
This is causing some of our tests to terminate abnormally with an error out of the Swift runtime: "freed pointer was not the last allocation". We're going to need `async` eventually to support swift-testing, but in the meantime, this fixes the broken projects while I investigate further. PiperOrigin-RevId: 662923954 (cherry picked from commit 9fb1eaa) Signed-off-by: Brentley Jones <[email protected]>
This was used as the principal class when `swift_test` built `MH_BUNDLE`-type binaries, but now that we build standard executables and register the observer manually, it's no longer needed or used. PiperOrigin-RevId: 662926687 (cherry picked from commit 722ec91) Signed-off-by: Brentley Jones <[email protected]>
…sync` There is what appears to be a strange bug on Darwin platforms here. If the calling context (e.g., `main`) is `async` and there are any `async` tests in the suite to be run, calling `XCTestSuite.run()` will cause the test process to terminate abnormally with the error "freed pointer was not the last allocation" out of the Swift runtime. We "avoid" (work around) this by wrapping the call in a detached task and then awaiting its result. The task must be detached so that we don't block the main actor, which the tests may also be depending on. PiperOrigin-RevId: 662944537 (cherry picked from commit d7f555d) Signed-off-by: Brentley Jones <[email protected]>
…e generator and just make them regular Swift classes in the observer module There's no compelling reason to generate all of this code; the only thing that actually needs to be generated is the list of tests for Linux, and the small `main` that invokes it. For simplicity, we generate it on Darwin platforms as well, even though it's not strictly necessary because all the discovery there is dynamic. This change also removes the detached `Task` hack that I introduced previously. This worked when all the tests pass (the common case), but if a test failed, XCTest complained that the `recordIssue` method wasn't being called on the main thread. I'll revisit this in a subsequent change to support swift-testing. PiperOrigin-RevId: 664761141 (cherry picked from commit 6d6727a) Signed-off-by: Brentley Jones <[email protected]>
PiperOrigin-RevId: 664764173 (cherry picked from commit 84c67e5) Signed-off-by: Brentley Jones <[email protected]>
Test discovery and execution is handled through the v0 JSON ABI entry point provided by the swift-testing framework. This requires version 0.11.0 or higher of the package (or Xcode 16 beta 5 or higher). PiperOrigin-RevId: 665308132 (cherry picked from commit 26e2624) Signed-off-by: Brentley Jones <[email protected]>
…arnings PiperOrigin-RevId: 667570499 (cherry picked from commit 1dd927f) Signed-off-by: Brentley Jones <[email protected]>
* Remove the module name from the test identifier used for filtering. * If no tests matched the filter, request explicit skipping of all tests so that the runner doesn't interpret the empty filter set as "run all tests". PiperOrigin-RevId: 670984261 (cherry picked from commit 7c149cf) Signed-off-by: Brentley Jones <[email protected]>
…compatibility Making all of the discovered tests returned by `static func`s instead of stored `static let`s removes some problematic `@Sendable`-related function conversions that were causing a runtime crash when Swift 6 more is enabled. SwiftPM avoids this problem by... not ever compiling the discovered test runner in Swift 6 mode even if the package requests it. 🫤 PiperOrigin-RevId: 673829386 (cherry picked from commit 6988259) Signed-off-by: Brentley Jones <[email protected]>
brentleyjones
force-pushed
the
bj/refactor-the-test-observer-to-extract-xunit-recording-from-the-xctest-observer
branch
from
6ecd0de
to
29433e5
Compare
brentleyjones
changed the title
brentleyjones
referenced
this pull request
Nov 18, 2024
This is causing some of our tests to terminate abnormally with an error out of the Swift runtime: "freed pointer was not the last allocation". We're going to need `async` eventually to support swift-testing, but in the meantime, this fixes the broken projects while I investigate further. PiperOrigin-RevId: 662923954
brentleyjones
referenced
this pull request
Nov 18, 2024
This was used as the principal class when `swift_test` built `MH_BUNDLE`-type binaries, but now that we build standard executables and register the observer manually, it's no longer needed or used. PiperOrigin-RevId: 662926687
brentleyjones
referenced
this pull request
Nov 18, 2024
…sync`. There is what appears to be a strange bug on Darwin platforms here. If the calling context (e.g., `main`) is `async` and there are any `async` tests in the suite to be run, calling `XCTestSuite.run()` will cause the test process to terminate abnormally with the error "freed pointer was not the last allocation" out of the Swift runtime. We "avoid" (work around) this by wrapping the call in a detached task and then awaiting its result. The task must be detached so that we don't block the main actor, which the tests may also be depending on. PiperOrigin-RevId: 662944537
brentleyjones
referenced
this pull request
Nov 18, 2024
…e generator and just make them regular Swift classes in the observer module. There's no compelling reason to generate all of this code; the only thing that actually needs to be generated is the list of tests for Linux, and the small `main` that invokes it. For simplicity, we generate it on Darwin platforms as well, even though it's not strictly necessary because all the discovery there is dynamic. This change also removes the detached `Task` hack that I introduced previously. This worked when all the tests pass (the common case), but if a test failed, XCTest complained that the `recordIssue` method wasn't being called on the main thread. I'll revisit this in a subsequent change to support swift-testing. PiperOrigin-RevId: 664761141
brentleyjones
referenced
this pull request
Nov 18, 2024
Test discovery and execution is handled through the v0 JSON ABI entry point provided by the swift-testing framework. This requires version 0.11.0 or higher of the package (or Xcode 16 beta 5 or higher). PiperOrigin-RevId: 665308132
brentleyjones
referenced
this pull request
Nov 18, 2024
* Remove the module name from the test identifier used for filtering. * If no tests matched the filter, request explicit skipping of all tests so that the runner doesn't interpret the empty filter set as "run all tests". PiperOrigin-RevId: 670984261
brentleyjones
referenced
this pull request
Nov 18, 2024
…compatibility. Making all of the discovered tests returned by `static func`s instead of stored `static let`s removes some problematic `@Sendable`-related function conversions that were causing a runtime crash when Swift 6 more is enabled. SwiftPM avoids this problem by... not ever compiling the discovered test runner in Swift 6 mode even if the package requests it. 🫤 PiperOrigin-RevId: 673829386
Signed-off-by: Brentley Jones <[email protected]>
Signed-off-by: Brentley Jones <[email protected]>
Signed-off-by: Brentley Jones <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Multiple commits related to
swift_testto make it compatible with Swift Testing.