Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gTests]Fix GTEST_SKIP kills all gtests in miopen_gtest #3220

Merged
merged 4 commits into from
Sep 4, 2024

Conversation

xinlipn
Copy link
Contributor

@xinlipn xinlipn commented Aug 27, 2024

miopen_gtest wouldn't start on e.g. Navi3x as below. This is because GTEST_SKIP() is invoked in a global environment setup. The fix is to skip the test in TEST_P(). Log with the fix is attached at the bottom.

root@aus-navi3x-09:/MIOpen/build/bin# ./miopen_gtest
Running main() from /opt/rocm/cget/build/tmp-c4f1a42bcee536579a1248d25169f834201170c187fc6d80e8c510a7d28f6f48/googletest-1.14.0/googletest/src/gtest_main .cc
/MIOpen/test/gtest/api_convbiasactiv.cpp:189: Skipped
Skipping fusion test on unsupported ASIC

PRNG seed: 12345678
[==========] Running 8939 tests from 394 test suites.
[----------] Global test environment set-up.
Skipping fusion test on unsupported ASIC

[----------] Global test environment tear-down
[==========] 8939 tests from 394 test suites ran. (0 ms total)
[ PASSED ] 8939 tests.

YOU HAVE 133 DISABLED TESTS

Log with the fix on Navi3x
navi3x.log

Comment on lines 154 to 156
const auto dev_name = get_handle().GetDeviceName();
#if WORKAROUND_ISSUE_2212
if(!miopen::StartsWith(dev_name, "gfx11") && !miopen::StartsWith(dev_name, "gfx94"))
Copy link
Collaborator

@BrianHarrisonAMD BrianHarrisonAMD Aug 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move the skip to the start of SetUp(), and use the device flags for skipping instead?

void SetUp() override
{
    if(!IsTestSupportedForDevice())
    {
         GTEST_SKIP();
    }

   // remaining Setup
}



bool IsTestSupportedForDevice()
{
#if WORKAROUND_ISSUE_2212
    using namespace miopen::debug;
    using e_mask = enabled<Gpu::gfx900, Gpu::gfx906, Gpu::gfx908, Gpu::gfx90A, Gpu::gfx103X>;
    using d_mask = disabled<Gpu::gfx110X, Gpu::gfx94X>;
   return ::IsTestSupportedForDevMask<d_mask, e_mask>();
#else
    return true;
#endif
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a part of overall gtest improvements https://github.com/ROCm/MIOpen/wiki/GTest-development#early-skip
Probably we should add some tickets here #3140

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move the skip to the start of SetUp(), and use the device flags for skipping instead?

void SetUp() override
{
    if(!IsTestSupportedForDevice())
    {
         GTEST_SKIP();
    }

   // remaining Setup
}



bool IsTestSupportedForDevice()
{
#if WORKAROUND_ISSUE_2212
    using namespace miopen::debug;
    using e_mask = enabled<Gpu::gfx900, Gpu::gfx906, Gpu::gfx908, Gpu::gfx90A, Gpu::gfx103X>;
    using d_mask = disabled<Gpu::gfx110X, Gpu::gfx94X>;
   return ::IsTestSupportedForDevMask<d_mask, e_mask>();
#else
    return true;
#endif
}

@BrianHarrisonAMD , thanks for the comments. Code updated

Comment on lines 188 to 196
void GatherCBATestCases(std::vector<CBATestCase>& cba_test_cases)
{
cba_test_cases.push_back(CBATestCase{
16, 128, 16, 16, 128, 3, 3, 0, 0, 1, 1, 1, 1, miopenActivationRELU, miopenConvolution});
}

// Extra layer of indirection introduced since GTEST_SKIP() cannot be called from non-void function.
std::vector<CBATestCase> GetTestValues()
{
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could combine these into a single method that returns the testcases, and I think the comment isn't applicable anymore with this fix.

std::vector<CBATestCase> GetTestValues()
{
    return {{ 16, 128, 16, 16, 128, 3, 3, 0, 0, 1, 1, 1, 1, miopenActivationRELU, miopenConvolution }};
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could combine these into a single method that returns the testcases, and I think the comment isn't applicable anymore with this fix.

std::vector<CBATestCase> GetTestValues()
{
    return {{ 16, 128, 16, 16, 128, 3, 3, 0, 0, 1, 1, 1, 1, miopenActivationRELU, miopenConvolution }};
}

@BrianHarrisonAMD , thanks for the comments. Cleaned up code

@BrianHarrisonAMD
Copy link
Collaborator

Couple minor comments, but looks good!

Interesting find!
I had no idea it worked that way hehe.

@xinlipn xinlipn changed the title [Test]Fix GTEST_SKIP kills all gtests in miopen_gtest [gTests]Fix GTEST_SKIP kills all gtests in miopen_gtest Aug 27, 2024
Copy link
Contributor

@atamazov atamazov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 LGTM

@junliume junliume merged commit e5109c2 into develop Sep 4, 2024
143 checks passed
@junliume junliume deleted the sl/gtest_skipped branch September 4, 2024 04:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants