Timing out SHMIP test in CEE nightly

[ikalash]

The SHMIP_suite_A1 test has been timing out in the CEE gcc nightly for a few days: https://sems-cdash-son.sandia.gov/cdash/test/3035824 . Can this test be shortened? It times out after 10 mins in the release build and takes 26 mins (!!!) in the debug build. This is too long for a regular test in Albany I think...

[bartgol]

Yeah, that's long. Unfortunately it can't be shortened, otherwise the comparison against SHMIP data is not possible. That test has very difficult parameters, which cause very slow convergence, and stopping earlier would likely produce useless results.

However, I think we can disable it by default, and maybe enable it only if certain CMake vars are ON (like ALBANY_ENABLE_LITERATURE_TESTS or something). In general, I think we could set a special testing category for tests that are meant to reproduce results from a MIP or from a paper or something like that. I don't think they need to be routinely run (though it might be helpful to run them at larger intervals, say once every couple of weeks or once a month).

As a side note, all subglacial hydrology tests are comparisons against literature, so it's relatively hard to shorten them. Yes, we could modify parameters and make them fast, but then if someone wanted to reproduce the papers/SHMIP results, they would have to know which parameters to modify and how. So, I'm in favor of disabling these, and enabling them only if the new cmake flag is on. To keep some hydrology testing active in our nightlies, I could cook up a new test, with parameters perhaps not interesting from the modeling point of view, but that can run quickly. That way, we continue to test the feature (to detect answer changing mods), while keeping the nightly testing burden reasonable.

What do you think?

[ikalash]

I see. I can just increase the timeout time for now, if the tests are important.

[bartgol]

I think 26m are way too many. Tests should run in less than 5m in DEBUG, and less than 1m in RELEASE, imho. I will work on this hopefully today, if not, tomorrow.

[ikalash]

To be fair, the 26 mins is in the debug build. I agree with you, but I'm OK with keeping these on the longer side for now until you've had a chance to add additional configure options.

[bartgol]

Ok, that will make my life easier. I will try to get to this in the short term anyways though.