Add Workflows, Fix Cycamore Build

[bennibbelink]

This replaces #544 and addresses #543. This PR adds testing workflows and fixes CMake issues.

The two testing workflows are adapted from Cyclus - one to push new images that have Cycamore and Cyclus installed, and one that pulls in the latest Cyclus image and builds/tests Cycamore against it.

The CMake fixes that I initially didn't think were necessary (since Cycamore doesn't use LibXML or SQLite) actually did need to be in place, since Cycamore is linked against libcyclus.so (build would fail on linking if I removed the Libxml and sqlite requirements)

[bennibbelink]

(@gonuke accidentally edited this instead of reply...)

I've omitted the nosetests for now, I can uncomment them but there were some import errors, and I think it may be less effort to just convert to pytest. Thoughts about this?

[bennibbelink]

I've omitted the nosetests for now, I can uncomment them but there were some import errors, and I think it may be less effort to just convert to pytest. Thoughts about this?

scratch that, I think its a problem with the environment. I'll try to get the python tests passing with nose before attempting a conversion to pytest

[gonuke]

We do want to move to pytest - we did on Cyclus. Nosetest is deprecated.

[bennibbelink]

Seeing an interesting segfault in the cycamore tests on the conda builds (apt builds pass all tests). You can replicate by simply running cyclus recycle.xml. Gdb output below:

...
Finished dumping matrix

Thread 1 "cyclus" received signal SIGSEGV, Segmentation fault.
0x00007ffff426d0ca in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_is_local (this=this@entry=0x55c956e44ad0) at /home/conda/feedstock_root/build_artifacts/gcc_compilers_1695216566116/work/build/x86_64-conda-linux-gnu/libstdc++-v3/include/bits/basic_string.h:264
264     /home/conda/feedstock_root/build_artifacts/gcc_compilers_1695216566116/work/build/x86_64-conda-linux-gnu/libstdc++-v3/include/bits/basic_string.h: No such file or directory.
(gdb) bt
#0  0x00007ffff426d0ca in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_is_local (this=this@entry=0x55c956e44ad0)
    at /home/conda/feedstock_root/build_artifacts/gcc_compilers_1695216566116/work/build/x86_64-conda-linux-gnu/libstdc++-v3/include/bits/basic_string.h:264
#1  0x00007ffff426d5b3 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::capacity (this=this@entry=0x55c956e44ad0)
    at /home/conda/feedstock_root/build_artifacts/gcc_compilers_1695216566116/work/build/x86_64-conda-linux-gnu/libstdc++-v3/include/bits/basic_string.h:1159
#2  0x00007ffff426d5e6 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_assign (this=0x55c956e44ad0, __str=...)
    at /home/conda/feedstock_root/build_artifacts/gcc_compilers_1695216566116/work/build/x86_64-conda-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc:279
#3  0x00007ffff0a6eac0 in CbcMain0(CbcModel&, CbcSolverUsefulData&) () from /opt/conda/lib/libCbcSolver.so.3
#4  0x00007ffff0a71932 in CbcMain0(CbcModel&) () from /opt/conda/lib/libCbcSolver.so.3
#5  0x00007ffff75bc6e1 in cyclus::SolveProg (si=0x555556d6deb8, greedy_obj=3.3000000000000002e+303, verbose=<optimized out>) at /cyclus/src/solver_factory.cc:131
#6  0x00007ffff75b8065 in cyclus::ProgSolver::SolveGraph (this=0x555556d2bb00) at /cyclus/src/prog_solver.cc:85
#7  0x00007ffff75a3aa1 in cyclus::ExchangeSolver::Solve (graph=0x555556d95b60, this=<optimized out>) at /cyclus/src/exchange_solver.h:45
#8  cyclus::ExchangeManager<cyclus::Material>::Execute (this=this@entry=0x7fffffffc530) at /cyclus/src/exchange_manager.h:57
#9  0x00007ffff75a49ec in cyclus::Timer::DoResEx (this=this@entry=0x7fffffffd180, matmgr=matmgr@entry=0x7fffffffc530, genmgr=genmgr@entry=0x7fffffffc540) at /cyclus/src/timer.cc:89
#10 0x00007ffff75a4bf8 in cyclus::Timer::RunSim (this=0x7fffffffd180) at /cyclus/src/timer.cc:36
#11 0x0000555555563755 in main (argc=<optimized out>, argv=<optimized out>) at /cyclus/src/sim_init.h:66

I'm not sure why there are references to /home/conda/feedstock_root..., have you seen any behavior like this before @gonuke? I've checked all the env vars for anything abnormal but everything I see references /opt/conda... like I would expect.

[gonuke]

I haven't seen this before, but it appears to be related to the CBC solvers in coin. We are getting those from the conda-forge channel, so maybe they are built against something that lives there? correctly? Incorrectly?

[gonuke]

It might be interesting to see if we test the Coin CBC solver in the Cyclus kernel tests and consider how it may be different from this test in Cycamore?

[bennibbelink]

I haven't seen this before, but it appears to be related to the CBC solvers in coin. We are getting those from the conda-forge channel, so maybe they are built against something that lives there? correctly? Incorrectly?

This has continued to stump me. Discussed it briefly at the hacker within meeting today and the consensus was that it is probably an issue with how that package is being built. I just opened an issue in the conda-forge feedstock repo, hopefully I get a response.

[bennibbelink]

It might be interesting to see if we test the Coin CBC solver in the Cyclus kernel tests and consider how it may be different from this test in Cycamore?

The CBC tests get skipped if MILPS is disabled. This appears to be the default install option, so I don't think we ever test the CBC solver in our CI.

Do you think this is a good time to tackle 1591 ?

[gonuke]

Adding code coverage should be high on the list, but it would be nice to have some build passing so we can use it for incoming PRs. We could temporarily back off the condo builds, for example.

[bennibbelink]

My current assessment of this issue: the problem is not with conda, but with the way that we operate the CBC Solver. I believe that we are operating the solver in an outdated way (using CbcMain0 and CbcMain1) and the current best practice is to use model.branchAndBound() to perform the solve. This would be consistent with the response I got from the issue I opened (that the segmentation fault is from user error, not environment configuration). I have struggled to get coincbc and cyclus built from source in the same environment to enable debugging and find the true source of the segmentation fault, but have a hunch that if we conform to current Cbc best practices then it will no longer be an issue.

I propose that in this PR I modify the python tests so that if MILPS is not enabled then we only test greedy solves (as it is in cyclus), then CI should be in a working state here in Cycamore. Additionally, an issue should be opened in Cyclus regarding the use Cbc, since it is not in a working state in Cyclus.

[bennibbelink]

Right now this does not include conda in the testing matrices, and all is passing on my fork. We could merge this now to get some CI in place, however if we wait to get #1613 merged in Cyclus, we can fill out the testing matrix in Cycamore and it will be more robust with respect to MILPS being enabled/disabled.

I'll defer to you @gonuke to make the call

[gonuke]

Let's get the cyclus merges in first, but I can't tell how 1613 and 1613 for together?

[bennibbelink]

I thought that I would separate the changes into 2 PRs, but I instead I need to just replace #1613 entirely with #1614 (including all the changes). I will get that done today and close 1613

[bennibbelink]

I manually re-ran the workflow in my fork. They now pull in the updated cyclus images with these COIN fixes, and all tests pass 👏👏

Marking this PR as ready for review

[gonuke]

Thanks @bennibbelink - this really helps get a stable footing for continued development

[gonuke]

Does this need to be staged as a branch on the upstream, at least for the build/publish workflow?

Also, I only see Circle tests here and no GitHub actions?

[bennibbelink]

Good point, I will make a branch on the upstream

[bennibbelink]

Closing in favor of #549