add CI workflow to run unit tests with Python 2 (again)

[boegel]

We stopped testing with Python 2 in #4267 because official support for using Python 2 was dropped in both GitHub Actions and actions/setup-python.

That has led to silently introducing bugs that make EasyBuild crash hard when running on top of Python 2 however, see for example #4328.

While support for running EasyBuild on top of Python 2 is deprecated (since EasyBuild 4.7.0 - Jan'23, cfr. #4136), we should do what we can to maintain compatibility with Python 2 until EasyBuild 5.0 is released.

[boegel]

Down to 4 failing tests, probably due to side-effects caused by running the test suite via sudo bash -l -c?

• test_regtest in test/framework/toy_build:

2023-08-17T08:42:56.8061932Z ERROR: test_regtest (test.framework.toy_build.ToyBuildTest)
2023-08-17T08:42:56.8062530Z Test use of --regtest.
2023-08-17T08:42:56.8063056Z ----------------------------------------------------------------------
2023-08-17T08:42:56.8063389Z Traceback (most recent call last):
2023-08-17T08:42:56.8063959Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/toy_build.py", line 2112, in test_regtest
2023-08-17T08:42:56.8064586Z     self.test_toy_build(extra_args=['--regtest', '--sequential'], verify=False)
2023-08-17T08:42:56.8065101Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/toy_build.py", line 202, in test_toy_build
2023-08-17T08:42:56.8065582Z     self.assertEqual(read_file(os.path.join(toy_install_path, 'README')), "TOY\n")
2023-08-17T08:42:56.8066105Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/easybuild/tools/filetools.py", line 219, in read_file
2023-08-17T08:42:56.8066465Z     raise EasyBuildError("Failed to read %s: %s", path, err)
2023-08-17T08:42:56.8067350Z EasyBuildError: "Failed to read /tmp/eb-TDA5dZ/eb-Qr5M24/eb-6TItku/tmp6YmlCM/software/toy/0.0/README: [Errno 2] No such file or directory: '/tmp/eb-TDA5dZ/eb-Qr5M24/eb-6TItku/tmp6YmlCM/software/toy/0.0/README'"

• test_toy_lock_cleanup_signals in test/framework/toy_build

2023-08-17T08:42:56.8068065Z ERROR: test_toy_lock_cleanup_signals (test.framework.toy_build.ToyBuildTest)
2023-08-17T08:42:56.8068404Z Test cleanup of locks after EasyBuild session gets a cancellation signal.
2023-08-17T08:42:56.8068784Z ----------------------------------------------------------------------
2023-08-17T08:42:56.8069206Z Traceback (most recent call last):
2023-08-17T08:42:56.8069714Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/toy_build.py", line 3648, in test_toy_lock_cleanup_signals
2023-08-17T08:42:56.8070081Z     raise_error=True, testing=False, raise_systemexit=True)
2023-08-17T08:42:56.8070586Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/easybuild/base/testing.py", line 170, in assertErrorRegex
2023-08-17T08:42:56.8070908Z     call(*args, **kwargs)
2023-08-17T08:42:56.8071365Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/toy_build.py", line 194, in test_toy_build
2023-08-17T08:42:56.8071651Z     raise myerr
2023-08-17T08:42:56.8072254Z EasyBuildError: "Undefined build option: 'silent'. Make sure you have set up the EasyBuild configuration using set_up_configuration() (from easybuild.tools.options) in case you're not using EasyBuild via the 'eb' CLI."

• test_download_instructions in test/framework/easyblock:

2023-08-17T08:42:56.8073001Z FAIL: test_download_instructions (test.framework.easyblock.EasyBlockTest)
2023-08-17T08:42:56.8073320Z Test use of download_instructions easyconfig parameter.
2023-08-17T08:42:56.8073693Z ----------------------------------------------------------------------
2023-08-17T08:42:56.8073959Z Traceback (most recent call last):
2023-08-17T08:42:56.8074452Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/easyblock.py", line 1532, in test_download_instructions
2023-08-17T08:42:56.8074861Z     self.assertErrorRegex(EasyBuildError, error_pattern, eb.fetch_step)
2023-08-17T08:42:56.8075396Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/easybuild/base/testing.py", line 178, in assertErrorRegex
2023-08-17T08:42:56.8076270Z     self.assertTrue(regex.search(msg), "Pattern '%s' is found in '%s'" % (regex.pattern, msg))
2023-08-17T08:42:56.8077886Z AssertionError: None is not true : Pattern '^Couldn't find file software_with_missing_sources-0.0.tar.gz anywhere, and downloading it didn't work either' is found in 'Failed to create directory /tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/sandbox/sources/s/software_with_missing_sources: [Errno 13] Permission denied: '/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/sandbox/sources/s''

• test_reproducibility in test/framework/toy_build:

2023-08-17T08:42:56.8079302Z FAIL: test_reproducibility (test.framework.toy_build.ToyBuildTest)
2023-08-17T08:42:56.8090104Z Test toy build produces expected reproducibility files
2023-08-17T08:42:56.8090574Z ----------------------------------------------------------------------
2023-08-17T08:42:56.8090844Z Traceback (most recent call last):
2023-08-17T08:42:56.8091349Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/toy_build.py", line 2145, in test_reproducibility
2023-08-17T08:42:56.8091895Z     self.test_toy_build(extra_args=['--minimal-toolchains', '--easyblock=EB_toytoy', '--hooks=%s' % hooks_file])
2023-08-17T08:42:56.8092459Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/toy_build.py", line 197, in test_toy_build
2023-08-17T08:42:56.8093060Z     self.check_toy(self.test_installpath, outtxt, name=name, versionsuffix=versionsuffix, error=myerr)
2023-08-17T08:42:56.8093615Z   File "/tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/toy_build.py", line 125, in check_toy
2023-08-17T08:42:56.8094114Z     self.assertTrue(success.search(outtxt), "COMPLETED message found in '%s'%s" % (outtxt, error_msg))
2023-08-17T08:42:56.8094650Z AssertionError: None is not true : COMPLETED message found in '
...
2023-08-17T08:42:56.9565365Z == 2023-08-17 08:17:51,527 build_log.py:171 ERROR EasyBuild crashed with an error (at easybuild/base/exceptions.py:126 in __init__): Build of /tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/easyconfigs/test_ecs/t/toy/toy-0.0.eb failed (err: "build failed (first 300 chars): Can't determine patch level for patch /tmp/296fcace019d5686f9da9785b4804a57149fd952/lib/python2.7/site-packages/test/framework/sandbox/sources/toy/toy-0.0_fix-silly-typo-in-printf-statement.patch from directory /tmp/eb-TDA5dZ/eb-Qr5M24/eb-xoEv4h/tmpjWHDt8/toy/0.0/system-system") (at easybuild/main.py:175 in build_and_install_software)
2023-08-17T08:42:56.9565450Z '

[boegel]

As suggested by @branfosj, I've skipped the 4 tests that are broken when running on top of Python 2 for now in c3fc220, so we can get this merged rather than spending more time to figure out why the tests are only broken with Python 2.

Very likely the problem with those tests is specific to using Python 2 in a nice feature of EasyBuild (like --regtest), which we don't care much about since support for using Python 2 has been deprecated since EasyBuild v4.7.0 (Jan'23) and the upcoming EasyBuild 5.x (ETA: by end of 2023, hopefully) will no longer support Python 2...