General config and run_reframe.sh for local and EESSI stack

[laraPPr]

The updates to the run_reframe.sh scripts as they will pickup the default enviroment which does:
unset MODULEPATH and sets EESSI enviroment

[casparvl]

Should we have an alternative to load a local module environment? Might not be needed on your systems. On our system we'd have to load a meta-module (e.g. 2023, or 2024) to make a software environment available.

[laraPPr]

On our system we have sticky module environments per partition/cluster. So right now I do that in the wrapper of the tun_reframe_wrapper.sh. But I can look into moving that to the hortense_local_ss/ci_config.sh

[laraPPr]

What could work is that we add something like LOCAL_CI_MODULE_ENV=2024 and in our case LOCAL_CI_MODULE_ENV=cpu_rome_256gb. Now I'm also hardcoding that because the fix in ReFrame is not part of a release yet. I'll add that now now and than you can see if it works.

[casparvl]

It would be good to print some more of these variables in the section that also prints reframe config etc. Makes things easier to debug.

[laraPPr]

I've now added these two

echo "Using EESSI: ${USE_EESSI_SOFTWARE_STACK}"
echo "MODULEPATH: ${MODULEPATH}"

[casparvl]

Some changes are needed to the run_reframe.sh to make things work on surf_snellius. I'll leave testing/reviewing the hortense-specific stuff to @smoors

[laraPPr]

Tested and working for local software stack is not gonna work for the EESSI software stack because it won't find the mympirun laucher when testing the EESSI modules. On tier-2 where the CI is already running I have two seperate configs because of this but maybe their is also another way?

[laraPPr]

Tested and working for local software stack is not gonna work for the EESSI software stack because it won't find the mympirun laucher when testing the EESSI modules. On tier-2 where the CI is already running I have two seperate configs because of this but maybe their is also another way?

The solution for this is actually in this pr. Completely forgot I did this.

[laraPPr]

I'm not sure if I also need this in the job environment. loading these in run_reframe.sh is probably all I need. Gonna test that on our tier2 because I have not seen this cause trouble on hortense.

[laraPPr]

@casparvl ready for another review

[laraPPr]

@smoors Feel free to test it yourself on Hortense. It is not gonna pick up the new config but you can set export RFM_CONFIG_FILES="PATH/TO/TEST-SUITE/config/vsc_hortense.py" before running the run_reframe_wrapper.sh.

Testing I set the following environment variables when testing on my account:
EESSI

export EESSI_CI_SYSTEM_NAME=hortense_EESSI_ss
export EESSI_CI_LOGDIR=PATH/TO/CI_logs
export TEST_SUITE_PARTITION=cpu_rome_256
export RFM_CONFIG_FILES="PATH/TO/test-suite/config/vsc_hortense.py"

Local

export EESSI_CI_SYSTEM_NAME=hortense_local_ss
export EESSI_CI_LOGDIR=$VSC_SCRATCH_WORK/CI_logs
export TEST_SUITE_PARTITION=cpu_rome_256
export RFM_CONFIG_FILES="PATH/TO/test-suite/config/vsc_hortense.py"

[smoors]

using the EESSI modules is works fine, but with the local modules i get the error below, which can be fixed by using the .format() method (see my suggestions)

  File "/data/brussel/100/vsc10009/reframe/eessitestsuite/test-suite/config/vsc_hortense.py", line 71, in <module>
    'modules': [mpi_module % 'cpu_rome'],
TypeError: not all arguments converted during string formatting

[smoors]

lgtm

requested changes done