You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Last iteration of builds seemed to fail due to a variety of issues (mostly building Qt5, one failed to build GObject-Introspection, another fail during OpenFOAM sanity check).
I've reconfigured the bot to use larger instances (16-core with 32GB of RAM, was 8-core with 16GB of RAM), let's see if that helps...
New job on instance eessi-bot-citc-aws for architecture x86_64-amd-zen2 in job dir /mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.01/pr_195/3582
date
job status
comment
Jan 31 22:09:41 UTC 2023
submitted
job id 3582 awaits release by job manager
Jan 31 22:10:12 UTC 2023
released
job awaits launch by Slurm scheduler
Jan 31 22:11:31 UTC 2023
running
job 3582 is running
Jan 31 22:17:16 UTC 2023
finished
😁 SUCCESS tarball eessi-2021.12-software-linux-x86_64-amd-zen2-1675203381.tar.gz (0.000 GiB) in job dir
New job on instance eessi-bot-citc-aws for architecture x86_64-amd-zen3 in job dir /mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.01/pr_195/3583
date
job status
comment
Jan 31 22:09:43 UTC 2023
submitted
job id 3583 awaits release by job manager
Jan 31 22:10:09 UTC 2023
released
job awaits launch by Slurm scheduler
Jan 31 22:11:28 UTC 2023
running
job 3583 is running
Jan 31 22:16:10 UTC 2023
finished
😁 SUCCESS tarball eessi-2021.12-software-linux-x86_64-amd-zen3-1675203324.tar.gz (0.000 GiB) in job dir
Feb 01 07:18:10 UTC 2023
uploaded
transfer of eessi-2021.12-software-linux-x86_64-amd-zen3-1675203324.tar.gz to S3 bucket succeeded
New job on instance eessi-bot-citc-aws for architecture aarch64-generic in job dir /mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.01/pr_195/3584
date
job status
comment
Jan 31 22:09:44 UTC 2023
submitted
job id 3584 awaits release by job manager
Jan 31 22:10:06 UTC 2023
released
job awaits launch by Slurm scheduler
Jan 31 22:11:26 UTC 2023
running
job 3584 is running
Feb 01 01:49:12 UTC 2023
finished
😁 SUCCESS tarball eessi-2021.12-software-linux-aarch64-generic-1675215914.tar.gz (1.140 GiB) in job dir
Feb 01 07:18:04 UTC 2023
uploaded
transfer of eessi-2021.12-software-linux-aarch64-generic-1675215914.tar.gz to S3 bucket succeeded
New job on instance eessi-bot-citc-aws for architecture aarch64-graviton2 in job dir /mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.01/pr_195/3585
date
job status
comment
Jan 31 22:09:46 UTC 2023
submitted
job id 3585 awaits release by job manager
Jan 31 22:10:04 UTC 2023
released
job awaits launch by Slurm scheduler
Jan 31 22:11:25 UTC 2023
running
job 3585 is running
Jan 31 22:16:08 UTC 2023
finished
😢 FAILURE
Found slurm output slurm-3585.out in job dir
Slurm output lacks message "No missing modules!".
edit (@boegel): build of GObject-Introspection-1.68.0-GCCcore-10.3.0.eb failed, error unclear, will re-trigger and see if it's consistent
New job on instance eessi-bot-citc-aws for architecture aarch64-graviton3 in job dir /mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.01/pr_195/3586
date
job status
comment
Jan 31 22:09:48 UTC 2023
submitted
job id 3586 awaits release by job manager
Jan 31 22:10:00 UTC 2023
released
job awaits launch by Slurm scheduler
Jan 31 22:11:23 UTC 2023
running
job 3586 is running
Feb 01 00:56:40 UTC 2023
finished
😁 SUCCESS tarball eessi-2021.12-software-linux-aarch64-graviton3-1675212769.tar.gz (1.138 GiB) in job dir
Feb 01 07:18:48 UTC 2023
uploaded
transfer of eessi-2021.12-software-linux-aarch64-graviton3-1675212769.tar.gz to S3 bucket succeeded
New job on instance eessi-bot-citc-aws for architecture aarch64-graviton2 in job dir /mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.02/pr_195/3587
date
job status
comment
Feb 02 08:29:38 UTC 2023
submitted
job id 3587 awaits release by job manager
Feb 02 08:30:10 UTC 2023
released
job awaits launch by Slurm scheduler
Feb 02 08:31:12 UTC 2023
running
job 3587 is running
Feb 02 08:37:21 UTC 2023
finished
😢 FAILURE
Found slurm output slurm-3587.out in job dir
Slurm output lacks message "No missing modules!".
Slurm output lacks message about created tarball.
No tarball matching eessi-*software-*.tar.gz found in job dir.
New job on instance eessi-bot-citc-aws for architecture aarch64-graviton2 in job dir /mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.02/pr_195/3588
New job on instance eessi-bot-citc-aws for architecture aarch64-graviton2 in job dir /mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.02/pr_195/3607
I've reproduced the build problem on graviton2 manually, and this is the culprit for the failing build of GObject-Introspection-1.68.0-GCCcore-10.3.0.eb:
I've reproduced the build problem on graviton2 manually, and this is the culprit for the failing build of GObject-Introspection-1.68.0-GCCcore-10.3.0.eb:
New job on instance bot-aws-PR155 for architecture aarch64-graviton2 for repository EESSI-pilot in job dir /mnt/shared/home/bot/PR155/jobs/2023.03/pr_195/4169
New job on instance bot-aws-PR155 for architecture x86_64-generic for repository eessi-2021.12 in job dir /mnt/shared/home/bot/PR155/jobs/2023.03/pr_195/4170
date
job status
comment
Mar 17 01:27:51 PM UTC 2023
submitted
job id 4170 awaits release by job manager
Mar 17 01:28:33 PM UTC 2023
released
job awaits launch by Slurm scheduler
Mar 17 01:31:38 PM UTC 2023
running
job 4170 is running
Mar 17 01:38:54 PM UTC 2023
finished
😁 SUCCESS tarball eessi-2021.12-software-linux-x86_64-generic-1679060321.tar.gz (0.000 GiB) in job dir
I may found the culprit for the Illegal instruction error...
OpenBLAS/0.3.15-GCC-10.3.0 was not correctly built, it wasn't using the tweaked OpenBLAS easyblock from easybuilders/easybuild-easyblocks#1946, which caused DYNAMIC_ARCH=1 to not be used in the options to make.
The build of OpenBLAS/0.3.15-GCC-10.3.0 for aarch64/generic was done on a c7g-8xlarge instance (Graviton 3), so that effectively leads to an OpenBLAS/0.3.15-GCC-10.3.0 installation that is incompatible with Graviton 2...
That was meant to fixed by #211, but it seems like the rebuild for aarch64/generic was done incorrectly, not using the required --include-easyblocks-from-pr 1946.
It's a perfect example of why the builds should be fully automated with the bot, rather than being done by a human who's prone to make mistakes...
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
boegel
added
the
bot:deploy
checklist:
aarch64/generic(done, ingested)aarch64/graviton2(building...)aarch64/graviton3(done, ingested)ppc64le/generic(done, ingested)ppc64le/power9le(done, ingested)x86_64/generic(done, ingested)x86_64/amd/zen2(done, ingested)x86_64/amd/zen3(done, ingested)x86_64/intel/haswell(done, ingested)x86_64/intel/skylake_avx512(done, ingested)