Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use proper Python dependency for OTF2 #21325

Merged

Conversation

Thyre
Copy link
Contributor

@Thyre Thyre commented Sep 5, 2024

Add Python as a dependency to OTF2 versions to ensure that the toolchain Python is used. To verify, a sanity check is added which ensures that the OTF2 Python module can be loaded. As six is required, newer toolchains also need Python-bundle-PyPI.

Resolves #21266

Add Python as a dependency to OTF2 versions to ensure that the toolchain
Python is used. To verify, a sanity check is added which ensures that
the OTF2 Python module can be loaded. As 'six' is required, newer
toolchains also need Python-bundle-PyPI.

Signed-off-by: Jan André Reuter <[email protected]>
@Thyre Thyre changed the title {perf} OTF2: Toolchain Python instead of system {perf} OTF2: Use toolchain Python instead of system one Sep 5, 2024
@SebastianAchilles SebastianAchilles added this to the release after 4.9.2 milestone Sep 6, 2024
@SebastianAchilles
Copy link
Member

@boegelbot please test @ jsc-zen3

@boegelbot
Copy link
Collaborator

@SebastianAchilles: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=21325 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_21325 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 4814

Test results coming soon (I hope)...

- notification for comment with ID 2334447417 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Collaborator

Test report by @boegelbot
SUCCESS
Build succeeded for 7 out of 7 (6 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.4, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18
See https://gist.github.com/boegelbot/08ec7e6407ed12dedb65680e4f96b0a7 for a full test report.

@SebastianAchilles
Copy link
Member

@boegelbot please test @ generoso
CORE_CNT=16

@boegelbot
Copy link
Collaborator

@SebastianAchilles: Request for testing this PR well received on login1

PR test command 'EB_PR=21325 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs /opt/software/slurm/bin/sbatch --job-name test_PR_21325 --ntasks="16" ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 14209

Test results coming soon (I hope)...

- notification for comment with ID 2334461861 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link
Collaborator

Test report by @boegelbot
SUCCESS
Build succeeded for 6 out of 6 (6 easyconfigs in total)
cnx1 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/122d6b325a7eced087d436631559ddd4 for a full test report.

Copy link
Member

@SebastianAchilles SebastianAchilles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@SebastianAchilles
Copy link
Member

Going in, thanks @Thyre!

@SebastianAchilles SebastianAchilles merged commit 6e5a0ea into easybuilders:develop Sep 6, 2024
9 checks passed

sanity_check_commands = ['%(namelower)s-config --help']
sanity_check_commands = ['%(namelower)s-config --help',
'python -c "import %(namelower)s"']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest to use -s here as otherwise the $HOME packages might be used causing failures/success:

-                         'python -c "import %(namelower)s"']
+                         'python -s -c "import %(namelower)s"']

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it makes sense to include that flag, just to make sure that nothing breaks because of user packages. Do you want to make a follow-up PR? I probably don't have the time for that today.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done: #21362 which also does the same for other existing ECs

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Flamefire This is relevant for several easyblocks too where we use python -c?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it is

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@boegel boegel changed the title {perf} OTF2: Use toolchain Python instead of system one use proper Python dependency for OTF2 Sep 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

OTF2 is using system Python instead of Python version of toolchain
5 participants