Update ALCF Sunspot and Aurora machine configs #6553
Conversation
|
|
i was tagged to review this. i have no opinion regarding these modules (but removed ones seem not to be used in scream configs). Maybe, it is useful to list what cases you were able to build/run with which compilers for this change? |
There was a problem hiding this comment.
@amametjanov , is there a specific reason why you are using the older software stack? The latest available module on sunspot is oneapi/eng-compiler/2024.04.15.002
|
Yes, just a draft to get basics working. I'm testing |
|
@mahf708 the CI testing is running even though this a draft PR. Can we change that? |
|
I don't think there's a way to prevent CI from draft PRs. At least I've never seen it. |
Yes We Can @xylar, I think specifying the type will do, but like you, I have never seen it or used it... https://github.com/orgs/community/discussions/25722#discussioncomment-5281953 Or we can apparently query the draftness: https://stackoverflow.com/a/68349262/22990681. |
|
@rljacob: I think we may want to move to labels-based testing (especially if we implement the self-hosted solution). (@xylar, for reference, https://github.com/mahf708/test-gh-runner-chrys/actions/runs/10409947659 ran on a chrysalis login node) |
|
Good to know! |
They lead to link errors in currently available oneapi versions.
|
Testing on Sunspot:
|
|
@amametjanov , did you run this branch with oneapi-ifxgpu ? To clarify, I think there need to be a few small patches in order for i.e. F2010-MMF1 to run on oneapi-ifxgpu. However, I wanted to confirm whether this would be important for the scope of this PR? |
Also export ZES_ENABLE_SYSMAN=1 to avoid ext_intel_free_memory run-time errors.
Also avoid a call to variadic printf in a SYCL kernel within an error diagnostic log.
There was a problem hiding this comment.
@amametjanov , your changes look good to me.
I would suggest considering the following changes to the oneapi-ifxgpu config:
<MAX_MPITASKS_PER_NODE compiler="oneapi-ifxgpu">48</MAX_MPITASKS_PER_NODE>
<env name="ZEX_NUMBER_OF_CCS">0:4,1:4,2:4,3:4:4:4,5:4,6:4,7:4</env>
This would allow you to run in "4-CCS mode", where you can use 4 MPI ranks per PVC tile with minimal performance differences on the device side (assuming that work is balanced between ranks), but significant speed-up on the host due to the increased number of processes.
amametjanov
changed the title
There was a problem hiding this comment.
@amametjanov , I am seeing some errors on your branch running on aurora with multiple openmp threads. Removing the "--cpu-bind depth" option to mpiexec appears to resolve things. Otherwise, things are looking great!
I wanted to note that I will be on bonding leave over the next two months, so my participation during this period will be sporadic starting on Monday.
|
Notes: Aurora currently down for network upgrade. Will wait for it to come back up and test again. |
Update ALCF Sunspot and Aurora machine configs. [BFB]
Update ALCF Sunspot and Aurora machine configs.
[BFB]