Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create cupid workflow #176

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from
Draft

Create cupid workflow #176

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
298c95c
Create cupid workflow
Jul 19, 2024
4834b4c
Get rid of code that loads conda and add code that will ensure the co…
Jul 24, 2024
59b6785
Merge tag 'ccs_config_cesm1.0.6' into add_cupid_workflow
mnlevy1981 Sep 7, 2024
95ae795
Move generate_cupid_config_file into CUPiD
mnlevy1981 Sep 7, 2024
cb6b713
Update template.cupid call to generate config
mnlevy1981 Sep 8, 2024
bbfb133
Changed API for CUPiD script
mnlevy1981 Sep 9, 2024
737fb64
Merge tag 'ccs_config_cesm1.0.8' into add_cupid_workflow
mnlevy1981 Nov 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions machines/casper/config_machines.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
<OS>LINUX</OS>
<COMPILERS>nvhpc,intel</COMPILERS>
<MPILIBS>openmpi</MPILIBS>
<CIME_OUTPUT_ROOT>/glade/scratch/$USER</CIME_OUTPUT_ROOT>
<CIME_OUTPUT_ROOT>/glade/derecho/scratch/$USER</CIME_OUTPUT_ROOT>
<DIN_LOC_ROOT>$ENV{CESMDATAROOT}/inputdata</DIN_LOC_ROOT>
<DIN_LOC_ROOT_CLMFORC>/glade/p/cgd/tss/CTSM_datm_forcing_data</DIN_LOC_ROOT_CLMFORC>
<DOUT_S_ROOT>$CIME_OUTPUT_ROOT/archive/$CASE</DOUT_S_ROOT>
Expand Down Expand Up @@ -108,7 +108,7 @@
<environment_variables>
<env name="MODULEPATH">/glade/u/apps/dav/modulefiles/default/compilers:/glade/u/apps/dav/modulefiles/default/idep</env>
<env name="OMP_STACKSIZE">256M</env>
<env name="TMPDIR">/glade/scratch/$USER</env>
<env name="TMPDIR">/glade/derecho/scratch/$USER</env>
<env name="CESMDATAROOT">/glade/p/cesmdata/cseg</env>
<env name="NETCDF_PATH">$ENV{NETCDF}</env>
</environment_variables>
Expand Down
13 changes: 13 additions & 0 deletions machines/config_workflow.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,19 @@
</job>
</workflow_jobs>

<workflow_jobs id="case.cupid" prepend="default">
<job name="case.cupid">
<template>template.cupid</template>
<dependency>case.st_archive</dependency>
<prereq>1</prereq>
<runtime_parameters MACH="derecho">
<task_count>1</task_count>
<tasks_per_node>1</tasks_per_node>
<walltime>0:20:00</walltime>
</runtime_parameters>
</job>
</workflow_jobs>

<!-- THIS CREATES THE JOB FOR THE LAUNCHING OF THE SMARTSIM ORCHESTRATOR -->
<workflow_jobs id="smartsim_db" prepend="default">
<job name="smartsim_db"> <!-- name here corresponds with scripts -->
Expand Down
1 change: 1 addition & 0 deletions machines/derecho/config_machines.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@
<command name="load">cesmdev/1.0</command>
<command name="load">ncarenv/23.09</command>
<command name="purge"/>
<command name="load">conda/latest</command>
<command name="load">craype</command>
</modules>
<modules compiler="intel">
Expand Down
10 changes: 10 additions & 0 deletions machines/template.cupid
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/bin/bash

# Batch system directives
{{ batchdirectives }}

module load conda
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jedwards4b even though @ingridc2051 added module load conda/latest to the derecho config_machines.xml file, without this line she gets a CondaError: Run 'conda init' before 'conda activate' error from the conda activate cupid-dev line. Do you happen to know why we seemingly need to load the module twice?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect that it's because the cesm environment is not being loaded by your script - let me look into how to do that and get back to you.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jedwards4b What if we replace the module load conda with source .env_mach_specific.sh? That should get everything from env_machines.xml into the environment of the compute node, right? @ingridc2051 can you try that when you get a chance?

- module load conda
+ source .env_mach_specific.sh

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - that should work - good idea.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It sounds like this worked given the test that @ingridc2051 performed.

conda activate cupid-dev
./generate_cupid_config_file.py

cupid-run