cheyenne and geyser quick start guide

Cheyenne and geyser Quick-Start Guide

Cheyenne/geyser users may follow these steps to get started in using the CESM python based post processing tools:

1. Set up your shell environment with variables POSTPROCESS_PATH and POSTPROCESS_PATH_GEYSER to point to the pre-installed virtualenv's and an alias to activate the virtualenv on cheyenne before running create_postprocess. This step only needs to be done once.

For tcsh users, in your .tcshrc add the following lines:

### CESM Postprocessing 
setenv POSTPROCESS_PATH /glade/p/cesm/postprocessing_ch
setenv POSTPROCESS_PATH_GEYSER /glade/p/cesm/postprocessing_geyser
setenv PATH /glade/apps/contrib/virtualenv/12.0.7:$PATH
alias cesm_pp_activate 'source $POSTPROCESS_PATH/cesm-env2/bin/activate.csh'

For bash or ksh users, in your .profile add the following lines:

### CESM Postprocessing 
export POSTPROCESS_PATH=/glade/p/cesm/postprocessing_ch
export POSTPROCESS_PATH_GEYSER=/glade/p/cesm/postprocessing_geyser
alias cesm_pp_activate='. $POSTPROCESS_PATH/cesm-env2/bin/activate'

2. Activate the virtualenv. Note: when the virtualenv is activated, your command line prompt will be prefixed by [cesm-env2].

module load python
cesm_pp_activate

3. Run create_postprocess from anywhere to add the postprocessing directory and files in an existing CESM caseroot or create a stand-alone post processing caseroot. Note - when specifying a postprocessing caseroot you need to specify a fully qualified path to the postprocessing caseroot.

create_postprocess --help

4. Deactivate virtualenv. This is optional. There should be no harm in leaving the virtualenv activated.

deactivate

5. `cd` to your post processing caseroot and modify the *.xml files to reflect your post processing configuration. Use the pp_config tool in your caseroot to query and change the XML settings to ensure the validity of any XML changes.

pp_config --help

Alternatively, if you edit an *.xml file directly, then you can validate it against an XML schema using the following command:

xmllint --schema env_file.xsd env_[filename].xml

6. Submit the scripts to the batch system

qsub ./[script_name]   (run on cheyenne)
sbatch ./[script_name]_geyser (submit on cheyenne to run on geyser)

Where script name is either timeseries, atm_averages, atm_diagnostics, ice_averages, ice_diagnostics, lnd_averages, lnd_regrid, lnd_diagnostics, ocn_averages, ocn_diagnostics, ilamb_initialize, iomb_initialize, ilamb_diagnostics, iomb_diagnostics, iconform or xconform.

You can submit multiple scripts concurrently but there are some dependencies that you need to be mindful of. For instance, the lnd_regrid script depends on the successful completion of the lnd_averages script. All the diagnostics scripts, except ilamb_diagnostics and iomb_diagnostics, depend on successful completion of the averages scripts. If you run the timeseries script first and want to use those files as inputs to the averages scripts, then the timeseries script needs to complete successfully.

7. Check for successful completion of submission scripts by looking at the batch script output files in $PP_CASE_PATH and then checking each of the log files in the $PP_CASE_PATH/logs directory.
8. Set up your ssh keys to a remote server that has access to a web storage data partition (note: NCAR users you can not post directly to a web server like cesmweb.cgd.ucar.edu but you can post to burnt.cgd.ucar.edu) so you can copy the plots and html files to a remote server without being prompted for a login/password. CGD provides a helper script to setup the ssh keys for you called `mk_user_ssh_keys.sh`. This script is not included with the post processing distribution so you will need to download it from the link below, copy it directly to your login account on yellowstone or cheyenne, and run it manually.

mk_user_ssh_keys.sh

9. Edit the env_postprocess.xml file and set the XML variables GLOBAL_WEBHOST, GLOBAL_WEBLOGIN, and GLOBAL_REMOTE_WEBDIR. For CGD web hosts, please follow the conventions for GLOBAL_REMOTE_WEBDIR settings at:

CGD Project Directory Structure

10. Run the copy_html utility from your PP_CASE_PATH:

./copy_html