Inquiry on SPINUP_LOOPS Parameter Settings and Parallel Computing Tutorial

[ZRYings]

Hi, I am planning to conduct simulations over a large area spanning 55 years. (1) Before the official simulation, I aim to conduct a spinup using the first 5 years of forcing data as a baseline. Could you kindly advise on how the SPINUP_LOOPS parameter should be set to achieve this?
(2) In addition, would it be possible to provide me a tutorial on setting up parallel computing as mentioned in this link (https://github.com/NCAR/hrldas/blob/master/tutorial/Note2_2D_NLDAS2_domain.ipynb)?

Thank you very much for your time!

[cenlinhe]

Hi, to answer your question:
(1) based on my understanding, you plan to use the first 5-year of your forcing to run a 5-year spin-up, right? Are you going to discard the first 5-year and only use the later 50 years' data? If so, you do not need to set up SPINUP_LOOPS (i.e., put it as the default value, SPINUP_LOOPS = 0) and just run a continuous 55-year simulation and not using the first 5-year data. Did I mis-understand what you want to do?
(2) The model already had parallel computing capabilities. To activate it, it really depends on your own machine setup (e.g., use parallel compiler such as mpi instead of ifort). Once you choose the parallel mode for your machine environment and compiler, then when you select the model during ./configure setup, choose the compiler option consistent with your machine environment (e.g., "Linux ifort compiler MPI" if using intel parallel compiler). Then edit the generated user_build_options file for netcdf and Jasper software path. And then compile the code. You can follow this instruction: https://github.com/NCAR/hrldas/blob/master/tutorial/Note0_Download_Compile.ipynb

[ZRYings]

Hi, thank you very much for your reply. (1) I apologize for the confusion regarding the SPINUP_LOOPS setting. I would like to use simulation data for the entire 55 years, but I found it is very time-consuming if setting SPINUP_LOOPS to a number n, cause the model runs the 55-year data (n+1) times. So, I am thinking to first conduct a spin-up using just the initial 5-year data and then run the simulation for the remaining 50 years. Would it be okay? If so, should I perform two separate runs? For example, in the first run, set the simulation duration to kday=365*5 and set SPINUP_LOOPS to the desired number, saving the last day as a restart file. For the second run, use this restart file and set SPINUP_LOOPS to 0 to simulate the remaining 50 years. I hope I have made myself clear, and I look forward to your advice.

Secondly, we have set up an MPI parallel environment according to the tutorial. But I saw section 4.2 in this link (https://github.com/NCAR/hrldas/blob/master/tutorial/Note2_2D_NLDAS2_domain.ipynb) mentioned that 'for larger domains and longer simulation times, it's encouraged to submit jobs to a supercomputer, e.g., Cheyenne, using a job submission script called runhrldas.csh and submitted via qsub runhrldas.csh'. Could you please share the script of the runhrldas.csh?

Thank you and looking forward to your reply.

[cenlinhe]

(1) How many total years do you want to do your spin-up? If you only need 5-year spin-up, then you do not need two separate runs. You can do a continuous 55-year simulation (SPINUP_LOOPS = 0), and then only analyze your recent 50-year results. If you need, for example, 15-year spin up, then you need two separate runs, the first run using the first 5-year forcing to run the model by setting SPINUP_LOOPS = 3, and use the produced restart file for the second 50-year run. You can set restart file output frequency to be daily.
(2) Our runhrldas.csh only works in our NCAR supercomputer environment. Are you running the model in NCAR supercomputer? It has to be specific about your own machine. Different machines use different job submission systems. You may want to check with your IT about the job submission system.

[ZRYings]

Hi, thank you very much for your detailed guidance.

[sudarsan3010]

Hi @cenlinhe,

For my study area, I plan to run the spinup several times for one particular initial year (say 2008) until the model gets stabilized. My query is that if I give the SPINUP_LOOPS = 20, will all the 20 simulation outputs be saved separately or will it be overwritten by the last run's output?

I am asking this because I need to check with the previous year's output and compute the percentage change.

[cenlinhe]

Yes, the output files will all have different names for each spinup loop with a suffix of loop number "_loop0001 ", like yyyymmddhh.LDOUT_DOMAIN1.loop*. However, the restart file will be overwritten after each loop to get updated.

[sudarsan3010]

Dear @cenlinhe,

Thank you so much for your quick reply. Kindly address the following querries as well.

I have created forcing for 15 years from 2008 to 2023 and the variables are present in LDASIN directory with a setup file 'HRLDAS_setup_2008010100_d1'.

1. I am planning to spinup the model over the year of 2008 for 'n' times, and start the simulation from 2009. In this case, the setup file can be same as that of 'HRLDAS_setup_2008010100_d1' or should I need a separate setup file for 2009?

2. Other than changing the below line, what are the other changes is to be made to run the model in 'Restart' mode.
   ! RESTART_FILENAME_REQUESTED = "/glade/scratch/cenlinhe/test/hrldas_output/RESTART.2000010100_DOMAIN1"