Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update jobid definition for all J-jobs #535

Merged
merged 2 commits into from
Oct 21, 2024

Conversation

BenjaminBlake-NOAA
Copy link
Contributor

DESCRIPTION OF CHANGES:

  • This PR builds off of Utilize jobid variable for non-DA tasks #512 and Add taskid and run deterministic and EnKF DA cycles. #524
  • In the NCO implementation standards, $jobid needs to be used when setting the run directory names. $jobid is defined at the job card level as $job.$PBS_JOBID. $job is the <jobname> tag in Rocoto (and whatever the equivalent is in ecflow), and $PBS_JOBID is a sequence of letters/numbers that is specific to each individual task. In this PR, we introduce a temporary variable called $taskid which is defined at the job card level in the Rocoto xml, and is passed into the J-jobs where $jobid is set and used to define the run directory ($DATA=$DATAROOT/$jobid).
  • Note: At the moment we shouldn't switch entirely to using $jobid = $job.$PBS_JOBID because then other jobs in the workflow cannot access the run directories for other tasks. Once workflow changes are made to utilize the umbrella data directory structure described in issue directory structures: com, umbrella, $DATA #498 then we can shift to using $jobid = $job.$PBS_JOBID everywhere, and the temporary $taskid variable will be removed.
  • The following line is added to all J-jobs but is commented out for now, because we do not want to switch to the standard $jobid definition just yet (see note above): export jobid=${jobid:-${taskid}}

TESTS CONDUCTED:

Completed a successful spinup cycle with the DA engineering test. @hu5970 has offered to assist with testing these changes.

Machines/Platforms:

  • WCOSS2
    • Cactus/Dogwood
    • Acorn
  • RDHPCS
    • Hera
    • Jet
    • Orion
    • Hercules

Test cases:

  • Engineering tests
    • Non-DA engineering test
    • DA engineering test
      • Retro
      • Ensemble
      • Parallel
  • RRFS fire weather
  • RRFS_A:
  • RRFS_B:
  • RTMA:
  • Others:

ISSUE:

Copy link
Contributor

@MatthewPyle-NOAA MatthewPyle-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passes my eyeball test, so will get this merged so it can be tested.

@MatthewPyle-NOAA MatthewPyle-NOAA merged commit acea734 into NOAA-EMC:main Oct 21, 2024
2 checks passed
@BenjaminBlake-NOAA BenjaminBlake-NOAA deleted the feature/jobid branch December 3, 2024 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants