-
Notifications
You must be signed in to change notification settings - Fork 118
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Community Fire Behavior Model #1139
Draft
mkavulich
wants to merge
59
commits into
ufs-community:develop
Choose a base branch
from
esmf-org:feature/ufs_fire_workflow
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Add Community Fire Behavior Model #1139
mkavulich
wants to merge
59
commits into
ufs-community:develop
from
esmf-org:feature/ufs_fire_workflow
+692
−203
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…essfully, but won't run correctly.
cleanup create_nems_configure_file.py to use arguments rather than environment variables, add fire capabilities
- Add remainder of configurable fire namelist options to config_defaults - Add fire namelist filenames and paths to config_defaults - Add LEVP (number of vertical levels) as a config variable, and explicitly set levp and npz as namelist variables. This will require some updates to documentation - Generate the majority of the UFS_FIRE namelist in the workflow generation script. The rest (start and end times) will be set in the run_fcst step to account for cycling
- Needed correct naming template for input ICS and LBCS in config.fire.yaml - Incorrectly formatted lines in create_nems_configure_file.py - Fix incorrect namelist location for levp - Forgot to ignore FIRE_INPUT_DIR when creating &fire namelist
- Fill in &time namelist section - Link namelist.fire to namelist.input for now (until we can get the name changed) - Link in geo_em file Stuck on namelist errors for now due to old weather model hash.
…file does not exist. This avoids weird error handling from calling an argparse error class.
- Force bash to output decimal numbers when creating fire namelist - Remove rogue ln command
- Create a new variable FIRE_NUM_TASKS to choose the number of tasks to assign to the fire behavior model. - Modify PE_MEMBER01 so it accounts for FIRE_NUM_TASKS if set - Check that FIRE_NUM_TASKS is positive if running the fire model - Remove last remnants of old namelist variables - Update nems.configure to assign weather model to first petlist, then the fire tasks last
- Remove deprecated namelist options "fire_grows_only" and "fire_lfn_ext_up" - Add functionality to create_diag_table_file.py that gives the ability to add additional entries to an existing diag_table - When UFS_FIRE is active, add line to diag_table for "fsmoke" tracer
…_workflow Resolved Conflicts: Externals.cfg parm/ufs.configure ush/create_ufs_configure_file.py ush/generate_FV3LAM_wflow.py ush/get_crontab_contents.py ush/set_namelist.py
…ack where it should be!
…ile. This gives ~5x speedup for create_symlink_to_file on derecho
…l (e.g. {fyyyy}, {fhh}, {fyyyymmdd}, etc) to supplement the current templates based on cycle date
…te to build library versions
- Rename ufs_srweather_app.settings --> build_settings.yaml, make yaml format for easy reading - Add "Application" and "Git hash" to build_settings.yaml - Read build_settings.yaml in setup.py to determine if App was built with fire capabilities
…hreading in current develop
… made this improvement months ago!
mkavulich
force-pushed
the
feature/ufs_fire_workflow
branch
from
October 16, 2024 14:27
60eb0a9
to
4076da5
Compare
… to run these tests locally...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Note: This PR is in draft mode while we investigate issues with the latest weather model (unrelated to this development) that are causing some tests to fail; see #1136 for details.
DESCRIPTION OF CHANGES:
This PR introduces the Community Fire Behavior Module (ufs-community/ufs-weather-model#2220) to the SRW App. In addition, there are a number of general improvements to the UFS SRW code and workflow. All of these changes are described in more detail below.
Community Fire Behavior Module
Documentation/updating the Users Guide is a work in progress, but the basics are that the fire build requires an additional build flag
and has an additional
fire:
section inconfig.yaml
. There is an example fire configuration available in the new example configconfig.fire.yaml
. Many settings are described in detail inconfig_defaults.yaml
, but I am working on additional details.Sample data for the case in the example file has been staged on Derecho (/glade/derecho/scratch/kavulich/FIRE/2024/test_data) and Hera (/scratch2/BMC/fv3lam/kavulich/UFS/workdir/UFS_FIRE/test_data). I will reach out to EPIC to see if they would be willing to officially stage this sample data on all Level 1 platforms for community use and potential future tests. This PR does not include a WE2E test because the testing system does not currently have a neat way to handle multiple build types, but in theory the new example case could be used.
Two new pre-defined domains
SUBCONUS_CO_1km
andSUBCONUS_CO_3km
, for use with high-resolution fire experiments, are added.build_settings.yaml
In
develop
there is currently no way to reference build settings at configure and run time. There was a vestigial capability in theufs_srweather_app.settings.in
which was added 4+ years ago, but it has never really been used or updated and it is in an ad-hoc format that cannot be easily used. I have leveraged this existing capability to create a new file,build_settings.yaml
in the build directory that contains useful build information (the application built, the compiler, the git hash, etc.) and can be easily read using uwtools in the existing workflows and run scripts. Specifically this is used in this PR to ensure that the app has been built with the fire behavior capability if fire options are selected, but this should prove very useful in other parts of the workflow going forward; for just a few examples:This new file and the unused (but potentially useful in the future)
ufs_srweather_app_meta.h.in
files are moved to thesorc
directory to keep the source code more organized and uncluttered.Improve capability to use a different set of vertical levels
Currently the instructions for running SRW with a different number of vertical levels from the default 65 is quite convoluted, involving the need to manually edit the template namelist file. This PR adds LEVP as a configuration variable to more easily allow for runs with different vertical levels. The users guide instructions have been updated accordingly.
Flexible config file name
Now rather than being forced to use the name
config.yaml
for your experiment config, you can reference any yaml file by using the-c
option for workflow generation:With these changes,
EXPT_CONFIG_FN
is removed as a configurable option, but this was uselessly self-referential anyway.Additional options for
retrieve_data.py
A big limitation in specifying filenames in the current app is that there is no capability for filenames to change based on the forecast hour vs the cycle hour. Therefore a new set of variables, identical to the originals but with the prefix
f
, can now be used when specifying filenames with respect to the forecast hour rather than the cycle hour.An example of this new usage can be seen in the newly added
config.fire.yaml
experiment:This instructs
retrieve_data.py
to look for filenames specific to the valid forecast hour for the specific lateral boundary condition file. So for forecast hour zero (valid 2020081318), the expected filename will be20200813.hrrr.t18z.wrfprsf00.grib2
. But at the 18 hour forecast, the LBC filename will be20200814.hrrr.t12z.wrfprsf00.grib2
. This was not previously possible without these new variables, and represents a vast improvement in flexibility for users.Various random improvements
SyntaxWarning: invalid escape sequence
boolify
to get significant speedup for this frequently used functionFC
variable, which is needed by the Fire Behavior component.Type of change
TESTS CONDUCTED:
A number of fundamental test runs have been conducted on Derecho, Hera, and Orion, both with and without the ATMF build. Not all are passing, but seems to be unrelated to this development; see the note at the top. This section will be updated once those issues are resolved.
DEPENDENCIES:
Pending resolution of whether this or #1136 will be merged first
DOCUMENTATION:
Updates in progress.
ISSUE:
None
CHECKLIST
CONTRIBUTORS (optional):
Thanks to the whole Fire Behavior team for bringing me on to this project, and their extensive efforts to get this capability ready for use by the scientific community: @danrosen25 @masih-e @pedro-jm and @mefrediani