Jgfouca/scream downstream 04 11 24 #6343

… sees

…-SCREAMv1 Automatically Merged using E3SM Pull Request AutoTester PR Title: Add F20TR-SCREAMv1 compset PR Author: brhillman PR LABELS: AT: AUTOMERGE

…fig/20240227 Automatically Merged using E3SM Pull Request AutoTester PR Title: Update machine config for Livermore Computing machines PR Author: AaronDonahue PR LABELS: BFB, Machine File, AT: AUTOMERGE, AT: Skip Stand-Alone Testing, AT: Skip v1 Testing

…-units Automatically Merged using E3SM Pull Request AutoTester PR Title: set_string for some radiation diagnostics PR Author: mahf708 PR LABELS: radiation, AT: AUTOMERGE, diagnostic

Use standard script format. Some minor fixes. Make sure it will work with py3.6.

check-hashes-ers: Bring in to eamxx/scripts ecosystem

Doing it in the baseline dir seems like a bad idea. One, potential races with other TAS runs. Two, on some machines like pmgpu, the baseline filesystem is MUCH slower than the scratch area.

…ng-add-logging Automatically Merged using E3SM Pull Request AutoTester PR Title: Make nudging log which file it's reading PR Author: bartgol PR LABELS: enhancement, I/O, AT: AUTOMERGE, nudging

…utput-map-files-to-inputdata Automatically Merged using E3SM Pull Request AutoTester PR Title: In buildnml, add map files needed for output to inputdata list PR Author: bartgol PR LABELS: AT: AUTOMERGE, scripts, CIME, AT: Skip v1 Testing

Keep the build if it fails.

…-cosp-cloud Automatically Merged using E3SM Pull Request AutoTester PR Title: Fix handling of cloud fraction for COSP PR Author: brhillman PR LABELS: AT: AUTOMERGE

Automatically Merged using E3SM Pull Request AutoTester PR Title: Change Implementation of get_vector_dim() in FieldLayout PR Author: mjs271 PR LABELS: AT: AUTOMERGE

…tamp-nstep-in-nc-file Automatically Merged using E3SM Pull Request AutoTester PR Title: Fix how we read/write timestamp to file PR Author: bartgol PR LABELS: I/O, AT: AUTOMERGE, bugfix

…_to_workdir test-all-scream: Move baseline build dir

…story The start ref in that case must be m_run_t0, not case_t0

…ogger-in-time-interp Automatically Merged using E3SM Pull Request AutoTester PR Title: In time interp util, set logger in new reader when we shift file PR Author: bartgol PR LABELS: AT: AUTOMERGE, bugfix

…dds MAM4xx CIME test to AT test suite

* origin/master: (25 commits) EAMxx: in time interp util, set logger in new reader when we shift file Fix mistakes EAMxx: fix bug in has_attribute scorpio util EAMxx: fix next/last write ts handling correct a couple of typos change function of get_vector_dim() and add get_vector_component_idx() Minor rethink of baseline situation Switch to cldfrac_rad for cosp_standalone test-all-scream: Move baseline build dir EAMxx: in buildnml, add map files needed for output to inputdata list EAMxx: allow to set a logger in the time interp utility EAMxx: better format logging in scorpio input/output EAMxx: read/write timestamp's nstep only if asked to do so EAMxx: change default value for TimeStamp::m_num_steps to very neg value EAMxx: read timestamp's nsteps only if present on file EAMxx: add possibility of querying whether a PIO file has an attribute Clean up condition and add documentation about unset fields Add doc string for do_subcol_sampling Add doc for cosp_subcolumns Default to no cosp subcolumns for ne256 and above ...

- additions to namelist defaults - SPA remap file to check "None" for IOP

assign default namelist based on compset

- Reload IOP data when initializing restart - Remove TODO comment (computing Q from Qdp *is* required)

…te_cime_case Automatically Merged using E3SM Pull Request AutoTester PR Title: Changes for running through CIME PR Author: tcclevenger PR LABELS: AT: AUTOMERGE, DP-SCREAM

…module Makes test_all_scream shorter to parse and separate concerns

It is much simpler than creating custom targets and using make Also, it allows to use resource-spec file, to efficiently split all baselines tests across resources

…cs_rrtmgp and dynamics_physics/homme_mam_optics.

There were a couple of copy+paste errors taking the change a different branch. Also, rrtmgp baseline generation was running at both baseline generation and test time.

…when-skipping-output-restart Automatically Merged using E3SM Pull Request AutoTester PR Title: Fix bug in IO when doing a restarted run without restarting history PR Author: bartgol PR LABELS: I/O, AT: AUTOMERGE, bugfix

…ST=ON

check-hashes-ers: Minor change so works with multi-case tests like PEM

…4_02_21 Jgfouca/upstream merge 2024 02 21

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4.7.0 to 5.0.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4.7.0...v5.0.0) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]>

…ve_unsupported_machines Delete cmake files for 2 machines we don't support on LC.

The prs are not going to impact EAMxx anyways, since they only impact gh actions

When appending to an empty array, avoid leading ','

* Make some params case insensitive * Add arg for filename prefix

…rs names YAML_GEN and YAML_EDIT were both referring to the same script, so use a single env var

…ream

Do not accept STOP_N/STOP_OPTION. These *do* change in an ERP/ERS test. Besides, these refer to when the run should stop, not when the output should be written. Instead, use HIST_N/HIST_OPTION, which are the same across both runs of an ERP/ERS test.

test-all-scream: Get working on pm-gpu nightly

…m-edit-script Add a script to generate/edit output stream yaml files

…reate-baselines-with-ctest-label Automatically Merged using E3SM Pull Request AutoTester PR Title: Use ctest to generate baselines PR Author: bartgol PR LABELS: testing, cmake, BFB, AT: AUTOMERGE, scripts

…ctions/actions/setup-python-5.0.0 Automatically Merged using E3SM Pull Request AutoTester PR Title: Bump actions/setup-python from 4.7.0 to 5.0.0 PR Author: dependabot[bot] PR LABELS: AT: AUTOMERGE, AT: Integrate Without Testing, AT: PRE-TEST INSPECTED, dependencies

…at-interaction Automatically Merged using E3SM Pull Request AutoTester PR Title: Do not run AT on dependabot PRs PR Author: bartgol PR LABELS: AT: AUTOMERGE, AT: Integrate Without Testing

* Do not allow to specify an arbitrary baseline ref. We always generate from HEAD, unless -i is used, which makes TAS use origin/master * Do not gen baselines at every run. Only gen baselines if -g is passed. Using -i or -u also implies -g * If generating, only allow to overwrite baselines in AUTO folder if doing an integration test (or if --force-baseline-regen is used) * Always allow to run with a dirty repo. If repo is dirty and -i is used, we create a bkp commit, which we later undo (softly). * For baseline-dir, add "LOCAL" as shortcut value for ctest-build/baseline * For baseline-dir, if not specified use: - LOCAL if generating (but not with -i) - AUTO otherwise

It was never used in our scripts. It only matters with 2+ builds, and only if -p is not used, which is a not common scenario.

Makes scripts-tests a bit faster

…filter Automatically Merged using E3SM Pull Request AutoTester PR Title: Run e3sm gh action only if repo name is e3sm PR Author: bartgol PR LABELS: AT: AUTOMERGE, AT: Integrate Without Testing

That way, you can set baseline_ref=original_commit if a backup commit was needed

You need to switch to baseline_ref when generating baselines, and switch back to original_commit when done

We build these tests without ekat (or kokkos even), for speed. So we must take care of linking to OpenMP/MPI ourselves.

… RANKS spread

1. SCREAM_TEST_DATA_DIR->SCREAM_BASELINES_DIR, with the caveat that the new var does *not* include '/data' 2. SCREAM_BASELINES_ONLY->SCREAM_GENERATE_BASELINES_ONLY, since the latter was not clear on whether we were generating or comparing

Some scripts test don't actually generate any baseline

…-to-scripts-tests-and-tas Some changes to test-all-scream and scripts-tests

Everytime a msg is logged with level >= flush_level, the logger will automatically flush. By default, it is set to warn.

…ests-baselines Add baseline tests for EAMxx standalone ad tests

…r-flush-level Automatically Merged using E3SM Pull Request AutoTester PR Title: EAMxx: add possibility of auto-flushing the atm logger PR Author: bartgol PR LABELS: infrastructure, AT: AUTOMERGE, code usability

If baselines tests are disabled, don't create the test

Automatically Merged using E3SM Pull Request AutoTester PR Title: buildnml: Write dbg XML when things fail PR Author: jgfouca PR LABELS: AT: AUTOMERGE, AT: Skip weaver, AT: Skip Stand-Alone Testing

…p-baseline-test-only-if-baseline-tests-enabled Automatically Merged using E3SM Pull Request AutoTester PR Title: Run rrtmgp standalone baseline tests only if baselines tests are enabled PR Author: bartgol PR LABELS: testing, AT: AUTOMERGE, bugfix

The word "coupled" is overloaded, so change to unambiguous "single-process" and "multi-process"

…ed-ad-tests-folders Automatically Merged using E3SM Pull Request AutoTester PR Title: Renamed AD tests folders to avoid confusion PR Author: bartgol PR LABELS: AT: AUTOMERGE, code cleanup, AT: Skip v1 Testing

…iop_file_read IOP fixes (shoc cell length and iop file read)

Automatically Merged using E3SM Pull Request AutoTester PR Title: test-all-scream: Get scripts-tests working again PR Author: jgfouca PR LABELS: AT: AUTOMERGE, scripts, AT: Skip v1 Testing

Do not cleanup unless there's a difference in commits. Otherwise, you lose all the unsaved changes!

…netcdf-in-tas Automatically Merged using E3SM Pull Request AutoTester PR Title: Add pnetcdf paths to config line when running test-all-scream PR Author: bartgol PR LABELS: testing, AT: AUTOMERGE, scripts, AT: Skip v1 Testing

Fixes #2565. This commit makes a set of changes to the Fortran and C++ versions of P3 used by SCREAM, all of which are intended to avoid redundant calculations and/or minimize special function evaluations. These changes are limited to `get_rain_dsd2` and its callers, and made due to the belief that `get_rain_dsd2` is relatively performance- critical code, since it is evaluated at each rain sedimentation time step (often much smaller than the overall physics time step). The changes are: 1. Remove calculation of `inv_dum`, since it is not used. 2. Simplify the conversions between `lamr` and `nr`. In particular, all special function calls can be removed from the calculation of the "adjusted" `nr` produced when `lamr` is limited. 3. Remove extra `log10` and `tgamma` calls when calculating `logn0r`, since it can be calculated more simply from the value of `cdistr` on the line above. The C++ version also has an additional change, which is separating the calculation of `cdistr` and `logn0r` into a separate function from `get_rain_dsd2`. This is done because these two values require a `tgamma` call and two `log10` calls to calculate, but they are only needed in a few places in the code. Thus it is better to only calculate them in the one place where they are actually needed, rather than in every `get_rain_dsd2` call. In particular, they are not needed in the rain sedimentation loop. These changes cause roundoff-level changes in P3. [non-BFB]

…ndalone testing

Adds some comments to clarify number of tracers in MAM4xx's CIME test. Removes repeated team barriers. Update set_refindex function's name to set_refindex_names.

Option was being ignored.

Using the same tools we do in CIME.

…aseline_fix Automatically Merged using E3SM Pull Request AutoTester PR Title: test-all-scream: Fix force_baseline_regen PR Author: jgfouca PR LABELS: AT: AUTOMERGE, scripts, AT: Skip v1 Testing

... without crashing the entire run. Just assume baselines are expired in that case.

…_fail test-all-scream: allow git symmetric diff to fail

All checks on rhist file attributes would open and close the file on the fly. For simplicity, open it once, so that all subsequent open/close are no op (they just incr/decr a ref count)

…lushing-of-output-files Automatically Merged using E3SM Pull Request AutoTester PR Title: Fix small bug in flushing file in scorpio interface PR Author: bartgol PR LABELS: I/O, AT: AUTOMERGE, bugfix

Speed up `get_rain_dsd2` calculations.

The fake-only tests don't launch any kernels which will cause all the compute-sanitizer runs to fail.

…scripts_tests Automatically Merged using E3SM Pull Request AutoTester PR Title: scripts-tests: Fix on weaver PR Author: jgfouca PR LABELS: AT: AUTOMERGE, AT: Integrate Without Testing

…date Automatically Merged using E3SM Pull Request AutoTester PR Title: Update EKAT PR Author: tcclevenger PR LABELS: AT: AUTOMERGE, EKAT

…u, aero_ssa_sw=aero_tau_ssa_sw/aero_tau_g_sw.

…, aero_ssa_sw_mam4 -> aero_ssa_sw_mam4.

not re-order swbands.

…mes.

…dy throw an exception when calling get.

Luca's recommendations.

Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.0.0 to 5.1.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v5.0.0...v5.1.0) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]>

…actions/setup-python-5.1.0 Bump actions/setup-python from 5.0.0 to 5.1.0

Only change is to remove the WSM omp=16 test.

We need the new ignore-diffs flag

Update CIME submodule

…_2024_04_02 Automatically Merged using E3SM Pull Request AutoTester PR Title: Update EKAT submodule PR Author: jgfouca PR LABELS: AT: AUTOMERGE, AT: Integrate Without Testing

They were handing due to YAKL pool running out, so make it bigger by setting GATOR_INITIAL_MB to 4GB in both the standalone and cime env for mappy.

…d_rrtmgp Automatically Merged using E3SM Pull Request AutoTester PR Title: Fix rrtmgp valgrind tests PR Author: jgfouca PR LABELS: AT: AUTOMERGE, AT: Integrate Without Testing

…merge_2024_04_04 * upstream/master: (173 commits) Update mappy toolchain to gcc-11.2.0 remove leftover debug statement skip previewing docs if dependabot pr Bump actions/setup-python from 5.0.0 to 5.1.0 Remove MOAB_PATH use MOABDEBUG flag for verbose message add perlmutter moab configuration Add explicit dep from coupler to MOAB if USE_MOAB Remove moab includes mpas ocean now needs moab path to be defined intel complained about variable not initialized move component_exch_moab to cplcomp_exchange_mod simplify data ocean model driver moab needs moab includes added to the cpl misplaced ifdef for HAVE_MOAB error message missing aream initialization issues and zero init l2x_am global ids for flux settings are saved locally changes for data ocean model Clean up the data-ocean model code and verify tags are propagated correctly ...

Fix standalone eamxx build on pm-gpu

get_field_out("qc") instead of get_field_in("qc").

…merge_2024_04_04 * upstream/master: Remove gnu9 1-off Update modules for intel,openmpi on Chrysalis. Add logic to catch issue when partition file assigns 0 cells to a pe add more comments for phys_grid_ctem implement zonal mean output for TEM diagnostics Removes nstep Adds a print statement for phosphorus fertilizer Fix Phosphorus Balance Error in crop model

Adds MAM4xx: Aerosol optics

…rge_2024_04_04 * origin/master: (122 commits) aerosol_optics - fixing name of variable: from tau_g_sw to tau_ssa_g_sw. add doc for namelist variables for MAM4 optics input files aerosol_optics - remove code. aerosol_optics - fixing run time error; we need to use get_field_out("qc") instead of get_field_in("qc"). aerosol_optics - remove old comment. aerosol_optics - remove aero_tau_forward from field manager. aerosol_optics - remove old comment. aerosol_optics - remove temporal variables from field manager. aerosol_optics - fix typo. aerosol_optics - rename: aero_tau_forward -> tau_f_lw . aerosol_optics - remove aero from aero_tau_sw. aerosol_optics - remove aero from aero_tau_ssa_sw. aerosol_optics - remove aero from variable name. aerosol_optics - removed duplicate variable. aerosol_optics - from Updated to Required. aerosol_optics - update comments. aerosol_optics - update comments. aerosol_optics - comment. aerosol_optics - update comment. aerosol_optics - rename layout. ...

…4_04_04 Jgfouca/upstream merge 2024 04 04

…stream_04_11_24 * scream/master: (539 commits) aerosol_optics - fixing name of variable: from tau_g_sw to tau_ssa_g_sw. add doc for namelist variables for MAM4 optics input files aerosol_optics - remove code. aerosol_optics - fixing run time error; we need to use get_field_out("qc") instead of get_field_in("qc"). aerosol_optics - remove old comment. aerosol_optics - remove aero_tau_forward from field manager. Add fallow-arg-mismatch to mappy aerosol_optics - remove old comment. aerosol_optics - remove temporal variables from field manager. aerosol_optics - fix typo. aerosol_optics - rename: aero_tau_forward -> tau_f_lw . aerosol_optics - remove aero from aero_tau_sw. aerosol_optics - remove aero from aero_tau_ssa_sw. aerosol_optics - remove aero from variable name. Switch mappy over to new toolchain aerosol_optics - removed duplicate variable. aerosol_optics - from Updated to Required. aerosol_optics - update comments. aerosol_optics - update comments. aerosol_optics - comment. ...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Jgfouca/scream downstream 04 11 24 #6343

Jgfouca/scream downstream 04 11 24 #6343

Commits on Feb 28, 2024

Commits on Feb 29, 2024

Commits on Mar 1, 2024

Commits on Mar 4, 2024

Commits on Mar 5, 2024

Commits on Mar 6, 2024

Commits on Mar 7, 2024

Commits on Mar 8, 2024

Commits on Mar 11, 2024

Commits on Mar 12, 2024

Commits on Mar 13, 2024

Commits on Mar 14, 2024

Commits on Mar 15, 2024

Commits on Mar 18, 2024

Commits on Mar 19, 2024

Commits on Mar 20, 2024

Commits on Mar 21, 2024

Commits on Mar 22, 2024

Commits on Mar 25, 2024

Commits on Mar 27, 2024

Commits on Mar 28, 2024

Commits on Mar 29, 2024

Commits on Mar 30, 2024

Commits on Apr 1, 2024

Commits on Apr 2, 2024

Commits on Apr 3, 2024

Commits on Apr 4, 2024

Commits on Apr 5, 2024

Commits on Apr 8, 2024

Commits on Apr 10, 2024

Commits on Apr 11, 2024