Merge branch 'main' of https://github.com/SpeedyWeather/SpeedyWeather.jl

 into mk/tracers

.github/workflows/CI.yml

@@ -13,9 +13,8 @@ jobs:
     strategy:
       matrix:
         version:
-          - '1.9'
           - '1.10'
-          - '~1.11.0-0'
+          - '1.11'
         os:
           - ubuntu-latest
         arch:

.github/workflows/Documenter.yml

@@ -14,7 +14,7 @@ jobs:
       - uses: actions/checkout@v4
       - uses: julia-actions/setup-julia@latest
         with:
-          version: '1.10'
+          version: '1.11'
       - name: Install dependencies
         run: julia --color=yes --project=docs/ -e 'using Pkg; Pkg.develop(PackageSpec(path=pwd())); Pkg.instantiate()'
       - name: Build and deploy

.github/workflows/draft-pdf.yml

@@ -21,8 +21,8 @@ jobs:
           # PDF. Note, this should be the same directory as the input
           # paper.md
           path: docs/joss/paper.pdf
-        # These lines have been added following
-        # https://github.com/openjournals/openjournals-draft-action/issues/15
+        # These lines have been added following
+        # https://github.com/openjournals/openjournals-draft-action/issues/15
       - name: save pdf to repo
         uses: stefanzweifel/git-auto-commit-action@v5
         with:

.github/workflows/extended_tests.yml

@@ -0,0 +1,29 @@
+name: Extended Test
+# Weekly extended tests
+on:
+  schedule:
+    - cron: '0 2 * * 0'  # Daily at 2 AM UTC every Sunday
+jobs:
+  test:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        version:
+          - '1.11'
+        os:
+          - ubuntu-latest
+        arch:
+          - x64
+    steps:
+      - uses: actions/checkout@v4
+      - uses: julia-actions/setup-julia@v2
+        with:
+          version: ${{ matrix.version }}
+          arch: ${{ matrix.arch }}
+      - uses: julia-actions/cache@v2
+      - uses: julia-actions/julia-buildpkg@v1
+      - uses: julia-actions/julia-runtest@v1
+        with:
+          test_args: 'extended_tests'
+        env:
+          GITHUB_AUTH: ${{ secrets.GITHUB_TOKEN }}

CHANGELOG.md

@@ -3,11 +3,53 @@
 ## Unreleased
 
 - Introduced n tracers that are passively advected
-- Introduced a new `set!` function that allows to set `PrognosticVariables` to new values with keyword arguments
+
+## v0.13
+
+- AbstractSurfacePerturbation introduced [#631](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/631)
+- AbstractStochasticPhysics and SPPT implemented [#629](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/629)
+- Introduced seperate extended tests that are not run on every commit [#628](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/628)
+- One-band longwave radiation [#624](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/624)
+- compat entry for FiniteDifferences.jl [#620](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/620)
+- Slab ocean and net surface fluxes [#613](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/613)
+- added compat entries for FiniteDifferences.jl and Enzyme.jl [#620](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/620) [#622](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/622)
+- update to GPUArrays v11, JLArrays v0.2 and remove CUDA from tests [#590](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/590)
+- Power spectrum for n-dim LowerTriangularArrays [#618](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/618)
+- Added custom EnzymeRules for the SpectralTransform and an extension for compatibility with FiniteDifferences.jl [#589](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/589) [#625](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/625) [#627](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/627)
+- Optical depth introduced and array-agnostic ColumnVariables [#606](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/606)
+- Include large-scale condensation tests [#615](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/615)
+- bugfix: large-scale condensation also at <100% [#609](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/609)
+
+## v0.12.1
+
+- ConstantLandTemperature implemented [#612](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/612)
+- set! for more boundary conditions [#611](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/611)
+- SpectralFilter for horizontal diffusion [#601](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/601)
+- GeoMakie weak dependency, globe function for 3D data visualisation [#600](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/600)
+- Zonal mean for AbstractGridArray [#603](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/603)
+- Rossby-Haurwitz wave with initial conditions for interface displacement for shallow water models[#604](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/604)
+
+## v0.12.0
+
+- OctaminimalGaussianArray/Grid to start with 4 points around the poles [#595](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/595)
+- Output both accumulated and precipitation rate as netCDF [#596](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/596)
+- Random processes for random pattern generation [#592](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/592)
+- Also allow SpectralGrid as positional argument to model constructors [#593](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/593)
+- De-interweave SpectralTransform [#587](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/587)
+- Rossby-Haurwitz wave initial conditions [#591](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/591)
+- Haversine formula and AbstractSphericalDistance [#588](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/588)
+- Array-agnostic SpectralTransform [#583](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/583)
+- Move CUDA dependency into extension [#586](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/586)
+- Stop supporting Julia v1.9 [#585](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/585)
+- `feedback.verbose` (de/activate the progressbar) is now set to `isinteractive()` to disable automatically for documentation [#582](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/582)
+
+## v0.11.0
+
+- Extend `set!` with `orography` keyword argument to set the orography with `set!` [#578](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/578)
+- Added a new benchmark suite for the dynamics functions [#577](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/577)
+- Introduced a new `set!` function that allows to set `PrognosticVariables` to new values with keyword arguments [#563](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/563)
 - Restructured dynamical core with prognostic/diagnostic variables array-agnostic and 3-dimensional [#525](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/525)
 - Modularised NetCDF output [#573](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/573)
 - Fixed a bug in RingGrids, now broadcasts are defined even when the dimensions mismatch in some cases [#568](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/568)
 - RingGrids: To wrap an Array with the horizontal dimension in matrix shape into a full grid, one has to use e.g. `FullGaussianGrid(map, input_as=Matrix)` now. [#572](https://github.com/SpeedyWeather/SpeedyWeather.jl/pull/572)
 - CompatHelper: Allow for JLD2.jl v0.5
-
-## v0.11.0

Project.toml

@@ -1,14 +1,13 @@
 name = "SpeedyWeather"
 uuid = "9e226e20-d153-4fed-8a5b-493def4f21a9"
 authors = ["Milan Klöwer and SpeedyWeather contributors"]
-version = "0.10.0"
+version = "0.13"
 
 [deps]
 AbstractFFTs = "621f4979-c628-5d54-868e-fcf4e3e8185c"
 Adapt = "79e6a3ab-5dfb-504d-930d-738a2a938a0e"
 AssociatedLegendrePolynomials = "2119f1ac-fb78-50f5-8cc0-dda848ebdb19"
 BitInformation = "de688a37-743e-4ac2-a6f0-bd62414d1aa7"
-CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba"
 CodecZlib = "944b1d66-785c-5afd-91f1-9de20f533193"
 Dates = "ade2ca70-3891-5945-98fb-dc099432e06a"
 DocStringExtensions = "ffbed154-4ef7-542d-bbb7-c09d3a79fcae"
@@ -29,10 +28,18 @@ TOML = "fa267f1f-6049-4f14-aa54-33bafae1ed76"
 UnicodePlots = "b8865327-cd53-5732-bb35-84acbb429228"
 
 [weakdeps]
+CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba"
+Enzyme = "7da242da-08ed-463a-9acd-ee780be4f1d9"
+FiniteDifferences = "26cc04aa-876d-5657-8c51-4c34ba976000"
+GeoMakie = "db073c08-6b98-4ee5-b6a4-5efafb3259c6"
 JLArrays = "27aeb0d3-9eb9-45fb-866b-73c2ecf80fcb"
 Makie = "ee78f7c6-11fb-53f2-987a-cfe4a2b5a57a"
 
 [extensions]
+SpeedyWeatherCUDAExt = "CUDA"
+SpeedyWeatherEnzymeExt = "Enzyme"
+SpeedyWeatherFiniteDifferencesExt = "FiniteDifferences"
+SpeedyWeatherGeoMakieExt = "GeoMakie"
 SpeedyWeatherJLArraysExt = "JLArrays"
 SpeedyWeatherMakieExt = "Makie"
 
@@ -43,30 +50,26 @@ AssociatedLegendrePolynomials = "1"
 BitInformation = "0.6"
 CUDA = "4, 5"
 CodecZlib = "0.7"
-Dates = "1.9"
+Dates = "1.10"
 DocStringExtensions = "0.9"
+Enzyme = "0.13"
 FFTW = "1"
 FastGaussQuadrature = "0.4, 0.5, 1"
-GPUArrays = "10"
+FiniteDifferences = "0.12"
+GPUArrays = "11"
 GenericFFT = "0.1"
-JLArrays = "0.1.4"
+GeoMakie = "0.7.6"
+JLArrays = "0.1.4, 0.2"
 JLD2 = "0.4, 0.5"
 KernelAbstractions = "0.9"
-LinearAlgebra = "1.9"
+LinearAlgebra = "1.10"
 Makie = "0.20, 0.21"
 NCDatasets = "0.12, 0.13, 0.14"
 Primes = "0.5"
-Printf = "1.9"
+Printf = "1.10"
 ProgressMeter = "1.7"
-Random = "1.9"
-Statistics = "1.9"
+Random = "1.10"
+Statistics = "1.10"
 TOML = "1"
 UnicodePlots = "3.3"
-julia = "1.9"
-
-[extras]
-JLArrays = "27aeb0d3-9eb9-45fb-866b-73c2ecf80fcb"
-Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
-
-[targets]
-test = ["Test", "JLArrays"]
+julia = "1.10"

README.md

@@ -1,26 +1,30 @@
-# SpeedyWeather.jl
+# SpeedyWeather.jl <img src="https://github.com/user-attachments/assets/977f5f46-ccd3-49d8-950a-8b619df863c3" width="100" />
+
 
 [![CI](https://github.com/SpeedyWeather/SpeedyWeather.jl/actions/workflows/CI.yml/badge.svg)](https://github.com/SpeedyWeather/SpeedyWeather.jl/actions/workflows/CI.yml) 
 [![status](https://joss.theoj.org/papers/515c81a4d6a69e31cc71ded65ac9c36a/status.svg)](https://joss.theoj.org/papers/515c81a4d6a69e31cc71ded65ac9c36a)
 [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.6510139.svg)](https://doi.org/10.5281/zenodo.6510139)  
 [![docs](https://img.shields.io/badge/documentation-latest_release-blue.svg)](https://speedyweather.github.io/SpeedyWeather.jl/stable/)
 [![docs](https://img.shields.io/badge/documentation-main-blue.svg)](https://speedyweather.github.io/SpeedyWeather.jl/dev/)
 
-SpeedyWeather.jl is a global spectral atmospheric model with simple physics which is developed as a research playground
-with an everything-flexible attitude as long as it is speedy. With minimal code redundancies it supports
+SpeedyWeather.jl is a global atmospheric model with simple physics developed as a research playground
+with an everything-flexible attitude as long as it is speedy. It is easy to use and easy to extend, making 
+atmospheric modelling an interactive experience -- in the terminal, in a notebook or conventionally through scripts.
+With minimal code redundancies it supports
 
 **Dynamics and physics**
-- Different physical equations (barotropic vorticity, shallow water, primitive equations)
+- Different physical equations (barotropic vorticity, shallow water, primitive equations, with and without humidity)
 - Particle advection in 2D for all equations
 - Physics parameterizations for convection, precipitation, boundary layer, etc.
 
 **Numerics and computing**
 - Different spatial grids (full and octahedral grids, Gaussian and Clenshaw-Curtis, HEALPix, OctaHEALPix)
 - Different resolutions (T31 to T1023 and higher, i.e. 400km to 10km using linear, quadratic or cubic truncation)
 - Different arithmetics: Float32 (default), Float64, and (experimental) BFloat16, stochastic rounding
-- multi-threading, layer-wise for dynamics, grid point-wise for physics
+- a very fast and flexible spherical harmonics transform library SpeedyTransforms
 
 **User interface**
+- Data visualisation: 2D, 3D, interactive (you can zoom and rotate!) powered by Makie
 - Extensibility: New model components (incl. parameterizations) can be externally defined
 - Modularity: Models are constructed from its components, non-defaults are passed on as argument
 - Interactivity: SpeedyWeather.jl runs in a notebook or in the REPL as well as from scripts
@@ -30,9 +34,9 @@ and Julia will compile to these choices just-in-time.
 
 For an overview of the functionality and explanation see the
 [documentation](https://speedyweather.github.io/SpeedyWeather.jl/dev).
-But as the documentation always lags behind our full functionality you are encouraged
-to [raise an issue](https://github.com/SpeedyWeather/SpeedyWeather.jl/issues) describing
-what you'd like to use SpeedyWeather for.
+You are always encouraged to [raise an issue](https://github.com/SpeedyWeather/SpeedyWeather.jl/issues)
+(even it is not actually an issue but an idea, a suggestion or really anything)
+describing what you'd like to use SpeedyWeather for. We're keen to help!
 
 ## Vision and roadmap
 
@@ -74,21 +78,20 @@ about dos and don'ts. Just express your interest to contribute and we'll be happ
 
 For a more comprehensive tutorial with several examples, see
 [Examples](https://speedyweather.github.io/SpeedyWeather.jl/dev/examples_2D/) in the documentation.
-The interface to SpeedyWeather.jl consist of 5 steps: define the grid, create model components,
+The basic interface to SpeedyWeather.jl consist of 4 steps: define the grid,
 construct the model, initialize, run
 
 ```julia
-spectral_grid = SpectralGrid(trunc=31, nlayers=8)          # define resolution
-orography = EarthOrography(spectral_grid)               # create non-default components
-model = PrimitiveWetModel(; spectral_grid, orography)   # construct model
-simulation = initialize!(model)                         # initialize all model components
-run!(simulation, period=Day(10), output=true)           # aaaand action!
+spectral_grid = SpectralGrid(trunc=31, nlayers=8)   # define resolution
+model = PrimitiveWetModel(spectral_grid)            # construct model
+simulation = initialize!(model)                     # initialize all model components
+run!(simulation, period=Day(10), output=true)       # aaaand action!
 ```
 and you will see
 
 <img src="https://github.com/SpeedyWeather/SpeedyWeather.jl/assets/25530332/a04fbb10-1cc1-4f77-93f2-7bdf047f277d" width="450"><br>
 
-Hurray🥳 In 5 seconds we just simulated 10 days of the Earth's atmosphere at a speed of 440 years per day.
+Hurray🥳 In a few seconds seconds we just simulated 10 days of the Earth's atmosphere at a speed of 440 years per day.
 This simulation used a T31 spectral resolution on an
 [octahedral Gaussian grid](https://speedyweather.github.io/SpeedyWeather.jl/dev/grids/#Implemented-grids)
 (~400km resolution) solving the primitive equations on 8 vertical levels.
@@ -139,6 +142,18 @@ T85 (150km) resolution and 8 vertical layers.
 
 https://github.com/SpeedyWeather/SpeedyWeather.jl/assets/25530332/a6192374-24d9-4065-9fcc-8b719190472f
 
+## Data visualisation
+
+Difficult to plot spherical data? SpeedyWeather also includes extensions for Makie and GeoMakie
+making it supereasy to create plots and interactively investigate a variables from a simulation.
+Two examples (screen recording those makes it a bit laggy, it's pretty smooth otherwise): Humidity plotted
+on a 50km HEALPix grid 
+
+https://github.com/user-attachments/assets/b02b31eb-e139-4193-89d1-7e277a2af5cc
+
+or the visualising cell centres and faces of the OctaminimalGaussianGrid 
+
+https://github.com/user-attachments/assets/6dfa212a-c5dc-4c54-b274-7755d5baf15c
 
 ## History
 
@@ -179,7 +194,7 @@ check out their documentation: [RingGrids](https://speedyweather.github.io/Speed
 
 SpeedyWeather.jl is registered in Julia's registry, so open the package manager with `]` and
 ```julia
-(@v1.10) pkg> add SpeedyWeather
+(@v1.11) pkg> add SpeedyWeather
 ```
 which will install the [latest release]([url](https://github.com/SpeedyWeather/SpeedyWeather.jl/releases))
 and all dependencies automatically. For more information see the
@@ -189,9 +204,19 @@ compatibilities with older versions are not guaranteed.
 
 ## Benchmarks
 
-The primitive equations at 400km resolution with 8 vertical layers are simulated by
-SpeedyWeather.jl at about 500 simulated years per day, i.e. one year takes about
-3min single-threaded on a CPU. Multi-threading will increase the speed typically by 2-4x.
+The primitive equations at 400km resolution with 8 vertical layers can be simulated by
+SpeedyWeather.jl at 1800 simulated years per day (SYPD) on a single core of newer CPUs with arm architecture
+(M-series MacBooks for example). At that speed, simulating one year takes about 50 seconds
+without output. The complex fused-multiply adds of the spectral transform compile efficiently to
+the large vector extensions of the newer arm chips in single precision.
+Another considerable speed-up comes from the reduced grids minimizing the number of columns for
+which expensive parameterizations like convection have to be computed. The parameterizations
+take up 40-60% of the total simulation time, depending on the grid. Particularly the
+`OctaminimalGaussianGrid`, `OctaHEALPixGrid` and the `HEALPixGrid` are increasingly faster,
+at a small accuracy sacrifice of the then inexact spectral transforms. 
+
+On older CPUs, like the Intel CPU MacBooks, the 1800 SYPD drop to about 500-600 SYPD,
+which is still 2x faster than Fortran SPEEDY which is reported to reach 240 SYPD.
 
 For an overview of typical simulation speeds a user can expect under different model setups see
 [Benchmarks](https://github.com/SpeedyWeather/SpeedyWeather.jl/blob/main/benchmark).

benchmark/Project.toml

@@ -0,0 +1,3 @@
+[deps]
+BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
+SpeedyWeather = "9e226e20-d153-4fed-8a5b-493def4f21a9"