-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reanalysis data as initial conditions #630
Comments
Thanks for reaching out and your interest! Yes, setting the initial conditions from ERA5 / some form of data assimilation is currently missing in SpeedyWeather and would require further attention. As you may have noticed SpeedyWeather’s current focus is more on idealised simulations, interactivity than on actually being good at the weather forecasting challenge. This can be changed but requires some effort. Many thanks for the praise, good to hear that a different approach to user and developer interface is appreciated. I guess a list from easy to complex of required features to make this possible
|
Thank you for your enthusiastic response. I plan to start with the easy way as your suggestion. using FFTW |
Yes we use |
Dear Dr. Klöwer
using SpeedyWeather
spectral_grid = SpectralGrid(trunc=31, Grid=FullGaussianGrid)
model = PrimitiveWetModel(spectral_grid)
simulation = initialize!(model)
#############################################
#change the arrays in simulation.prognostic_variables
##############################################
u = rand(2, 48, 96, 8) #[nstep, lat ,lon, lev]
v = rand(2, 48, 96, 8)
u = permutedims(u, (1, 3, 2, 4))
v = permutedims(v, (1, 2, 3, 4))
vor= simulation.prognostic_variables.vor
for i in 1:2
for k in 1:8
u_grid =FullGaussianGrid(u[i,:,:,k], input_as=Matrix)
v_grid = FullGaussianGrid(v[i,:,:,k], input_as=Matrix)
vor[i][:,k] = curl(u_grid, v_grid, radius=spectral_grid.radius)
end
end
run!(simulation) |
We don't have such a functionality yet, you are more than welcome to implement it, you essentially only need to reverse this #268
Yes, it's call
Arrays in Julia are column-major so we generally use lon-lat-X-Y in that order where X, Y can be any non-horizontal dimensions. E.g. vertical or time. If you want to create random data on a grid you can just use In your case of setting initial conditions, you only have to provide initial conditions for the first leapfrog time step. nlat_half = 24
set!(simulation, u = rand(FullGaussianGrid, 24, 8), v = rand(FullGaussianGrid, 24, 8)) this will leave the 2nd leapfrog time step at zero, as the first time step of the model is a small Euler step anyway, it would overwrite the 2nd leapfrog time step anyway.
This then just becomes julia> u, v = rand(FullGaussianGrid, 24, 8), rand(FullGaussianGrid, 24, 8);
julia> divergence(u, v)
560×8 LowerTriangularArray{ComplexF64, 2, Matrix{ComplexF64}}:
... or |
Dear Dr. Klöwer using SpeedyWeather
spectral_grid = SpectralGrid(trunc=31, Grid=FullGaussianGrid)
#48X96
model = PrimitiveWetModel(spectral_grid)
simulation = initialize!(model)
set!(simulation, u = rand(FullGaussianGrid, 24, 8), v = rand(FullGaussianGrid, 24, 8)) Segment B using SpeedyWeather
spectral_grid = SpectralGrid(trunc=31, Grid=FullGaussianGrid)
#48X96
model = PrimitiveWetModel(spectral_grid)
simulation = initialize!(model)
set!(simulation, u = rand(FullGaussianGrid, 24, 8), v = rand(FullGaussianGrid, 24, 8), temp = rand(FullGaussianGrid, 24, 8)) |
(please put code into |
Mhhh that's a bug, you currently need to provide the same number format julia> set!(simulation, u = rand(FullGaussianGrid{Float32}, 24, 8), v = rand(FullGaussianGrid{Float32}, 24, 8))
julia> simulation.prognostic_variables.vor[1]
560×8 LowerTriangularArray{ComplexF32, 2, Matrix{ComplexF32}}:
0.0+0.0im 0.0+0.0im … 0.0+0.0im 0.0+0.0im
3.77433f-7+0.0im 3.77419f-7+0.0im 3.80322f-7+0.0im 3.82084f-7+0.0im
1.29421f-8+0.0im 9.57814f-11+0.0im 1.35997f-9+0.0im 3.46717f-10+0.0im
2.21733f-7+0.0im 2.18971f-7+0.0im 2.21314f-7+0.0im 2.04825f-7+0.0im
-1.4568f-8+0.0im 1.14279f-8+0.0im -1.21748f-8+0.0im -3.39043f-9+0.0im
1.52779f-7+0.0im 1.80171f-7+0.0im … 1.77427f-7+0.0im 1.72589f-7+0.0im and similar for
same here |
To bug originates from here as we don't convert
I suggest to add the following in the line before grids = convert.(eltype(S), grids) and define Base.eltype(S::SpectralTransform{NF}) where NF = NF generally in |
Ideally we would have a fully number format flexible code but unfortunately FFTW isn't happy with that and so we need to iron those bits out where it otherwise would complain |
Thank you for your help. Now It can work using following code using SpeedyWeather
spectral_grid = SpectralGrid(trunc=31, Grid=FullGaussianGrid)
model = PrimitiveWetModel(spectral_grid)
simulation = initialize!(model)
set!(simulation, u = Float32.(rand(FullGaussianGrid, 24, 8)), v = Float32.(rand(FullGaussianGrid, 24, 8)), temp = Float32.(rand(FullGaussianGrid, 24, 8))) |
When I add code set!(simulation, u = Float32.(rand(FullGaussianGrid, 24, 8)), v = Float32.(rand(FullGaussianGrid, 24, 8))) And check using code simulation.prognostic_variables.vor[1] I find vor[1] has changed. I assume that during initialization, I only need to modify u and v in set! and do not need to calculate vor and div separately. I would like to ask if my idea is correct. |
Dear Dr. Klöwer # Initialize spectral grid and model
spectral_grid = SpectralGrid(trunc=10, Grid=FullGaussianGrid)
model = PrimitiveWetModel(spectral_grid)
simulation = initialize!(model)
# Load data from NetCDF files
ds_u = NCDataset("uwnd.2023.nc")
u1 = ds_u["uwnd"][:,:,:,1]
ds_v = NCDataset("vwnd.2023.nc")
v1 = ds_v["vwnd"][:,:,:,1]
ds_t = NCDataset("air.2023.nc")
t1 = ds_t["air"][:,:,:,1]
ds_rh = NCDataset("rhum.2023.nc")
rh1 = ds_rh["rhum"][:,:,:,1]
ds_pres = NCDataset("pres.sfc.2023.nc")
pres1 = ds_pres["pres"][:,:,1]
function bilinear_interpolation(source::AbstractMatrix, target_size::Tuple{Int,Int})
src_rows, src_cols = size(source)
target_rows, target_cols = target_size
target = zeros(eltype(source), target_rows, target_cols)
row_scale = (src_rows - 1) / (target_rows - 1)
col_scale = (src_cols - 1) / (target_cols - 1)
for i in 1:target_rows
for j in 1:target_cols
src_row = (i - 1) * row_scale
src_col = (j - 1) * col_scale
r1 = floor(Int, src_row) + 1
r2 = min(r1 + 1, src_rows)
c1 = floor(Int, src_col) + 1
c2 = min(c1 + 1, src_cols)
w_r = src_row - (r1 - 1)
w_c = src_col - (c1 - 1)
target[i, j] = (1 - w_r) * (1 - w_c) * source[r1, c1] +
w_r * (1 - w_c) * source[r2, c1] +
(1 - w_r) * w_c * source[r1, c2] +
w_r * w_c * source[r2, c2]
end
end
return target
end
source = pres1
target_size = (32, 16)
pres2 = bilinear_interpolation(source, target_size)
u2 = zeros(Float32, 32, 16, 17)
for i in 1:17
u2[:, :, i] = bilinear_interpolation(u1[:, :, i], target_size)
end
v2 = zeros(Float32, 32, 16, 17)
for i in 1:17
v2[:, :, i] = bilinear_interpolation(v1[:, :, i], target_size)
end
t2 = zeros(Float32, 32, 16, 17)
for i in 1:17
t2[:, :, i] = bilinear_interpolation(t1[:, :, i], target_size)
end
rh2 = zeros(Float32, 32, 16, 8)
for i in 1:8
rh2[:, :, i] = bilinear_interpolation(rh1[:, :, i], target_size)
end
using Interpolations
p_c=[1000,925,850,700,600,500,400,300,250,200,150,100,70,50,30,20,10]
sigma_c=collect(0.9375:-0.125:0.0625)
function p_to_sigma(var_p, p_levels, pres_surf, sigma_levels)
var_p = convert(Array, var_p)
p_levels = convert(Array, p_levels)
pres_surf = convert(Array, pres_surf)
sigma_levels = convert(Array, sigma_levels)
nlon, nlat, nplev = size(var_p)
nsiglev = length(sigma_levels)
var_sigma = zeros(nlon, nlat, nsiglev)
p_levels_sorted = reverse(p_levels)
for i in 1:nlon
for j in 1:nlat
p_sigma = sigma_levels .* pres_surf[i, j]
var_p_sorted = reverse(var_p[i, j, :])
itp = interpolate((p_levels_sorted,), var_p_sorted, Gridded(Linear()))
for k in 1:nsiglev
if p_sigma[k] < p_levels_sorted[1]
var_sigma[i, j, k] = var_p_sorted[1]
elseif p_sigma[k] > p_levels_sorted[end]
var_sigma[i, j, k] = var_p_sorted[end]
else
var_sigma[i, j, k] = itp(p_sigma[k])
end
end
end
end
return var_sigma
end
u3 = p_to_sigma(u2, p_c, pres2, sigma_c)
v3 = p_to_sigma(v2, p_c, pres2, sigma_c)
t3 = p_to_sigma(t2, p_c, pres2, sigma_c)
rh3 = p_to_sigma(rh2, [1000, 925, 850, 700, 600, 500, 400, 300], pres2, sigma_c)
u_grid = Float32.(FullGaussianGrid(u3, input_as=Matrix))
v_grid = Float32.(FullGaussianGrid(v3, input_as=Matrix))
t_grid = Float32.(FullGaussianGrid(t3, input_as=Matrix))
rh_grid = Float32.(FullGaussianGrid(rh3, input_as=Matrix))
pres_grid = Float32.(FullGaussianGrid(pres2, input_as=Matrix))
set!(simulation, u=u_grid, v=v_grid, temp=t_grid, humid=rh_grid, pres=pres_grid)
run!(simulation) The NCEP data can be download from |
the vertical coordinate in SpeedyWeather (and also in ECMWF's model) is sorted top to bottom, so k=1 is the uppermost layer, k=nlayers is the lowermost layer.
temperature has to be in K, relative humidity in kg/kg, pressure is actually the logarithm of surface pressure. Maybe we should convert that automatically but currently we don't. So put a |
Yes, if you set u, v then it computes the curl and divergence for you. that happens here SpeedyWeather.jl/src/dynamics/prognostic_variables.jl Lines 482 to 483 in 4f07e4d
|
Dear Dr. Klöwer using SpeedyWeather
using NCDatasets
# Initialize spectral grid and model
spectral_grid = SpectralGrid(trunc=10, Grid=FullGaussianGrid)
model = PrimitiveWetModel(spectral_grid)
simulation = initialize!(model)
# Load data from NetCDF files
ds_u = NCDataset("uwnd.2023.nc")
u1 = ds_u["uwnd"][:,:,:,1]
ds_v = NCDataset("vwnd.2023.nc")
v1 = ds_v["vwnd"][:,:,:,1]
ds_t = NCDataset("air.2023.nc")
t1 = ds_t["air"][:,:,:,1]
ds_sh = NCDataset("shum.2023.nc")
sh1 = ds_sh["shum"][:,:,:,1]
ds_pres = NCDataset("pres.sfc.2023.nc")
pres1 = ds_pres["pres"][:,:,1]
function bilinear_interpolation(source::AbstractMatrix, target_size::Tuple{Int,Int})
src_rows, src_cols = size(source)
target_rows, target_cols = target_size
target = zeros(eltype(source), target_rows, target_cols)
row_scale = (src_rows - 1) / (target_rows - 1)
col_scale = (src_cols - 1) / (target_cols - 1)
for i in 1:target_rows
for j in 1:target_cols
src_row = (i - 1) * row_scale
src_col = (j - 1) * col_scale
r1 = floor(Int, src_row) + 1
r2 = min(r1 + 1, src_rows)
c1 = floor(Int, src_col) + 1
c2 = min(c1 + 1, src_cols)
w_r = src_row - (r1 - 1)
w_c = src_col - (c1 - 1)
target[i, j] = (1 - w_r) * (1 - w_c) * source[r1, c1] +
w_r * (1 - w_c) * source[r2, c1] +
(1 - w_r) * w_c * source[r1, c2] +
w_r * w_c * source[r2, c2]
end
end
return target
end
source = pres1
target_size = (32, 16)
pres2 = bilinear_interpolation(source, target_size)
u2 = zeros(Float32, 32, 16, 17)
for i in 1:17
u2[:, :, i] = bilinear_interpolation(u1[:, :, i], target_size)
end
v2 = zeros(Float32, 32, 16, 17)
for i in 1:17
v2[:, :, i] = bilinear_interpolation(v1[:, :, i], target_size)
end
t2 = zeros(Float32, 32, 16, 17)
for i in 1:17
t2[:, :, i] = bilinear_interpolation(t1[:, :, i], target_size)
end
sh2 = zeros(Float32, 32, 16, 8)
for i in 1:8
sh2[:, :, i] = bilinear_interpolation(sh1[:, :, i], target_size)
end
using Interpolations
p_c=[1000,925,850,700,600,500,400,300,250,200,150,100,70,50,30,20,10]
sigma_c=collect(0.9375:-0.125:0.0625)
function p_to_sigma(var_p, p_levels, pres_surf, sigma_levels)
var_p = convert(Array, var_p)
p_levels = convert(Array, p_levels)
pres_surf = convert(Array, pres_surf)
sigma_levels = convert(Array, sigma_levels)
nlon, nlat, nplev = size(var_p)
nsiglev = length(sigma_levels)
var_sigma = zeros(nlon, nlat, nsiglev)
p_levels_sorted = reverse(p_levels)
for i in 1:nlon
for j in 1:nlat
p_sigma = sigma_levels .* pres_surf[i, j]
var_p_sorted = reverse(var_p[i, j, :])
itp = interpolate((p_levels_sorted,), var_p_sorted, Gridded(Linear()))
for k in 1:nsiglev
if p_sigma[k] < p_levels_sorted[1]
var_sigma[i, j, k] = var_p_sorted[1]
elseif p_sigma[k] > p_levels_sorted[end]
var_sigma[i, j, k] = var_p_sorted[end]
else
var_sigma[i, j, k] = itp(p_sigma[k])
end
end
end
end
return var_sigma
end
u3 = p_to_sigma(u2, p_c, pres2, sigma_c)
v3 = p_to_sigma(v2, p_c, pres2, sigma_c)
t3 = p_to_sigma(t2, p_c, pres2, sigma_c)
sh3 = p_to_sigma(sh2, [1000, 925, 850, 700, 600, 500, 400, 300], pres2, sigma_c)
u5 = sort(u3, dims=3, rev=true)
v5 = sort(v3, dims=3, rev=true)
t5 = sort(t3, dims=3, rev=true)
sh5 = sort(sh3, dims=3, rev=true)
pres5 = log.(pres2)
u_grid = Float32.(FullGaussianGrid(u5, input_as=Matrix))
v_grid = Float32.(FullGaussianGrid(v5, input_as=Matrix))
t_grid = Float32.(FullGaussianGrid(t5, input_as=Matrix))
sh_grid = Float32.(FullGaussianGrid(sh5, input_as=Matrix))
pres_grid = Float32.(FullGaussianGrid(pres5, input_as=Matrix))
set!(simulation, u=u_grid, v=v_grid, temp=t_grid, humid=sh_grid, pres=pres_grid)
run!(simulation)
|
using SpeedyWeather
grid24 = rand(FullClenshawGrid, 24)
grid12 = RingGrids.interpolate(FullClenshawGrid, 12, grid) would interpolate from a 96x47 grid to a 48x23.
julia> grid = rand(FullClenshawGrid, 2, 3)
24×3, 3-ring FullClenshawArray{Float64, 2, Matrix{Float64}}:
0.933543 0.301979 0.571856
0.21249 0.951522 0.893032
0.67024 0.555407 0.791283
0.50855 0.802504 0.250079
0.358565 0.933129 0.222025
0.833646 0.143867 0.285908
0.275013 0.982804 0.785195
0.0837254 0.924352 0.300314
0.258964 0.857231 0.221561
0.888978 0.0955061 0.452848
0.709174 0.59125 0.597493
0.986523 0.107532 0.60472
0.471154 0.417332 0.705191
0.0497819 0.682505 0.144499
0.984909 0.906613 0.299998
0.302946 0.077889 0.88048
0.233419 0.700507 0.782197
0.846485 0.255638 0.935966
0.341638 0.355736 0.541445
0.0896589 0.022498 0.383306
0.339466 0.853529 0.969174
0.618453 0.755872 0.610549
0.155821 0.0678199 0.596454
0.137743 0.0420721 0.351357
julia> grid[:, end:-1:1]
24×3 Matrix{Float64}:
0.571856 0.301979 0.933543
0.893032 0.951522 0.21249
0.791283 0.555407 0.67024
0.250079 0.802504 0.50855
0.222025 0.933129 0.358565
0.285908 0.143867 0.833646
0.785195 0.982804 0.275013
0.300314 0.924352 0.0837254
0.221561 0.857231 0.258964
0.452848 0.0955061 0.888978
0.597493 0.59125 0.709174
0.60472 0.107532 0.986523
0.705191 0.417332 0.471154
0.144499 0.682505 0.0497819
0.299998 0.906613 0.984909
0.88048 0.077889 0.302946
0.782197 0.700507 0.233419
0.935966 0.255638 0.846485
0.541445 0.355736 0.341638
0.383306 0.022498 0.0896589
0.969174 0.853529 0.339466
0.610549 0.755872 0.618453
0.596454 0.0678199 0.155821
0.351357 0.0420721 0.137743
|
julia> grid[:, end:-1:1]
24×3 Matrix{Float64}: Mhhh this escapes the grid, just wrap it back in one, like |
Dear Dr. Klöwer |
Sorry this was automatically closed because I linked the pull request, we can continue the discussion here, might be better |
From #643 Dear Dr. Klöwer I find there are small changes in temp, humid and pres, but there are large changes in vor and div. |
Dear Dr. Klöwer,
I am writing to express my admiration for your work on the SpeedyWeather.jl model, which I came across recently.
Your development of SpeedyWeather.jl has truly revolutionized the way we think about traditional climate models. The modular approach you have adopted allows researchers like myself to easily adapt and explore various aspects of climate model. This innovation is particularly commendable, as it significantly enhances the flexibility and usability of climate modeling tools.
I am particularly interested in using reanalysis data as the initial data in SpeedyWeather.jl. However, I am facing some challenges in implementing this. I would greatly appreciate your guidance on how to effectively incorporate reanalysis data into the model as an initial data.
Thank you very much for considering my request. I look forward to the possibility of engaging with your expertise and potentially contributing to the ongoing discourse in climate modeling.
The text was updated successfully, but these errors were encountered: