Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Long compile time for differentiation transform test in order SH -> Grid -> SH #626

Open
maximilian-gelbrecht opened this issue Dec 2, 2024 · 1 comment
Labels
differentiability 🤖 Making the model differentiable via AD testing 🧪 How we test things and continuous integration

Comments

@maximilian-gelbrecht
Copy link
Member

maximilian-gelbrecht commented Dec 2, 2024

For some reason the unit test of the the differentiability of the full transform needs a lot of compilation time when starting in spectral space with the OctahedralGaussianGrid.

function transform_identity!(x::LowerTriangularArray{Complex{T}}, S::SpectralTransform{T}) where T
x_grid = zeros(S.Grid{T}, S.nlat_half, S.nlayers)
transform!(x_grid, x, S)
transform!(x, x_grid, S)
return nothing
end
spec = transform(grid, S)
spec_copy = deepcopy(spec)
transform_identity!(spec, S)
@test isapprox(spec, spec_copy)
dspec = similar(spec)
fill!(dspec, 1+im)
autodiff(Reverse, transform_identity!, Const, Duplicated(spec, dspec), Duplicated(S, dS))

This is not an issue when starting in grid space, or with the FullGaussianGrid. The results are correct, and every subsequent execution is also fast. It's the compilation that takes so long.

Not sure why that is. As every subsequent execution is fast, I am not too worried, and I'll just exclude this test from the CI for now, but this might be something to look at it again later.

@milankl
Copy link
Member

milankl commented Dec 6, 2024

I'm surprised that there's a difference between the full/octahedral grids. From a software engineering perspective they are super similar. Same definition, same constructors, same place in the type hierarchy just on another branch, same Legendre polynomials, it's just that FFTW plans either 24 (at default resolution) identical plans (full) or 24 different plans (octahedral). But why would that matter?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
differentiability 🤖 Making the model differentiable via AD testing 🧪 How we test things and continuous integration
Projects
None yet
Development

No branches or pull requests

2 participants