Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tempfiles do not live long #9

Open
jhardenberg opened this issue Mar 15, 2023 · 1 comment
Open

tempfiles do not live long #9

jhardenberg opened this issue Mar 15, 2023 · 1 comment

Comments

@jhardenberg
Copy link
Owner

jhardenberg commented Mar 15, 2023

the function cdo_generate_weights uses the tempfile to generate a tempfile weight_file for the weights, which is then read to an xarray with

 weights = xarray.open_dataset(weight_file.name, engine="netcdf4")

The problem is that this tempfile at this point, when exiting the function appears to the system not to be needed anymore, so that it will be erased at the first best occasion (at best at the next garbage collection). If we open it with xarray using open_dataset though, this is a lazy opening and relies on the file still existing.

The solution would be to open the xarray with xarray.load_dataset but this will load everything in memory. Might this be an issue for very large weight files?

@oloapinivad
Copy link
Collaborator

there is a delete=False option that can be used, but it will require that the removal is applied in a later stage

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants