Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add minimum package versions for ESRI/GDAL nodata fix #288

Merged
merged 7 commits into from
Jul 19, 2024
Merged

Conversation

robbibt
Copy link
Member

@robbibt robbibt commented Jul 16, 2024

This PR pins minimum versions of odc-geo and eodatasets to ensure we have completely resolved the ESRI/GDAL nodata issue. Key changes include:

eodatasets3 >= 0.30.6  # min version required for ESRI/GDAL nodata fix
datacube[performance,s3] >= 1.8.19  # min version required for ESRI/GDAL nodata fix
odc-geo >= 0.4.8  # min version required for ESRI/GDAL nodata fix

@robbibt
Copy link
Member Author

robbibt commented Jul 17, 2024

Hey @emmaai @SpacemanPaul, any idea what is causing the integration tests to fail here? It seems like they get skipped before they even run, and I can't see anything obvious in the logs...
image

@SpacemanPaul
Copy link

I can't see anything in the logs to explain the failure either. Have you tried re-running?

@emmaai
Copy link
Contributor

emmaai commented Jul 17, 2024

Is it necessary to copy that much auxiliary data? it blows up the disk, only 15G is available for a GitHub workflow if my memory is reliable. there is no room to pull docker image or compile one. Also the stdout is too much, it shouldn't be allowed unless debugging is required, 10s of thousands lines output will blow up any ephemeral storage for logs.

@robbibt
Copy link
Member Author

robbibt commented Jul 17, 2024

Is it necessary to copy that much auxiliary data? it blows up the disk, only 15G is available for a GitHub workflow if my memory is reliable. there is no room to pull docker image or compile one. Also the stdout is too much, it shouldn't be allowed unless debugging is required, 10s of thousands lines output will blow up any ephemeral storage for logs.

Ah, I know what is going wrong - we copied some extra data into the tide model directory yesterday. I'll update the tests to only copy a small portion here

@robbibt
Copy link
Member Author

robbibt commented Jul 17, 2024

Thanks @emmaai, that fixed it

@emmaai
Copy link
Contributor

emmaai commented Jul 18, 2024

Borrowed the pr to install awscliv2 from conda, I meant to make the change a while ago. thanks!

@emmaai
Copy link
Contributor

emmaai commented Jul 18, 2024

Thanks @emmaai, that fixed it

not sure if it did, the indentation of yaml was not right and hence the action failed. fixed in the commits

@robbibt robbibt marked this pull request as ready for review July 19, 2024 05:51
@robbibt robbibt requested a review from emmaai July 19, 2024 05:51
@robbibt robbibt merged commit 2953bd0 into develop Jul 19, 2024
5 checks passed
@robbibt robbibt deleted the nodata_fixes branch July 19, 2024 05:58
@emmaai
Copy link
Contributor

emmaai commented Jul 22, 2024

is there any reason that geopandas<1.0? It'll cause compatible issue with Athena query of aws.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants