Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor VCE RARE transform to reduce peak memory usage #3925

Closed
Tracked by #3914
jdangerx opened this issue Oct 23, 2024 · 0 comments · Fixed by #3959
Closed
Tracked by #3914

Refactor VCE RARE transform to reduce peak memory usage #3925

jdangerx opened this issue Oct 23, 2024 · 0 comments · Fixed by #3959
Assignees
@zschira
Labels
dagster Issues related to our use of the Dagster orchestrator nightly-builds Anything having to do with nightly builds or continuous deployment. performance Make PUDL run faster! vcerare Pertaining to Vibrant Clean Energy's Resource Adequacy Renewable Energy Power datasets

Comments

@jdangerx
Copy link
Member

No description provided.

@jdangerx jdangerx mentioned this issue Oct 23, 2024
Open
@jdangerx jdangerx moved this from New to Backlog in Catalyst Megaproject Oct 23, 2024
@zaneselvans zaneselvans added performance Make PUDL run faster! dagster Issues related to our use of the Dagster orchestrator nightly-builds Anything having to do with nightly builds or continuous deployment. vcerare Pertaining to Vibrant Clean Energy's Resource Adequacy Renewable Energy Power datasets labels Oct 23, 2024
@zaneselvans zaneselvans changed the title Refactor transform to reduce peak memory use so we can use the 8CPU/64GB VM in nightly builds Refactor VCE transform to reduce peak memory usage Oct 23, 2024
@zaneselvans zaneselvans changed the title Refactor VCE transform to reduce peak memory usage Refactor VCE RARE transform to reduce peak memory usage Oct 23, 2024
@zschira zschira mentioned this issue Nov 14, 2024
Merged
@github-project-automation github-project-automation bot moved this from In progress to Done in Catalyst Megaproject Nov 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zschira zschira

Labels
dagster Issues related to our use of the Dagster orchestrator nightly-builds Anything having to do with nightly builds or continuous deployment. performance Make PUDL run faster! vcerare Pertaining to Vibrant Clean Energy's Resource Adequacy Renewable Energy Power datasets
Projects
Catalyst Megaproject
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Reduce peak memory use of VCE RARE assets catalyst-cooperative/pudl
3 participants
@zaneselvans @jdangerx @zschira