Memory Leak with LocalCUDACluster due to Excessive Objective Instances

[Guinan-Su]

Hi！

When using LocalCUDACluster with worker=4, I’ve noticed a memory leak due to the repeated creation of objective instances during evaluation. Currently, it appears that each evaluation step creates a new instance without releasing memory from the old ones, which eventually leads to high memory usage.

Ideally, I’d like a way to:

1. Limit Instance Creation: Restrict the number of objective instances to the number specified by the worker setting and reuse these old instances ? (e.g., 4).
2. Implement a cleanup mechanism for old instances

I’ve looked through the documentation but haven’t found any existing features or configuration options that would allow this behavior. Is there an existing approach or best practice within the framework to achieve this?

Thank you for any insights or suggestions

Guinan

[benjamc]

Hi Guinan,

by instances, do you mean the evaluation of the target function or are you doing algorithm configuration?
Can you release the memory at the end of your target function?
This sounds a bit dask/cuda related

Best wishes