You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I was getting strange resource exhausted bug when running xlearn fm model predict method for a while.
When I profiled the processes via htop, I have noticed that the number of threads gradually increases by 8 when invoking model.predict("model/model.out", f"output/output.txt") which leads to resource exhausted when the number of threads reaches a critical level.
One solution, I found to solve this problem is invoke the model.predict in a separate process via the multiprocessing module, however this solution is extremely slow in cases when model.predict needs to be invoked many times.
Is there a way to kill the created threads after the execution of the predict method has completed?
The text was updated successfully, but these errors were encountered:
HovhannesManushyan
changed the title
xlearn's .predict method doesn't kill the created threads after execution in python API, which leads to resource exhausted.
xlearn python API's .predict method in doesn't kill the created threads after execution in python API, which leads to resource exhausted.
Aug 24, 2021
I was getting strange resource exhausted bug when running xlearn fm model predict method for a while.
When I profiled the processes via htop, I have noticed that the number of threads gradually increases by 8 when invoking
model.predict("model/model.out", f"output/output.txt")
which leads to resource exhausted when the number of threads reaches a critical level.One solution, I found to solve this problem is invoke the model.predict in a separate process via the multiprocessing module, however this solution is extremely slow in cases when model.predict needs to be invoked many times.
Is there a way to kill the created threads after the execution of the predict method has completed?
The text was updated successfully, but these errors were encountered: