You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Using dask for grid search can lead to huge performance improvements over multiprocessing due to better use of data locality. However, this raises a question of interface. Dask is a much larger tool, with many more intricacies. For example, I've had dask hang when it's used with multithreaded numba code. I've had other problems monitoring output when dask was launched through a notebook. This was mostly a year ago, so I'm unsure if any of these problems have been resolved.
What is the simplest way to provide a fool-proof dask based clustering backend?
One solution is to just provide it as a script, but this isn't the nicest api. Ideally it could be run through a notebook, and I'd like users to be able to provide their own client. I need to investigate how plausible this is.
The text was updated successfully, but these errors were encountered:
Using dask for grid search can lead to huge performance improvements over multiprocessing due to better use of data locality. However, this raises a question of interface. Dask is a much larger tool, with many more intricacies. For example, I've had dask hang when it's used with multithreaded numba code. I've had other problems monitoring output when dask was launched through a notebook. This was mostly a year ago, so I'm unsure if any of these problems have been resolved.
What is the simplest way to provide a fool-proof dask based clustering backend?
One solution is to just provide it as a script, but this isn't the nicest api. Ideally it could be run through a notebook, and I'd like users to be able to provide their own client. I need to investigate how plausible this is.
The text was updated successfully, but these errors were encountered: