Flexible Dask architecture

[benbovy]

Related to this blog post and this thread.

A good way to support those architectures in xpublish would be to have a get_dask_client FastAPI dependency. Like the other resources (dataset, cache, etc.), this dependency would be overridden when initializing the FastAPI application instance in Rest with a dependency function implementing one of the architectures mentioned in the blog post. This would also be extensible to any user-defined architecture.

Perhaps the dask client and cache resources can be independent or tightly coupled to each other, depending on the architecture? I guess accessing those resources as nested dependencies would probably help supporting both cases.

Any thoughts @andersy005 @jhamman?