Add spatial slice parallelization tool

[keflavich]

This depends on #506. It splits out the parallelization stuff, and the memory mapped application, into spatial and spectral components

[keflavich]

I believe this works as is, but we don't use it anywhere yet. More work is needed to get something like reproject working, since the parallelizer presently assumes the input and output cubes are the same shape. Smoothing should be easy to parallelize though...

[keflavich]

@astrofrog @e-koch want to give this a quick review? I think the concept is reasonably solid. I could use tips on how to reasonably test this... we probably want some performance tests to go along with the correctness tests.

@astrofrog should this be booted in favor of dasktropyarrayquantity magic?

[e-koch]

The parallel framework looks good to me. Apart from the tests, we also need some way to update the mask. Can we use a LazyMask on the memmap file?

For the tests, is specifying parallel=True with one core enough to test the joblib parts? The unfortunate part is that we probably need each of these methods tested with and without parallelizing.

What did you have in mind for performance tests?

[keflavich]

I don't have a clear plan for performance tests.

What are you thinking about in terms of modifying the mask? I don't think we necessarily need any way to update the mask on the fly within the parallelized loop. We can just update it after the fact. No?

[e-koch]

I agree with updating the mask after the operation. Much simpler. I can't remember if there was a specific operation that I thought would require updating the mask within the loop.

[keflavich]

By default, I don't think we should do anything to the mask, then. We should leave it to the user (or the specific operation) to update the mask after running apply_function_parallel_spatial. At the moment, we don't have any functions using apply_function_parallel_spatial, but we should come up with some so we have some practical use and test cases in hand.

[e-koch]

Alright. It's probably worth showing the mask updating step in one of the examples, then.

How about converting convolve_to to use the new spatial function? spatial_smooth and spatial_median_smooth could be, too. Then we could just expand the existing test cases and docs for those functions.

[keflavich]

Good idea. Next time I have a few hours to put into this, I'll do it.

[coveralls]

Coverage increased (+0.2%) to 74.887% when pulling 5dcded1 on keflavich:spatial_parallelization into 28ff5c2 on radio-astro-tools:master.

[keflavich]

Since the kernel changes with slice in VaryingResolutionSpectralCube.convolve_to, it is not straightforward to convert that function, unfortunately. If you have ideas on how to do that, please chime in, but we'll have to punt that to a new issue for now.

[keflavich]

@e-koch the smooth tests now actually test this. I think it's good to merge?

[e-koch]

LGTM!