Attempt to load avi throws importerror

[EricThomson]

In Windows 10, dev install, dev branch, demo_online_cnmfE.ipynb, when I hit the line mc.motion_correct(save_movie=True), I get the following:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[5], line 2
      1 mc = MotionCorrect(fnames, dview=dview, **opts.get_group('motion'))
----> 2 mc.motion_correct(save_movie=True)

File c:\users\eric\development\dev_packages\caiman\caiman\motion_correction.py:248, in MotionCorrect.motion_correct(self, template, save_movie)
    245         self.min_mov = mi
    246     else:
    247         self.min_mov = np.array([high_pass_filter_space(m_, self.gSig_filt)
--> 248             for m_ in cm.load(self.fname[0], var_name_hdf5=self.var_name_hdf5,
    249                               subindices=slice(400))]).min()
    251 if self.pw_rigid:
    252     self.motion_correct_pwrigid(template=template, save_movie=save_movie)

File c:\users\eric\development\dev_packages\caiman\caiman\base\movies.py:1479, in load(file_name, fr, start_time, meta_data, subindices, shape, var_name_hdf5, in_memory, is_behavior, bottom, top, left, right, channel, outtype, is3D)
   1476 def rgb2gray(rgb):
   1477     return np.dot(rgb[..., :3], [0.299, 0.587, 0.114])
-> 1479 pims_movie = pims.Video(file_name)
   1480 length = len(pims_movie)
   1481 height, width = pims_movie.frame_shape[0:2]    #shape is (h,w,channels)

File ~\miniconda3\envs\caiman_dev\lib\site-packages\pims\api.py:24, in not_available.<locals>.raiser(*args, **kwargs)
     23 def raiser(*args, **kwargs):
---> 24     raise ImportError(
     25         "This reader requires {0}.".format(requirement))

ImportError: This reader requires PyAV, MoviePy, or ImageIO.

I have ImageIO installed.

[EricThomson]

Note I also have pyav installed.

[EricThomson]

I'm looking at this fairly old issue suggesting pims/pyav versioning, but I have most recent releases of each:
soft-matter/pims#311

[EricThomson]

I reproduced this problem in an independent environment (not caiman), and can't load this or other avi files, and am getting this same error. So it seems to be a caiman-independent problem (at least on Windows 10). I'll open an issue over at the pims library. I'm now confused why I didn't see this problem earlier when I tested this notebook on the old master branch.

[pgunn]

Actually, I can reproduce this too (if I force pims to load the movie, bypassing opencv). I wonder why opencv is failing for you)

[EricThomson]

Yes I guess that is the bigger more annoying question.

I was able to get pims to work by installing with av<10.0 : there seem to be some issues with pyav (relatively) new move from 9->10, even on Linux (soft-matter/trackpy#737).

[EricThomson]

Ok for this issue I'm going to just work on what happens after fixing this problem: in my dev env, I mamba install -c conda-forge "av<10.0" and the errors stopped happening with pims. But this means, we are now testing the branches in the code when open-cv fails.

Which is I think a useful test, as other things are starting to fail. I'll go through and fix them and do a PR, and we can get back to the weird opencv fail after.

[EricThomson]

Fixing the cv_failed branch will take some work, as the dims variable is used in the if subindices is not None conditional for patch processing in motion correction (starting on line 1433 of movies.py), which throws an error when they are 0, as dims is not re-defined in the pims movie load branch that things are shunted to when the opencv loader doesn't work.

CaImAn/caiman/base/movies.py

Line 1433 in b3e4b72

if subindices is not None:

So I think we basically have two things to fix here: loading avi with opencv in windows, and fix the cv_failed use case. I have a few other things I'm trying to finish this week, so won't be able to sprint through these quickly.

[EricThomson]

It looks like opencv will load avi correctly in Windows in version 4.7 of opencv, which just came out recently (a little over two weeks ago). It may be a little while (e.g., release after next) before this is ready for prime time.

[pgunn]

Interesting that on conda-forge, OpenCV seems to have skipped a python 3.10 build. I wonder why things worked out that way. Their build matrix is 3.8/3.9/3.11.

[pgunn]

Actually nevermind that, I misread the binary list.

[pgunn]

(already covered on slack, updating the ticket)

Right now tensorflow is going to hold us back from OpenCV 4.7, but only on Windows. conda-forge sometime back migrated from libjpeg to libjpeg-turbo (which was a fork of libjpeg, but not of the final version of it, so the APIs differ a bit - long story) . Most software in the repo got rebuilt. OpenCV hasn't had a release for a very long time (they're very behind in general and have a complex release history), but about half a month ago did their most recent conda-forge release (of code that's considerably older - their next source release came out at about the same time), which finally put them on libjpeg-turbo.

We get tensorflow from conda-forge on 2 platforms - osx and linux, and on those platforms it uses libjpeg-turbo. Unfortunately, its least-supported platform, windows, hasn't been available on conda-forge for a long time now, so we get it from defaults there, where it's still built with classic libjpeg. Which is the root of the problem. If you try to include both tensorflow and the "new" opencv in the same project, they won't want to coexist because libjpeg and libjpeg-turbo don't want to coexist in the same environment and that ripples up.

I filed an issue against the tensorflow recipe with ContinuumIO (or whatever they're calling themselves nowadays), which despite appearances actually is still how tensorflow is built nowadays (they just use branches weird - look at the non-master branches and you'll see current activity elsewhere). No idea how long if they're open to or even technically could change over just their tensorflow or if this would have to be the entire defaults channel. Either way, let's not hold our breath.

A notebook is a promise that things will work. Let's not break it - this is a release blocker, so we'll need to find another way to make this go.

[pgunn]

The issue I filed:
AnacondaRecipes/tensorflow_recipes#46

[pgunn]

Other options to get the next release out the door:

1. Fix the pims code path, possibly by adding more dependencies
2. Find another opensource library that can decode avis
3. Temporarily-or-not change the demo code to use some other file format, and hope users don't have avi-formatted data until this all gets sorted out

[EricThomson]

Fixed in most recent release!