atlas_commons.exceptions.AtlasCommonsError when making glial density

[HDictus]

when I run the command glia-cell-densities, I get the following error:

  File "/gpfs/bbp.cscs.ch/home/dictus/py39/lib/python3.9/site-packages/atlas_commons/app_utils.py",                                                                                                        
line 77, in compare_all                                                                                                                                                                                    
    res = all(comp(fun(data_sets[0]), fun(other)) for other in data_sets[1:])                                                                                                                              
  File "/gpfs/bbp.cscs.ch/home/dictus/py39/lib/python3.9/site-packages/atlas_commons/app_utils.py",                                                                                                        
line 77, in <genexpr>                                                                                                                                                                                      
    res = all(comp(fun(data_sets[0]), fun(other)) for other in data_sets[1:])                                                                                                                              
  File "<__array_function__ internals>", line 180, in allclose                                                                                                                                             
  File "/gpfs/bbp.cscs.ch/home/dictus/py39/lib/python3.9/site-packages/numpy/core/numeric.py", line                                                                                                        
2251, in allclose                                                                                                                                                                                          
    res = all(isclose(a, b, rtol=rtol, atol=atol, equal_nan=equal_nan))                                                                                                                                    
  File "<__array_function__ internals>", line 180, in isclose                                                                                                                                              
  File "/gpfs/bbp.cscs.ch/home/dictus/py39/lib/python3.9/site-packages/numpy/core/numeric.py", line                                                                                                        
2361, in isclose                                                                                                                                                                                           
    return within_tol(x, y, atol, rtol)                                                                                                                                                                    
  File "/gpfs/bbp.cscs.ch/home/dictus/py39/lib/python3.9/site-packages/numpy/core/numeric.py", line                                                                                                        
2342, in within_tol                                                                                                                                                                                        
    return less_equal(abs(x-y), atol + rtol * abs(y))                                                                                                                                                      
ValueError: operands could not be broadcast together with shapes (4,) (3,)                                                                                                                                 
                                                                                                                                                                                                           
The above exception was the direct cause of the following exception:                                                                                                                                       
                                                                                                                                                                                                           
Traceback (most recent call last):                                                                                                                                                                         
  File "/gpfs/bbp.cscs.ch/home/dictus/py39/bin/atlas-densities", line 33, in <module>                                                                                                                      
    sys.exit(load_entry_point('atlas-densities', 'console_scripts', 'atlas-densities')())                                                                                                                  
  File "/gpfs/bbp.cscs.ch/project/proj148/atlas/atlas-densities/atlas_densities/app/cli.py", line 24                                                                                                       
, in cli                                                                                                                                                                                                   
    app()                                                                                                                                                                                                  
  File "/gpfs/bbp.cscs.ch/home/dictus/py39/lib/python3.9/site-packages/click/core.py", line 1130, in                                                                                                       
 __call__                                                                                                                                                                                                  
    return self.main(*args, **kwargs)                                                                                                                                                                      
  File "/gpfs/bbp.cscs.ch/home/dictus/py39/lib/python3.9/site-packages/click/core.py", line 1055, in                                                                                                       
 main                                                                                                                                                                                                      
    rv = self.invoke(ctx)                                                                                                                                                                                  
  File "/gpfs/bbp.cscs.ch/home/dictus/py39/lib/python3.9/site-packages/click/core.py", line 1657, in                                                                                                       
 invoke                                                                                                                                                                                                    
    return _process_result(sub_ctx.command.invoke(sub_ctx))                                                                                                                                                
  File "/gpfs/bbp.cscs.ch/home/dictus/py39/lib/python3.9/site-packages/click/core.py", line 1657, in                                                                                                       
 invoke                                                                                                                                                                                                    
    return _process_result(sub_ctx.command.invoke(sub_ctx))                                                                                                                                                
  File "/gpfs/bbp.cscs.ch/home/dictus/py39/lib/python3.9/site-packages/click/core.py", line 1404, in                                                                                                       
 invoke                                                                                                                                                                                                                                                                                                                                                                                                      
    return ctx.invoke(self.callback, **ctx.params)                                                                                                                                                         
  File "/gpfs/bbp.cscs.ch/home/dictus/py39/lib/python3.9/site-packages/click/core.py", line 760, in                                                                                                        
invoke                                                                                                                                                                                                     
    return __callback(*args, **kwargs)                                                                                                                                                                     
  File "/gpfs/bbp.cscs.ch/home/dictus/py39/lib/python3.9/site-packages/atlas_commons/app_utils.py",                                                                                                        
line 51, in wrapper                                                                                                                                                                                        
    function(*args, **kw)                                                                                                                                                                                  
  File "/gpfs/bbp.cscs.ch/project/proj148/atlas/atlas-densities/atlas_densities/app/cell_densities.p                                                                                                       
y", line 297, in glia_cell_densities                                                                                                                                                                       
    assert_properties(atlases)                                                                                                                                                                             
  File "/gpfs/bbp.cscs.ch/home/dictus/py39/lib/python3.9/site-packages/atlas_commons/app_utils.py",                                                                                                        
line 93, in assert_properties                                                                                                                                                                              
    if not compare_all(atlases, lambda x: x.raw.shape, comp=np.allclose):                                                                                                                                  
  File "/gpfs/bbp.cscs.ch/home/dictus/py39/lib/python3.9/site-packages/atlas_commons/app_utils.py",                                                                                                        
line 79, in compare_all                                                                                                                                                                                    
    raise AtlasCommonsError("[compare_all] Bad operation during comparing") from error_                                                                                                                    
atlas_commons.exceptions.AtlasCommonsError: [compare_all] Bad operation during comparing

Can anyone easily tell me what may be wrong here? If so, maybe we can also put a more informative error in place.

[eleftherioszisis]

Hi @HDictus . At a first glance, it seems like the input volume datasets do not have the same dimensions/shape. Could you please verify if this is the cases?

[HDictus]

all the voxeldata have the same shape, however the underlying numpy array for the astrocyte, microglia, and oligodendrocyte relative density do have 3 channels to my surprise (shape (528, 320, 456, 3))
These were output by combine-markers step, is this the shape we expect them in from that?

[eleftherioszisis]

This is peculiar because combine-markers should output 3D volumes, one per type of glial cell.

[HDictus]

The problem seems to originate in Deep-atlas, the interpolated volumes have this shape

[drodarie]

DeepAtlas produces RGB volumes, because antspy was designed for color images. The ISH datasets however do not have colors, ISH expression uses a single dimension.
You can average the RGB values of the DeepAtlas volumes to obtain the correct ISH signal. Either way the RGB values should be closed to each other.

[HDictus]

That seems to do the trick, thanks!
I'll make a quick PR with a check and error for the volume's dimensions