Fixes to Perens and Kim mouse atlases

[viktorpm]

Description

This PR fixes IndexError: tuple index out of range of perens_lsfm_mouse.py (see: #249) and the orientation and scaling issue of perens_lsfm_moue and kim_mouse atlases mentioned in #265

What is this PR

• Bug fix
• Addition of a new feature
• Other

What does this PR do?

fixes #265

References

#265

How has this PR been tested?

tested on HPC and locally

Is this a breaking change?

no

Does this PR require an update to the documentation?

Yes, when new versions of the atlases are published

Checklist:

• The code has been tested locally
• Tests have been added to cover all new functionality (unit & integration)
• The documentation has been updated to reflect any changes
• The code has been formatted with pre-commit

[viktorpm]

Errors to be solved:

Script - error number	Error	Note	Solved
admba_3d_dev_mouse.py - 2	gets stuck at region 112892417 of admba_3d_p28_mouse	Creating mesh for region 112892417 No labels found for r10LimCG (112892417)	no
admba_3d_dev_mouse.py - 3	doesn't skip already generated atlases	FileExistsError: [Errno 17] File exists:	no
humanatlas.py - 1	FileNotFoundError: No such file or no access: 'D:\Dropbox (UCL -SWC...'	hardcoded paths in the code	no, needs separate PR
mpin_zfish.py - 2	SSLError: HTTPSConnectionPool(...)	-	no
perens_lsfm_mouse.py - 1	IndexError: tuple index out of range		yes

perens IndexError: tuple index out of range error is fixed

[viktorpm]

perens orientation issue is fixed

[viktorpm]

Atlas generation on HPC:

#!/bin/bash

#SBATCH -p cpu # partition (queue)
#SBATCH -N 1   # number of nodes
#SBATCH --mem 160G # memory pool for all cores
#SBATCH -n 10
#SBATCH -t 0-8:0 # time (D-HH:MM)
#SBATCH -o atlas_regeneration.out
#SBATCH -e atlas_regeneration.err
#SBATCH --mail-type=ALL
#SBATCH [email protected]



# Load mamba module
module load mamba

# Initialize mamba
mamba init

# Create environment
#mamba create -n bg-atlasapi python=3.11

# Activate environment
mamba activate bg-atlasapi

cd _projects

# Clone repository
git clone https://github.com/brainglobe/brainglobe-atlasapi.git

# Navigate to repository directory
cd brainglobe-atlasapi/

# Checkout specific branch
git checkout issue_265

# Install required packages
pip install -e ".[allenmouse, dev]"
# pip install .
# pip install .[allenmouse,dev]
# pip install brainglobe-atlasapi
# pip install brainglobe-atlasapi[allenmouse,dev]
pip install scikit-image
pip install imio
pip install brainio

# Navigate to specific directory
cd brainglobe_atlasapi/atlas_generation/atlas_scripts

# Define resolutions
resolutions=(100 50 25 10)

# Loop over resolutions and run the script for each
for resolution in "${resolutions[@]}"; do
    python kim_mouse.py --resolution=$resolution
done

[viktorpm]

New atlases were generated using the updated script. Once they pass the validation functions, they can be published.

kim_mouse_25um:

[alessandrofelder]

I am confused about the argument parser introduced here.
Otherwise looks great :)

[alessandrofelder]

Why are we adding an argument parser here?
I think it should be a loop over the resolutions we'd like to provide?

[adamltyson]

Yeah I think the atlas generation scripts are meant to (?) loop over resolutions.

[viktorpm]

It didn't do it. Only one atlas was generated when I submitted a job on the HPC and I didn't want to change the resolution value in the code manually, push the new version and submit a new job for each resolution. I might be missing something.

[alessandrofelder]

I didn't want to change the resolution value in the code manually, push the new version and submit a new job for each resolution.

I agree this is a worthy goal - my point is that I think this should not be achieved by a command line argument, but rather with a for loop in the code (so the script generates all four atlases in the same run)

[adamltyson]

I thought this was already in the code, but it's what the main script (see #273) coordinates.

[viktorpm]

I see what you mean now. Will I still have the option to easily specify which atlas I want to generate? For example, if I only need the 100um one, I wouldn't want to wait for the script to generate all the others.

[alessandrofelder]

I don't think you will, no.
If you'd like that flexibility, you probably want to clone the repository on the HPC, and edit the script via the command line?

[adamltyson]

Having thought about this, I think it's best to:

• Stick with what you have now
• Later on, remove it and sort out the main script (unless anyone has the capacity to address Decide what to do with atlas_generation/main_script #273 now?

@alessandrofelder?

[adamltyson]

I think we commented at the same time @alessandrofelder. I think we ideally shouldn't add the parser to the script, but getting fixed versions of the atlases out should be a priority.

[viktorpm]

Passed the validation functions (without the additional_references functions, see #275):
validation_results.json

• kim_mouse_25um
• kim_mouse_50um
• kim_mouse_100um
• perens_lsfm_mouse_20um

kim_mouse_10um is still being regenerated

[viktorpm]

kim_mouse_10um passed the validation functions

[adamltyson]

Does this also close #247?

[viktorpm]

Yes, it does

[alessandrofelder]

• Quick check that locally the issue is gone was successful ✅
• Opened [Feature] flexibly generate atlases with different resolutions  #280 to think further about what the packager's interface should be for multi-res atlases