Skip to content

Commit

Permalink
Merge branch 'main' into dev.
Browse files Browse the repository at this point in the history
This reconciles the two divergent histories of the old `brainglobe-scripts` and `cellfinder` (CLI tool).
  • Loading branch information
willGraham01 committed Nov 15, 2023
2 parents 9d31eb9 + e69e83e commit 4a95ca4
Show file tree
Hide file tree
Showing 20 changed files with 1,553 additions and 140 deletions.
17 changes: 10 additions & 7 deletions .github/workflows/test_and_deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,20 +32,22 @@ jobs:
- uses: neuroinformatics-unit/actions/check_manifest@v2

test:
needs: [linting,manifest]
needs: [linting, manifest]
name: ${{ matrix.os }} py${{ matrix.python-version }}
runs-on: ${{ matrix.os }}
env:
INPUT_COREDEV: ${{ github.event.inputs.coredev }}
strategy:
fail-fast: false
matrix:
# Run across a mixture of Python versions and operating systems
# Run tests on ubuntu across all supported versions
python-version: ["3.9", "3.10"]
os: [ubuntu-latest]
# Include at least one MacOS and Windows test
include:
- os: macos-latest
python-version: "3.10"
- os: windows-latest
python-version: "3.9"
- os: ubuntu-latest
python-version: "3.10"
steps:
- uses: neuroinformatics-unit/actions/test@v2
Expand All @@ -60,7 +62,8 @@ jobs:
steps:
- uses: neuroinformatics-unit/actions/build_sdist_wheels@v2

deploy:
upload_all:
name: Publish build distributions
needs: [build_sdist_wheels]
runs-on: ubuntu-latest
if: github.event_name == 'push' && github.ref_type == 'tag'
Expand All @@ -75,15 +78,15 @@ jobs:
password: ${{ secrets.TWINE_API_KEY }}

build-and-push-docker:
needs: deploy
needs: [upload_all]
runs-on: ubuntu-latest
permissions:
contents: read
packages: write

steps:
- name: Checkout repository
uses: actions/checkout@v2
uses: actions/checkout@v4

- name: Log in to the Container registry
uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
Expand Down
21 changes: 13 additions & 8 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -77,10 +77,17 @@ doc/build/
_build/
mkdocs.yml

# MkDocs documentation
site/

# PyBuilder
target/

# Jupyter Notebook
# Pycharm and VSCode
.idea/
.vscode/

# IPython Notebook
.ipynb_checkpoints

# pyenv
Expand Down Expand Up @@ -109,18 +116,16 @@ venv.bak/
# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/

.idea/

*.~lock.*

pip-wheel-metadata/

*.DS_Store
# OS
.DS_Store

*.vscode/
# written by setuptools_scm
**/_version.py
benchmarks/results/*
6 changes: 6 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,9 @@ repos:
additional_dependencies:
- types-setuptools
- types-requests
- repo: https://github.com/mgedmin/check-manifest
rev: "0.49"
hooks:
- id: check-manifest
args: [--no-build-isolation]
additional_dependencies: [setuptools-scm]
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
BSD 3-Clause License

Copyright (c) 2020, University College London
Copyright (c) 2023, BrainGlobe developers.
All rights reserved.

Redistribution and use in source and binary forms, with or without
Expand Down
9 changes: 8 additions & 1 deletion MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,16 +1,23 @@
include README.md
include LICENSE
include brainglobe_workflows/cellfinder/default_config.json

exclude .pre-commit-config.yaml
exclude *.yaml
exclude *.yml
exclude Dockerfile
exclude *.ini
exclude asv.conf.json

recursive-include brainglobe_workflows *.py
include brainglobe_workflows/config/*

recursive-exclude * __pycache__
recursive-exclude * *.py[co]

global-include *.pxd

prune benchmarks
prune docs
prune tests
prune resources

Expand Down
187 changes: 88 additions & 99 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,150 +1,139 @@
[![Python Version](https://img.shields.io/pypi/pyversions/cellfinder.svg)](https://pypi.org/project/cellfinder)
[![PyPI](https://img.shields.io/pypi/v/cellfinder.svg)](https://pypi.org/project/cellfinder)
[![Downloads](https://pepy.tech/badge/cellfinder)](https://pepy.tech/project/cellfinder)
[![Wheel](https://img.shields.io/pypi/wheel/cellfinder.svg)](https://pypi.org/project/cellfinder)
[![Development Status](https://img.shields.io/pypi/status/cellfinder.svg)](https://github.com/brainglobe/cellfinder)
[![Tests](https://img.shields.io/github/workflow/status/brainglobe/cellfinder/tests)](
https://github.com/brainglobe/cellfinder/actions)
[![codecov](https://codecov.io/gh/brainglobe/cellfinder/branch/master/graph/badge.svg?token=s3MweEFPhl)](https://codecov.io/gh/brainglobe/cellfinder)
[![Python Version](https://img.shields.io/pypi/pyversions/brainglobe-workflows.svg)](https://pypi.org/project/brainglobe-workflows)
[![PyPI](https://img.shields.io/pypi/v/brainglobe-workflows.svg)](https://pypi.org/project/brainglobe-workflows)
[![Downloads](https://pepy.tech/badge/brainglobe-workflows)](https://pepy.tech/project/brainglobe-workflows)
[![Wheel](https://img.shields.io/pypi/wheel/brainglobe-workflows.svg)](https://pypi.org/project/brainglobe-workflows)
[![Development Status](https://img.shields.io/pypi/status/brainglobe-workflows.svg)](https://github.com/brainglobe/brainglobe-workflows)
[![Tests](https://img.shields.io/github/workflow/status/brainglobe/brainglobe-workflows/tests)](
https://github.com/brainglobe/brainglobe-workflows/actions)
[![codecov](https://codecov.io/gh/brainglobe/brainglobe-workflows/branch/master/graph/badge.svg?token=s3MweEFPhl)](https://codecov.io/gh/brainglobe/brainglobe-workflows)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/python/black)
[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336)](https://pycqa.github.io/isort/)
[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit)
[![Contributions](https://img.shields.io/badge/Contributions-Welcome-brightgreen.svg)](https://docs.brainglobe.info/cellfinder/contributing)
[![Website](https://img.shields.io/website?up_message=online&url=https%3A%2F%2Fbrainglobe.info)](https://brainglobe.info/documentation/cellfinder/index.html)
[![Contributions](https://img.shields.io/badge/Contributions-Welcome-brightgreen.svg)](https://brainglobe.info/developers/index.html)
[![Website](https://img.shields.io/website?up_message=online&url=https%3A%2F%2Fbrainglobe.info)](https://brainglobe.info/documentation/brainglobe-workflows/index.html)
[![Twitter](https://img.shields.io/twitter/follow/brain_globe?style=social)](https://twitter.com/brain_globe)

# BrainGlobe Workflows

`brainglobe-workflows` is a package that provides users with a number of out-of-the-box data analysis workflows employed in neuroscience, implemented using BrainGlobe tools.

At present, the package currently offers the following workflows:

- [cellfinder](#cellfinder): Whole-brain detection, registration, and analysis. The successor to the old [cellfinder CLI](TODO:permalnk to deprecated cellfinder tag on repo) TODO: rename tool appropriately and give flavour text

## Installation

`brainglobe-workflows` comes packaged with version 1 of BrainGlobe, so the easiest way to make sure you get the latest release and stay up to date is to install that package - [follow this link to see the install instructions](TODO: link me!).

If you want to install BrainGlobe workflows as a standalone tool, you can run `pip install` in your desired environment:

```bash
pip install brainglobe-workflows
```

## Contributing

Contributions to BrainGlobe are more than welcome.
Please see the [developers guide](https://brainglobe.info/developers/index.html).

## Citing `brainglobe-workflows`

**If you use any tools in the [brainglobe suite](https://brainglobe.info/documentation/index.html), please [let us know](mailto:[email protected]?subject=cellfinder), and we'd be happy to promote your paper/talk etc.**

If you find [`cellfinder`](#cellfinder) useful, and use it in your research, please cite the paper outlining the cell detection algorithm:
> Tyson, A. L., Rousseau, C. V., Niedworok, C. J., Keshavarzi, S., Tsitoura, C., Cossell, L., Strom, M. and Margrie, T. W. (2021) “A deep learning algorithm for 3D cell detection in whole mouse brain image datasets’ PLOS Computational Biology, 17(5), e1009074
[https://doi.org/10.1371/journal.pcbi.1009074](https://doi.org/10.1371/journal.pcbi.1009074)
>
If you use any of the image registration functions in `cellfinder`, please also cite [`brainreg`](https://github.com/brainglobe/brainreg#citing-brainreg).

---

# Cellfinder

**TODO: move this information to an appropriate place on the website**

Whole-brain cell detection, registration and analysis.

**N.B. If you want to just use the cell detection part of cellfinder, please
see the standalone [cellfinder-core](https://github.com/brainglobe/cellfinder-core)
package, or the [cellfinder plugin](https://github.com/brainglobe/cellfinder-napari)
for [napari](https://napari.org/).**
**N.B. If you want to just use the cell detection part of cellfinder, please see the standalone [cellfinder-core](https://github.com/brainglobe/cellfinder-core) package, or the [cellfinder plugin](https://github.com/brainglobe/cellfinder-napari) for [napari](https://napari.org/).**

---
`cellfinder` is a collection of tools developed by [Adam Tyson](https://github.com/adamltyson), [Charly Rousseau](https://github.com/crousseau) and [Christian Niedworok](https://github.com/cniedwor) in the [Margrie Lab](https://www.sainsburywellcome.org/web/groups/margrie-lab), generously supported by the [Sainsbury Wellcome Centre](https://www.sainsburywellcome.org/web/).

`cellfinder` is a designed for the analysis of whole-brain imaging data such as
[serial-section imaging](https://sainsburywellcomecentre.github.io/OpenSerialSection/)
and lightsheet imaging in cleared tissue. The aim is to provide a single solution for:

* Cell detection (initial cell candidate detection and refinement using
deep learning) (using [cellfinder-core](https://github.com/brainglobe/cellfinder-core))
* Atlas registration (using [brainreg](https://github.com/brainglobe/brainreg))
* Analysis of cell positions in a common space
`cellfinder` is a designed for the analysis of whole-brain imaging data such as [serial-section imaging](https://sainsburywellcomecentre.github.io/OpenSerialSection/) and lightsheet imaging in cleared tissue.
The aim is to provide a single solution for:

---
Installation is with
`pip install cellfinder`
- Cell detection (initial cell candidate detection and refinement using deep learning) (using [cellfinder-core](https://github.com/brainglobe/cellfinder-core)),
- Atlas registration (using [brainreg](https://github.com/brainglobe/brainreg)),
- Analysis of cell positions in a common space.

---
Basic usage:

```bash
cellfinder -s signal_images -b background_images -o output_dir --metadata metadata
```
Full documentation can be
found [here](https://brainglobe.info/documentation/cellfinder/index.html).

This software is at a very early stage, and was written with our data in mind.
Over time we hope to support other data types/formats. If you have any issues, please get in touch [on the forum](https://forum.image.sc/tag/brainglobe) or by
[raising an issue](https://github.com/brainglobe/cellfinder/issues/new/choose).
Full documentation can be found [here](https://brainglobe.info/documentation/cellfinder/index.html).

This software is at a very early stage, and was written with our data in mind.
Over time we hope to support other data types/formats.
If you have any issues, please get in touch [on the forum](https://forum.image.sc/tag/brainglobe) or by [raising an issue](https://github.com/brainglobe/cellfinder/issues/new/choose).

---
## Illustration

### Introduction
cellfinder takes a stitched, but otherwise raw whole-brain dataset with at least
two channels:
* Background channel (i.e. autofluorescence)
* Signal channel, the one with the cells to be detected:

![raw](https://raw.githubusercontent.com/brainglobe/cellfinder/master/resources/raw.png)
**Raw coronal serial two-photon mouse brain image showing labelled cells**
cellfinder takes a stitched, but otherwise raw whole-brain dataset with at least two channels:

- Background channel (i.e. autofluorescence),
- Signal channel, the one with the cells to be detected:

![Raw coronal serial two-photon mouse brain image showing labelled cells](https://raw.githubusercontent.com/brainglobe/cellfinder/master/resources/raw.png)

### Cell candidate detection
Classical image analysis (e.g. filters, thresholding) is used to find
cell-like objects (with false positives):

![raw](https://raw.githubusercontent.com/brainglobe/cellfinder/master/resources/detect.png)
**Candidate cells (including many artefacts)**
Classical image analysis (e.g. filters, thresholding) is used to find cell-like objects (with false positives):

![Candidate cells (including many artefacts)](https://raw.githubusercontent.com/brainglobe/cellfinder/master/resources/detect.png)

### Cell candidate classification
A deep-learning network (ResNet) is used to classify cell candidates as true
cells or artefacts:

![raw](https://raw.githubusercontent.com/brainglobe/cellfinder/master/resources/classify.png)
**Cassified cell candidates. Yellow - cells, Blue - artefacts**
A deep-learning network (ResNet) is used to classify cell candidates as true cells (yellow) or artefacts (blue):

![Cassified cell candidates. Yellow - cells, Blue - artefacts](https://raw.githubusercontent.com/brainglobe/cellfinder/master/resources/classify.png)

### Registration and segmentation (brainreg)
Using [brainreg](https://github.com/brainglobe/brainreg),
cellfinder aligns a template brain and atlas annotations (e.g.
the Allen Reference Atlas, ARA) to the sample allowing detected cells to be assigned
a brain region.
### Registration and segmentation (`brainreg`)

This transformation can be inverted, allowing detected cells to be
transformed to a standard anatomical space.
Using [`brainreg`](https://github.com/brainglobe/brainreg), `cellfinder` aligns a template brain and atlas annotations (e.g. the Allen Reference Atlas, ARA) to the sample allowing detected cells to be assigned a brain region.

![raw](https://raw.githubusercontent.com/brainglobe/cellfinder/master/resources/register.png)
**ARA overlaid on sample image**
This transformation can be inverted, allowing detected cells to be transformed to a standard anatomical space.

![ARA overlaid on sample image](https://raw.githubusercontent.com/brainglobe/cellfinder/master/resources/register.png)

### Analysis of cell positions in a common anatomical space
Registration to a template allows for powerful group-level analysis of cellular
disributions. *(Example to come)*

Registration to a template allows for powerful group-level analysis of cellular distributions.
*(Example to come)*

## Examples

*(more to come)*

### Tracing of inputs to retrosplenial cortex (RSP)
Input cell somas detected by cellfinder, aligned to the Allen Reference Atlas,
and visualised in [brainrender](https://github.com/brainglobe/brainrender) along

Input cell somas detected by cellfinder, aligned to the Allen Reference Atlas, and visualised in [brainrender](https://github.com/brainglobe/brainrender) along
with RSP.

![brainrender](https://raw.githubusercontent.com/brainglobe/cellfinder/master/resources/brainrender.png)

Data courtesy of Sepiedeh Keshavarzi and Chryssanthi Tsitoura. [Details here](https://www.youtube.com/watch?v=pMHP0o-KsoQ)
Data courtesy of Sepiedeh Keshavarzi and Chryssanthi Tsitoura.
[Details here](https://www.youtube.com/watch?v=pMHP0o-KsoQ)

## Visualisation

cellfinder comes with a plugin ([brainglobe-napari-io](https://github.com/brainglobe/brainglobe-napari-io)) for [napari](https://github.com/napari/napari) to view your data

#### Usage
* Open napari (however you normally do it, but typically just type `napari` into your terminal, or click on your desktop icon)

#### Load cellfinder XML file
* Load your raw data (drag and drop the data directories into napari, one at a time)
* Drag and drop your cellfinder XML file (e.g. `cell_classification.xml`) into napari.
You can view your data using the [brainglobe-napari-io](https://github.com/brainglobe/brainglobe-napari-io) plugin for [napari](https://github.com/napari/napari).

#### Load cellfinder directory
* Load your raw data (drag and drop the data directories into napari, one at a time)
* Drag and drop your cellfinder output directory into napari.

The plugin will then load your detected cells (in yellow) and the rejected cell
candidates (in blue). If you carried out registration, then these results will be
overlaid (similarly to the loading brainreg data, but transformed to the
coordinate space of your raw data).

![load_data](https://raw.githubusercontent.com/brainglobe/brainglobe-napari-io/master/resources/load_data.gif)
**Loading raw data**

![load_data](https://raw.githubusercontent.com/brainglobe/brainglobe-napari-io/master/resources/load_results.gif)
**Loading cellfinder results**


## Contributing
Contributions to cellfinder are more than welcome. Please see the [developers guide](https://brainglobe.info/developers/index.html).


## Citing cellfinder

If you find cellfinder useful, and use it in your research, please cite the paper outlining the cell detection algorithm:
> Tyson, A. L., Rousseau, C. V., Niedworok, C. J., Keshavarzi, S., Tsitoura, C., Cossell, L., Strom, M. and Margrie, T. W. (2021) “A deep learning algorithm for 3D cell detection in whole mouse brain image datasets’ PLOS Computational Biology, 17(5), e1009074
[https://doi.org/10.1371/journal.pcbi.1009074](https://doi.org/10.1371/journal.pcbi.1009074)
>
If you use any of the image registration functions in cellfinder, please also cite [brainreg](https://github.com/brainglobe/brainreg#citing-brainreg).
- Open napari (however you normally do it, but typically just type `napari` into your terminal, or click on your desktop icon).
- Load your raw data (drag and drop the data directories into napari, one at a time). ![Loading raw data](https://raw.githubusercontent.com/brainglobe/brainglobe-napari-io/master/resources/load_data.gif)
- Drag and drop your cellfinder XML file (e.g. `cell_classification.xml`) and/or cellfinder output directory into napari. ![Loading cellfinder results](https://raw.githubusercontent.com/brainglobe/brainglobe-napari-io/master/resources/load_results.gif)

**If you use this, or any other tools in the brainglobe suite, please
[let us know](mailto:[email protected]?subject=cellfinder), and
we'd be happy to promote your paper/talk etc.**
The plugin will then load your detected cells (in yellow) and the rejected cell candidates (in blue).
If you carried out registration, then these results will be overlaid (similarly to the loading `brainreg` data, but transformed to the coordinate space of your raw data).
Loading

0 comments on commit 4a95ca4

Please sign in to comment.