Merge pull request #55 from Janelia-Trackathon-2023/documentation_update

README update

README.md

@@ -1,4 +1,4 @@
-# traccuracy
+# traccuracy: Evaluate Cell Tracking Solutions
 
 [![License](https://img.shields.io/pypi/l/traccuracy.svg?color=green)](https://github.com/Janelia-Trackathon-2023/traccuracy/raw/main/LICENSE)
 [![PyPI](https://img.shields.io/pypi/v/traccuracy.svg?color=green)](https://pypi.org/project/traccuracy)
@@ -7,52 +7,50 @@
 [![Documentation Status](https://readthedocs.org/projects/traccuracy/badge/?version=latest)](https://traccuracy.readthedocs.io/en/latest/?badge=latest)
 
 
-`traccuracy` provides a suite of benchmarking functions that can be used to evaluate tracking solutions against ground truth annotations. It includes the standard metrics that are calculated by the [Cell Tracking Challenge](http://celltrackingchallenge.net/evaluation-methodology/).
+`traccuracy` provides a suite of benchmarking functions that can be used to evaluate cell
+tracking solutions against ground truth annotations. The goal of this library is to provide a convenient way to run rigorous evaluation and to document and consolidate the wide variety of
+metrics used in the field.
 
-Learn more in the [documentation](https://traccuracy.readthedocs.io/en/latest/).
+`traccuracy` can compute a comprehensive set of metrics for evaluating the cell linking and division performance, and can compute biologically meaningful metrics such as number of correctly reconstructed lineages over N frames and cell cycle length accuracy. As matching ground truth and
+predicted lineages is a crucial step for performing evaluation, `traccuracy` includes
+a number of algorithms for matching ground truth and predicted lineages, both with
+and without segmentation masks.
+
+Learn more in the [documentation](https://traccuracy.readthedocs.io/en/latest/) or check out the [source code](https://github.com/Janelia-Trackathon-2023/traccuracy).
 
 ## Installation
 `pip install traccuracy`
 
-## Development
-For local development, clone the repo and install in editable mode.
-```
-git clone https://github.com/Janelia-Trackathon-2023/traccuracy.git
-cd traccuracy
-pip install -e .[dev]
-```
-
-### Testing
-Install testing requirements
-```
-pip install -e .[test]
-```
-Run tests
-```
-python -m pytest tests
-```
-### Style
-We utilize `pre-commit` with black (formatting) and ruff (linting). If you would like to run `pre-commit` locally:
-```
-pip install -e .[dev]
-pre-commit install
-```
-Alternatively [pre-commit.ci](https://pre-commit.ci/), will run and commit changes on any open PRs.
-
-### Releases
-In order to deploy a new version, tag the commit with a version number and push it to github. This will trigger a github action that will build and deploy to PyPI. (see the "deploy" step in workflows/ci.yml). The version number is determined automatically based on the tag.
-```
-git tag -a v0.1.0 -m v0.1.0
-git push --follow-tags
-```
-
-### Documentation
-Documentation is built with Sphinx using `sphinx-autoapi` to automatically generate API documentation at build time. Docs are hosted on ReadTheDocs and build automatically after each push to main. Documentation can be built locally by running the following:
-```
-# Install docs requirements
-pip install -e .[docs]
-# Run the build
-sphinx-build docs/source docs/_build
-```
-
-You can view the documentation by opening `docs/_build/index.html` in your browser.
+## Getting Started
+The `traccuracy` library has three main components: loaders, matchers, and metrics.
+Loaders load tracking graphs from other formats, such as the CTC format, into a [TrackingGraph](https://traccuracy.readthedocs.io/en/latest/autoapi/traccuracy/index.html#traccuracy.TrackingGraph) object.
+A TrackingGraph is a spatiotemporal graph.
+Nodes represent a single cell in a given time point, and are annotated with a time and a location.
+Edges point from a node representing a cell in time point `t` to the same cell or its daughter in `t+1`.
+To load TrackingGraphs from a custom format, you will likely need to implement a loader: see
+documentation [here](https://traccuracy.readthedocs.io/en/latest/autoapi/traccuracy/loaders/index.html#module-traccuracy.loaders) for more information.
+
+Matchers take a ground truth and a predicted TrackingGraph with optional segmentation masks and match the nodes and edges to allow evaluation to occur.
+Metrics are then computed on the matched graphs, and a summary is printed out.
+
+The `traccuracy` library has a command line interface for running common metrics
+pipelines, documented [here](https://traccuracy.readthedocs.io/en/latest/cli.html), and a flexible Python API, shown in [this](https://traccuracy.readthedocs.io/en/latest/examples/ctc.html) example notebook.
+
+
+## Implemented Metrics
+
+ - CTC-DET from the [Cell Tracking Challenge Evaluation Methodology](http://celltrackingchallenge.net/evaluation-methodology/)
+ - CTC-TRA from the [Cell Tracking Challenge Evaluation Methodology](http://celltrackingchallenge.net/evaluation-methodology/)
+ - Acyclic Oriented Graph Metric (AOGM) from [Matula et al. 2015](https://doi.org/10.1371/journal.pone.0144959). A generalized form the CTC metrics where you can supply different weights for each component of the overall metric.
+ - Division Precision. Optionally allows detection within N frames of ground truth division.
+ - Division Recall. Optionally allows detection within N frames of ground truth division.
+ - Division F1 score. Optionally allows detection within N frames of ground truth division.
+ - Mitotic Branching Correctness from [Ulicna et al. 2021](https://doi.org/10.3389/fcomp.2021.734559). TP / (TP + FP + FN). Optionally allows detection within N frames of ground truth division.
+
+## Glossary
+
+**Tracklet**
+: A single non-dividing cell tracked over time. In graph terms, this is the connected component of a track between divisions (daughter to next parent). Tracklets can also start or end with a non-dividing cell at the beginning and end of the captured time or if the track leaves the field of view.
+
+**Track**
+: A single cell and all of its progeny. In graph terms, a connected component including divisions.

developer_notes.md

@@ -0,0 +1,46 @@
+# Traccuracy Developer Notes
+
+## Development
+For local development, clone the repo and install in editable mode.
+```
+git clone https://github.com/Janelia-Trackathon-2023/traccuracy.git
+cd traccuracy
+pip install -e ".[dev]"
+```
+
+### Testing
+Install testing requirements
+```
+pip install -e ".[test]"
+```
+Run tests
+```
+python -m pytest tests
+```
+### Style
+We utilize `pre-commit` with black (formatting) and ruff (linting). If you would like to run `pre-commit` locally:
+```
+pip install -e ".[dev]"
+pre-commit install
+```
+Alternatively [pre-commit.ci](https://pre-commit.ci/), will run and commit changes on any open PRs.
+
+### Releases
+In order to deploy a new version, tag the commit with a version number and push it to github. This will trigger a github action that will build and deploy to PyPI. (see the "deploy" step in workflows/ci.yml). The version number is determined automatically based on the tag.
+```
+git tag -a v0.1.0 -m v0.1.0
+git push --follow-tags
+```
+
+### Documentation
+Documentation is built with Sphinx using `sphinx-autoapi` to automatically generate API documentation at build time. Docs are hosted on ReadTheDocs and build automatically after each push to main. Documentation can be built locally by running the following:
+```
+# Install docs requirements
+pip install -e ".[docs]"
+# Run the build
+sphinx-build docs/source docs/_build
+```
+
+Note that running documentation locally requires Pandoc to be installed as well - https://pandoc.org/installing.html.
+
+You can view the documentation by opening `docs/_build/index.html` in your browser.

pyproject.toml

@@ -59,13 +59,16 @@ dev = [
 docs = [
     "mock",
     "Sphinx<6",
-    "sphinx-rtd-theme",
-    "m2r2",
+    "sphinx-rtd-theme>=0.5.2 ",
+    "m2r2<0.3.3",
     "sphinx-autoapi",
     "nbsphinx",
     "nbsphinx-link",
     "sphinx-click",
-    "ipython"
+    "ipython",
+    "docutils~=0.16.0"
+    # had to pin versions because of sphinx list rendering bug
+    # https://stackoverflow.com/questions/67542699/readthedocs-sphinx-not-rendering-bullet-list-from-rst-file
 ]
 
 [project.urls]