Merge branch 'main' into future

Janelia-Trackathon-2023 · Nov 28, 2023 · dc9db8a · dc9db8a
2 parents cef06a3 + 2ed94df
commit dc9db8a
Show file tree

Hide file tree

Showing 31 changed files with 925 additions and 454 deletions.
diff --git a/.github/ISSUE_TEMPLATE.md → .github/ISSUE_TEMPLATE/bugs.md b/.github/ISSUE_TEMPLATE.md → .github/ISSUE_TEMPLATE/bugs.md
@@ -1,15 +1,29 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: bug
+assignees: ''
+
+---
+
 * traccuracy version:
 * Python version:
 * Operating System:
 
-### Description
+# Description
 
 Describe what you were trying to get done.
 Tell us what happened, what went wrong, and what you expected to happen.
 
-### What I Did
+# Minimal example to reproduce the bug
 
 ```
 Paste the command(s) you ran and the output.
 If there was a crash, please include the traceback here.
 ```
+
+# Severity
+- [ ] Unusable
+- [ ] Annoying, but still functional
+- [ ] Very minor
diff --git a/.github/ISSUE_TEMPLATE/features.md b/.github/ISSUE_TEMPLATE/features.md
@@ -0,0 +1,36 @@
+---
+name: Feature
+about: Suggest an idea for this project
+title: ''
+labels: enhancement
+assignees: ''
+
+---
+
+# Description
+
+Please describe the feature that you would like to see implemented in `traccuracy`.
+
+# Topics
+
+What types of changes are you suggesting? Put an x in the boxes that apply.
+- [ ] New feature or enhancement
+- [ ] Documentation update
+- [ ] Tests and benchmarks
+- [ ] Maintenance (e.g. dependencies, CI, releases, etc.)
+
+Which topics does your change affect? Put an x in the boxes that apply.
+- [ ] Loaders
+- [ ] Matchers
+- [ ] Track Errors
+- [ ] Metrics
+- [ ] Core functionality (e.g. `TrackingGraph`, `run_metrics`, `cli`, etc.)
+
+# Priority
+- [ ] This is an essential feature
+- [ ] Nice to have
+- [ ] Future idea
+
+# Are you interested in contributing?
+- [ ] Yes! :tada:
+- [ ] No :slightly_frowning_face:
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
@@ -0,0 +1,32 @@
+If you are implementing a new matcher or metric, please append this `&template=new_matcher_metric.md` to your url to load the correct template.
+
+# Proposed Change
+Briefly describe the contribution. If it resolves an issue or feature request, be sure to link to that issue.
+
+# Types of Changes
+What types of changes does your code introduce? Put an x in the boxes that apply.
+- [ ] Bugfix (non-breaking change which fixes an issue)
+- [ ] New feature or enhancement
+- [ ] Documentation update
+- [ ] Tests and benchmarks
+- [ ] Maintenance (e.g. dependencies, CI, releases, etc.)
+
+Which topics does your change affect? Put an x in the boxes that apply.
+- [ ] Loaders
+- [ ] Matchers
+- [ ] Track Errors
+- [ ] Metrics
+- [ ] Core functionality (e.g. `TrackingGraph`, `run_metrics`, `cli`, etc.)
+
+# Checklist
+Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.
+
+- [ ] I have read the developer/contributing docs.
+- [ ] I have added tests that prove that my feature works in various situations or tests the bugfix (if appropriate).
+- [ ] I have checked that I maintained or improved code coverage.
+- [ ] I have checked the benchmarking action to verify that my changes did not adversely affect performance.
+- [ ] I have written docstrings and checked that they render correctly in the Read The Docs build (created after the PR is opened).
+- [ ] I have updated the general documentation including Metric descriptions and example notebooks if necessary.
+
+# Further Comments
+If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...
diff --git a/.github/PULL_REQUEST_TEMPLATE/new_matcher_metric.md b/.github/PULL_REQUEST_TEMPLATE/new_matcher_metric.md
@@ -0,0 +1,18 @@
+# Proposed Matcher or Metric Addition
+- [ ] Matcher
+- [ ] Metric
+
+Briefly describe your new Matcher or Metric class, including links to publication or other source code if relevant. A full description should be included in the documentation. If it resolves a feature request, be sure to link to that issue.
+
+# Checklist
+Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.
+
+- [ ] I have read the developer/contributing docs.
+- [ ] I have added tests that prove that my feature works in various situations.
+- [ ] I have checked that I maintained or improved code coverage.
+- [ ] I have added benchmarking functions for my change `tests/bench.py`.
+- [ ] I have added a page to the documentation with a complete description of my matcher/metric including any references.
+- [ ] I have written docstrings and checked that they render correctly in the Read The Docs build (created after the PR is opened).
+
+# Further Comments
+If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -5,12 +5,12 @@ ci:
 
 repos:
   - repo: https://github.com/crate-ci/typos
-    rev: v1.16.21
+    rev: v1.16.23
     hooks:
       - id: typos
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.1.3
+    rev: v0.1.4
     hooks:
       - id: ruff
         args: [--fix]

diff --git a/examples/ctc.ipynb b/examples/ctc.ipynb
@@ -24,7 +24,7 @@
     "\n",
     "from traccuracy import run_metrics\n",
     "from traccuracy.loaders import load_ctc_data\n",
-    "from traccuracy.matchers import CTCMatched, IOUMatched\n",
+    "from traccuracy.matchers import CTCMatcher, IOUMatcher\n",
     "from traccuracy.metrics import CTCMetrics, DivisionMetrics\n",
     "\n",
     "pp = pprint.PrettyPrinter(indent=4)"
@@ -63,7 +63,14 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "Fluo-N2DL-HeLa.zip: 191MB [00:18, 10.2MB/s]                              \n"
+      "Fluo-N2DL-HeLa.zip: 0.00B [00:00, ?B/s]"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Fluo-N2DL-HeLa.zip: 191MB [00:15, 12.1MB/s]                               \n"
      ]
     }
    ],
@@ -96,8 +103,8 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "Loading TIFFs: 100%|██████████| 92/92 [00:00<00:00, 374.71it/s]\n",
-      "Loading TIFFs: 100%|██████████| 92/92 [00:00<00:00, 824.06it/s]\n"
+      "Loading TIFFs: 100%|██████████| 92/92 [00:00<00:00, 388.26it/s]\n",
+      "Loading TIFFs: 100%|██████████| 92/92 [00:00<00:00, 640.22it/s]\n"
      ]
     }
    ],
@@ -130,60 +137,59 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "Matching frames: 100%|██████████| 92/92 [00:13<00:00,  6.65it/s]\n",
-      "Evaluating nodes: 100%|██████████| 92/92 [00:00<00:00, 10573.68it/s]\n",
-      "Evaluating edges: 100%|██████████| 8535/8535 [00:06<00:00, 1359.15it/s]\n"
+      "Matching frames: 100%|██████████| 92/92 [00:00<00:00, 93.42it/s] \n",
+      "Evaluating nodes: 100%|██████████| 8600/8600 [00:00<00:00, 721911.19it/s]\n",
+      "Evaluating FP edges: 100%|██████████| 8535/8535 [00:00<00:00, 968440.00it/s]\n",
+      "Evaluating FN edges: 100%|██████████| 8562/8562 [00:00<00:00, 1054425.71it/s]\n"
      ]
     },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "{   'CTCMetrics': {   'AOGM': 631.5,\n",
-      "                      'DET': 0.9954855886097927,\n",
-      "                      'TRA': 0.9936361895740329,\n",
-      "                      'fn_edges': 87,\n",
-      "                      'fn_nodes': 39,\n",
-      "                      'fp_edges': 60,\n",
-      "                      'fp_nodes': 0,\n",
-      "                      'ns_nodes': 0,\n",
-      "                      'ws_edges': 51},\n",
-      "    'DivisionMetrics': {   'Frame Buffer 0': {   'Division F1': 0.76,\n",
-      "                                                 'Division Precision': 0.7169811320754716,\n",
-      "                                                 'Division Recall': 0.8085106382978723,\n",
-      "                                                 'False Negative Divisions': 18,\n",
-      "                                                 'False Positive Divisions': 30,\n",
-      "                                                 'Mitotic Branching Correctness': 0.6129032258064516,\n",
-      "                                                 'Total GT Divisions': 94,\n",
-      "                                                 'True Positive Divisions': 76},\n",
-      "                           'Frame Buffer 1': {   'Division F1': 0.76,\n",
-      "                                                 'Division Precision': 0.7169811320754716,\n",
-      "                                                 'Division Recall': 0.8085106382978723,\n",
-      "                                                 'False Negative Divisions': 18,\n",
-      "                                                 'False Positive Divisions': 30,\n",
-      "                                                 'Mitotic Branching Correctness': 0.6129032258064516,\n",
-      "                                                 'Total GT Divisions': 94,\n",
-      "                                                 'True Positive Divisions': 76},\n",
-      "                           'Frame Buffer 2': {   'Division F1': 0.76,\n",
-      "                                                 'Division Precision': 0.7169811320754716,\n",
-      "                                                 'Division Recall': 0.8085106382978723,\n",
-      "                                                 'False Negative Divisions': 18,\n",
-      "                                                 'False Positive Divisions': 30,\n",
-      "                                                 'Mitotic Branching Correctness': 0.6129032258064516,\n",
-      "                                                 'Total GT Divisions': 94,\n",
-      "                                                 'True Positive Divisions': 76}}}\n"
+      "[   {   'metric': {   'e_weights': {'fn': 1.5, 'fp': 1, 'ws': 1},\n",
+      "                      'name': 'CTCMetrics',\n",
+      "                      'v_weights': {'fn': 10, 'fp': 1, 'ns': 5}},\n",
+      "        'results': {   'AOGM': 627.5,\n",
+      "                       'DET': 0.9954855886097927,\n",
+      "                       'TRA': 0.993676498745377,\n",
+      "                       'fn_edges': 87,\n",
+      "                       'fn_nodes': 39,\n",
+      "                       'fp_edges': 60,\n",
+      "                       'fp_nodes': 0,\n",
+      "                       'ns_nodes': 0,\n",
+      "                       'ws_edges': 47}},\n",
+      "    {   'metric': {'frame_buffer': (0, 1, 2), 'name': 'DivisionMetrics'},\n",
+      "        'results': {   'Frame Buffer 0': {   'Division F1': 0.76,\n",
+      "                                             'Division Precision': 0.7169811320754716,\n",
+      "                                             'Division Recall': 0.8085106382978723,\n",
+      "                                             'False Negative Divisions': 18,\n",
+      "                                             'False Positive Divisions': 30,\n",
+      "                                             'Mitotic Branching Correctness': 0.6129032258064516,\n",
+      "                                             'True Positive Divisions': 76},\n",
+      "                       'Frame Buffer 1': {   'Division F1': 0.76,\n",
+      "                                             'Division Precision': 0.7169811320754716,\n",
+      "                                             'Division Recall': 0.8085106382978723,\n",
+      "                                             'False Negative Divisions': 18,\n",
+      "                                             'False Positive Divisions': 30,\n",
+      "                                             'Mitotic Branching Correctness': 0.6129032258064516,\n",
+      "                                             'True Positive Divisions': 76},\n",
+      "                       'Frame Buffer 2': {   'Division F1': 0.76,\n",
+      "                                             'Division Precision': 0.7169811320754716,\n",
+      "                                             'Division Recall': 0.8085106382978723,\n",
+      "                                             'False Negative Divisions': 18,\n",
+      "                                             'False Positive Divisions': 30,\n",
+      "                                             'Mitotic Branching Correctness': 0.6129032258064516,\n",
+      "                                             'True Positive Divisions': 76}}}]\n"
      ]
     }
    ],
    "source": [
     "ctc_results = run_metrics(\n",
     "    gt_data=gt_data, \n",
     "    pred_data=pred_data, \n",
-    "    matcher=CTCMatched, \n",
-    "    metrics=[CTCMetrics, DivisionMetrics],\n",
-    "    metrics_kwargs=dict(\n",
-    "        frame_buffer=(0,1,2)\n",
-    "    )\n",
+    "    matcher=CTCMatcher(), \n",
+    "    metrics=[CTCMetrics(), DivisionMetrics(frame_buffer=(0,1,2))],\n",
     ")\n",
     "pp.pprint(ctc_results)"
    ]
@@ -198,55 +204,61 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 7,
    "metadata": {},
    "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Matching frames: 100%|██████████| 92/92 [00:15<00:00,  6.03it/s]\n"
+     ]
+    },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "{   'DivisionMetrics': {   'Frame Buffer 0': {   'Division F1': 0,\n",
-      "                                                 'Division Precision': 0.0,\n",
-      "                                                 'Division Recall': 0.0,\n",
-      "                                                 'False Negative Divisions': 94,\n",
-      "                                                 'False Positive Divisions': 93,\n",
-      "                                                 'Mitotic Branching Correctness': 0.0,\n",
-      "                                                 'Total GT Divisions': 94,\n",
-      "                                                 'True Positive Divisions': 0},\n",
-      "                           'Frame Buffer 1': {   'Division F1': 0.44837758112094395,\n",
-      "                                                 'Division Precision': 0.44970414201183434,\n",
-      "                                                 'Division Recall': 0.4470588235294118,\n",
-      "                                                 'False Negative Divisions': 94,\n",
-      "                                                 'False Positive Divisions': 93,\n",
-      "                                                 'Mitotic Branching Correctness': 0.2889733840304182,\n",
-      "                                                 'Total GT Divisions': 94,\n",
-      "                                                 'True Positive Divisions': 76},\n",
-      "                           'Frame Buffer 2': {   'Division F1': 0.44837758112094395,\n",
-      "                                                 'Division Precision': 0.44970414201183434,\n",
-      "                                                 'Division Recall': 0.4470588235294118,\n",
-      "                                                 'False Negative Divisions': 94,\n",
-      "                                                 'False Positive Divisions': 93,\n",
-      "                                                 'Mitotic Branching Correctness': 0.2889733840304182,\n",
-      "                                                 'Total GT Divisions': 94,\n",
-      "                                                 'True Positive Divisions': 76}}}\n"
+      "[   {   'metric': {'frame_buffer': (0, 1, 2), 'name': 'DivisionMetrics'},\n",
+      "        'results': {   'Frame Buffer 0': {   'Division F1': 0.711340206185567,\n",
+      "                                             'Division Precision': 0.69,\n",
+      "                                             'Division Recall': 0.7340425531914894,\n",
+      "                                             'False Negative Divisions': 25,\n",
+      "                                             'False Positive Divisions': 31,\n",
+      "                                             'Mitotic Branching Correctness': 0.552,\n",
+      "                                             'True Positive Divisions': 69},\n",
+      "                       'Frame Buffer 1': {   'Division F1': 0.711340206185567,\n",
+      "                                             'Division Precision': 0.69,\n",
+      "                                             'Division Recall': 0.7340425531914894,\n",
+      "                                             'False Negative Divisions': 25,\n",
+      "                                             'False Positive Divisions': 31,\n",
+      "                                             'Mitotic Branching Correctness': 0.552,\n",
+      "                                             'True Positive Divisions': 69},\n",
+      "                       'Frame Buffer 2': {   'Division F1': 0.711340206185567,\n",
+      "                                             'Division Precision': 0.69,\n",
+      "                                             'Division Recall': 0.7340425531914894,\n",
+      "                                             'False Negative Divisions': 25,\n",
+      "                                             'False Positive Divisions': 31,\n",
+      "                                             'Mitotic Branching Correctness': 0.552,\n",
+      "                                             'True Positive Divisions': 69}}}]\n"
      ]
     }
    ],
    "source": [
     "iou_results = run_metrics(\n",
     "    gt_data=gt_data, \n",
     "    pred_data=pred_data, \n",
-    "    matcher=IOUMatched, \n",
-    "    matcher_kwargs=dict(\n",
-    "        iou_threshold=0.5\n",
-    "    ),\n",
-    "    metrics=[DivisionMetrics], \n",
-    "    metrics_kwargs=dict(\n",
-    "        frame_buffer=(0,1,2)\n",
-    "    )\n",
+    "    matcher=IOUMatcher(iou_threshold=0.1), \n",
+    "    metrics=[DivisionMetrics(frame_buffer=(0,1,2))], \n",
     ")\n",
     "pp.pprint(iou_results)"
    ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
   }
  ],
  "metadata": {
@@ -265,7 +277,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.9"
+   "version": "3.10.12"
   },
   "orig_nbformat": 4
  },