From 6ffb671f78c13f0eae0db01c57c615cc173a642a Mon Sep 17 00:00:00 2001 From: DavidHerreros <40200657+DavidHerreros@users.noreply.github.com> Date: Thu, 27 Jun 2024 09:22:41 +0200 Subject: [PATCH 01/12] Update README.md --- README.md | 48 +++++++++++++++++++++++++++++++++++++----------- 1 file changed, 37 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index 48ecc20..a7ccf0b 100644 --- a/README.md +++ b/README.md @@ -1,23 +1,24 @@

Cryo-EM Heterogeniety Challenge

-This repository contains the code used to analyse the submissions for the [Inaugrural Flatirion Cryo-EM Heteorgeneity Challenge](https://www.simonsfoundation.org/flatiron/center-for-computational-biology/structural-and-molecular-biophysics-collaboration/heterogeneity-in-cryo-electron-microscopy/). +.. image:: https://img.shields.io/badge/Supported Python Versions-3.8 - 3.9 - 3.10 - 3.11) + :alt: Supported Python versions -## Scope +This repository contains the code used to analyse the submissions for the [Inaugrural Flatirion Cryo-EM Heteorgeneity Challenge](https://www.simonsfoundation.org/flatiron/center-for-computational-biology/structural-and-molecular-biophysics-collaboration/heterogeneity-in-cryo-electron-microscopy/). +# Scope This repository explains how to preprocess a submission (80 maps and corresponding probability distribution), and analyze it. Challenge participants can benchmark their submissions locally against the ground truth and other submissions that are available on the cloud via the Open Science Foundation project [The Inaugural Flatiron Institute Cryo-EM Heterogeneity Community Challenge](https://osf.io/8h6fz/). -## Warning - -This is a work in progress, while the code will probably not change, we are still writting better tutorials, documentation, and other ideas for analyzing the data. We are also in the process of making it easier for other people to contribute with their own metrics and methods. We are also in the process of distributiing the code to PyPi - - -## Accesing the data +# Warning +This is a work in progress, while the code will probably not change, we are still writting better tutorials, documentation, and other ideas for analyzing the data. We are also in the process of making it easier for other people to contribute with their own metrics and methods. We are also in the process of distributiing the code to PyPi. +# Accesing the data The data is available via the Open Science Foundation project [The Inaugural Flatiron Institute Cryo-EM Heterogeneity Community Challenge](https://osf.io/8h6fz/). You can download via a webbroswer, or programatically with wget as per [this script](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/blob/main/tests/scripts/fetch_test_data.sh). +NOTE: We recommend downloadaing the data with the script and wget as the downloads from the web browser might be unstable. -## Installation +# Installation +## Stable installation Installing this repository is simply. We recommend creating a virtual environment (using conda or pyenv), since we have dependencies such as PyTorch or Aspire, which are better dealt with in an isolated environment. After creating your environment, make sure to activate it and run ```bash @@ -25,7 +26,25 @@ cd /path/to/Cryo-EM-Heterogeneity-Challenge-1 pip install . ``` -## Running +## Devel installation +If you are interested in testing the programs previously installed, please, install the repository in development mode with the following commands: + +```bash +cd /path/to/Cryo-EM-Heterogeneity-Challenge-1 +pip install .[dev] +``` + +The test included in the repo can be executed with PyTest as shown below: + +```bash +cd /path/to/Cryo-EM-Heterogeneity-Challenge-1 +pytest tests/test_preprocessing.py +pytest tests/test_svd.py +pytest tests/test_map_to_map.py +pytest tests/test_distribution_to_distribution.py +``` + +# Running If you want to run our code, please check the notebooks in the [tutorials folder](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/tree/main/tutorials). The tutorials explain how to setup the config files, and run the commands @@ -36,6 +55,13 @@ cryo_challenge run_map2map_pipeline --config config_files/conf cryo_challenge run_distribution2distribution_pipeline --config config_files/config_distribution_to_distribution.yaml ``` -## Acknowledgements +# Contributing +If you find any bug or have a suggestion on the code feel free to open an issue [here]([https://www.simonsfoundation.org/flatiron/center-for-computational-biology/structural-and-molecular-biophysics-collaboration/heterogeneity-in-cryo-electron-microscopy/](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/issues) + +We also welcome any help with the development of this repository. If you want to contribute with your own suggestions, code, or fixes, we recommend creating a fork of this repository to avoid any incompatibilities with newer versions of the software. Once you are happy with your new code, please, make a PR from your fork to this repository. + +We are also working on pipelines to simplify the exentension of the code with new metrics or functionalities, stay tuned! + +# Acknowledgements * Miro A. Astore, Geoffrey Woollard, David Silva-Sánchez, Wenda Zhao, Khanh Dao Duc, Nikolaus Grigorieff, Pilar Cossio, and Sonya M. Hanson. "The Inaugural Flatiron Institute Cryo-EM Heterogeneity Community Challenge". 9 June 2023. DOI:10.17605/OSF.IO/8H6FZ * [David Herreros](https://github.com/DavidHerreros) for testing and CI and debugging in this repo From 629e63688c01565e6c0ffc0a19db0876c16c3ae0 Mon Sep 17 00:00:00 2001 From: DavidHerreros <40200657+DavidHerreros@users.noreply.github.com> Date: Thu, 27 Jun 2024 09:28:46 +0200 Subject: [PATCH 02/12] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index a7ccf0b..411c5b8 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@

Cryo-EM Heterogeniety Challenge

-.. image:: https://img.shields.io/badge/Supported Python Versions-3.8 - 3.9 - 3.10 - 3.11) +.. image:: https://img.shields.io/badge/Supported_Python_Versions-3.8_%7C_3.9_%7C_3.10_%7C_3.11-blue :alt: Supported Python versions This repository contains the code used to analyse the submissions for the [Inaugrural Flatirion Cryo-EM Heteorgeneity Challenge](https://www.simonsfoundation.org/flatiron/center-for-computational-biology/structural-and-molecular-biophysics-collaboration/heterogeneity-in-cryo-electron-microscopy/). From 0f3881e9b37b7c3614ca16f3e1566cada1900798 Mon Sep 17 00:00:00 2001 From: DavidHerreros <40200657+DavidHerreros@users.noreply.github.com> Date: Thu, 27 Jun 2024 09:36:20 +0200 Subject: [PATCH 03/12] Update README.md --- README.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 411c5b8..d7dc003 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,13 @@

Cryo-EM Heterogeniety Challenge

-.. image:: https://img.shields.io/badge/Supported_Python_Versions-3.8_%7C_3.9_%7C_3.10_%7C_3.11-blue - :alt: Supported Python versions - +

+ +Supported Python versions +GitHub Downloads (all assets, all releases) +GitHub branch check runs + +<\p> + This repository contains the code used to analyse the submissions for the [Inaugrural Flatirion Cryo-EM Heteorgeneity Challenge](https://www.simonsfoundation.org/flatiron/center-for-computational-biology/structural-and-molecular-biophysics-collaboration/heterogeneity-in-cryo-electron-microscopy/). # Scope From 3e08e4096b8b5b65651fffa39a0477cec354c7d9 Mon Sep 17 00:00:00 2001 From: DavidHerreros <40200657+DavidHerreros@users.noreply.github.com> Date: Thu, 27 Jun 2024 10:51:36 +0200 Subject: [PATCH 04/12] Fix typo --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d7dc003..4662f6f 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ GitHub Downloads (all assets, all releases) GitHub branch check runs -<\p> +

This repository contains the code used to analyse the submissions for the [Inaugrural Flatirion Cryo-EM Heteorgeneity Challenge](https://www.simonsfoundation.org/flatiron/center-for-computational-biology/structural-and-molecular-biophysics-collaboration/heterogeneity-in-cryo-electron-microscopy/). From 687a5695ff3cd5a7964708596c8ecf698ce68e43 Mon Sep 17 00:00:00 2001 From: DavidHerreros <40200657+DavidHerreros@users.noreply.github.com> Date: Thu, 27 Jun 2024 10:59:43 +0200 Subject: [PATCH 05/12] Update README.md --- README.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/README.md b/README.md index 4662f6f..c9ab1e3 100644 --- a/README.md +++ b/README.md @@ -5,8 +5,17 @@ Supported Python versions GitHub Downloads (all assets, all releases) GitHub branch check runs +GitHub License

+ +

+ +Cryo-EM Heterogeneity Challenge + +

+ + This repository contains the code used to analyse the submissions for the [Inaugrural Flatirion Cryo-EM Heteorgeneity Challenge](https://www.simonsfoundation.org/flatiron/center-for-computational-biology/structural-and-molecular-biophysics-collaboration/heterogeneity-in-cryo-electron-microscopy/). @@ -67,6 +76,17 @@ We also welcome any help with the development of this repository. If you want to We are also working on pipelines to simplify the exentension of the code with new metrics or functionalities, stay tuned! +# License +MIT License + +Copyright (c) 2024 Flatiron Institute + +Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + # Acknowledgements * Miro A. Astore, Geoffrey Woollard, David Silva-Sánchez, Wenda Zhao, Khanh Dao Duc, Nikolaus Grigorieff, Pilar Cossio, and Sonya M. Hanson. "The Inaugural Flatiron Institute Cryo-EM Heterogeneity Community Challenge". 9 June 2023. DOI:10.17605/OSF.IO/8H6FZ * [David Herreros](https://github.com/DavidHerreros) for testing and CI and debugging in this repo From da17f203ab3b1d56a4c89f85f4092982ba650179 Mon Sep 17 00:00:00 2001 From: DavidHerreros <40200657+DavidHerreros@users.noreply.github.com> Date: Thu, 27 Jun 2024 11:01:37 +0200 Subject: [PATCH 06/12] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index c9ab1e3..a89c72e 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,7 @@ This is a work in progress, while the code will probably not change, we are stil # Accesing the data The data is available via the Open Science Foundation project [The Inaugural Flatiron Institute Cryo-EM Heterogeneity Community Challenge](https://osf.io/8h6fz/). You can download via a webbroswer, or programatically with wget as per [this script](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/blob/main/tests/scripts/fetch_test_data.sh). -NOTE: We recommend downloadaing the data with the script and wget as the downloads from the web browser might be unstable. +**_NOTE_**: We recommend downloadaing the data with the script and wget as the downloads from the web browser might be unstable. # Installation From cda61ff6baf3bde7bd3fbca28a1b48df55e34834 Mon Sep 17 00:00:00 2001 From: DavidHerreros <40200657+DavidHerreros@users.noreply.github.com> Date: Thu, 27 Jun 2024 13:24:45 +0200 Subject: [PATCH 07/12] Correct typo --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index a89c72e..13f7dc6 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ This repository contains the code used to analyse the submissions for the [Inaug This repository explains how to preprocess a submission (80 maps and corresponding probability distribution), and analyze it. Challenge participants can benchmark their submissions locally against the ground truth and other submissions that are available on the cloud via the Open Science Foundation project [The Inaugural Flatiron Institute Cryo-EM Heterogeneity Community Challenge](https://osf.io/8h6fz/). # Warning -This is a work in progress, while the code will probably not change, we are still writting better tutorials, documentation, and other ideas for analyzing the data. We are also in the process of making it easier for other people to contribute with their own metrics and methods. We are also in the process of distributiing the code to PyPi. +This is a work in progress, while the code will probably not change, we are still writting better tutorials, documentation, and other ideas for analyzing the data. We are also in the process of making it easier for other people to contribute with their own metrics and methods. We are also in the process of distributing the code to PyPi. # Accesing the data The data is available via the Open Science Foundation project [The Inaugural Flatiron Institute Cryo-EM Heterogeneity Community Challenge](https://osf.io/8h6fz/). You can download via a webbroswer, or programatically with wget as per [this script](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/blob/main/tests/scripts/fetch_test_data.sh). From 118e9fc498d50a9cf95cab9ca7a5ad5338eff433 Mon Sep 17 00:00:00 2001 From: DavidHerreros <40200657+DavidHerreros@users.noreply.github.com> Date: Thu, 27 Jun 2024 13:25:42 +0200 Subject: [PATCH 08/12] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 13f7dc6..6feb275 100644 --- a/README.md +++ b/README.md @@ -70,7 +70,7 @@ cryo_challenge run_distribution2distribution_pipeline --config config_files/conf ``` # Contributing -If you find any bug or have a suggestion on the code feel free to open an issue [here]([https://www.simonsfoundation.org/flatiron/center-for-computational-biology/structural-and-molecular-biophysics-collaboration/heterogeneity-in-cryo-electron-microscopy/](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/issues) +If you find any bug or have a suggestion on the code feel free to open an issue [here](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/issues) We also welcome any help with the development of this repository. If you want to contribute with your own suggestions, code, or fixes, we recommend creating a fork of this repository to avoid any incompatibilities with newer versions of the software. Once you are happy with your new code, please, make a PR from your fork to this repository. From bcd6a37e609ac100f7fbbbb52e4fc3a6c0470df2 Mon Sep 17 00:00:00 2001 From: DavidHerreros <40200657+DavidHerreros@users.noreply.github.com> Date: Thu, 27 Jun 2024 13:26:00 +0200 Subject: [PATCH 09/12] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 6feb275..3552b4e 100644 --- a/README.md +++ b/README.md @@ -70,7 +70,7 @@ cryo_challenge run_distribution2distribution_pipeline --config config_files/conf ``` # Contributing -If you find any bug or have a suggestion on the code feel free to open an issue [here](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/issues) +If you find any bug or have a suggestion on the code feel free to open an issue [here](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/issues). We also welcome any help with the development of this repository. If you want to contribute with your own suggestions, code, or fixes, we recommend creating a fork of this repository to avoid any incompatibilities with newer versions of the software. Once you are happy with your new code, please, make a PR from your fork to this repository. From a974d225445ade92def6d3f822e0fef461a26004 Mon Sep 17 00:00:00 2001 From: DavidHerreros <40200657+DavidHerreros@users.noreply.github.com> Date: Thu, 27 Jun 2024 16:00:46 +0200 Subject: [PATCH 10/12] Remove license section --- README.md | 11 ----------- 1 file changed, 11 deletions(-) diff --git a/README.md b/README.md index 3552b4e..4d858fe 100644 --- a/README.md +++ b/README.md @@ -76,17 +76,6 @@ We also welcome any help with the development of this repository. If you want to We are also working on pipelines to simplify the exentension of the code with new metrics or functionalities, stay tuned! -# License -MIT License - -Copyright (c) 2024 Flatiron Institute - -Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - # Acknowledgements * Miro A. Astore, Geoffrey Woollard, David Silva-Sánchez, Wenda Zhao, Khanh Dao Duc, Nikolaus Grigorieff, Pilar Cossio, and Sonya M. Hanson. "The Inaugural Flatiron Institute Cryo-EM Heterogeneity Community Challenge". 9 June 2023. DOI:10.17605/OSF.IO/8H6FZ * [David Herreros](https://github.com/DavidHerreros) for testing and CI and debugging in this repo From e6c9fcae7031fd7740e3143211f63d511280a2ce Mon Sep 17 00:00:00 2001 From: geoffwoollard Date: Tue, 2 Jul 2024 17:02:20 -0400 Subject: [PATCH 11/12] Update README.md --- README.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index b64513f..17ebc71 100644 --- a/README.md +++ b/README.md @@ -40,7 +40,7 @@ cd /path/to/Cryo-EM-Heterogeneity-Challenge-1 pip install . ``` -## Devel installation +## Developer installation If you are interested in testing the programs previously installed, please, install the repository in development mode with the following commands: ```bash @@ -52,6 +52,7 @@ The test included in the repo can be executed with PyTest as shown below: ```bash cd /path/to/Cryo-EM-Heterogeneity-Challenge-1 +sh tests/scripts/fetch_test_data.sh # download test data from OSF pytest tests/test_preprocessing.py pytest tests/test_svd.py pytest tests/test_map_to_map.py @@ -59,9 +60,14 @@ pytest tests/test_distribution_to_distribution.py ``` # Running -If you want to run our code, please check the notebooks in the [tutorials folder](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/tree/main/tutorials). +If you want to run our code on the full challenge data, or you own local data, please complete the following steps -The tutorials explain how to setup the config files, and run the commands +### 1. Dowload the full challenge data from [The Inaugural Flatiron Institute Cryo-EM Heterogeneity Community Challenge](https://osf.io/8h6fz/) +You can do this through the web browser, or programatically with wget (you can get inspiration from [this script](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/blob/main/tests/scripts/fetch_test_data.sh), which is just for the test data, not the full datasets) + +### 2. Modify the config files and run the commands on the full challenge data +Point to the path where the data is locally +The [tutorial notebooks](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/tree/main/tutorials) explain how to setup the config files, and run the commands ``` cryo_challenge run_preprocessing --config config_files/config_preproc.yaml cryo_challenge run_svd --config config_files/config_svd.yaml @@ -69,6 +75,7 @@ cryo_challenge run_map2map_pipeline --config config_files/conf cryo_challenge run_distribution2distribution_pipeline --config config_files/config_distribution_to_distribution.yaml ``` + # Contributing If you find any bug or have a suggestion on the code feel free to open an issue [here](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/issues). From 9199bd229f568474c70cacb0f4f5074ef71e6401 Mon Sep 17 00:00:00 2001 From: sonyahanson Date: Tue, 9 Jul 2024 18:35:29 -0400 Subject: [PATCH 12/12] fixed some typos --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 17ebc71..9dfb755 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -

Cryo-EM Heterogeniety Challenge

+

Cryo-EM Heterogeneity Challenge

@@ -62,7 +62,7 @@ pytest tests/test_distribution_to_distribution.py # Running If you want to run our code on the full challenge data, or you own local data, please complete the following steps -### 1. Dowload the full challenge data from [The Inaugural Flatiron Institute Cryo-EM Heterogeneity Community Challenge](https://osf.io/8h6fz/) +### 1. Download the full challenge data from [The Inaugural Flatiron Institute Cryo-EM Heterogeneity Community Challenge](https://osf.io/8h6fz/) You can do this through the web browser, or programatically with wget (you can get inspiration from [this script](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/blob/main/tests/scripts/fetch_test_data.sh), which is just for the test data, not the full datasets) ### 2. Modify the config files and run the commands on the full challenge data