Merge pull request #37 from flatironinstitute/dev

update main. mainly readme

README.md

@@ -1,4 +1,4 @@
-<h1 align='center'>Cryo-EM Heterogeniety Challenge</h1>
+<h1 align='center'>Cryo-EM Heterogeneity Challenge</h1>
 
 <p align="center">
         
@@ -40,7 +40,7 @@ cd /path/to/Cryo-EM-Heterogeneity-Challenge-1
 pip install .
 ```
 
-## Devel installation
+## Developer installation
 If you are interested in testing the programs previously installed, please, install the repository in development mode with the following commands:
 
 ```bash
@@ -52,23 +52,30 @@ The test included in the repo can be executed with PyTest as shown below:
 
 ```bash
 cd /path/to/Cryo-EM-Heterogeneity-Challenge-1
+sh tests/scripts/fetch_test_data.sh # download test data from OSF
 pytest tests/test_preprocessing.py
 pytest tests/test_svd.py
 pytest tests/test_map_to_map.py
 pytest tests/test_distribution_to_distribution.py
 ```
 
 # Running
-If you want to run our code, please check the notebooks in the [tutorials folder](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/tree/main/tutorials).
+If you want to run our code on the full challenge data, or you own local data, please complete the following steps
 
-The tutorials explain how to setup the config files, and run the commands
+### 1. Download the full challenge data from [The Inaugural Flatiron Institute Cryo-EM Heterogeneity Community Challenge](https://osf.io/8h6fz/)
+You can do this through the web browser, or programatically with wget (you can get inspiration from [this script](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/blob/main/tests/scripts/fetch_test_data.sh), which is just for the test data, not the full datasets)
+
+### 2. Modify the config files and run the commands on the full challenge data
+Point to the path where the data is locally
+The [tutorial notebooks](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/tree/main/tutorials) explain how to setup the config files, and run the commands
 ```
 cryo_challenge run_preprocessing                      --config config_files/config_preproc.yaml
 cryo_challenge run_svd                                --config config_files/config_svd.yaml
 cryo_challenge run_map2map_pipeline                   --config config_files/config_map_to_map.yaml
 cryo_challenge run_distribution2distribution_pipeline --config config_files/config_distribution_to_distribution.yaml
 ```
 
+
 # Contributing
 If you find any bug or have a suggestion on the code feel free to open an issue [here](https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/issues).
 

tutorials/README.md

@@ -0,0 +1,28 @@
+# Brief overview
+
+There are notebook tutorials for the each step in the analysis pipeline.
+
+### `1_tutorial_preprocessing.ipynb`
+This notebok walks through how to align the raw submissions, and the various options in this step.
+- input: folders with respective 80 `.mrc` files and a `populations.txt` file
+- output: anonymized `submission_?.pt` files with aligned populations
+
+
+### `2_tutorial_svd.ipynb`
+This notebook walks through generating and analyzing (plots) the SVD results.
+- input: `submission_?.pt` files
+- output: `svd_results.pt`
+
+
+### `3_tutorial_map2map.ipynb`
+This notebook walks through generating and analyzing (plots) the map to map distance matrix results.
+- input: one `submission_?.pt` file
+- output: a `.pkl` file
+
+
+### `4_tutorial_distribution2distribution.ipynb`
+- input: one `.pkl` file from the map2map step
+- output: a `.pkl` file
+
+### `5_tutorial_plotting.ipynb`
+This notebook walks through parsing and analyzing (plots) the map to map and distribution to distribution results.