From 2eea11d76402c7c70f8efb161a6f06652680e48d Mon Sep 17 00:00:00 2001 From: Jon Clucas Date: Tue, 21 Apr 2020 10:07:07 -0400 Subject: [PATCH 1/7] :books: Bind data directories for `data_config build` Ref https://github.com/FCP-INDI/C-PAC/issues/1252 --- docs/user/_sources/subject_list_config.txt | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/docs/user/_sources/subject_list_config.txt b/docs/user/_sources/subject_list_config.txt index 9937951f9..7bd8100bd 100644 --- a/docs/user/_sources/subject_list_config.txt +++ b/docs/user/_sources/subject_list_config.txt @@ -29,7 +29,7 @@ If you don't already have a data settings YAML file, you can generate one in you or:: - docker run -i --rm --user $(id -u) -v /path/to/data_config:/scratch -w="/scratch" fcpindi/c-pac:latest bids_dir outputs_dir cli -- utils data_config new_settings_template + docker run -i --rm --user $(id -u):$(id -g) -v /path/to/data_config:/scratch -w="/scratch" fcpindi/c-pac:latest bids_dir outputs_dir cli -- utils data_config new_settings_template For either of the above commands, ``bids_dir`` and ``outputs_dir`` are required but not used, so any non-null string will work for those positional arguments. @@ -46,13 +46,20 @@ Generate configuration files for individual-level and group-level analyses Once you have a template, you can then configure the file as needed. You will need to use a text editor to fill in at least ``dataFormat``, ``bidsBaseDir`` (if using BIDS data), ``outputSubjectListLocation``, and ``subjectListName`` in your settings file before you build. -Once your ``data_settings.yml`` file is ready, you can generate your data configuration file by running:: +Once your ``data_settings.yml`` file is ready, you can generate your data configuration file by running the following commands, binding any directories in your ``data_settings.yml`` to the same locations in the container:: - singularity run C-PAC_latest.sif bids_dir outputs_dir cli -- utils data_config build /path/to/data_settings.yml + singularity run -B /path/to/data/in/data_settings/1 \ + -B /path/to/data/in/data_settings/2 \ + C-PAC_latest.sif bids_dir outputs_dir cli -- utils \ + data_config build /path/to/data_settings.yml or:: - docker run -i --rm --user $(id -u) -v /path/to/data_config:/scratch -w="/scratch" fcpindi/c-pac:latest bids_dir outputs_dir cli -- utils data_config build /path/to/data_settings.yml + docker run -i --rm --user $(id -u):$(id -g) -v /path/to/data/in/data_settings/1:/path/to/data/in/data_settings/1 \ + -v /path/to/data/in/data_settings/2:/path/to/data/in/data_settings/2 \ + -v /path/to/data_config:/scratch -w="/scratch" \ + fcpindi/c-pac:latest bids_dir outputs_dir cli -- utils \ + data_config build /path/to/data_settings.yml Continue below for some example use cases. From 2e8ab2caf7ad4adb596af57fce4e07f2a966f754 Mon Sep 17 00:00:00 2001 From: Jon Clucas Date: Wed, 22 Apr 2020 12:37:29 -0400 Subject: [PATCH 2/7] :pencil: /path/to/bids_data, /path/to/outputs Ref https://github.com/FCP-INDI/fcp-indi.github.com/pull/214#discussion_r412257153 --- docs/user/_sources/subject_list_config.txt | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/docs/user/_sources/subject_list_config.txt b/docs/user/_sources/subject_list_config.txt index 7bd8100bd..4b9895db3 100644 --- a/docs/user/_sources/subject_list_config.txt +++ b/docs/user/_sources/subject_list_config.txt @@ -48,17 +48,23 @@ Once you have a template, you can then configure the file as needed. You will ne Once your ``data_settings.yml`` file is ready, you can generate your data configuration file by running the following commands, binding any directories in your ``data_settings.yml`` to the same locations in the container:: - singularity run -B /path/to/data/in/data_settings/1 \ + singularity run \ + -B /path/to/bids_dir \ + -B /path/to/outputs \ + -B /path/to/data/in/data_settings/1 \ -B /path/to/data/in/data_settings/2 \ - C-PAC_latest.sif bids_dir outputs_dir cli -- utils \ + C-PAC_latest.sif /path/to/bids_dir /path/to/outputs cli -- utils \ data_config build /path/to/data_settings.yml or:: - docker run -i --rm --user $(id -u):$(id -g) -v /path/to/data/in/data_settings/1:/path/to/data/in/data_settings/1 \ + docker run -i --rm --user $(id -u):$(id -g) \ + -v /path/to/bids_dir:/path/to/bids_dir \ + -v /path/to/outputs:/path/to/outputs \ + -v /path/to/data/in/data_settings/1:/path/to/data/in/data_settings/1 \ -v /path/to/data/in/data_settings/2:/path/to/data/in/data_settings/2 \ -v /path/to/data_config:/scratch -w="/scratch" \ - fcpindi/c-pac:latest bids_dir outputs_dir cli -- utils \ + fcpindi/c-pac:latest /path/to/bids_dir /path/to/outputs cli -- utils \ data_config build /path/to/data_settings.yml Continue below for some example use cases. From 5c80d6ef5a3d9335301633e188befece698dc185 Mon Sep 17 00:00:00 2001 From: Jon Clucas Date: Sat, 9 May 2020 12:25:41 -0400 Subject: [PATCH 3/7] :ambulance: Correct path to C-PAC for Circle build --- .circleci/config.yml | 2 +- docs/user/_sources/run/help.txt | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index 69dd9687a..09780ed63 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -15,7 +15,7 @@ jobs: apt-get update && apt-get -y install git rsync python-dev graphviz libgraphviz-dev pkg-config python3-sphinx pip install --user -r https://raw.githubusercontent.com/FCP-INDI/C-PAC/develop/requirements.txt pip install --user git+https://github.com/FCP-INDI/C-PAC.git sphinx m2r numpydoc PyGithub sphinxcontrib-programoutput - git clone https://github.com/FCP-INDI/C-PAC.git + git clone https://github.com/FCP-INDI/C-PAC.git /build/C-PAC - run: name: Build docs command: | diff --git a/docs/user/_sources/run/help.txt b/docs/user/_sources/run/help.txt index 59db90661..36479d32c 100644 --- a/docs/user/_sources/run/help.txt +++ b/docs/user/_sources/run/help.txt @@ -1,4 +1,4 @@ -cpac run --help -``````````````` +Usage +````` -.. program-output:: python -m /build/C-PAC/dev/docker_data/run --help +.. program-output:: python /build/C-PAC/dev/docker_data/run.py --help From 8d5ceacf239f28605bf6187696cda273bd4aaad4 Mon Sep 17 00:00:00 2001 From: Jon Clucas Date: Thu, 14 May 2020 17:53:18 -0400 Subject: [PATCH 4/7] :fire: Remove wxPython GUI references --- docs/user/_sources/docker.txt | 2 - docs/user/_sources/group_fsl_feat.txt | 167 +------------------------ docs/user/_sources/group_isc.txt | 17 --- docs/user/_sources/group_mdmr.txt | 19 --- docs/user/_sources/group_pybasc.txt | 49 -------- docs/user/_sources/group_qpp.txt | 23 ---- docs/user/_sources/pipeline_config.txt | 7 -- docs/user/_sources/quick.txt | 6 - 8 files changed, 1 insertion(+), 289 deletions(-) diff --git a/docs/user/_sources/docker.txt b/docs/user/_sources/docker.txt index 00362d37d..719532158 100644 --- a/docs/user/_sources/docker.txt +++ b/docs/user/_sources/docker.txt @@ -83,8 +83,6 @@ Note that any of the optional arguments above will over-ride any pipeline settin * You can run only anatomical preprocessing easily, without modifying your data or pipeline configuration files, by providing the ``--anat_only`` flag. -* A GUI can be invoked to assist in pipeline customization by specifying the ``GUI`` command line argument, as opposed to ``participant`` (this currently only works for Singularity containers). - * As stated, the default behavior is to read data that is organized in the BIDS format. This includes data that is in Amazon AWS S3 by using the format ``s3:///`` for the ``bids_dir`` command line argument. Outputs can be written to S3 using the same format for the ``output_dir``. Credentials for accessing these buckets can be specified on the command line (using ``--aws_input_creds`` or ``--aws_output_creds``). * When the app is run, a data configuration file is written to the working directory. This file can be passed into subsequent runs, which avoids the overhead of re-parsing the BIDS input directory on each run (i.e. for cluster or cloud runs). These files can be generated without executing the C-PAC pipeline using the test_run command line argument. diff --git a/docs/user/_sources/group_fsl_feat.txt b/docs/user/_sources/group_fsl_feat.txt index db3129f2f..aa1466482 100644 --- a/docs/user/_sources/group_fsl_feat.txt +++ b/docs/user/_sources/group_fsl_feat.txt @@ -3,7 +3,7 @@ Overview ^^^^^^^^ -C-PAC uses the `FSL/FEAT `_ tool to compare findings across groups. You can construct models using a participant list and a phenotype file, select derivatives to be predicted by the model, and define contrasts between conditions using either the GUI or a custom csv file. Then FSL/FEAT will run a second-level `General Linear Model (GLM) `_ for you. +C-PAC uses the `FSL/FEAT `_ tool to compare findings across groups. You can construct models using a participant list and a phenotype file, select derivatives to be predicted by the model, and define contrasts between conditions using a custom CSV file. Then FSL/FEAT will run a second-level `General Linear Model (GLM) `_ for you. In addition, FMRIB's FSL Randomise package is also available in C-PAC for the execution of non-parametric permutation inference. FSL-Randomise can use the same group models you generate or build/edit via the C-PAC model builder for FSL-FEAT. @@ -29,8 +29,6 @@ The presets that are generated: * can be modified to your liking either by using the standard Group Analysis Model Builder, or by hand via text editor * are all in one place in the output directory you specify in the Preset Generator -To configure group-level analysis using a preset from the main screen of the GUI, click the "Generate FSL-FEAT Presets" button on the bottom left. This will allow you to generate a pre-set model. - **Preset options**: Currently, there are 5 presets to choose from. More are on their way. If you have any commonly-used or useful group model designs you'd like to see as a preset, `please let us know `_! The available presets are: @@ -41,69 +39,6 @@ The available presets are: * `Paired Two-Group Difference (Two-Sample Paired T-Test) `_ * `Tripled Two-Group Difference ('Tripled' T-Test) `_ -Using the GUI -------------- - -.. figure:: /_images/ga_main_presets.png - -Start by opening the FSL group model presets from the main window of the GUI. You'll see a window like this open: - -.. figure:: /_images/group_presets_one.png - -#. **Choose Preset:** Select the type of preset you'd like to generate. The preset generator will prompt you for more information relevant to the type of preset you selected on the next window (except for the Single Group Average (One-Sample T-Test), which needs no additional information). - -#. **Participant List - [path]:** Full path to a list of subjects to be included in the model. This should be a text file with one subject per line. A list in this format containing all subjects run through CPAC was generated along with the main CPAC subject list (see the subject list in `Overview`). Another easy way to manually create this file is to copy the subjects column from your Regressor/EV spreadsheet. - -#. **Select Derivatives - [checkboxes]:** Select which derivatives you would like to include when running group analysis. When including Dual Regression, make sure to correct your P-value for the number of maps you are comparing. When including Multiple Regression SCA, you must have more degrees of freedom (subjects) than there were time series. - -#. **Z Threshold - [decimal]:** Only voxels with a Z-score higher than this value will be considered significant. - -#. **Cluster Significance Threshold - [decimal]:** Significance threshold (P-value) to use when doing cluster correction for multiple comparisons. - -#. **Model Name - [text]:** Specify a name for the new model. - -#. **Output Directory - [path]:** Full path to the directory where CPAC should place the model files (.mat, .con, .grp, .csv) and the outputs of group analysis. The input files and the group analysis configuration YAML file generated by the preset generator will also be written here. - -.. figure:: /_images/group_presets_two.png - -This window opens if you have selected any of the presets that require phenotypic information [Single Group Average with Additional Covariate, Unpaired Two-Group Difference (Two-Sample Unpaired T-Test)]. - -#. **Phenotype/EV File -[path]:** Full path to a .csv or .tsv file containing EV information for each subject. Levels for categorical variables in this file can be expressed as words ('ADHD'/'TD') or numerical values (0/1) depending on your preferences. - -#. **Participant Column Name [text]:** Name/label of the subjects column in your phenotype file. - -#. **Two groups from pheno to compare [text]:** Enter the names/labels of the two columns from your phenotype file, separated by a comma, that specify the two groups (categorical EVs) you wish to compare, OR, if the two groups are encoded in one column only (ex. Sex: M,F,F,M,.. or Diagnosis: 1,1,0,0,..), simply enter the name of that one column. - -.. figure:: /_images/group_presets_three.png - -This window opens if you have selected any of the presets that compare paired/linked groups or conditions [Paired Two-Group Difference (Two-Sample Paired T-Test), Tripled Two-Group Difference ('Tripled' T-Test)]. - -#. **Conditions: Sessions or Series/Scans?:** Choose whether the linked groups or conditions you wish to compare are separated in the data as Sessions (multiple visits into the scanner) or Series/Scans (multiple acquisitions during a single session). - -#. **Session or Series/Scan IDs - [dialogue: list of names]:** Enter the session or series/scan ID names you wish to compare. If they are sessions, they can be found under 'unique_id' in the individual-level data configuration YAML file for the data in question, or in the 'participant_session' IDs that you would see in the individual-level analysis output directory (ex. '3005_1' would be participant 3005, session 1). If they are series/scans, they will be the functional scan names, which can be found in the data configuration YAML file nested under 'func'. - -Once you click "Generate Model", all of the associated files will be generated. Most importantly, it will generate a group configuration file that you can load into the 'Pipelines' box. You can check the terminal window for progress and details. Once complete, you can review the model generated to make sure it meets your expectations. If you wish to tweak the model generated by a preset, you can load the group configuration file into the model builder (more details below) and make any modifications necessary. - -When you are happy with your model, you can go ahead to build the model for each of your selected derivatives: - -.. figure:: /_images/ga_main_fsl-build.png - -Once this is complete, a design matrix will have been generated for each derivative, session, scan, and nuisance regression strategy you had in your pipeline directory. These are available for review in the group-level analysis output directory you specified in your group configuration file (or during preset generation). In addition, a ``contrasts.csv`` file will be generated in the top level of the FSL group model directory (in your selected group-level analysis output directory). This contrasts file will be set up in accordance with the design matrices created for your pipeline outputs, and will look like this: - -.. figure:: /_images/ga_contrast_csv.png - -The first column should be filled with labels for the contrasts that you can define - these do not have to follow any particular convention, and can be whatever works best for your experiment. The remainder of the cells can be populated with contrast weights according to your needs. - -If you would like to add f-tests, add each f-test as a column to the CSV with the label ``f_test_#`` and the assign weights to each contrast to be included in the f-test. - -.. figure:: /_images/ga_contrast_ftest.png - -Once you have specified your contrasts, you can start FSL FEAT, Randomise, or both from the GUI like so: - -.. figure:: /_images/ga_main_fsl-run.png - -Note, if you have group configuration options for other types of group-level analyses (ex. PyBASC, MDMR, ISC/ISFC) in the same group configuration file, C-PAC will run all of your analyses for you. You can have all of your group-level analysis setups in one group configuration file, or you can separate them if preferred- both will work. - From Terminal ------------- You can generate any of these presets using the C-PAC command-line interface (CLI): @@ -193,106 +128,6 @@ C-PAC Group Analysis Model Builder ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Use the model builder to create an FSL FEAT/Randomise model from scratch, or to modify any of the generated presets. -To configure group-level analysis from the main screen of the GUI, click the "New Group" button to open the group-level analysis configuration editor. Navigate to the "FSL FEAT Settings" tab to specify your model. Note that this model will also work with Randomise. Specific Randomise options (number of permutations, etc.) are available on the next tab, "FSL Randomise Settings". - -From the GUI ------------- - -General Group-Level Analysis Settings -------------------------------------- - -.. figure:: /_images/ga_general.png - -#. **Pipeline Directory - [path]:** The main input of group-level analysis- the output directory of your individual-level analysis pipeline run (pre-processing & derivatives for each participant). This should be a path to your C-PAC individual-level run's pipeline folder, which includes the sub-directories labeled with the participant IDs. - -#. **Participant List - [path]:** (Optional) Full path to a list of participants to be included in the model. You can use this to easily prune participants from your model. In group-level analyses involving phenotype files, this allows you to prune participants without removing them from the phenotype CSV/TSV file. This should be a text file with one subject per line. An easy way to manually create this file is to copy the participant ID column from your phenotype file. - -#. **Output Directory - [path]:** Full path to the directory where CPAC should write out the results of group analysis, and also place the model files (.mat, .con, .grp, .csv). The CSV file is a human-readable version of the .mat file used by FLAME that you can use to examine the exact model that C-PAC generated. - -#. **Working Directory - [path]:** Much like the working directory for individual-level analysis, this is where the intermediate and working files will be stored during your run. This directory can be deleted later on. However, saving this directory allows the group analysis run to skip steps that have been already completed, in the case of re-runs. - -#. **Log Directory - [path]:** Where to write out log information for your group analysis run. - -#. **FSL Directory - [text/path]:** The path to your FSL installation directory. This can be left as 'FSLDIR' to grab your system's default FSL installation. However, if you prefer to use a specific install of FSL, you can enter the path here. - -Specifying Models to Run ------------------------- - -.. figure:: /_images/ga_fsl_model1.png - -#. **Run FEAT - [Off, On]:** Whether to run FSL-FEAT. Only applies to when you run group analysis through the GUI. This allows you to select multiple types of group-level analyses to run in sequence. - -#. **Model Name - [text]:** Specify a name for the new model. - -#. **Phenotype/EV File -[path]:** Full path to a .csv or .tsv file containing EV information for each subject. Levels for categorical variables in this file can be expressed as words ('ADHD'/'TD') or numerical values (0/1) depending on your preferences. - -#. **Participant Column Name [text]:** Name of the subjects column in your EV file. - -#. **Model Setup - [checkboxes]:** A list of EVs from your phenotype file will populate in this window. From here, you can select whether the EVs should be treated as categorical or if they should be demeaned (continuous/non-categorical EVs only). 'MeanFD' and 'Measure Mean' will also appear in this window automatically as options to be used as regressors that can be included in your model design. Note that the MeanFD and mean of measure values are automatically calculated and supplied by C-PAC via individual-level analysis. Measure mean is calculated using the mean signal from raw data rather than z-score data, which is then demeaned. Also, MeanFD and mean of measure values are automatically demeaned prior to being inserted into the group analysis model. - -#. **Design Matrix Formula - [Patsy formula]:** Specify the formula to describe your model design. Essentially, including EVs in this formula inserts them into the model. The most basic format to include each EV you select would be 'EV + EV + EV + ..', etc. You can also select to include MeanFD, Measure_Mean, or an intercept here (by adding ' + Measure_Mean', ' + MeanFD_' or ' + Intercept ' respectively). Note that when you add an intercept to your formula categorical variables will automatically be demeaned since this is a requirement for FLAME to run properly. This design formula is pre-generated for the user depending on the EVs in the phenotype file, but can be edited at any time. C-PAC uses the Python library Patsy to generate the design matrices, so more information on how to format your design formula for specific designs can be found here- `Patsy formula documentation `_. If you have used R in the past, Patsy's formula syntax should be familiar. - -#. **Custom ROI Mean Mask (optional) - [path]:** Use a binarized mask with one or more ROIs to add averages for those ROIs to the model as EVs. Mask file must be in NifTI format. These averages will be calculated using the raw data, rather than z-scored data, and will then be demeaned afterwards. - -#. **Select Derivatives - [checkboxes]:** Select which derivatives you would like to include when running group analysis. When including Dual Regression, make sure to correct your P-value for the number of maps you are comparing. When including Multiple Regression SCA, you must have more degrees of freedom (subjects) than there were time series. - -.. figure:: /_images/ga_fsl_model2.png - -#. **Coding Scheme - [Treatment, Sum]:** Select the encoding for your design matrix. For more details, see Patsy's pages on `Treatment `_ and `Sum `_ coding. - -#. **Mask for Means Calculation - [Group Mask, Individual Mask]:** C-PAC can add the average voxel intensity for a derivative as an EV in the model. If this average voxel intensity is present in the model, this menu allows you to select either a group-level or individual-level mask. Otherwise, this menu can be ignored. - -#. **Use z-score Standardized Derivatives - [True, False]:** Run model on a z-score standardized version of individual-level outputs or the raw versions. - -#. **Z Threshold - [decimal]:** Only voxels with a Z-score higher than this value will be considered significant. - -#. **Cluster Significance Threshold - [decimal]:** Significance threshold (P-value) to use when doing cluster correction for multiple comparisons. - -#. **Model Group Variances Separately - [Off, On]:** Specify whether FSL should model the variance for each group separately. If this option is enabled, you must specify a grouping variable below. - -#. **Grouping Variable - [text]:** The name of the EV that should be used to group subjects when modeling variances. If you do not wish to model group variances separately, set this value to None. - -#. **Sessions (Repeated Measures Only) - [dialogue: list of session names]:** Enter the session names in your dataset that you wish to include within the same model (this is for repeated measures/ within-subject designs). These will be the names listed as "unique_id" in the original individual-level participant list, or the labels in the original data directories you marked as {session} while creating the C-PAC participant list. Do not adjust your phenotype CSV- C-PAC will re-formulate it internally before passing it to FLAME. - -#. **Series/Scans (Repeated Measures Only) - [dialogue: list of scan names]:** Enter the series names in your dataset that you wish to include within the same model (this is for repeated measure / within-subject designs). These will be the labels listed under "rest:" in the original individual-level participant list, or the labels in the original data directories you marked as {series} while creating the C-PAC participant list. Do not adjust your phenotype CSV- C-PAC will re-formulate it internally before passing it to FLAME. - -Once you have designed your model, the model builder will save your design in a group configuration YAMl file. As described above, you can then build your models for the derivatives in the pipeline directory selected, from the GUI: - -.. figure:: /_images/ga_main_fsl-build.png - -Once this is complete, a design matrix will have been generated for each derivative, session, scan, and nuisance regression strategy you had in your pipeline directory. These are available for review in the group-level analysis output directory you specified in your group configuration file (or during preset generation). In addition, a ``contrasts.csv`` file will be generated in the top level of the FSL group model directory (in your selected group-level analysis output directory). This contrasts file will be set up in accordance with the design matrices created for your pipeline outputs, and will look like this: - -.. figure:: /_images/ga_contrast_csv.png - -The first column should be filled with labels for the contrasts that you can define - these do not have to follow any particular convention, and can be whatever works best for your experiment. The remainder of the cells can be populated with contrast weights according to your needs. - -If you would like to add f-tests, add each f-test as a column to the CSV with the label ``f_test_#`` and the assign weights to each contrast to be included in the f-test. - -.. figure:: /_images/ga_contrast_ftest.png - -Once you have specified your contrasts, you can start FSL FEAT, Randomise, or both from the GUI like so: - -.. figure:: /_images/ga_main_fsl-run.png - -Note, if you have group configuration options for other types of group-level analyses (ex. PyBASC, MDMR, ISC/ISFC) in the same group configuration file, C-PAC will run all of your analyses for you. You can have all of your group-level analysis setups in one group configuration file, or you can separate them if preferred- both will work. - -FSL Randomise Settings ----------------------- - -You can set up FSL Randomise models by using the FSL-FEAT model builder described above. The options in the FSL Randomise Settings tab supplies the configuration decisions for options that are exclusive to FSL Randomise. More information is available on FMRIB's `Randomise User Guide `_. - -.. figure:: /_images/ga_fsl_randomise.png - -#. **Run Randomise - [Off, On]:** Whether to run FSL Randomise. Only applies to when you run group analysis through the GUI. This allows you to select multiple types of group-level analyses to run in sequence. - -#. **Permutations - [integer]:** Number of permutations you would like to use when building up the null distribution to test against. - -#. **Threshold - [integer]:** Cluster-based thresholding corrected for multiple comparisons by using the null distribution of the max (across the image) cluster mask. - -#. **Demean - [True/False]:** Demean data temporally before model fitting. - -#. **Threshold-Free Cluster Enhancement - [True/False]:** From the FMRIB FSL-Randomise user guide: TFCE (Threshold-Free Cluster Enhancement) is a new method for finding 'clusters' in your data with out having to define clusters in a binary way. Cluster-like structures are enhanced but the image remains fundamentally voxelwise. - From the terminal ----------------- diff --git a/docs/user/_sources/group_isc.txt b/docs/user/_sources/group_isc.txt index eb8ef4657..d36593f50 100644 --- a/docs/user/_sources/group_isc.txt +++ b/docs/user/_sources/group_isc.txt @@ -52,23 +52,6 @@ https://dataspace.princeton.edu/jspui/handle/88435/dsp015d86p269k Configuring ISC/ISFC ^^^^^^^^^^^^^^^^^^^^ -Using the GUI -""""""""""""" - -.. figure:: /_images/isfc.png - -#. **Run ISC - [Off, On]:** Used to determine if Inter-subject Correlation (ISC) will be added to the pipeline or not. - -#. **Run ISFC - [Off, On]:** Used to determine if Inter-subject Functional Correlation (ISFC) will be added to the pipeline or not. - -#. **ROI analysis - [Off, On]:** Used to determine if the ISC and ISFC will run in the ROI level. - -#. **Voxel-wise analysis - [Off, On]:** Used to determine if the ISC and ISFC will run in the voxel level. Depending on the image resolution, it may take several hours and consume a great amount of available memory. - -#. **Voxel-wise analysis: Standard Deviation filter - [number]:** Filter out voxels that, in the correlation distribution, is greater then the informed standard deviation. Zero value will disable the filter. - -#. **Permutations - [integer]:** Number of permutation tests to run on the Pseudo-F statistics. - Configuration Using a YAML File """"""""""""""""""""""""""""""" diff --git a/docs/user/_sources/group_mdmr.txt b/docs/user/_sources/group_mdmr.txt index 1c69b48e2..e7a7a3a44 100644 --- a/docs/user/_sources/group_mdmr.txt +++ b/docs/user/_sources/group_mdmr.txt @@ -34,25 +34,6 @@ As CWAS examines connectivity across the whole brain, it may have reduced sensit Configuring CWAS ^^^^^^^^^^^^^^^^^ -Using the GUI -""""""""""""" -.. figure:: /_images/cwas.png - -#. **Run CWAS - [Off, On]:** Used to determine if CWAS will be added to the pipeline or not. - -#. **Mask ROI File - [path]:** Path to a mask file. Voxels outside of the mask will be excluded from CWAS. - -#. **Regressor File - [path]:** Path to a text file containing the phenotypic regressor. - -#. **Regressor Participant Column Name - [string]:** Name of the participants column in your regressor file. - -#. **Regressor of Interest columns - [string]:** Columns of the regressor of interest in your regressor file. - -#. **Permutations - [integer]:** Number of permutation tests to run on the Pseudo-F statistics. - -#. **Parallel Nodes - [integer]:** Number of Nipype nodes created while computing CWAS. Dependent upon computing resources. - - Configuration Using a YAML File """""""""""""""""""""""""""""""" To configure CWAS options within a YAML file, add the following lines to your file (with appropriate substitutions for paths): diff --git a/docs/user/_sources/group_pybasc.txt b/docs/user/_sources/group_pybasc.txt index 24e146a2d..f3c3c4e99 100644 --- a/docs/user/_sources/group_pybasc.txt +++ b/docs/user/_sources/group_pybasc.txt @@ -15,55 +15,6 @@ When running BASC, C-PAC employs the Python-based PyBASC package implemented and PyBASC allows users to create individual and group level clustering solutions and compare the reliability and reproducibility of these clustering solutions across a wide variety of methods. C-PAC makes it easy to directly supply the preprocessed functional timeseries of your participants directly into PyBASC, for each nuisance strategy, session, and scan you have in your pipeline output folder. -Using the GUI -------------- - -General Group-Level Analysis Settings -------------------------------------- - -.. figure:: /_images/ga_general.png - -#. **Pipeline Directory - [path]:** The main input of group-level analysis- the output directory of your individual-level analysis pipeline run (pre-processing & derivatives for each participant). This should be a path to your C-PAC individual-level run's pipeline folder, which includes the sub-directories labeled with the participant IDs. - -#. **Participant List - [path]:** (Optional) Full path to a list of participants to be included in the model. You can use this to easily prune participants from your model. In group-level analyses involving phenotype files, this allows you to prune participants without removing them from the phenotype CSV/TSV file. This should be a text file with one subject per line. An easy way to manually create this file is to copy the participant ID column from your phenotype file. - -#. **Output Directory - [path]:** Full path to the directory where CPAC should write out the results of group analysis, and also place the model files (.mat, .con, .grp, .csv). The CSV file is a human-readable version of the .mat file used by FLAME that you can use to examine the exact model that C-PAC generated. - -#. **Working Directory - [path]:** Much like the working directory for individual-level analysis, this is where the intermediate and working files will be stored during your run. This directory can be deleted later on. However, saving this directory allows the group analysis run to skip steps that have been already completed, in the case of re-runs. - -#. **Log Directory - [path]:** Where to write out log information for your group analysis run. - -#. **FSL Directory - [text/path]:** The path to your FSL installation directory. This can be left as 'FSLDIR' to grab your system's default FSL installation. However, if you prefer to use a specific install of FSL, you can enter the path here. - -PyBASC Specific Settings ------------------------- - -.. figure:: /_images/pybasc_gui.png - -#. **Run BASC - [Off, On]:** Whether to run PyBASC. Only applies to when you run group analysis through the GUI. This allows you to select multiple types of group-level analyses to run in sequence. - -#. **ROI Mask file - [path]:** Full path to a mask file to be used when running BASC. Voxels outside this mask will be excluded from analysis. This is the region that you’d like to parcellate. - -#. **ROI Mask file for Cross-Clustering - [path]:** If cross clustering is enabled, then clustering of the first region will be calculated based on pairwise similarity between the timeseries of the ROI Mask File, and this second ROI. - -#. **Similarity metric - ['correlation', 'euclidean', 'cityblock', 'cosine']:** The metric used to compare similarity between voxel timeseries. - -#. **Number of timeseries bootstraps - [integer]:** How many times individual level circular block bootstrapping of the timeseries will be applied. - -#. **Number of dataset Bootstraps - [integer]:** How many times the group level bootstrapping across individuals will be applied. - -#. **Number of clusters - [integer]:** How many parcels you’d like to use. - -#. **Affinity Threshold - [decimal]:** The similarity threshold at which the similarity matrices will be set to 0. - -#. **Output sizes - [integer]:** This is the amount of feature agglomeration that will be applied. Smaller values mean more feature agglomeration. - -#. **Run Cross Clustering - [True/False]:** If set to true, then the ROI Mask file parcellation will be based on the similarity between ROI Mask file voxels based on their connectivity to each voxel in ROI mask file for cross-clustering. - -#. **Blocklength - [integer]:** This parameter determines the width of the time window used in the circular block bootstrap. - -#. **Group dimension reduce - [True/False]:** If this is set to true, the all individuals will have feature agglomeration applied together, resulting in the same mapping across subjects. Use this only when memory demands limit ability to process ROIs with a high number of voxels. - From terminal ------------- diff --git a/docs/user/_sources/group_qpp.txt b/docs/user/_sources/group_qpp.txt index d81feb3b7..7cc71f23a 100644 --- a/docs/user/_sources/group_qpp.txt +++ b/docs/user/_sources/group_qpp.txt @@ -24,29 +24,6 @@ It is important to note that QPP is computationally intensive, efficiency is imp Configuring QPP ^^^^^^^^^^^^^^^ -Using the GUI -^^^^^^^^^^^^^ - -#. **Run QPP - [Off, On]:** Run Quasi-Periodic Pattern analysis. - -#. **Scan Inclusion (optional) - list of scans or None:** Include specific scans in your analysis. If you choose this option, only these scans will be included in performing concatenation of the time courses. - -#. **Session Inclusion (optional) - list of sessions or None:** Include specific sessions in your analysiS. If you choose this option, only these sessions will be included in performing concatenation of the time courses. - -#. **Scan/Sessions Stratification: ["Session and Scan", "Session", "Scan", "None"]:** This represents the grouping stratergy for concatenating the time courses to generate the input for QPP. This will run QPP for each session, (which may contain multiple scans within it) if 'Sessions' is selected. It will run QPP for each included scans, (which may contain multiple sessions within it) if 'Scans' is selected. If no grouping strategy is selected(None), then it will not group by sessions or scans, and will run once for all sessions and all scans. - -#. **Window Length - [integer]:** The window length that is used to search the time course for some patterns. - -#. **Permutations - [integer]:** The window length that is used to search the time course for some patterns. - -#. **Initial Correlation Threshold: [number]:** This represents the early threshold for determining the peaks/signals representing a important - -#. **Final Correlation Threshold: [number]:** This represents the early threshold for determining the peaks/signals representing a important - -#. **Number of iterations to use initial correlation threshold: [number]:** This represents the early threshold for determining the peaks/signals representing a important - -#. **Maximum number of iterations: [number]:** This represents the early threshold for determining the peaks/signals representing a important - Configuration Using a YAML File ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To configure QPP options within a YAML file, add the following lines to your file (with appropriate substitutions for paths): diff --git a/docs/user/_sources/pipeline_config.txt b/docs/user/_sources/pipeline_config.txt index 384a64fc1..206a8113d 100644 --- a/docs/user/_sources/pipeline_config.txt +++ b/docs/user/_sources/pipeline_config.txt @@ -95,10 +95,3 @@ Derivatives * :doc:`Regional Homogeneity (ReHo) ` - Measure the similarity of activity patterns across neighboring voxels. * :doc:`Network Centrality ` - Analyze the structure of functional networks. -.. toctree:: - :hidden: - - /glossary/derivative.txt - /glossary/pipeline.txt - /glossary/strategy.txt - /glossary/workflow.txt diff --git a/docs/user/_sources/quick.txt b/docs/user/_sources/quick.txt index 7a0d8b0c8..f5e4c0660 100644 --- a/docs/user/_sources/quick.txt +++ b/docs/user/_sources/quick.txt @@ -116,9 +116,3 @@ References 23. Lancaster, J. L., Woldorff, M. G., Parsons, L. M., Liotti, M., Freitas, C. S., Rainey, L., … Fox, P. T. (2000). Automated Talairach atlas labels for functional brain mapping. Human Brain Mapping, 10(3), 120–31. Retrieved from `http://www.ncbi.nlm.nih.gov/pubmed/10912591 `_ 24. Craddock, R. C., James, G. A., Holtzheimer, P. E., Hu, X. P., & Mayberg, H. S. (2011). A whole brain fMRI atlas generated via spatially constrained spectral clustering. Human Brain Mapping, 0(July 2010). `http://doi.org/10.1002/hbm.21333 `_ - -.. toctree:: - :hidden: - - /docker.txt - /singularity.txt From ed9114e6e8402ceef74b0a51f65db3dfdcd4a9c1 Mon Sep 17 00:00:00 2001 From: Jon Clucas Date: Fri, 15 May 2020 09:54:47 -0400 Subject: [PATCH 5/7] :pencil: Fix "Design a Pipeline" import --- docs/user/_sources/pipeline_config.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user/_sources/pipeline_config.txt b/docs/user/_sources/pipeline_config.txt index 206a8113d..d68e4df32 100644 --- a/docs/user/_sources/pipeline_config.txt +++ b/docs/user/_sources/pipeline_config.txt @@ -25,7 +25,7 @@ Definitions .. include:: /glossary/derivative.txt :start-line: 3 -..include:: design_a_pipeline.txt +.. include:: design_a_pipeline.txt Using a Text Editor ------------------- From 34b4a499d8d46c77ffd411cb86f9868e340d417f Mon Sep 17 00:00:00 2001 From: Jon Clucas Date: Fri, 15 May 2020 10:22:39 -0400 Subject: [PATCH 6/7] :fire: Remove wxPython GUI references --- docs/user/_sources/_static/params/group_config.csv | 4 ++-- docs/user/_sources/appendix.txt | 3 --- docs/user/_sources/futuredocs/basc.txt | 2 +- docs/user/_sources/running.txt | 9 +++------ 4 files changed, 6 insertions(+), 12 deletions(-) diff --git a/docs/user/_sources/_static/params/group_config.csv b/docs/user/_sources/_static/params/group_config.csv index a51b85d9b..a310b3f92 100644 --- a/docs/user/_sources/_static/params/group_config.csv +++ b/docs/user/_sources/_static/params/group_config.csv @@ -2,7 +2,7 @@ participant_list,A list of subjects to be included in the model. This should be pheno_file,A csv file containing EV information for each subject.,"A path (e.g., ‘/data/my_analysis/ev.csv’)." participant_id_label,Name of the subjects column in your EV file.,A string. ev_selections,"Specify which EVs from your phenotype are categorical or numerical. Of those which are numerical, specify which are to be demeaned.","A dictionary with two keys, ‘demean’ and ‘categorical’. Each of these keys has a list as its value, with the names of the EVs that are categorical and which are to be demeaned. For example: {‘demean’: [‘age’], ‘categorical’: [‘sex’, ‘diagnosis’]}" -design_formula,"Specify the formula to describe your model design. Essentially, including EVs in this formula inserts them into the model. The most basic format to include each EV you select would be ‘EV + EV + EV + ..’, etc. You can also select to include MeanFD, Measure_Mean, and Custom_ROI_Mean here. See the GUI instructions for more details.","A formula (e.g., ‘sex + diagnosis + age + MeanFD_Jenkinson’)." +design_formula,"Specify the formula to describe your model design. Essentially, including EVs in this formula inserts them into the model. The most basic format to include each EV you select would be ‘EV + EV + EV + ..’, etc. You can also select to include MeanFD, Measure_Mean, and Custom_ROI_Mean here.","A formula (e.g., ‘sex + diagnosis + age + MeanFD_Jenkinson’)." mean_mask,Choose whether to use a group mask or individual-specific mask when calculating the output means to be used as a regressor. This only takes effect if you include the ‘Measure_Mean’ regressor in your Design Matrix Formula.,A string within a list- can be [‘Group Mask’] or [‘Individual Mask’]. custom_roi_mask,Full path to a NIFTI file containing one or more ROI masks. The means of the masked regions will then be computed for each subject’s output and will be included in the model as regressors (one for each ROI in the mask file) if you include ‘Custom_ROI_Mean’ in the Design Matrix Formula.,"A path (e.g., ‘/data/my_analysis/ROI.nii.gz’) or None." derivative_list,"Choose the derivatives to run the group model on. These must be written out as a list, and must be one of the options listed below this table.","A list (e.g., [‘alff_to_standard_smooth_zstd’, ‘sca_roi_files_to_standard_smooth_fisher_zstd’] )." @@ -13,7 +13,7 @@ z_threshold,Only voxels with a Z-score higher than this value will be considered p_threshold,Significance threshold (P-value) to use when doing cluster correction for multiple comparisons.,"A list containing a decimal value as a string (e.g., [‘0.05’])." sessions_list,For repeated measures only. This is a list of session names that you wish to include in a single model to run repeated measures or within-subject analysis. Repeated measures will run automatically if this is non-empty.,"A list containing a session names as strings (e.g., [‘session_1’,'session_2'])." series_list,For repeated measures only. This is a list of series/scan names that you wish to include in a single model to run repeated measures or within-subject analysis. Repeated measures will run automatically if this is non-empty.,"A list containing a scan names as strings (e.g., [‘scan_1’,'scan_2'])." -contrasts,The contrasts to be run as part of the group analysis. See the GUI instructions for more details.,"A list of contrast descriptions (e.g., [‘C(diagnosis)[T.ADHD] - C(diagnosis)[T.Typical] = 0’, 'age = 0'])." +contrasts,The contrasts to be run as part of the group analysis.,"A list of contrast descriptions (e.g., [‘C(diagnosis)[T.ADHD] - C(diagnosis)[T.Typical] = 0’, 'age = 0'])." f_tests,"An optional list of f-test strings containing contrasts. If you do not wish to run f-tests, leave this blank.","A list of strings containing all the contrasts to be included in the f-test separated by commas (e.g., [‘C(diagnosis)[T.ADHD] - C(diagnosis)[T.Typical] = 0, age = 0’])." custom_contrasts,An optional path to a CSV file which specifies the contrasts you wish to run in group analysis. See ‘Custom Contrasts’ section below.,"A path (e.g., ‘/data/my_analysis/custom_contrasts.csv’)." model_name,A name for the new model.,A string. diff --git a/docs/user/_sources/appendix.txt b/docs/user/_sources/appendix.txt index 5be926c9e..cc6faa3ab 100644 --- a/docs/user/_sources/appendix.txt +++ b/docs/user/_sources/appendix.txt @@ -136,7 +136,6 @@ Note: Specific maximum versions are required for the following dependencies. C-P * `Nipype `__ - version 1.1.2 * `NetworkX `__ - version 1.11 * `Jinja2 `__ - version 2.7.2 -* `wxPython 3 `__ - version 3.0.0 (if using the GUI) These specific versions can be installed via pip: @@ -440,6 +439,4 @@ Some network centrality features will not be available without compiling the C-b python setup.py build_ext --inplace -**Note:** Unfortunately, it is not possible at this time to use the C-PAC GUI without installing C-PAC. - .. include:: benchmark.txt \ No newline at end of file diff --git a/docs/user/_sources/futuredocs/basc.txt b/docs/user/_sources/futuredocs/basc.txt index 395641b38..2ae9e5a3f 100644 --- a/docs/user/_sources/futuredocs/basc.txt +++ b/docs/user/_sources/futuredocs/basc.txt @@ -5,7 +5,7 @@ Introduction & Background ^^^^^^^^^^^^^^^^^^^^^^^^^ Bootstrap Analysis of Stable Clusters (BASC) is a statistical framework that quantifies the stability of Resting State Networks (RSNs) by comparing the results of cluster analysis across replications (Bellec et al. 2010) As it is often impossible or impractical to repeat data collection for an experiment multiple times, BASC utilizes bootstrapping to generate a large number of samples from the existing data. BASC can be used to characterize stable RSNs at both the individual and group levels. -**Important Note: BASC is currently not fully implemented in C-PAC. This page is a reference for how to use it in future releases. An interface is available in the C-PAC GUI, but it is not functional.** +**Important Note: BASC is currently not fully implemented in C-PAC. This page is a reference for how to use it in future releases.** Computation and Analysis Considerations ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/docs/user/_sources/running.txt b/docs/user/_sources/running.txt index 44a7ba20b..56611fd23 100644 --- a/docs/user/_sources/running.txt +++ b/docs/user/_sources/running.txt @@ -3,10 +3,7 @@ Running C-PAC Overview -------- -As with configuring the subject list, pipeline configuration, and group analysis files, there are two ways of executing a C-PAC run: - -* Using C-PAC's command line interface -* Using the main dialog in the C-PAC GUI +As with configuring the subject list, pipeline configuration, and group analysis files, execute a C-PAC run by using C-PAC's command line interface. In addition to running C-PAC traditionally on your own local computer or on a server, there are three other avenues through which you can run C-PAC without going through the install process: @@ -169,7 +166,7 @@ When you are done, your session configuration should look similar to the followi .. figure:: /_images/cloud_x2go.png -Note: If X2Go does not work on your computer, you can also access the C-PAC GUI by adding the ``-X`` flag to the ssh command to enable X11 port forwarding (i.e., the ssh command would be ``ssh -X -i /path/to/pem/file ubuntu@``). X11 port forwarding is very slow compared to X2Go, however, so it is recommended that you troubleshoot X2Go further before turning to this option. +Note: If X2Go does not work on your computer, you can add the ``-X`` flag to the ssh command to enable X11 port forwarding (i.e., the ssh command would be ``ssh -X -i /path/to/pem/file ubuntu@``). X11 port forwarding is very slow compared to X2Go, however, so it is recommended that you troubleshoot X2Go further before turning to this option. Uploading Data to Your Instance ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -298,7 +295,7 @@ To add additional nodes to your C-PAC HPC cluster, use the following Starcluster Accessing the Head Node ''''''''''''''''''''''' -If you wish to use the C-PAC GUI while accessing the head node, type the following command: +If you wish to access the head node, type the following command: .. code-block:: console From 9380b9f8b9c466222892f8654e9fc583de35cc3c Mon Sep 17 00:00:00 2001 From: Jon Clucas Date: Thu, 25 Jun 2020 17:30:09 -0400 Subject: [PATCH 7/7] :rocket: Update release notes from GH