From 55dd18684debd30cf05d77a2233a597ad0fc38f5 Mon Sep 17 00:00:00 2001 From: Robert Oostenveld Date: Wed, 25 Sep 2024 08:59:26 +0200 Subject: [PATCH] converted dos to unix format --- example/reproducescript.md | 372 +++---- example/reproducescript_andersen.md | 230 ++-- example/reproducescript_group.md | 1542 +++++++++++++-------------- 3 files changed, 1072 insertions(+), 1072 deletions(-) diff --git a/example/reproducescript.md b/example/reproducescript.md index f82f1c3e1..86d78e995 100644 --- a/example/reproducescript.md +++ b/example/reproducescript.md @@ -1,186 +1,186 @@ ---- -title: Making your analysis pipeline reproducible using reproducescript -category: example -tags: [reproducescript, script] ---- - -# Making your analysis pipeline reproducible using reproducescript - -This example script will introduce you to functionality in the FieldTrip toolbox designed to aid in making your analysis pipeline - including code, data and results - easily reproducible and shareable. It is based on the manuscript [Reducing the efforts to create reproducible analysis code with FieldTrip](http://dx.doi.org/10.21105/joss.05566). - -## Description - -### Reproduciblity in neuroimaging - -In recent years, unsound scientific practices have led to a replication crisis, which lead to an increased demand for greater methodological transparency and robustness in results in the scientific community. As part of this, researchers are more and more encouraged, or even obligated, by funders and journals to publish their results and analysis pipelines along with the published journal article. - -The analysis of M/EEG data, and neuroimaging data in general, typically requires many calculations in which the raw data is transformed step by step. In FieldTrip and most other analysis software, these steps are written down in code in an analysis script. The more complex the analysis pipeline is, the more steps are required and the longer the analysis script becomes. Because of this, a full analysis pipeline is usually not written in a single script, but in a set of scripts and functions that are interdependent and have to be called in a certain order. - -Many researchers are not formally trained as software engineers or computer scientists. Thus, the quality, readability, and generalizability of analysis scripts is highly dependent on the individual researcher's coding style and expertise. Unfortunately, the variability in the quality of analysis scripts might compromise the reproducibility of results. Re-running the analysis pipeline might lead to different results, or the pipeline doesn't run on someone else's computer. - - -### Reproducescript - -In order to encourage code and data sharing, and to ensure that the shared material can reliably reproduce results, we added a functionality into FieldTrip, called _reproducescript_. In short: the researcher adds one additional flag to the configuration options in each FieldTrip function in the pipeline, which results in the analysis pipeline and data dependencies to be exported to a standardized representation that resembles the format of the FieldTrip tutorials. The generated scripts and corresponding data have minimal to no ambiguity. This code and data will then immediately be ready for sharing (though we encourage to add comments to the analysis scripts in order to explain what is happening). - -To explain the new reproducescript functionality, we will demonstrate its use with a simple example pipeline for a single-subject analysis that comprises only a few analysis steps. Second, we demonstrate its application in a complete pipeline with preprocessing for multiple subjects, followed by a group analysis. The original idiosyncratic scripts that we selected for these first two examples are relatively clean and transparent, which means they are easily reproducible even without the new reproducescript functionality. As a final, third, example, we will apply _reproducescript_ to an already published analysis pipeline that contains more complexity, and thus benefits more from the _reproducescript_ functionality. The analysis code and data used in these examples are publicly available in the [Donders Repository](https://doi.org/10.34973/21pa-dg13). - - -## Example 1 - -To show how the reproducescript functionality works, we apply it to a script from the tutorial on [Segmenting and reading trial-based EEG and MEG data](/tutorial/preprocessing). Note that before calling **[ft_topoplotER](/reference/ft_topoplotER)**, we changed the units from T to fT. This is usually not done, but in this instance it serves as an example for how reproducescript handles analysis steps that were performed outside the FieldTrip ecosystem (i.e., arbitrary code). - - %% LISTING 1 - - data_dir = '../rawdata/'; - results_dir = 'analysis/'; - - % extract epochs - cfg = []; - cfg.dataset = fullfile(data_dir, 'Subject01.ds'); - cfg.trialfun = 'ft_trialfun_general'; - cfg.trialdef.eventtype = 'backpanel trigger'; - cfg.trialdef.eventvalue = 3; - cfg.trialdef.prestim = 1; - cfg.trialdef.poststim = 2; - cfg = ft_definetrial(cfg); - - % loading data and basic preprocessing - cfg.channel = {'MEG' 'EOG'}; - cfg.continuous = 'yes'; % see https://www.fieldtriptoolbox.org/faq/continuous/ - dataFIC = ft_preprocessing(cfg); - - % time-lock analysis - cfg = []; - avgFIC = ft_timelockanalysis(cfg, dataFIC); - - % let's make a manual change to the data that is not caputured in the provenance - avgFIC.avg = avgFIC.avg * 1e15; % convert from T to fT - - % save time-locked data - save(fullfile(results_dir, 'timelock.mat'), 'avgFIC') - - % plot the results - cfg = []; - cfg.xlim = [0.3 0.5]; - cfg.layout = 'CTF151_helmet.mat'; - ft_topoplotER(cfg, avgFIC); - - % save the figure - savefig(gcf, fullfile(results_dir, 'topoplot')) - -The way our files are organised can be seen in this directory tree: - -{% include image src="/assets/img/example/reproducescript/filedir_example1_analysis.jpg" width="155" %} - -The _reproducescript_ functionality is initiated with the following code, combined with the pipeline above. The _reproducescript_ option is enabled at the top of the script. We specify the directory to which the standard script and intermediate data are written in the reproducescript field of the global `ft_default` variable. `ft_default` is the structure in which global configuration defaults are stored; it is used throughout all FieldTrip functions and global options are at the start of the function merged with the user-supplied options in the cfg structure specific to the function. Note that we are additionally specifying `ft_default.checksize = inf`, which instructs FieldTrip to never remove (large) fields from any cfg-structure, thus ensuring perfect reproducibility. We recommend enabling this additional option whenever reproducescript is used. - -{% include markup/red %} -Using _reproducescript_ can lead to a lot of data being written to disk. Be mindful of where you save it! -{% include markup/end %} - - clear - close all - - global ft_default - ft_default = []; - ft_default.checksize = inf; - - % enable reproducescript - ft_default.reproducescript = 'reproduce/'; - - % the original source code from listing 1 goes here. - - % disable reproducescript - ft_default.reproducescript = []; - -When applied to the original source code it generates a `reproduce` folder containing input and output data files with unique file identifiers, a MATLAB script, and a `hashes.mat` file: - -{% include image src="/assets/img/example/reproducescript/filedir_example1_reproducescript.jpg" width="423" %} - -The reproducescript functionality traces the steps to each FieldTrip function call, and recreates human-readable read-eval-print loops (REPL) code from scratch in the `script.m` file. For this example, `script.m` looks like the code below. - -The reproducescript functionality copies the input data to a function and the output data it generates, and gives them a unique identifier/filename. Pointers to these identifiers end up in the standardized script as `cfg.inputfile` and `cfg.outputfile`. Note that this means that no input or output data structures as they normally appear in the MATLAB workspace appear in the standardized script; these are all handled using data on disk corresponding with `cfg.inputfile` and `cfg.outputfile`. Similarly, if the function’s output is a figure (e.g., in **[ft_topoplotER](/reference/ft_topoplotER)**) the figure is also directly saved to disk in .png (bitmap) and .fig (MATLAB figure) formats. - - %% - - cfg = []; - cfg.dataset = '../rawdata/Subject01.ds'; - cfg.trialfun = 'ft_trialfun_general'; - cfg.trialdef.eventtype = 'backpanel trigger'; - cfg.trialdef.eventvalue = 3; - cfg.trialdef.prestim = 1; - cfg.trialdef.poststim = 2; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg = ft_definetrial(cfg); - - %% - - cfg = []; - cfg.dataset = '../rawdata/Subject01.ds'; - cfg.trialfun = 'ft_trialfun_general'; - cfg.trialdef.eventtype = 'backpanel trigger'; - cfg.trialdef.eventvalue = 3; - cfg.trialdef.prestim = 1; - cfg.trialdef.poststim = 2; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.datafile = '../rawdata/Subject01.ds/Subject01.meg4'; - cfg.headerfile = '../rawdata/Subject01.ds/Subject01.res4'; - cfg.dataformat = 'ctf_meg4'; - cfg.headerformat = 'ctf_res4'; - cfg.representation = 'numeric'; - cfg.trl = 'reproduce/20210112T113326_ft_preprocessing_largecfginput_trl.mat'; - cfg.outputfile = { 'reproduce/20210112T113326_ft_preprocessing_output_data.mat' }; - cfg.channel = {'MEG', 'EOG'}; - cfg.continuous = 'yes'; - ft_preprocessing(cfg); - - %% - - cfg = []; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce/20210112T113326_ft_preprocessing_output_data.mat' }; - cfg.outputfile = { 'reproduce/20210112T113332_ft_timelockanalysis_output_timelock.mat' }; - ft_timelockanalysis(cfg); - - %% - - % a new input variable is entering the pipeline here: 20210112T113333_ft_topoplotER_input_varargin_1.mat - - cfg = []; - cfg.xlim = [0.3 0.5]; - cfg.layout = 'CTF151_helmet.mat'; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { - 'reproduce/20210112T113333_ft_topoplotER_input_varargin_1.mat' }; - cfg.outputfile = 'reproduce/20210112T113338_ft_topoplotER_output'; - figure; - ft_topoplotER(cfg); - -Note that the fields from the cfg input to **[ft_definetrial](/reference/ft_definetrial)** are repeated as input to **[ft_preprocessing](/reference/ft_preprocessing)** because the configuration in the original script was not emptied. There are also additional fields created by **[ft_definetrial](/reference/ft_definetrial)**. If these fields exceed a certain printed size, which would make them unwieldy to include inline in a script (e.g., `cfg.trl`, which normally consists of a [Ntrials x 3] matrix specifying the relevant sections of the data on disk), these too are saved on disk instead of being printed in the standardized script. One last thing that should stand out is the comment “a new input variable is entering the pipeline here ...”. This points to the mat-file subsequently specified in cfg.inputfile to **[ft_topoplotER](/reference/ft_topoplotER)**. The data structure in this file was not originally created by a FieldTrip function but comes from another source: in this case it consists of the data in which originates from the T to fT unit conversion step (see the original code at the top of the page). Thus, this comment puts an emphasis on the fact that a data structure with unknown provenance enters the pipeline. See the section below on [Note on using functions outside of the FieldTrip ecosystem](/example/reproducescript/#note-on-using-functions-outside-of-the-fieldtrip-ecosystem) for how to work with _reproducescript_ and non-FieldTrip code. - -Finally, the reproduce folder contains a file named `hashes.mat`. This is a file containing MD5 hashes for bookkeeping all input and output files. It allows reproducescript to match the output files of any one step to the input files of any subsequent step. For example, the output from **[ft_preprocessing](/reference/ft_preprocessing)** is used as input to **[ft_timelockanalysis](/reference/ft_preprocessing)**, which means that the data structure only needs to be stored once and `xxx_ft_timelockanalysis_input_timelock.mat` does not have to be additionally saved to disk. If the output data from one function and the input data to the next function are slightly different, they are both saved under different file names. This happens when the researcher modified the data using custom code (as in the example when converting channel units). The `hashes.mat` file furthermore allows any researcher to check the integrity of all the intermediate and final result files of the pipeline. - -### Using functions outside the FieldTrip ecosystem - -All analysis steps that do not use FieldTrip functions will create such comments and save the data structure. Importantly, the pipeline thus remains reproducible without relying on external code. However, this does mean that it will be important to annotate script.m after it's created and note where the data with unknown provenance comes from. Even if the pipeline exclusively uses FieldTrip functions, some FieldTrip functions evaluate custom-written code. For example, a user can specify custom code to select trials in **[ft_definetrial](/reference/ft_definetrial)** (i.e., cfg.trialfun). If this code were not shared, this particular analysis step could not be re-executed, but since intermediate results are stored as well (in the example of cfg.trialfun, cfg.trl is stored), it is always possible to skip a particular step and continue with the rest of the pipeline. - -If a researcher wishes that _every_ analysis step can be re-executed, including non-FieldTrip code, a user can "FieldTrip-ify" their non-FieldTrip functions by writing a wrapper around it (see **[ft_examplefunction](/reference/ft_examplefunction)**) and [Implementing a new high-level function](/development/guideline/code/#implementing-a-new-high-level-function), such that there are no unknowns in the data provenance. Under the hood, this wrapper function uses low level FieldTrip bookkeeping functions (see [Toolbox architecture and organization of the source code](/development/architecture/#toolbox-architecture-and-organization-of-the-source-code) for more information). - -## Conclusion - -FieldTrip provides researchers with a tool to easily share complete analysis pipelines that use the FieldTrip toolbox. This is especially aimed at researchers with limited coding experience, that nevertheless want to share their analysis code and/or data with the confidence that their code is reproducible. - -Here we applied _reproducescript_ to the simplest analysis pipeline, which is example 1 in [Reducing the efforts to create reproducible analysis code with FieldTrip](http://dx.doi.org/10.21105/joss.05566). For more complicated analysis pipelines, have a look at example [2](/example/reproducescript_group/#example-2) and [3](/example/reproducescript_andersen/#example-3) in the links in the [Suggested further reading](#suggested-further-reading). - -Note that there are other strategies for improving shareability and reproducibility, and we don't assume that _reproducescript_ is the best way in every scenario. Rather, it is one of many tools that can aid the researcher to improve the community's standard in methodological transparency and robustness of results. For other strategies, we refer the reader to the pre-print in which we introduce the _reproducescript_ functionality. - -## Suggested further reading - -- [Reducing the efforts to create reproducible analysis code with FieldTrip](http://dx.doi.org/10.21105/joss.05566) -- [Using reproducescript for a group analysis](/example/reproducescript_group) -- [Using reproducescript on a full study](/example/reproducescript_andersen) +--- +title: Making your analysis pipeline reproducible using reproducescript +category: example +tags: [reproducescript, script] +--- + +# Making your analysis pipeline reproducible using reproducescript + +This example script will introduce you to functionality in the FieldTrip toolbox designed to aid in making your analysis pipeline - including code, data and results - easily reproducible and shareable. It is based on the manuscript [Reducing the efforts to create reproducible analysis code with FieldTrip](http://dx.doi.org/10.21105/joss.05566). + +## Description + +### Reproduciblity in neuroimaging + +In recent years, unsound scientific practices have led to a replication crisis, which lead to an increased demand for greater methodological transparency and robustness in results in the scientific community. As part of this, researchers are more and more encouraged, or even obligated, by funders and journals to publish their results and analysis pipelines along with the published journal article. + +The analysis of M/EEG data, and neuroimaging data in general, typically requires many calculations in which the raw data is transformed step by step. In FieldTrip and most other analysis software, these steps are written down in code in an analysis script. The more complex the analysis pipeline is, the more steps are required and the longer the analysis script becomes. Because of this, a full analysis pipeline is usually not written in a single script, but in a set of scripts and functions that are interdependent and have to be called in a certain order. + +Many researchers are not formally trained as software engineers or computer scientists. Thus, the quality, readability, and generalizability of analysis scripts is highly dependent on the individual researcher's coding style and expertise. Unfortunately, the variability in the quality of analysis scripts might compromise the reproducibility of results. Re-running the analysis pipeline might lead to different results, or the pipeline doesn't run on someone else's computer. + + +### Reproducescript + +In order to encourage code and data sharing, and to ensure that the shared material can reliably reproduce results, we added a functionality into FieldTrip, called _reproducescript_. In short: the researcher adds one additional flag to the configuration options in each FieldTrip function in the pipeline, which results in the analysis pipeline and data dependencies to be exported to a standardized representation that resembles the format of the FieldTrip tutorials. The generated scripts and corresponding data have minimal to no ambiguity. This code and data will then immediately be ready for sharing (though we encourage to add comments to the analysis scripts in order to explain what is happening). + +To explain the new reproducescript functionality, we will demonstrate its use with a simple example pipeline for a single-subject analysis that comprises only a few analysis steps. Second, we demonstrate its application in a complete pipeline with preprocessing for multiple subjects, followed by a group analysis. The original idiosyncratic scripts that we selected for these first two examples are relatively clean and transparent, which means they are easily reproducible even without the new reproducescript functionality. As a final, third, example, we will apply _reproducescript_ to an already published analysis pipeline that contains more complexity, and thus benefits more from the _reproducescript_ functionality. The analysis code and data used in these examples are publicly available in the [Donders Repository](https://doi.org/10.34973/21pa-dg13). + + +## Example 1 + +To show how the reproducescript functionality works, we apply it to a script from the tutorial on [Segmenting and reading trial-based EEG and MEG data](/tutorial/preprocessing). Note that before calling **[ft_topoplotER](/reference/ft_topoplotER)**, we changed the units from T to fT. This is usually not done, but in this instance it serves as an example for how reproducescript handles analysis steps that were performed outside the FieldTrip ecosystem (i.e., arbitrary code). + + %% LISTING 1 + + data_dir = '../rawdata/'; + results_dir = 'analysis/'; + + % extract epochs + cfg = []; + cfg.dataset = fullfile(data_dir, 'Subject01.ds'); + cfg.trialfun = 'ft_trialfun_general'; + cfg.trialdef.eventtype = 'backpanel trigger'; + cfg.trialdef.eventvalue = 3; + cfg.trialdef.prestim = 1; + cfg.trialdef.poststim = 2; + cfg = ft_definetrial(cfg); + + % loading data and basic preprocessing + cfg.channel = {'MEG' 'EOG'}; + cfg.continuous = 'yes'; % see https://www.fieldtriptoolbox.org/faq/continuous/ + dataFIC = ft_preprocessing(cfg); + + % time-lock analysis + cfg = []; + avgFIC = ft_timelockanalysis(cfg, dataFIC); + + % let's make a manual change to the data that is not caputured in the provenance + avgFIC.avg = avgFIC.avg * 1e15; % convert from T to fT + + % save time-locked data + save(fullfile(results_dir, 'timelock.mat'), 'avgFIC') + + % plot the results + cfg = []; + cfg.xlim = [0.3 0.5]; + cfg.layout = 'CTF151_helmet.mat'; + ft_topoplotER(cfg, avgFIC); + + % save the figure + savefig(gcf, fullfile(results_dir, 'topoplot')) + +The way our files are organised can be seen in this directory tree: + +{% include image src="/assets/img/example/reproducescript/filedir_example1_analysis.jpg" width="155" %} + +The _reproducescript_ functionality is initiated with the following code, combined with the pipeline above. The _reproducescript_ option is enabled at the top of the script. We specify the directory to which the standard script and intermediate data are written in the reproducescript field of the global `ft_default` variable. `ft_default` is the structure in which global configuration defaults are stored; it is used throughout all FieldTrip functions and global options are at the start of the function merged with the user-supplied options in the cfg structure specific to the function. Note that we are additionally specifying `ft_default.checksize = inf`, which instructs FieldTrip to never remove (large) fields from any cfg-structure, thus ensuring perfect reproducibility. We recommend enabling this additional option whenever reproducescript is used. + +{% include markup/red %} +Using _reproducescript_ can lead to a lot of data being written to disk. Be mindful of where you save it! +{% include markup/end %} + + clear + close all + + global ft_default + ft_default = []; + ft_default.checksize = inf; + + % enable reproducescript + ft_default.reproducescript = 'reproduce/'; + + % the original source code from listing 1 goes here. + + % disable reproducescript + ft_default.reproducescript = []; + +When applied to the original source code it generates a `reproduce` folder containing input and output data files with unique file identifiers, a MATLAB script, and a `hashes.mat` file: + +{% include image src="/assets/img/example/reproducescript/filedir_example1_reproducescript.jpg" width="423" %} + +The reproducescript functionality traces the steps to each FieldTrip function call, and recreates human-readable read-eval-print loops (REPL) code from scratch in the `script.m` file. For this example, `script.m` looks like the code below. + +The reproducescript functionality copies the input data to a function and the output data it generates, and gives them a unique identifier/filename. Pointers to these identifiers end up in the standardized script as `cfg.inputfile` and `cfg.outputfile`. Note that this means that no input or output data structures as they normally appear in the MATLAB workspace appear in the standardized script; these are all handled using data on disk corresponding with `cfg.inputfile` and `cfg.outputfile`. Similarly, if the function’s output is a figure (e.g., in **[ft_topoplotER](/reference/ft_topoplotER)**) the figure is also directly saved to disk in .png (bitmap) and .fig (MATLAB figure) formats. + + %% + + cfg = []; + cfg.dataset = '../rawdata/Subject01.ds'; + cfg.trialfun = 'ft_trialfun_general'; + cfg.trialdef.eventtype = 'backpanel trigger'; + cfg.trialdef.eventvalue = 3; + cfg.trialdef.prestim = 1; + cfg.trialdef.poststim = 2; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg = ft_definetrial(cfg); + + %% + + cfg = []; + cfg.dataset = '../rawdata/Subject01.ds'; + cfg.trialfun = 'ft_trialfun_general'; + cfg.trialdef.eventtype = 'backpanel trigger'; + cfg.trialdef.eventvalue = 3; + cfg.trialdef.prestim = 1; + cfg.trialdef.poststim = 2; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.datafile = '../rawdata/Subject01.ds/Subject01.meg4'; + cfg.headerfile = '../rawdata/Subject01.ds/Subject01.res4'; + cfg.dataformat = 'ctf_meg4'; + cfg.headerformat = 'ctf_res4'; + cfg.representation = 'numeric'; + cfg.trl = 'reproduce/20210112T113326_ft_preprocessing_largecfginput_trl.mat'; + cfg.outputfile = { 'reproduce/20210112T113326_ft_preprocessing_output_data.mat' }; + cfg.channel = {'MEG', 'EOG'}; + cfg.continuous = 'yes'; + ft_preprocessing(cfg); + + %% + + cfg = []; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce/20210112T113326_ft_preprocessing_output_data.mat' }; + cfg.outputfile = { 'reproduce/20210112T113332_ft_timelockanalysis_output_timelock.mat' }; + ft_timelockanalysis(cfg); + + %% + + % a new input variable is entering the pipeline here: 20210112T113333_ft_topoplotER_input_varargin_1.mat + + cfg = []; + cfg.xlim = [0.3 0.5]; + cfg.layout = 'CTF151_helmet.mat'; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { + 'reproduce/20210112T113333_ft_topoplotER_input_varargin_1.mat' }; + cfg.outputfile = 'reproduce/20210112T113338_ft_topoplotER_output'; + figure; + ft_topoplotER(cfg); + +Note that the fields from the cfg input to **[ft_definetrial](/reference/ft_definetrial)** are repeated as input to **[ft_preprocessing](/reference/ft_preprocessing)** because the configuration in the original script was not emptied. There are also additional fields created by **[ft_definetrial](/reference/ft_definetrial)**. If these fields exceed a certain printed size, which would make them unwieldy to include inline in a script (e.g., `cfg.trl`, which normally consists of a [Ntrials x 3] matrix specifying the relevant sections of the data on disk), these too are saved on disk instead of being printed in the standardized script. One last thing that should stand out is the comment “a new input variable is entering the pipeline here ...”. This points to the mat-file subsequently specified in cfg.inputfile to **[ft_topoplotER](/reference/ft_topoplotER)**. The data structure in this file was not originally created by a FieldTrip function but comes from another source: in this case it consists of the data in which originates from the T to fT unit conversion step (see the original code at the top of the page). Thus, this comment puts an emphasis on the fact that a data structure with unknown provenance enters the pipeline. See the section below on [Note on using functions outside of the FieldTrip ecosystem](/example/reproducescript/#note-on-using-functions-outside-of-the-fieldtrip-ecosystem) for how to work with _reproducescript_ and non-FieldTrip code. + +Finally, the reproduce folder contains a file named `hashes.mat`. This is a file containing MD5 hashes for bookkeeping all input and output files. It allows reproducescript to match the output files of any one step to the input files of any subsequent step. For example, the output from **[ft_preprocessing](/reference/ft_preprocessing)** is used as input to **[ft_timelockanalysis](/reference/ft_preprocessing)**, which means that the data structure only needs to be stored once and `xxx_ft_timelockanalysis_input_timelock.mat` does not have to be additionally saved to disk. If the output data from one function and the input data to the next function are slightly different, they are both saved under different file names. This happens when the researcher modified the data using custom code (as in the example when converting channel units). The `hashes.mat` file furthermore allows any researcher to check the integrity of all the intermediate and final result files of the pipeline. + +### Using functions outside the FieldTrip ecosystem + +All analysis steps that do not use FieldTrip functions will create such comments and save the data structure. Importantly, the pipeline thus remains reproducible without relying on external code. However, this does mean that it will be important to annotate script.m after it's created and note where the data with unknown provenance comes from. Even if the pipeline exclusively uses FieldTrip functions, some FieldTrip functions evaluate custom-written code. For example, a user can specify custom code to select trials in **[ft_definetrial](/reference/ft_definetrial)** (i.e., cfg.trialfun). If this code were not shared, this particular analysis step could not be re-executed, but since intermediate results are stored as well (in the example of cfg.trialfun, cfg.trl is stored), it is always possible to skip a particular step and continue with the rest of the pipeline. + +If a researcher wishes that _every_ analysis step can be re-executed, including non-FieldTrip code, a user can "FieldTrip-ify" their non-FieldTrip functions by writing a wrapper around it (see **[ft_examplefunction](/reference/ft_examplefunction)**) and [Implementing a new high-level function](/development/guideline/code/#implementing-a-new-high-level-function), such that there are no unknowns in the data provenance. Under the hood, this wrapper function uses low level FieldTrip bookkeeping functions (see [Toolbox architecture and organization of the source code](/development/architecture/#toolbox-architecture-and-organization-of-the-source-code) for more information). + +## Conclusion + +FieldTrip provides researchers with a tool to easily share complete analysis pipelines that use the FieldTrip toolbox. This is especially aimed at researchers with limited coding experience, that nevertheless want to share their analysis code and/or data with the confidence that their code is reproducible. + +Here we applied _reproducescript_ to the simplest analysis pipeline, which is example 1 in [Reducing the efforts to create reproducible analysis code with FieldTrip](http://dx.doi.org/10.21105/joss.05566). For more complicated analysis pipelines, have a look at example [2](/example/reproducescript_group/#example-2) and [3](/example/reproducescript_andersen/#example-3) in the links in the [Suggested further reading](#suggested-further-reading). + +Note that there are other strategies for improving shareability and reproducibility, and we don't assume that _reproducescript_ is the best way in every scenario. Rather, it is one of many tools that can aid the researcher to improve the community's standard in methodological transparency and robustness of results. For other strategies, we refer the reader to the pre-print in which we introduce the _reproducescript_ functionality. + +## Suggested further reading + +- [Reducing the efforts to create reproducible analysis code with FieldTrip](http://dx.doi.org/10.21105/joss.05566) +- [Using reproducescript for a group analysis](/example/reproducescript_group) +- [Using reproducescript on a full study](/example/reproducescript_andersen) diff --git a/example/reproducescript_andersen.md b/example/reproducescript_andersen.md index d17fe4bf0..a0051e441 100644 --- a/example/reproducescript_andersen.md +++ b/example/reproducescript_andersen.md @@ -1,115 +1,115 @@ ---- -title: Using reproducescript on a full study -category: example -tags: [reproducescript] ---- - -# Using reproducescript on a full study - -This example script will introduce you to functionality in the FieldTrip toolbox designed to aid in making your analysis pipeline - including code, data and results - easily reproducible and shareable. It is based on the manuscript [Reducing the efforts to create reproducible analysis code with FieldTrip](http://dx.doi.org/10.21105/joss.05566). We assume that you already had a look at the examples on [Making your analysis pipeline reproducible using reproducescript](/example/reproducescript) and [Using reproducescript for a group analysis](/example/reproducescript_group). - -## Example 3 - -Example [1](/example/reproducescript/#example-1) and [2](/example/reproducescript_group/#example-2) were fairly simple analysis pipelines that didn't particularly benefit from _reproducescript_ because the original scripts and data organisation were already clear. Those examples are intended to show how _reproducescript_ works and how it's used. In this example, we apply _reproducescript_ to a published analysis pipeline of MEG data by [Andersen (2018)](https://doi.org/10.3389/fnins.2018.00261). We hope that this will help you to set up your own analysis pipeline using _reproducescript_. - -### Original analysis - -The analysis pipeline in Andersen (2018) is well-documented and itself a good demonstration of a reproducible analysis pipeline in the FieldTrip ecosystem. Nevertheless, it consists of a complex set of 10 analysis scripts and 46 functions, which, without the extensive documentation that has been provided by the author, would be challenging to reuse and reproduce the results. This makes it particularly suited to demonstrate the effectiveness and simplicity of reproducescript. - -Andersen describes an analysis pipeline from raw single-subject MEG data to group-level statistics in source space. Each of the custom-written scripts has a specific purpose: - -{% include image src="/assets/img/example/reproducescript/fnins-12-00261-t002.jpg" width="495" %} -_Figure copied from [Andersen (2018)](https://doi.org/10.3389/fnins.2018.00261) with permission from the author._ - -Still, multiple analysis steps in separate functions are required for the purpose of one script (see figure 6 for the full analysis pipeline), creating a complex hierarchy of scripts and functions: - -{% include image src="/assets/img/example/reproducescript/fnins-12-00261-g002.jpg" width="500" %} -_Figure copied from [Andersen (2018)](https://doi.org/10.3389/fnins.2018.00261) with permission from the author._ - -To keep the computational time and storage requirements low, we applied the full analysis pipeline to two subjects only. Both the original source code from Andersen and the standardized scripts generated by reproducescript are available on [GitHub](https://github.com/matsvanes/reproducescript). Using the documentation in Andersen (2018), we wrote the master script `run_all.m` , which calls all relevant functions in Andersen's analysis pipeline in the correct order. Here we added the `reproducescript` option to `ft_default` in the same way as in the [group analysis](/example/reproducescript_group) example, i.e., by initialising `reproducescript` separately on each subject, and then once for the group analysis (see below). - -#### run_all.m with reproducescript enabled - -This is the original code for `run_all.m` with the only difference that reproducescript is enabled. It is also available on [GitHub](https://github.com/matsvanes/reproducescript/blob/master/example3_andersen/omission_frontiers/FieldTrip/run_all.m). - - clear - close all - - global ft_default - ft_default = []; - ft_default.checksize = inf; - - %% Single subject analysis - datainfo; - for do_subject = 1:numel(all_subjects) - reproduce_dir = [home_dir, sprintf('reproduce%02d/', do_subject)]; - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - % enable reproducescript - ft_default.reproducescript = reproduce_dir; - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - - % Create all relevant directories where all data and all figures will be saved - create_MEG_BIDS_data_structure - - % Go from raw MEG data to a time-frequency representation - sensor_space_analysis - - % Go from raw MRI data to a volume conductor and a forward model - mr_preprocessing - - % Extract fourier transforms and do beamformer source reconstructions - source_space_analysis - - % Plot all steps in the sensor space analysis - plot_sensor_space - - % Plot all steps in the MR processing - plot_processed_mr - - % Plot all steps in the source space analysis - plot_source_space - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - % disable reproducescript - ft_default.reproducescript = []; - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - end - - %% Group analysis - datainfo; - reproduce_dir = [home_dir, 'reproduce_group']; - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - % enable reproducescript - ft_default.reproducescript = reproduce_dir; - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - - % Do grand averages across subjects for both sensor and source spaces - grand_averages - - % Do statistics on time-frequency representations and beamformer source reconstructions - statistics - - % Plot grand averages in both the sensor and source spaces, with and without statistical masking - plot_grand_averages - - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - % disable reproducescript - ft_default.reproducescript = []; - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -### Reproduced analysis - -Please head over to [GitHub](https://github.com/matsvanes/reproducescript/tree/master/example3_andersen) to see what the reproduced `script.m` look like for the two subjects and for the group level. - -## Conclusion - -In three examples we have shown how the _reproducescript_ functionality can be applied to any analysis pipeline that is based on the FieldTrip ecosystem. This functionality can be applied without much effort on the researcher's side, and it generates all code, original and intermediate data, as well as final results in a format in which it's readily shareable and reproducible. - -Note that there are other strategies for improving shareability and reproducibility, and we don't claim that _reproducescript_ is the best way in every scenario. Rather, it is one of many tools that can aid the researcher to improve the community's standard in methodological transparency and robustness of results. For other strategies, we refer the reader to the pre-print in which we first described _reproducescript_. - -## Suggested further reading - -- [Reducing the efforts to create reproducible analysis code with FieldTrip](http://dx.doi.org/10.21105/joss.05566) -- Andersen (2018). [Group Analysis in FieldTrip of Time-Frequency Responses](https://doi.org/10.3389/fnins.2018.00261) +--- +title: Using reproducescript on a full study +category: example +tags: [reproducescript] +--- + +# Using reproducescript on a full study + +This example script will introduce you to functionality in the FieldTrip toolbox designed to aid in making your analysis pipeline - including code, data and results - easily reproducible and shareable. It is based on the manuscript [Reducing the efforts to create reproducible analysis code with FieldTrip](http://dx.doi.org/10.21105/joss.05566). We assume that you already had a look at the examples on [Making your analysis pipeline reproducible using reproducescript](/example/reproducescript) and [Using reproducescript for a group analysis](/example/reproducescript_group). + +## Example 3 + +Example [1](/example/reproducescript/#example-1) and [2](/example/reproducescript_group/#example-2) were fairly simple analysis pipelines that didn't particularly benefit from _reproducescript_ because the original scripts and data organisation were already clear. Those examples are intended to show how _reproducescript_ works and how it's used. In this example, we apply _reproducescript_ to a published analysis pipeline of MEG data by [Andersen (2018)](https://doi.org/10.3389/fnins.2018.00261). We hope that this will help you to set up your own analysis pipeline using _reproducescript_. + +### Original analysis + +The analysis pipeline in Andersen (2018) is well-documented and itself a good demonstration of a reproducible analysis pipeline in the FieldTrip ecosystem. Nevertheless, it consists of a complex set of 10 analysis scripts and 46 functions, which, without the extensive documentation that has been provided by the author, would be challenging to reuse and reproduce the results. This makes it particularly suited to demonstrate the effectiveness and simplicity of reproducescript. + +Andersen describes an analysis pipeline from raw single-subject MEG data to group-level statistics in source space. Each of the custom-written scripts has a specific purpose: + +{% include image src="/assets/img/example/reproducescript/fnins-12-00261-t002.jpg" width="495" %} +_Figure copied from [Andersen (2018)](https://doi.org/10.3389/fnins.2018.00261) with permission from the author._ + +Still, multiple analysis steps in separate functions are required for the purpose of one script (see figure 6 for the full analysis pipeline), creating a complex hierarchy of scripts and functions: + +{% include image src="/assets/img/example/reproducescript/fnins-12-00261-g002.jpg" width="500" %} +_Figure copied from [Andersen (2018)](https://doi.org/10.3389/fnins.2018.00261) with permission from the author._ + +To keep the computational time and storage requirements low, we applied the full analysis pipeline to two subjects only. Both the original source code from Andersen and the standardized scripts generated by reproducescript are available on [GitHub](https://github.com/matsvanes/reproducescript). Using the documentation in Andersen (2018), we wrote the master script `run_all.m` , which calls all relevant functions in Andersen's analysis pipeline in the correct order. Here we added the `reproducescript` option to `ft_default` in the same way as in the [group analysis](/example/reproducescript_group) example, i.e., by initialising `reproducescript` separately on each subject, and then once for the group analysis (see below). + +#### run_all.m with reproducescript enabled + +This is the original code for `run_all.m` with the only difference that reproducescript is enabled. It is also available on [GitHub](https://github.com/matsvanes/reproducescript/blob/master/example3_andersen/omission_frontiers/FieldTrip/run_all.m). + + clear + close all + + global ft_default + ft_default = []; + ft_default.checksize = inf; + + %% Single subject analysis + datainfo; + for do_subject = 1:numel(all_subjects) + reproduce_dir = [home_dir, sprintf('reproduce%02d/', do_subject)]; + + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + % enable reproducescript + ft_default.reproducescript = reproduce_dir; + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + + % Create all relevant directories where all data and all figures will be saved + create_MEG_BIDS_data_structure + + % Go from raw MEG data to a time-frequency representation + sensor_space_analysis + + % Go from raw MRI data to a volume conductor and a forward model + mr_preprocessing + + % Extract fourier transforms and do beamformer source reconstructions + source_space_analysis + + % Plot all steps in the sensor space analysis + plot_sensor_space + + % Plot all steps in the MR processing + plot_processed_mr + + % Plot all steps in the source space analysis + plot_source_space + + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + % disable reproducescript + ft_default.reproducescript = []; + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + end + + %% Group analysis + datainfo; + reproduce_dir = [home_dir, 'reproduce_group']; + + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + % enable reproducescript + ft_default.reproducescript = reproduce_dir; + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + + % Do grand averages across subjects for both sensor and source spaces + grand_averages + + % Do statistics on time-frequency representations and beamformer source reconstructions + statistics + + % Plot grand averages in both the sensor and source spaces, with and without statistical masking + plot_grand_averages + + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + % disable reproducescript + ft_default.reproducescript = []; + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +### Reproduced analysis + +Please head over to [GitHub](https://github.com/matsvanes/reproducescript/tree/master/example3_andersen) to see what the reproduced `script.m` look like for the two subjects and for the group level. + +## Conclusion + +In three examples we have shown how the _reproducescript_ functionality can be applied to any analysis pipeline that is based on the FieldTrip ecosystem. This functionality can be applied without much effort on the researcher's side, and it generates all code, original and intermediate data, as well as final results in a format in which it's readily shareable and reproducible. + +Note that there are other strategies for improving shareability and reproducibility, and we don't claim that _reproducescript_ is the best way in every scenario. Rather, it is one of many tools that can aid the researcher to improve the community's standard in methodological transparency and robustness of results. For other strategies, we refer the reader to the pre-print in which we first described _reproducescript_. + +## Suggested further reading + +- [Reducing the efforts to create reproducible analysis code with FieldTrip](http://dx.doi.org/10.21105/joss.05566) +- Andersen (2018). [Group Analysis in FieldTrip of Time-Frequency Responses](https://doi.org/10.3389/fnins.2018.00261) diff --git a/example/reproducescript_group.md b/example/reproducescript_group.md index 8cce5fe8d..6f6ce649b 100644 --- a/example/reproducescript_group.md +++ b/example/reproducescript_group.md @@ -1,771 +1,771 @@ ---- -title: Using reproducescript for a group analysis -category: example -tags: [reproducescript] ---- - -# Using reproducescript for a group analysis - -This example script will introduce you to functionality in the FieldTrip toolbox designed to aid in making your analysis pipeline - including code, data and results - more easily reproducible and shareable. It is based on the manuscript [Reducing the efforts to create reproducible analysis code with FieldTrip](http://dx.doi.org/10.21105/joss.05566). We assume that you have already had a look at the example on [Making your analysis pipeline reproducible using reproducescript](/example/reproducescript). - -## Example 2 - -The first example contained only a few analysis steps in a single subject. More realistic data analysis pipelines consist of many more steps in which often the same (or similar) pipelines are used for multiple subjects. In this section, we will show how the reproducescript functionality applies in such a case. - -### Original analysis - -The analysis example follows the strategy outlined in the [Literate Programming](https://www-cs-faculty.stanford.edu/~knuth/lp.html) book by Donald Knuth and starts with a single subject analysis pipeline that is repeated for four subjects. The directory is structured as depicted below. - -{% include image src="/assets/img/example/reproducescript/filedir_example2_analysis.jpg" width="237" %} - -After the single-subject analysis, all single-subject results are used in a group analysis. The single-subject and group analyses are executed from the master script `analyze.m`: - -#### analyze.m - - clear - close all - - subjlist = { - 'Subject01' - 'Subject02' - 'Subject03' - 'Subject04' - }; - - %% Loop single-subject analyses over subjects - for i=1:numel(subjlist) - subj = subjlist{i}; - doSingleSubjectAnalysis(subj); - end - - %% Group analysis - doGroupAnalysis(subjlist); - -This is the control script from which the relevant analysis scripts and functions are called. The master script relies on two functions: `doSingleSubjectAnalysis` and `doGroupAnalysis`, which are each stored in separate m-files. The original source code for these scripts can be found below. - -#### doSingleSubjectAnalysis.m - - function doSingleSubjectAnalysis(subj) - - % the details of each subject are in separate files - % details_Subject01.m - % details_Subject02.m - % details_Subject03.m - % details_Subject04.m - - fprintf('evaluating single subject analysis for %s\n', subj); - eval(['details_' subj]); - - % this is for artifact detection - interactive = false; - - %% - - cfg = []; - cfg.dataset = fullfile(datadir, filename); - cfg.trialfun = 'ft_trialfun_general'; - cfg.trialdef.eventtype = 'backpanel trigger'; - cfg.trialdef.eventvalue = [triggerFIC triggerIC triggerFC]; - cfg.trialdef.prestim = 1; - cfg.trialdef.poststim = 2; - cfg = ft_definetrial(cfg); - - % the EOG channel has a different name in the different datasets - cfg.channel = {'MEG' eogchannel}; - cfg.continuous = 'yes'; % see https://www.fieldtriptoolbox.org/faq/continuous/ - data = ft_preprocessing(cfg); - - %% - - if interactive - % visually identify the artifacts - cfg = []; - cfg.channel = eogchannel; - cfg.method = 'channel'; - dummy1 = ft_rejectvisual(cfg, data); - - cfg = []; - cfg.channel = 'MEG'; - cfg.method = 'summary'; - dummy2 = ft_rejectvisual(cfg, data); - - % combine the artifacts that have been detected - artifact = [ - dummy1.cfg.artfctdef.channel.artifact - dummy2.cfg.artfctdef.summary.artifact - ]; - - % print them and copy them to the subject details file - disp(artifact); - - % use the MATLAB debugger to wait on this line - disp('please copy these artifacts to the subject details file'); - keyboard - end - - % remove the artifacts that were previously detected - cfg = []; - cfg.artfctdef.visual.artifact = artifact; - data_clean = ft_rejectartifact(cfg, data); - - %% - - cfg = []; - cfg.trials = data_clean.trialinfo==triggerFIC; - avgFIC = ft_timelockanalysis(cfg, data_clean); - - cfg.trials = data_clean.trialinfo==triggerFC; - avgFC = ft_timelockanalysis(cfg, data_clean); - - cfg.trials = data_clean.trialinfo==triggerIC; - avgIC = ft_timelockanalysis(cfg, data_clean); - - %% - - cfg = []; - cfg.showlabels = 'no'; - cfg.fontsize = 6; - cfg.layout = 'CTF151_helmet.mat'; - cfg.baseline = [-0.2 0]; - cfg.xlim = [-0.2 1.0]; - cfg.ylim = [-3e-13 3e-13]; - ft_multiplotER(cfg, avgFC, avgIC, avgFIC); - - %% - - cfg = []; - cfg.feedback = 'yes'; - cfg.method = 'template'; - cfg.neighbours = ft_prepare_neighbours(cfg, avgFIC); - cfg.planarmethod = 'sincos'; - avgFICplanar = ft_megplanar(cfg, avgFIC); - avgFCplanar = ft_megplanar(cfg, avgFC); - avgICplanar = ft_megplanar(cfg, avgIC); - - %% - - cfg = []; - avgFICplanarComb = ft_combineplanar(cfg, avgFICplanar); - avgFCplanarComb = ft_combineplanar(cfg, avgFCplanar); - avgICplanarComb = ft_combineplanar(cfg, avgICplanar); - - %% - - cfg = []; - cfg.xlim = [0.3 0.5]; - cfg.zlim = 'maxmin'; - cfg.colorbar = 'yes'; - cfg.layout = 'CTF151_helmet.mat'; - subplot(2,3,1); ft_topoplotER(cfg, avgFIC) - subplot(2,3,2); ft_topoplotER(cfg, avgFC) - subplot(2,3,3); ft_topoplotER(cfg, avgIC) - - cfg.zlim = 'maxabs'; - cfg.layout = 'CTF151_helmet.mat'; - subplot(2,3,4); ft_topoplotER(cfg, avgFICplanarComb) - subplot(2,3,5); ft_topoplotER(cfg, avgFCplanarComb) - subplot(2,3,6); ft_topoplotER(cfg, avgICplanarComb) - - %% - - % save the results to disk - outputdir = ['result_' subj]; - mkdir(outputdir) - save(fullfile(outputdir, 'avgFIC'), 'avgFIC'); - save(fullfile(outputdir, 'avgFC'), 'avgFC'); - save(fullfile(outputdir, 'avgIC'), 'avgIC'); - save(fullfile(outputdir, 'avgFICplanarComb'), 'avgFICplanarComb'); - save(fullfile(outputdir, 'avgFCplanarComb'), 'avgFCplanarComb'); - save(fullfile(outputdir, 'avgICplanarComb'), 'avgICplanarComb'); - -#### doGroupAnalysis.m - - function doGroupAnalysis(allsubj) - - avgFIC = cell(size(allsubj)); - avgFC = cell(size(allsubj)); - avgIC = cell(size(allsubj)); - - % load the results from disk - for i=1:numel(allsubj) - subj = allsubj{i}; - fprintf('loading data for subject %s\n', subj); - - inputdir = ['result_' subj]; - tmp = load(fullfile(inputdir, 'avgFIC')); avgFIC{i} = tmp.avgFIC; - tmp = load(fullfile(inputdir, 'avgFC')); avgFC{i} = tmp.avgFC; - tmp = load(fullfile(inputdir, 'avgIC')); avgIC{i} = tmp.avgIC; - clear tmp - end - - %% - - cfg = []; - cfg.showlabels = 'no'; - cfg.fontsize = 6; - cfg.layout = 'CTF151_helmet.mat'; - cfg.baseline = [-0.2 0]; - cfg.xlim = [-0.2 1.0]; - cfg.ylim = [-3e-13 3e-13]; - figure - ft_multiplotER(cfg, avgFIC{:}); - title('Fully incongruent condition'); - - figure - ft_multiplotER(cfg, avgFC{:}); - title('Fully congruent condition'); - - figure - ft_multiplotER(cfg, avgIC{:}); - title('Initially congruent condition'); - - %% - - avgFICvsFC = cell(size(allsubj)); - for i=1:numel(allsubj) - cfg = []; - cfg.parameter = 'avg'; - cfg.operation = 'x1-x2'; - avgFICvsFC{i} = ft_math(cfg, avgFIC{i}, avgFC{i}); - end - - cfg = []; - cfg.showlabels = 'no'; - cfg.fontsize = 6; - cfg.layout = 'CTF151_helmet.mat'; - cfg.baseline = [-0.2 0]; - cfg.xlim = [-0.2 1.0]; - cfg.ylim = [-3e-13 3e-13]; - ft_multiplotER(cfg, avgFICvsFC{:}); - title('FIC minus FC'); - - %% - - % let's make a manual change to the data that is not caputured in the provenance - for i=1:numel(allsubj) - avgFIC{i}.avg = avgFIC{i}.avg * 1e15; % convert from T to fT - avgFC{i}.avg = avgFC{i}.avg * 1e15; % convert from T to fT - avgIC{i}.avg = avgIC{i}.avg * 1e15; % convert from T to fT - end - - %% - - cfg = []; - grandavgFIC = ft_timelockgrandaverage(cfg, avgFIC{:}); - grandavgFC = ft_timelockgrandaverage(cfg, avgFC{:}); - grandavgIC = ft_timelockgrandaverage(cfg, avgIC{:}); - - %% - - % save the results to disk - outputdir = 'result_Group'; - mkdir(outputdir) - save(fullfile(outputdir, 'grandavgFIC'), 'grandavgFIC'); - save(fullfile(outputdir, 'grandavgFC'), 'grandavgFC'); - save(fullfile(outputdir, 'grandavgIC'), 'grandavgIC'); - -### Using reproducescript - -To create a standard script from the analysis pipeline, the `ft_default` variable is initialized at the top of the `analyze.m` script. Note that we do not immediately initiate reproducescript, this is done in the loop just prior to `doSingleSubjectAnalysis` and just prior to `doGroupAnalysis`; this allows specifying unique directories for each subject and for the group. In fact, reproducescript can be stopped and restarted between different subjects, or even in between analysis steps, which is especially convenient in pipelines that require a lot of compute resources which you would rather split up to allow for parallel execution on a compute cluster. - -#### analyze.m with reproducescript enabled - - clear - close all - - % initialize the global variable - global ft_default - ft_default = []; - ft_default.checksize = inf; - - subjlist = { - 'Subject01' - 'Subject02' - 'Subject03' - 'Subject04' - }; - - %% Loop single-subject analysis over subjects - for i=1:numel(subjlist) - subj = subjlist{i}; - % initiate reproducescript - ft_default.reproducescript = ['reproduce_' subj]; - doSingleSubjectAnalysis(subj); - ft_default.reproducescript = []; % disable - end - - %% Group analysis - % initiate reproducescript - ft_default.reproducescript = 'reproduce_Group'; - doGroupAnalysis(subjlist); - ft_default.reproducescript = []; % disable - -### Reproduced analysis - -{% include image src="/assets/img/example/reproducescript/filedir_example2_reproducescript.jpg" width="435" %} - -We devoted a specific folder to the reproducescript content of each subject, and one for the group analysis. Thus, upon execution of the master script in `analyze.m`, folders are created for each of the subjects, and for the group analysis. These all contain the intermediate data, a standardized script, and a `hashes.mat` file for the bookkeeping. The reproducescript standardized scripts for the single-subject analysis and group analysis can be found below. Note that these are quite lengthy, but they unambiguously list all analysis steps. - -#### Single subject reproduced analysis - -This is the resulting code in the file `project/example2/reproduce_Subject01/script.m` - - %% - - cfg = []; - cfg.dataset = '../rawdata/Subject01.ds'; - cfg.trialfun = 'ft_trialfun_general'; - cfg.trialdef.eventtype = 'backpanel trigger'; - cfg.trialdef.eventvalue = [3 5 9]; - cfg.trialdef.prestim = 1; - cfg.trialdef.poststim = 2; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg = ft_definetrial(cfg); - - %% - - cfg = []; - cfg.dataset = '../rawdata/Subject01.ds'; - cfg.trialfun = 'ft_trialfun_general'; - cfg.trialdef.eventtype = 'backpanel trigger'; - cfg.trialdef.eventvalue = [3 5 9]; - cfg.trialdef.prestim = 1; - cfg.trialdef.poststim = 2; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.datafile = '../rawdata/Subject01.ds/Subject01.meg4'; - cfg.headerfile = '../rawdata/Subject01.ds/Subject01.res4'; - cfg.dataformat = 'ctf_meg4'; - cfg.headerformat = 'ctf_res4'; - cfg.representation = 'numeric'; - cfg.trl = 'reproduce_Subject01/20210112T113604_ft_preprocessing_largecfginput_trl.mat'; - cfg.outputfile = { 'reproduce_Subject01/20210112T113604_ft_preprocessing_output_data.mat' }; - cfg.channel = {'MEG', 'EOG'}; - cfg.continuous = 'yes'; - ft_preprocessing(cfg); - - %% - - cfg = []; - cfg.artfctdef.visual.artifact = [8101 9000; - 68401 69300; - 99001 99900; - ... - 228601 229500]; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113604_ft_preprocessing_output_data.mat' }; - cfg.outputfile = { 'reproduce_Subject01/20210112T113611_ft_rejectartifact_output_data.mat' }; - ft_rejectartifact(cfg); - - %% - - cfg = []; - cfg.trials = logical([true false ... false]); - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113611_ft_rejectartifact_output_data.mat' }; - cfg.outputfile = { 'reproduce_Subject01/20210112T113616_ft_timelockanalysis_output_timelock.mat' }; - ft_timelockanalysis(cfg); - - %% - - cfg = []; - cfg.trials = logical([false false ... false]); - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113611_ft_rejectartifact_output_data.mat' }; - cfg.outputfile = { 'reproduce_Subject01/20210112T113621_ft_timelockanalysis_output_timelock.mat' }; - ft_timelockanalysis(cfg); - - %% - - cfg = []; - cfg.trials = logical([false true ... true]); - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113611_ft_rejectartifact_output_data.mat' }; - cfg.outputfile = { 'reproduce_Subject01/20210112T113625_ft_timelockanalysis_output_timelock.mat' }; - ft_timelockanalysis(cfg); - - %% - - cfg = []; - cfg.showlabels = 'no'; - cfg.fontsize = 6; - cfg.layout = 'CTF151_helmet.mat'; - cfg.baseline = [-0.2 0]; - cfg.xlim = [-0.2 1]; - cfg.ylim = [-3e-13 3e-13]; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { - 'reproduce_Subject01/20210112T113621_ft_timelockanalysis_output_timelock.mat', 'reproduce_Subject01/20210112T113625_ft_timelockanalysis_output_timelock.mat', 'reproduce_Subject01/20210112T113616_ft_timelockanalysis_output_timelock.mat' - }; - cfg.outputfile = 'reproduce_Subject01/20210112T113634_ft_multiplotER_output'; - figure; - ft_multiplotER(cfg); - - %% - - cfg = []; - cfg.feedback = 'yes'; - cfg.method = 'template'; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113616_ft_timelockanalysis_output_timelock.mat' }; - cfg.outputfile = { 'reproduce_Subject01/20210112T113643_ft_prepare_neighbours_output_neighbours.mat' }; - ft_prepare_neighbours(cfg); - - %% - - cfg = []; - cfg.feedback = 'yes'; - cfg.method = 'template'; - cfg.neighbours = 'reproduce_Subject01/20210112T113643_ft_megplanar_largecfginput_neighbours.mat'; - cfg.planarmethod = 'sincos'; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113616_ft_timelockanalysis_output_timelock.mat' }; - cfg.outputfile = { 'reproduce_Subject01/20210112T113647_ft_megplanar_output_data.mat' }; - ft_megplanar(cfg); - - %% - - cfg = []; - cfg.feedback = 'yes'; - cfg.method = 'template'; - cfg.neighbours = 'reproduce_Subject01/20210112T113643_ft_megplanar_largecfginput_neighbours.mat'; - cfg.planarmethod = 'sincos'; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113621_ft_timelockanalysis_output_timelock.mat' }; - cfg.outputfile = { 'reproduce_Subject01/20210112T113650_ft_megplanar_output_data.mat' }; - ft_megplanar(cfg); - - %% - - cfg = []; - cfg.feedback = 'yes'; - cfg.method = 'template'; - cfg.neighbours = 'reproduce_Subject01/20210112T113643_ft_megplanar_largecfginput_neighbours.mat'; - cfg.planarmethod = 'sincos'; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113625_ft_timelockanalysis_output_timelock.mat' }; - cfg.outputfile = { 'reproduce_Subject01/20210112T113653_ft_megplanar_output_data.mat' }; - ft_megplanar(cfg); - - %% - - cfg = []; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113647_ft_megplanar_output_data.mat' }; - cfg.outputfile = { 'reproduce_Subject01/20210112T113657_ft_combineplanar_output_data.mat' }; - ft_combineplanar(cfg); - - %% - - cfg = []; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113650_ft_megplanar_output_data.mat' }; - cfg.outputfile = { 'reproduce_Subject01/20210112T113701_ft_combineplanar_output_data.mat' }; - ft_combineplanar(cfg); - - %% - - cfg = []; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113653_ft_megplanar_output_data.mat' }; - cfg.outputfile = { 'reproduce_Subject01/20210112T113704_ft_combineplanar_output_data.mat' }; - ft_combineplanar(cfg); - - %% - - cfg = []; - cfg.xlim = [0.3 0.5]; - cfg.zlim = 'maxmin'; - cfg.colorbar = 'yes'; - cfg.layout = 'CTF151_helmet.mat'; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113616_ft_timelockanalysis_output_timelock.mat' }; - cfg.outputfile = 'reproduce_Subject01/20210112T113708_ft_topoplotER_output'; - figure; - ft_topoplotER(cfg); - - %% - - cfg = []; - cfg.xlim = [0.3 0.5]; - cfg.zlim = 'maxmin'; - cfg.colorbar = 'yes'; - cfg.layout = 'CTF151_helmet.mat'; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113621_ft_timelockanalysis_output_timelock.mat' }; - cfg.outputfile = 'reproduce_Subject01/20210112T113712_ft_topoplotER_output'; - figure; - ft_topoplotER(cfg); - - %% - - cfg = []; - cfg.xlim = [0.3 0.5]; - cfg.zlim = 'maxmin'; - cfg.colorbar = 'yes'; - cfg.layout = 'CTF151_helmet.mat'; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113625_ft_timelockanalysis_output_timelock.mat' }; - cfg.outputfile = 'reproduce_Subject01/20210112T113716_ft_topoplotER_output'; - figure; - ft_topoplotER(cfg); - - %% - - cfg = []; - cfg.xlim = [0.3 0.5]; - cfg.zlim = 'maxabs'; - cfg.colorbar = 'yes'; - cfg.layout = 'CTF151_helmet.mat'; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113657_ft_combineplanar_output_data.mat' }; - cfg.outputfile = 'reproduce_Subject01/20210112T113721_ft_topoplotER_output'; - figure; - ft_topoplotER(cfg); - - %% - - cfg = []; - cfg.xlim = [0.3 0.5]; - cfg.zlim = 'maxabs'; - cfg.colorbar = 'yes'; - cfg.layout = 'CTF151_helmet.mat'; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113701_ft_combineplanar_output_data.mat' }; - cfg.outputfile = 'reproduce_Subject01/20210112T113726_ft_topoplotER_output'; - figure; - ft_topoplotER(cfg); - - %% - - cfg = []; - cfg.xlim = [0.3 0.5]; - cfg.zlim = 'maxabs'; - cfg.colorbar = 'yes'; - cfg.layout = 'CTF151_helmet.mat'; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { 'reproduce_Subject01/20210112T113704_ft_combineplanar_output_data.mat' }; - cfg.outputfile = 'reproduce_Subject01/20210112T113731_ft_topoplotER_output'; - figure; - ft_topoplotER(cfg); - -#### Group reproduced analysis - -This is the resulting code in the file `project/example2/reproduce_Group/script.m` - - %% - - % a new input variable is entering the pipeline here: 20210112T114236_ft_multiplotER_input_varargin_1.mat - % a new input variable is entering the pipeline here: 20210112T114236_ft_multiplotER_input_varargin_2.mat - % a new input variable is entering the pipeline here: 20210112T114236_ft_multiplotER_input_varargin_3.mat - % a new input variable is entering the pipeline here: 20210112T114236_ft_multiplotER_input_varargin_4.mat - - cfg = []; - cfg.showlabels = 'no'; - cfg.fontsize = 6; - cfg.layout = 'CTF151_helmet.mat'; - cfg.baseline = [-0.2 0]; - cfg.xlim = [-0.2 1]; - cfg.ylim = [-3e-13 3e-13]; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { - 'reproduce_Group/20210112T114236_ft_multiplotER_input_varargin_1.mat', 'reproduce_Group/20210112T114236_ft_multiplotER_input_varargin_2.mat', 'reproduce_Group/20210112T114236_ft_multiplotER_input_varargin_3.mat', 'reproduce_Group/20210112T114236_ft_multiplotER_input_varargin_4.mat' - }; - cfg.outputfile = 'reproduce_Group/20210112T114246_ft_multiplotER_output'; - figure; - ft_multiplotER(cfg); - - %% - - % a new input variable is entering the pipeline here: 20210112T114253_ft_multiplotER_input_varargin_1.mat - % a new input variable is entering the pipeline here: 20210112T114253_ft_multiplotER_input_varargin_2.mat - % a new input variable is entering the pipeline here: 20210112T114253_ft_multiplotER_input_varargin_3.mat - % a new input variable is entering the pipeline here: 20210112T114253_ft_multiplotER_input_varargin_4.mat - - cfg = []; - cfg.showlabels = 'no'; - cfg.fontsize = 6; - cfg.layout = 'CTF151_helmet.mat'; - cfg.baseline = [-0.2 0]; - cfg.xlim = [-0.2 1]; - cfg.ylim = [-3e-13 3e-13]; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { - 'reproduce_Group/20210112T114253_ft_multiplotER_input_varargin_1.mat', 'reproduce_Group/20210112T114253_ft_multiplotER_input_varargin_2.mat', 'reproduce_Group/20210112T114253_ft_multiplotER_input_varargin_3.mat', 'reproduce_Group/20210112T114253_ft_multiplotER_input_varargin_4.mat' - }; - cfg.outputfile = 'reproduce_Group/20210112T114303_ft_multiplotER_output'; - figure; - ft_multiplotER(cfg); - - %% - - % a new input variable is entering the pipeline here: 20210112T114310_ft_multiplotER_input_varargin_1.mat - % a new input variable is entering the pipeline here: 20210112T114310_ft_multiplotER_input_varargin_2.mat - % a new input variable is entering the pipeline here: 20210112T114310_ft_multiplotER_input_varargin_3.mat - % a new input variable is entering the pipeline here: 20210112T114310_ft_multiplotER_input_varargin_4.mat - - cfg = []; - cfg.showlabels = 'no'; - cfg.fontsize = 6; - cfg.layout = 'CTF151_helmet.mat'; - cfg.baseline = [-0.2 0]; - cfg.xlim = [-0.2 1]; - cfg.ylim = [-3e-13 3e-13]; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { - 'reproduce_Group/20210112T114310_ft_multiplotER_input_varargin_1.mat', 'reproduce_Group/20210112T114310_ft_multiplotER_input_varargin_2.mat', 'reproduce_Group/20210112T114310_ft_multiplotER_input_varargin_3.mat', 'reproduce_Group/20210112T114310_ft_multiplotER_input_varargin_4.mat' - }; - cfg.outputfile = 'reproduce_Group/20210112T114321_ft_multiplotER_output'; - figure; - ft_multiplotER(cfg); - - %% - - cfg = []; - cfg.parameter = 'avg'; - cfg.operation = 'x1-x2'; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { - 'reproduce_Group/20210112T114236_ft_multiplotER_input_varargin_1.mat', 'reproduce_Group/20210112T114253_ft_multiplotER_input_varargin_1.mat' - }; - cfg.outputfile = { 'reproduce_Group/20210112T114330_ft_math_output_data.mat' }; - ft_math(cfg); - - %% - - cfg = []; - cfg.parameter = 'avg'; - cfg.operation = 'x1-x2'; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { - 'reproduce_Group/20210112T114236_ft_multiplotER_input_varargin_2.mat', 'reproduce_Group/20210112T114253_ft_multiplotER_input_varargin_2.mat' - }; - cfg.outputfile = { 'reproduce_Group/20210112T114333_ft_math_output_data.mat' }; - ft_math(cfg); - - %% - - cfg = []; - cfg.parameter = 'avg'; - cfg.operation = 'x1-x2'; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { - 'reproduce_Group/20210112T114236_ft_multiplotER_input_varargin_3.mat', 'reproduce_Group/20210112T114253_ft_multiplotER_input_varargin_3.mat' - }; - cfg.outputfile = { 'reproduce_Group/20210112T114337_ft_math_output_data.mat' }; - ft_math(cfg); - - %% - - cfg = []; - cfg.parameter = 'avg'; - cfg.operation = 'x1-x2'; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { - 'reproduce_Group/20210112T114236_ft_multiplotER_input_varargin_4.mat', 'reproduce_Group/20210112T114253_ft_multiplotER_input_varargin_4.mat' - }; - cfg.outputfile = { 'reproduce_Group/20210112T114340_ft_math_output_data.mat' }; - ft_math(cfg); - - %% - - cfg = []; - cfg.showlabels = 'no'; - cfg.fontsize = 6; - cfg.layout = 'CTF151_helmet.mat'; - cfg.baseline = [-0.2 0]; - cfg.xlim = [-0.2 1]; - cfg.ylim = [-3e-13 3e-13]; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { - 'reproduce_Group/20210112T114330_ft_math_output_data.mat', 'reproduce_Group/20210112T114333_ft_math_output_data.mat', 'reproduce_Group/20210112T114337_ft_math_output_data.mat', 'reproduce_Group/20210112T114340_ft_math_output_data.mat' - }; - cfg.outputfile = 'reproduce_Group/20210112T114351_ft_multiplotER_output'; - figure; - ft_multiplotER(cfg); - - %% - - % a new input variable is entering the pipeline here: 20210112T114358_ft_timelockgrandaverage_input_varargin_1.mat - % a new input variable is entering the pipeline here: 20210112T114358_ft_timelockgrandaverage_input_varargin_2.mat - % a new input variable is entering the pipeline here: 20210112T114358_ft_timelockgrandaverage_input_varargin_3.mat - % a new input variable is entering the pipeline here: 20210112T114358_ft_timelockgrandaverage_input_varargin_4.mat - - cfg = []; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { - 'reproduce_Group/20210112T114358_ft_timelockgrandaverage_input_varargin_1.mat', 'reproduce_Group/20210112T114358_ft_timelockgrandaverage_input_varargin_2.mat', 'reproduce_Group/20210112T114358_ft_timelockgrandaverage_input_varargin_3.mat', 'reproduce_Group/20210112T114358_ft_timelockgrandaverage_input_varargin_4.mat' - }; - cfg.outputfile = { 'reproduce_Group/20210112T114403_ft_timelockgrandaverage_output_grandavg.mat' }; - ft_timelockgrandaverage(cfg); - - %% - - % a new input variable is entering the pipeline here: 20210112T114404_ft_timelockgrandaverage_input_varargin_1.mat - % a new input variable is entering the pipeline here: 20210112T114404_ft_timelockgrandaverage_input_varargin_2.mat - % a new input variable is entering the pipeline here: 20210112T114404_ft_timelockgrandaverage_input_varargin_3.mat - % a new input variable is entering the pipeline here: 20210112T114404_ft_timelockgrandaverage_input_varargin_4.mat - - cfg = []; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { - 'reproduce_Group/20210112T114404_ft_timelockgrandaverage_input_varargin_1.mat', 'reproduce_Group/20210112T114404_ft_timelockgrandaverage_input_varargin_2.mat', 'reproduce_Group/20210112T114404_ft_timelockgrandaverage_input_varargin_3.mat', 'reproduce_Group/20210112T114404_ft_timelockgrandaverage_input_varargin_4.mat' - }; - cfg.outputfile = { 'reproduce_Group/20210112T114409_ft_timelockgrandaverage_output_grandavg.mat' }; - ft_timelockgrandaverage(cfg); - - %% - - % a new input variable is entering the pipeline here: 20210112T114411_ft_timelockgrandaverage_input_varargin_1.mat - % a new input variable is entering the pipeline here: 20210112T114411_ft_timelockgrandaverage_input_varargin_2.mat - % a new input variable is entering the pipeline here: 20210112T114411_ft_timelockgrandaverage_input_varargin_3.mat - % a new input variable is entering the pipeline here: 20210112T114411_ft_timelockgrandaverage_input_varargin_4.mat - - cfg = []; - cfg.tracktimeinfo = 'yes'; - cfg.trackmeminfo = 'yes'; - cfg.inputfile = { - 'reproduce_Group/20210112T114411_ft_timelockgrandaverage_input_varargin_1.mat', 'reproduce_Group/20210112T114411_ft_timelockgrandaverage_input_varargin_2.mat', 'reproduce_Group/20210112T114411_ft_timelockgrandaverage_input_varargin_3.mat', 'reproduce_Group/20210112T114411_ft_timelockgrandaverage_input_varargin_4.mat' - }; - cfg.outputfile = { 'reproduce_Group/20210112T114415_ft_timelockgrandaverage_output_grandavg.mat' }; - ft_timelockgrandaverage(cfg); - -## Conclusion - -This example demonstrated how _reproducescript_ can be applied to a group analysis by enabling it separately for each subject, and then once for the group analysis. It would be possible as well to enable reproducescript only once at the top of the master script, but we found the current solution to have a more clear organisation. There is one more example in which we apply reproducescript to full study published by Andersen et al. This shows how _reproducescript_ would be applied in a more complex scenario. - -Note that there are other strategies for improving shareability and reproducibility, and we don't claim that _reproducescript_ is the best way in every scenario. Rather, it is one of many tools that can aid the researcher to improve the community's standard in methodological transparency and robustness of results. For other strategies, we refer the reader to the pre-print in which we first described _reproducescript_. - -## Suggested further reading - -- [Reducing the efforts to create reproducible analysis code with FieldTrip](http://dx.doi.org/10.21105/joss.05566) -- [Using reproducescript on a full study](/example/reproducescript_andersen) +--- +title: Using reproducescript for a group analysis +category: example +tags: [reproducescript] +--- + +# Using reproducescript for a group analysis + +This example script will introduce you to functionality in the FieldTrip toolbox designed to aid in making your analysis pipeline - including code, data and results - more easily reproducible and shareable. It is based on the manuscript [Reducing the efforts to create reproducible analysis code with FieldTrip](http://dx.doi.org/10.21105/joss.05566). We assume that you have already had a look at the example on [Making your analysis pipeline reproducible using reproducescript](/example/reproducescript). + +## Example 2 + +The first example contained only a few analysis steps in a single subject. More realistic data analysis pipelines consist of many more steps in which often the same (or similar) pipelines are used for multiple subjects. In this section, we will show how the reproducescript functionality applies in such a case. + +### Original analysis + +The analysis example follows the strategy outlined in the [Literate Programming](https://www-cs-faculty.stanford.edu/~knuth/lp.html) book by Donald Knuth and starts with a single subject analysis pipeline that is repeated for four subjects. The directory is structured as depicted below. + +{% include image src="/assets/img/example/reproducescript/filedir_example2_analysis.jpg" width="237" %} + +After the single-subject analysis, all single-subject results are used in a group analysis. The single-subject and group analyses are executed from the master script `analyze.m`: + +#### analyze.m + + clear + close all + + subjlist = { + 'Subject01' + 'Subject02' + 'Subject03' + 'Subject04' + }; + + %% Loop single-subject analyses over subjects + for i=1:numel(subjlist) + subj = subjlist{i}; + doSingleSubjectAnalysis(subj); + end + + %% Group analysis + doGroupAnalysis(subjlist); + +This is the control script from which the relevant analysis scripts and functions are called. The master script relies on two functions: `doSingleSubjectAnalysis` and `doGroupAnalysis`, which are each stored in separate m-files. The original source code for these scripts can be found below. + +#### doSingleSubjectAnalysis.m + + function doSingleSubjectAnalysis(subj) + + % the details of each subject are in separate files + % details_Subject01.m + % details_Subject02.m + % details_Subject03.m + % details_Subject04.m + + fprintf('evaluating single subject analysis for %s\n', subj); + eval(['details_' subj]); + + % this is for artifact detection + interactive = false; + + %% + + cfg = []; + cfg.dataset = fullfile(datadir, filename); + cfg.trialfun = 'ft_trialfun_general'; + cfg.trialdef.eventtype = 'backpanel trigger'; + cfg.trialdef.eventvalue = [triggerFIC triggerIC triggerFC]; + cfg.trialdef.prestim = 1; + cfg.trialdef.poststim = 2; + cfg = ft_definetrial(cfg); + + % the EOG channel has a different name in the different datasets + cfg.channel = {'MEG' eogchannel}; + cfg.continuous = 'yes'; % see https://www.fieldtriptoolbox.org/faq/continuous/ + data = ft_preprocessing(cfg); + + %% + + if interactive + % visually identify the artifacts + cfg = []; + cfg.channel = eogchannel; + cfg.method = 'channel'; + dummy1 = ft_rejectvisual(cfg, data); + + cfg = []; + cfg.channel = 'MEG'; + cfg.method = 'summary'; + dummy2 = ft_rejectvisual(cfg, data); + + % combine the artifacts that have been detected + artifact = [ + dummy1.cfg.artfctdef.channel.artifact + dummy2.cfg.artfctdef.summary.artifact + ]; + + % print them and copy them to the subject details file + disp(artifact); + + % use the MATLAB debugger to wait on this line + disp('please copy these artifacts to the subject details file'); + keyboard + end + + % remove the artifacts that were previously detected + cfg = []; + cfg.artfctdef.visual.artifact = artifact; + data_clean = ft_rejectartifact(cfg, data); + + %% + + cfg = []; + cfg.trials = data_clean.trialinfo==triggerFIC; + avgFIC = ft_timelockanalysis(cfg, data_clean); + + cfg.trials = data_clean.trialinfo==triggerFC; + avgFC = ft_timelockanalysis(cfg, data_clean); + + cfg.trials = data_clean.trialinfo==triggerIC; + avgIC = ft_timelockanalysis(cfg, data_clean); + + %% + + cfg = []; + cfg.showlabels = 'no'; + cfg.fontsize = 6; + cfg.layout = 'CTF151_helmet.mat'; + cfg.baseline = [-0.2 0]; + cfg.xlim = [-0.2 1.0]; + cfg.ylim = [-3e-13 3e-13]; + ft_multiplotER(cfg, avgFC, avgIC, avgFIC); + + %% + + cfg = []; + cfg.feedback = 'yes'; + cfg.method = 'template'; + cfg.neighbours = ft_prepare_neighbours(cfg, avgFIC); + cfg.planarmethod = 'sincos'; + avgFICplanar = ft_megplanar(cfg, avgFIC); + avgFCplanar = ft_megplanar(cfg, avgFC); + avgICplanar = ft_megplanar(cfg, avgIC); + + %% + + cfg = []; + avgFICplanarComb = ft_combineplanar(cfg, avgFICplanar); + avgFCplanarComb = ft_combineplanar(cfg, avgFCplanar); + avgICplanarComb = ft_combineplanar(cfg, avgICplanar); + + %% + + cfg = []; + cfg.xlim = [0.3 0.5]; + cfg.zlim = 'maxmin'; + cfg.colorbar = 'yes'; + cfg.layout = 'CTF151_helmet.mat'; + subplot(2,3,1); ft_topoplotER(cfg, avgFIC) + subplot(2,3,2); ft_topoplotER(cfg, avgFC) + subplot(2,3,3); ft_topoplotER(cfg, avgIC) + + cfg.zlim = 'maxabs'; + cfg.layout = 'CTF151_helmet.mat'; + subplot(2,3,4); ft_topoplotER(cfg, avgFICplanarComb) + subplot(2,3,5); ft_topoplotER(cfg, avgFCplanarComb) + subplot(2,3,6); ft_topoplotER(cfg, avgICplanarComb) + + %% + + % save the results to disk + outputdir = ['result_' subj]; + mkdir(outputdir) + save(fullfile(outputdir, 'avgFIC'), 'avgFIC'); + save(fullfile(outputdir, 'avgFC'), 'avgFC'); + save(fullfile(outputdir, 'avgIC'), 'avgIC'); + save(fullfile(outputdir, 'avgFICplanarComb'), 'avgFICplanarComb'); + save(fullfile(outputdir, 'avgFCplanarComb'), 'avgFCplanarComb'); + save(fullfile(outputdir, 'avgICplanarComb'), 'avgICplanarComb'); + +#### doGroupAnalysis.m + + function doGroupAnalysis(allsubj) + + avgFIC = cell(size(allsubj)); + avgFC = cell(size(allsubj)); + avgIC = cell(size(allsubj)); + + % load the results from disk + for i=1:numel(allsubj) + subj = allsubj{i}; + fprintf('loading data for subject %s\n', subj); + + inputdir = ['result_' subj]; + tmp = load(fullfile(inputdir, 'avgFIC')); avgFIC{i} = tmp.avgFIC; + tmp = load(fullfile(inputdir, 'avgFC')); avgFC{i} = tmp.avgFC; + tmp = load(fullfile(inputdir, 'avgIC')); avgIC{i} = tmp.avgIC; + clear tmp + end + + %% + + cfg = []; + cfg.showlabels = 'no'; + cfg.fontsize = 6; + cfg.layout = 'CTF151_helmet.mat'; + cfg.baseline = [-0.2 0]; + cfg.xlim = [-0.2 1.0]; + cfg.ylim = [-3e-13 3e-13]; + figure + ft_multiplotER(cfg, avgFIC{:}); + title('Fully incongruent condition'); + + figure + ft_multiplotER(cfg, avgFC{:}); + title('Fully congruent condition'); + + figure + ft_multiplotER(cfg, avgIC{:}); + title('Initially congruent condition'); + + %% + + avgFICvsFC = cell(size(allsubj)); + for i=1:numel(allsubj) + cfg = []; + cfg.parameter = 'avg'; + cfg.operation = 'x1-x2'; + avgFICvsFC{i} = ft_math(cfg, avgFIC{i}, avgFC{i}); + end + + cfg = []; + cfg.showlabels = 'no'; + cfg.fontsize = 6; + cfg.layout = 'CTF151_helmet.mat'; + cfg.baseline = [-0.2 0]; + cfg.xlim = [-0.2 1.0]; + cfg.ylim = [-3e-13 3e-13]; + ft_multiplotER(cfg, avgFICvsFC{:}); + title('FIC minus FC'); + + %% + + % let's make a manual change to the data that is not caputured in the provenance + for i=1:numel(allsubj) + avgFIC{i}.avg = avgFIC{i}.avg * 1e15; % convert from T to fT + avgFC{i}.avg = avgFC{i}.avg * 1e15; % convert from T to fT + avgIC{i}.avg = avgIC{i}.avg * 1e15; % convert from T to fT + end + + %% + + cfg = []; + grandavgFIC = ft_timelockgrandaverage(cfg, avgFIC{:}); + grandavgFC = ft_timelockgrandaverage(cfg, avgFC{:}); + grandavgIC = ft_timelockgrandaverage(cfg, avgIC{:}); + + %% + + % save the results to disk + outputdir = 'result_Group'; + mkdir(outputdir) + save(fullfile(outputdir, 'grandavgFIC'), 'grandavgFIC'); + save(fullfile(outputdir, 'grandavgFC'), 'grandavgFC'); + save(fullfile(outputdir, 'grandavgIC'), 'grandavgIC'); + +### Using reproducescript + +To create a standard script from the analysis pipeline, the `ft_default` variable is initialized at the top of the `analyze.m` script. Note that we do not immediately initiate reproducescript, this is done in the loop just prior to `doSingleSubjectAnalysis` and just prior to `doGroupAnalysis`; this allows specifying unique directories for each subject and for the group. In fact, reproducescript can be stopped and restarted between different subjects, or even in between analysis steps, which is especially convenient in pipelines that require a lot of compute resources which you would rather split up to allow for parallel execution on a compute cluster. + +#### analyze.m with reproducescript enabled + + clear + close all + + % initialize the global variable + global ft_default + ft_default = []; + ft_default.checksize = inf; + + subjlist = { + 'Subject01' + 'Subject02' + 'Subject03' + 'Subject04' + }; + + %% Loop single-subject analysis over subjects + for i=1:numel(subjlist) + subj = subjlist{i}; + % initiate reproducescript + ft_default.reproducescript = ['reproduce_' subj]; + doSingleSubjectAnalysis(subj); + ft_default.reproducescript = []; % disable + end + + %% Group analysis + % initiate reproducescript + ft_default.reproducescript = 'reproduce_Group'; + doGroupAnalysis(subjlist); + ft_default.reproducescript = []; % disable + +### Reproduced analysis + +{% include image src="/assets/img/example/reproducescript/filedir_example2_reproducescript.jpg" width="435" %} + +We devoted a specific folder to the reproducescript content of each subject, and one for the group analysis. Thus, upon execution of the master script in `analyze.m`, folders are created for each of the subjects, and for the group analysis. These all contain the intermediate data, a standardized script, and a `hashes.mat` file for the bookkeeping. The reproducescript standardized scripts for the single-subject analysis and group analysis can be found below. Note that these are quite lengthy, but they unambiguously list all analysis steps. + +#### Single subject reproduced analysis + +This is the resulting code in the file `project/example2/reproduce_Subject01/script.m` + + %% + + cfg = []; + cfg.dataset = '../rawdata/Subject01.ds'; + cfg.trialfun = 'ft_trialfun_general'; + cfg.trialdef.eventtype = 'backpanel trigger'; + cfg.trialdef.eventvalue = [3 5 9]; + cfg.trialdef.prestim = 1; + cfg.trialdef.poststim = 2; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg = ft_definetrial(cfg); + + %% + + cfg = []; + cfg.dataset = '../rawdata/Subject01.ds'; + cfg.trialfun = 'ft_trialfun_general'; + cfg.trialdef.eventtype = 'backpanel trigger'; + cfg.trialdef.eventvalue = [3 5 9]; + cfg.trialdef.prestim = 1; + cfg.trialdef.poststim = 2; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.datafile = '../rawdata/Subject01.ds/Subject01.meg4'; + cfg.headerfile = '../rawdata/Subject01.ds/Subject01.res4'; + cfg.dataformat = 'ctf_meg4'; + cfg.headerformat = 'ctf_res4'; + cfg.representation = 'numeric'; + cfg.trl = 'reproduce_Subject01/20210112T113604_ft_preprocessing_largecfginput_trl.mat'; + cfg.outputfile = { 'reproduce_Subject01/20210112T113604_ft_preprocessing_output_data.mat' }; + cfg.channel = {'MEG', 'EOG'}; + cfg.continuous = 'yes'; + ft_preprocessing(cfg); + + %% + + cfg = []; + cfg.artfctdef.visual.artifact = [8101 9000; + 68401 69300; + 99001 99900; + ... + 228601 229500]; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113604_ft_preprocessing_output_data.mat' }; + cfg.outputfile = { 'reproduce_Subject01/20210112T113611_ft_rejectartifact_output_data.mat' }; + ft_rejectartifact(cfg); + + %% + + cfg = []; + cfg.trials = logical([true false ... false]); + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113611_ft_rejectartifact_output_data.mat' }; + cfg.outputfile = { 'reproduce_Subject01/20210112T113616_ft_timelockanalysis_output_timelock.mat' }; + ft_timelockanalysis(cfg); + + %% + + cfg = []; + cfg.trials = logical([false false ... false]); + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113611_ft_rejectartifact_output_data.mat' }; + cfg.outputfile = { 'reproduce_Subject01/20210112T113621_ft_timelockanalysis_output_timelock.mat' }; + ft_timelockanalysis(cfg); + + %% + + cfg = []; + cfg.trials = logical([false true ... true]); + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113611_ft_rejectartifact_output_data.mat' }; + cfg.outputfile = { 'reproduce_Subject01/20210112T113625_ft_timelockanalysis_output_timelock.mat' }; + ft_timelockanalysis(cfg); + + %% + + cfg = []; + cfg.showlabels = 'no'; + cfg.fontsize = 6; + cfg.layout = 'CTF151_helmet.mat'; + cfg.baseline = [-0.2 0]; + cfg.xlim = [-0.2 1]; + cfg.ylim = [-3e-13 3e-13]; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { + 'reproduce_Subject01/20210112T113621_ft_timelockanalysis_output_timelock.mat', 'reproduce_Subject01/20210112T113625_ft_timelockanalysis_output_timelock.mat', 'reproduce_Subject01/20210112T113616_ft_timelockanalysis_output_timelock.mat' + }; + cfg.outputfile = 'reproduce_Subject01/20210112T113634_ft_multiplotER_output'; + figure; + ft_multiplotER(cfg); + + %% + + cfg = []; + cfg.feedback = 'yes'; + cfg.method = 'template'; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113616_ft_timelockanalysis_output_timelock.mat' }; + cfg.outputfile = { 'reproduce_Subject01/20210112T113643_ft_prepare_neighbours_output_neighbours.mat' }; + ft_prepare_neighbours(cfg); + + %% + + cfg = []; + cfg.feedback = 'yes'; + cfg.method = 'template'; + cfg.neighbours = 'reproduce_Subject01/20210112T113643_ft_megplanar_largecfginput_neighbours.mat'; + cfg.planarmethod = 'sincos'; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113616_ft_timelockanalysis_output_timelock.mat' }; + cfg.outputfile = { 'reproduce_Subject01/20210112T113647_ft_megplanar_output_data.mat' }; + ft_megplanar(cfg); + + %% + + cfg = []; + cfg.feedback = 'yes'; + cfg.method = 'template'; + cfg.neighbours = 'reproduce_Subject01/20210112T113643_ft_megplanar_largecfginput_neighbours.mat'; + cfg.planarmethod = 'sincos'; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113621_ft_timelockanalysis_output_timelock.mat' }; + cfg.outputfile = { 'reproduce_Subject01/20210112T113650_ft_megplanar_output_data.mat' }; + ft_megplanar(cfg); + + %% + + cfg = []; + cfg.feedback = 'yes'; + cfg.method = 'template'; + cfg.neighbours = 'reproduce_Subject01/20210112T113643_ft_megplanar_largecfginput_neighbours.mat'; + cfg.planarmethod = 'sincos'; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113625_ft_timelockanalysis_output_timelock.mat' }; + cfg.outputfile = { 'reproduce_Subject01/20210112T113653_ft_megplanar_output_data.mat' }; + ft_megplanar(cfg); + + %% + + cfg = []; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113647_ft_megplanar_output_data.mat' }; + cfg.outputfile = { 'reproduce_Subject01/20210112T113657_ft_combineplanar_output_data.mat' }; + ft_combineplanar(cfg); + + %% + + cfg = []; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113650_ft_megplanar_output_data.mat' }; + cfg.outputfile = { 'reproduce_Subject01/20210112T113701_ft_combineplanar_output_data.mat' }; + ft_combineplanar(cfg); + + %% + + cfg = []; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113653_ft_megplanar_output_data.mat' }; + cfg.outputfile = { 'reproduce_Subject01/20210112T113704_ft_combineplanar_output_data.mat' }; + ft_combineplanar(cfg); + + %% + + cfg = []; + cfg.xlim = [0.3 0.5]; + cfg.zlim = 'maxmin'; + cfg.colorbar = 'yes'; + cfg.layout = 'CTF151_helmet.mat'; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113616_ft_timelockanalysis_output_timelock.mat' }; + cfg.outputfile = 'reproduce_Subject01/20210112T113708_ft_topoplotER_output'; + figure; + ft_topoplotER(cfg); + + %% + + cfg = []; + cfg.xlim = [0.3 0.5]; + cfg.zlim = 'maxmin'; + cfg.colorbar = 'yes'; + cfg.layout = 'CTF151_helmet.mat'; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113621_ft_timelockanalysis_output_timelock.mat' }; + cfg.outputfile = 'reproduce_Subject01/20210112T113712_ft_topoplotER_output'; + figure; + ft_topoplotER(cfg); + + %% + + cfg = []; + cfg.xlim = [0.3 0.5]; + cfg.zlim = 'maxmin'; + cfg.colorbar = 'yes'; + cfg.layout = 'CTF151_helmet.mat'; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113625_ft_timelockanalysis_output_timelock.mat' }; + cfg.outputfile = 'reproduce_Subject01/20210112T113716_ft_topoplotER_output'; + figure; + ft_topoplotER(cfg); + + %% + + cfg = []; + cfg.xlim = [0.3 0.5]; + cfg.zlim = 'maxabs'; + cfg.colorbar = 'yes'; + cfg.layout = 'CTF151_helmet.mat'; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113657_ft_combineplanar_output_data.mat' }; + cfg.outputfile = 'reproduce_Subject01/20210112T113721_ft_topoplotER_output'; + figure; + ft_topoplotER(cfg); + + %% + + cfg = []; + cfg.xlim = [0.3 0.5]; + cfg.zlim = 'maxabs'; + cfg.colorbar = 'yes'; + cfg.layout = 'CTF151_helmet.mat'; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113701_ft_combineplanar_output_data.mat' }; + cfg.outputfile = 'reproduce_Subject01/20210112T113726_ft_topoplotER_output'; + figure; + ft_topoplotER(cfg); + + %% + + cfg = []; + cfg.xlim = [0.3 0.5]; + cfg.zlim = 'maxabs'; + cfg.colorbar = 'yes'; + cfg.layout = 'CTF151_helmet.mat'; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { 'reproduce_Subject01/20210112T113704_ft_combineplanar_output_data.mat' }; + cfg.outputfile = 'reproduce_Subject01/20210112T113731_ft_topoplotER_output'; + figure; + ft_topoplotER(cfg); + +#### Group reproduced analysis + +This is the resulting code in the file `project/example2/reproduce_Group/script.m` + + %% + + % a new input variable is entering the pipeline here: 20210112T114236_ft_multiplotER_input_varargin_1.mat + % a new input variable is entering the pipeline here: 20210112T114236_ft_multiplotER_input_varargin_2.mat + % a new input variable is entering the pipeline here: 20210112T114236_ft_multiplotER_input_varargin_3.mat + % a new input variable is entering the pipeline here: 20210112T114236_ft_multiplotER_input_varargin_4.mat + + cfg = []; + cfg.showlabels = 'no'; + cfg.fontsize = 6; + cfg.layout = 'CTF151_helmet.mat'; + cfg.baseline = [-0.2 0]; + cfg.xlim = [-0.2 1]; + cfg.ylim = [-3e-13 3e-13]; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { + 'reproduce_Group/20210112T114236_ft_multiplotER_input_varargin_1.mat', 'reproduce_Group/20210112T114236_ft_multiplotER_input_varargin_2.mat', 'reproduce_Group/20210112T114236_ft_multiplotER_input_varargin_3.mat', 'reproduce_Group/20210112T114236_ft_multiplotER_input_varargin_4.mat' + }; + cfg.outputfile = 'reproduce_Group/20210112T114246_ft_multiplotER_output'; + figure; + ft_multiplotER(cfg); + + %% + + % a new input variable is entering the pipeline here: 20210112T114253_ft_multiplotER_input_varargin_1.mat + % a new input variable is entering the pipeline here: 20210112T114253_ft_multiplotER_input_varargin_2.mat + % a new input variable is entering the pipeline here: 20210112T114253_ft_multiplotER_input_varargin_3.mat + % a new input variable is entering the pipeline here: 20210112T114253_ft_multiplotER_input_varargin_4.mat + + cfg = []; + cfg.showlabels = 'no'; + cfg.fontsize = 6; + cfg.layout = 'CTF151_helmet.mat'; + cfg.baseline = [-0.2 0]; + cfg.xlim = [-0.2 1]; + cfg.ylim = [-3e-13 3e-13]; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { + 'reproduce_Group/20210112T114253_ft_multiplotER_input_varargin_1.mat', 'reproduce_Group/20210112T114253_ft_multiplotER_input_varargin_2.mat', 'reproduce_Group/20210112T114253_ft_multiplotER_input_varargin_3.mat', 'reproduce_Group/20210112T114253_ft_multiplotER_input_varargin_4.mat' + }; + cfg.outputfile = 'reproduce_Group/20210112T114303_ft_multiplotER_output'; + figure; + ft_multiplotER(cfg); + + %% + + % a new input variable is entering the pipeline here: 20210112T114310_ft_multiplotER_input_varargin_1.mat + % a new input variable is entering the pipeline here: 20210112T114310_ft_multiplotER_input_varargin_2.mat + % a new input variable is entering the pipeline here: 20210112T114310_ft_multiplotER_input_varargin_3.mat + % a new input variable is entering the pipeline here: 20210112T114310_ft_multiplotER_input_varargin_4.mat + + cfg = []; + cfg.showlabels = 'no'; + cfg.fontsize = 6; + cfg.layout = 'CTF151_helmet.mat'; + cfg.baseline = [-0.2 0]; + cfg.xlim = [-0.2 1]; + cfg.ylim = [-3e-13 3e-13]; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { + 'reproduce_Group/20210112T114310_ft_multiplotER_input_varargin_1.mat', 'reproduce_Group/20210112T114310_ft_multiplotER_input_varargin_2.mat', 'reproduce_Group/20210112T114310_ft_multiplotER_input_varargin_3.mat', 'reproduce_Group/20210112T114310_ft_multiplotER_input_varargin_4.mat' + }; + cfg.outputfile = 'reproduce_Group/20210112T114321_ft_multiplotER_output'; + figure; + ft_multiplotER(cfg); + + %% + + cfg = []; + cfg.parameter = 'avg'; + cfg.operation = 'x1-x2'; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { + 'reproduce_Group/20210112T114236_ft_multiplotER_input_varargin_1.mat', 'reproduce_Group/20210112T114253_ft_multiplotER_input_varargin_1.mat' + }; + cfg.outputfile = { 'reproduce_Group/20210112T114330_ft_math_output_data.mat' }; + ft_math(cfg); + + %% + + cfg = []; + cfg.parameter = 'avg'; + cfg.operation = 'x1-x2'; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { + 'reproduce_Group/20210112T114236_ft_multiplotER_input_varargin_2.mat', 'reproduce_Group/20210112T114253_ft_multiplotER_input_varargin_2.mat' + }; + cfg.outputfile = { 'reproduce_Group/20210112T114333_ft_math_output_data.mat' }; + ft_math(cfg); + + %% + + cfg = []; + cfg.parameter = 'avg'; + cfg.operation = 'x1-x2'; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { + 'reproduce_Group/20210112T114236_ft_multiplotER_input_varargin_3.mat', 'reproduce_Group/20210112T114253_ft_multiplotER_input_varargin_3.mat' + }; + cfg.outputfile = { 'reproduce_Group/20210112T114337_ft_math_output_data.mat' }; + ft_math(cfg); + + %% + + cfg = []; + cfg.parameter = 'avg'; + cfg.operation = 'x1-x2'; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { + 'reproduce_Group/20210112T114236_ft_multiplotER_input_varargin_4.mat', 'reproduce_Group/20210112T114253_ft_multiplotER_input_varargin_4.mat' + }; + cfg.outputfile = { 'reproduce_Group/20210112T114340_ft_math_output_data.mat' }; + ft_math(cfg); + + %% + + cfg = []; + cfg.showlabels = 'no'; + cfg.fontsize = 6; + cfg.layout = 'CTF151_helmet.mat'; + cfg.baseline = [-0.2 0]; + cfg.xlim = [-0.2 1]; + cfg.ylim = [-3e-13 3e-13]; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { + 'reproduce_Group/20210112T114330_ft_math_output_data.mat', 'reproduce_Group/20210112T114333_ft_math_output_data.mat', 'reproduce_Group/20210112T114337_ft_math_output_data.mat', 'reproduce_Group/20210112T114340_ft_math_output_data.mat' + }; + cfg.outputfile = 'reproduce_Group/20210112T114351_ft_multiplotER_output'; + figure; + ft_multiplotER(cfg); + + %% + + % a new input variable is entering the pipeline here: 20210112T114358_ft_timelockgrandaverage_input_varargin_1.mat + % a new input variable is entering the pipeline here: 20210112T114358_ft_timelockgrandaverage_input_varargin_2.mat + % a new input variable is entering the pipeline here: 20210112T114358_ft_timelockgrandaverage_input_varargin_3.mat + % a new input variable is entering the pipeline here: 20210112T114358_ft_timelockgrandaverage_input_varargin_4.mat + + cfg = []; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { + 'reproduce_Group/20210112T114358_ft_timelockgrandaverage_input_varargin_1.mat', 'reproduce_Group/20210112T114358_ft_timelockgrandaverage_input_varargin_2.mat', 'reproduce_Group/20210112T114358_ft_timelockgrandaverage_input_varargin_3.mat', 'reproduce_Group/20210112T114358_ft_timelockgrandaverage_input_varargin_4.mat' + }; + cfg.outputfile = { 'reproduce_Group/20210112T114403_ft_timelockgrandaverage_output_grandavg.mat' }; + ft_timelockgrandaverage(cfg); + + %% + + % a new input variable is entering the pipeline here: 20210112T114404_ft_timelockgrandaverage_input_varargin_1.mat + % a new input variable is entering the pipeline here: 20210112T114404_ft_timelockgrandaverage_input_varargin_2.mat + % a new input variable is entering the pipeline here: 20210112T114404_ft_timelockgrandaverage_input_varargin_3.mat + % a new input variable is entering the pipeline here: 20210112T114404_ft_timelockgrandaverage_input_varargin_4.mat + + cfg = []; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { + 'reproduce_Group/20210112T114404_ft_timelockgrandaverage_input_varargin_1.mat', 'reproduce_Group/20210112T114404_ft_timelockgrandaverage_input_varargin_2.mat', 'reproduce_Group/20210112T114404_ft_timelockgrandaverage_input_varargin_3.mat', 'reproduce_Group/20210112T114404_ft_timelockgrandaverage_input_varargin_4.mat' + }; + cfg.outputfile = { 'reproduce_Group/20210112T114409_ft_timelockgrandaverage_output_grandavg.mat' }; + ft_timelockgrandaverage(cfg); + + %% + + % a new input variable is entering the pipeline here: 20210112T114411_ft_timelockgrandaverage_input_varargin_1.mat + % a new input variable is entering the pipeline here: 20210112T114411_ft_timelockgrandaverage_input_varargin_2.mat + % a new input variable is entering the pipeline here: 20210112T114411_ft_timelockgrandaverage_input_varargin_3.mat + % a new input variable is entering the pipeline here: 20210112T114411_ft_timelockgrandaverage_input_varargin_4.mat + + cfg = []; + cfg.tracktimeinfo = 'yes'; + cfg.trackmeminfo = 'yes'; + cfg.inputfile = { + 'reproduce_Group/20210112T114411_ft_timelockgrandaverage_input_varargin_1.mat', 'reproduce_Group/20210112T114411_ft_timelockgrandaverage_input_varargin_2.mat', 'reproduce_Group/20210112T114411_ft_timelockgrandaverage_input_varargin_3.mat', 'reproduce_Group/20210112T114411_ft_timelockgrandaverage_input_varargin_4.mat' + }; + cfg.outputfile = { 'reproduce_Group/20210112T114415_ft_timelockgrandaverage_output_grandavg.mat' }; + ft_timelockgrandaverage(cfg); + +## Conclusion + +This example demonstrated how _reproducescript_ can be applied to a group analysis by enabling it separately for each subject, and then once for the group analysis. It would be possible as well to enable reproducescript only once at the top of the master script, but we found the current solution to have a more clear organisation. There is one more example in which we apply reproducescript to full study published by Andersen et al. This shows how _reproducescript_ would be applied in a more complex scenario. + +Note that there are other strategies for improving shareability and reproducibility, and we don't claim that _reproducescript_ is the best way in every scenario. Rather, it is one of many tools that can aid the researcher to improve the community's standard in methodological transparency and robustness of results. For other strategies, we refer the reader to the pre-print in which we first described _reproducescript_. + +## Suggested further reading + +- [Reducing the efforts to create reproducible analysis code with FieldTrip](http://dx.doi.org/10.21105/joss.05566) +- [Using reproducescript on a full study](/example/reproducescript_andersen)