Skip to content

Manual

Jump to bottom
Garikoitz Lerma-Usabiaga edited this page Apr 24, 2023 · 15 revisions

General workflow of launchcontainers

By now, we assume that you have already git cloned this repository at github/launchcontainers and installed all the required software.

There are two types of config files.

  • The .yaml file that is is used to control launchcontainers. This file will provide path information, which container to launch, in which HPC cluster and with what parameters.
  • The config files specific to each container. They are required to control the analyses within the container. They are usually json files, but they can be yaml or other formats as well (we plan to migrate some of the json files to yaml in the future. IMPORTANT: some containers might have MORE THAN ONE CONFIG FILES. For example, rtp-pipeline, which has config.json and tractparams.csv.

There are

These are the main steps to configure launchcontainers and run analyses within containers:

  1. Create a project folder      At your base directory:

$ mkdir /TESTDATA/PROJECT_NAME and put your dicoms in the dicom/ directory

  1. Convert your dicoms with heudiconv (heudiconv right now is not part of launchcontainers, remove this when added), and use NORDIC or presurfer or whatever you need to use to have the files in nifti format. RESULT: you should have a PROJECT_NAME/nifti/ folder, with all subjects and sessions there with all the nifti files organized in BIDS format

  2. Copy an launchcontainers/example_config.yaml to /nifti folder and edit it with your paths and containers versions and HPC requirements. Follow the instructions on the yaml file to know what every field means. How to edit the yaml file should be auto-explained in the file itself.

Create the symlinks by running 'python launchcontainers.py -lcc .... -cc .... -ssl ....' (always without --run_lc) Check that all the symlinks are successfully created and that the configs are copied (use 'tree' command with color coding to check that there are not broken links) 'cd' to nifti/derivatives/container_version/analysis-x/ Edit config.json (tractparams.csv for the case of rtp-pipeline, etc..) Run the container by running 'python launchcontainers.py -lcc ./config_lc.yaml --run_lc'

2. Prepare required config files for launchcontainer. We will take the example of the anatROIS container in this practical tutorial.

     Copy generic configs from github/laucncontainer/example_configs

$ scp ~/github/launchcontainers/example_configs/example_lc_config.yaml lc_config.yaml

$ scp ~/github/launchcontainers/example_configs/example_subSesList.txt subSesList.txt

     Copy container specific configs from github/laucncontainer/example_configs/container_especific_configs

$ scp ~/github/launchcontainers/example_configs/container_especific_configs/anatrois/4.2.7_7.1.1/example_config.json config.json

  1. Edit the three config files based on your analysis demands. More documentation on how to edit configs are in progress.

  2. (optional) Check the documentation and help for launchcontainer, it will give you some explanation on the flags:

$ python ~/github/launchcontainers/launchcontainers/launchcontainers.py -h

5. Use the prepare mode of launchcontainer to create symlinks and auto-check if all the required files are in place.

$ python ~/github/launchcontainers/launchcontainers/launchcontainers.py -lcc lc_config.yaml -ssl subSesList.txt -cc config.json

     (Hint: use tab to autofill the name of file and directory to reduce human error)

6. Navigate to analysis folder~/TESTDATA/PROJ01/nifti/derivatives/anatrois_4.2.7_7.1.1/analysis-#### to check if all three config files are copied and are the same as you wish.

7. After checking the configs and preparing the input file, you can use the launch mode to finally launch your jobs to cluster:

$ python ~/github/launchcontainers/launchcontainers/launchcontainers.py -lcc lc_config.yaml -ssl subSesList.txt -cc config.json --run_lc

Clone this wiki locally