Expand documentation

docs/crop/crop.md

@@ -6,4 +6,49 @@ nav_order: 3
 
 # Cropping
 
-Documentation on the cropping module
+The `crop` module removes pixels from the sides of an image. In most cases, this module is not necessary. However, in some experimental set ups, it maybe useful to remove some pixels before processing, so save on both processing time and final image storage. As an example, there may be a need to leave a large margin around the sample during imaging if sample migration is expected. After the time lapse is completed, if the sample only moved to the left, the empty margins on the right could be removed.
+
+When used though the pipeline command, the only parameters to be specified are the amount of pixels to crop from each side of the image. These should be the number of pixels to be removed, not the size of either the initial or final image. Thus, to remove the right 10 pixels from a 512 x 1500 x 201 pixel image stack, set `"cropLeft": 10`, which will lead to the output of a 502 x 1500 x 201 pixel image stack. `cropLeft` and `cropRight` remove pixels in the x-direction, `cropTop` and `cropBottom` remove pixels in the y-direction, and `cropFront` and `cropBack` remove pixels in z (i.e., remove z-slices). If an input stack is opened in ImageJ or FIJI, there will be a z-slider at the bottom of the image; `cropFront` removes slices starting from ImageJ's z = 1, while `cropBack` removes slices starting from the final z-slice. If any values are excluded, they are set to zero.
+
+If directly using the crop module on the command line, these cropping values are provided as one list in the input variables. You must manually specify the paths to the input and output files, as well as metadata such as the z-step size.
+
+
+# Usage
+
+### Pipeline: Configuration File
+The example below removes top 10 pixels in y, no and no pixels from the bottom in y. It removes 15 pixels from the left of the image in x. Because `cropRight` is excluded, no pixels are removed from the right. It also removes the first 100 z-slices and the last 50 z-slices.
+
+```json
+"crop": {
+    "cropTop": 10,
+    "cropBottom": 0,
+    "cropLeft": 15,
+    "cropFront": 100,
+    "cropBack": 50
+}
+```
+
+### Command Line Example
+The following command removes 40 pixels from the top and no other cropping is performed. The input file is `/path/to/experiment/scan_CamA_ch0_CAM1_stack0000_488nm_0000000msec_0004732481msecAbs_000x_000y_000z_0000t.tif` and the output (`-o`) will be saved to `/path/to/experiment/crop/scan_Cam1_ch0_tile0_t0000_crop.tif`. The input image has a z-step size of 0.25 (`-s 0.25`) and any existing output will be overwritten (`-w`)
+```c
+crop -c 40,0,0,0,0,0 -w -s 0.25 -o /path/to/experiment/crop/scan_Cam1_ch0_tile0_t0000_crop.tif  /path/to/experiment/scan_CamA_ch0_CAM1_stack0000_488nm_0000000msec_0004732481msecAbs_000x_000y_000z_0000t.tif
+```
+
+### Crop Options
+
+```
+crop: cropping boundary pixels
+usage: crop [options] path
+
+Allowed options:
+  -h [ --help ]                    display this help message
+  -c [ --crop ] arg (=0,0,0,0,0,0) boundary pixel numbers 
+                                   (top,bottom,left,right,front,back)
+  -o [ --output ] arg              output file path
+  -x [ --xy-rez ] arg (=-1)        x/y resolution (um/px)
+  -s [ --step ] arg (=-1)          step/interval (um)
+  -b [ --bit-depth ] arg (=16)     bit depth (8, 16, or 32) of output image
+  -w [ --overwrite ]               overwrite output if it exists
+  -v [ --verbose ]                 display progress and debug information
+  --version                        display the version number
+```

docs/decon/decon.md

@@ -6,6 +6,45 @@ nav_order: 5
 
 # Deconvolution
 
+The `decon` function performs a [Richardson-Lucy deconvolution](https://en.wikipedia.org/wiki/Richardson%E2%80%93Lucy_deconvolution) on the input image. The input image can be a deskewed image (in the case of stage scanning) or a raw input image (in the case of objective scanning). The function requires a centered PSF kernel, the PSF z-step, and the image z-step.
+
+The PSF is input to the pipeline by including it in the paths portion of the configuration file. The directory containing PSF files is is specified, and then specific file names inside that folder are linked to the laser lines. Any filename is appropriate as long as it is specified, but the laser values themselves must exactly match the `Settings.txt file` (e.g., don't use 561 when it should be 560). The input PSF must be centered, and ideally cropped to a square. We provide an [ImageJ macro](https://github.com/aicjanelia/LLSM/blob/master/src/imagej/center-psf.ijm) that will take an image stack and perform this cropping for you. There must be a matching settings file for each PSF and it must have a matching name. For example, a PSF file named `cropped_560_PSF.tif` must have a corresponding file  `cropped_560_PSF_Settings.txt`.
+
+If using the pipeline, the image and PSF z-steps will be automatically parsed. If you using this module directly, those values can be determined from the image and PSF settings files (see [metadata](https://aicjanelia.github.io/LLSM/pipeline/bdv_save.html#metadata)).
+
+ Other inputs are `n`, the number of Richardson-Lucy iterations, and `subtract`, which subtracts a uniform value from the deconvolved data. For the AIC LLSM and MOSAIC, this subtract value should be 100 (based on the properties of our cameras).
+
+# Usage
+
+### Pipeline: Configuration File
+This is an example portion of a configuration file to use deconvolution.
+
+```json
+"paths": {
+    "root": "/path/to/experiment/",
+    "psf": {
+        "dir": "calibration",
+        "laser": {
+            "560": "cropped_560_PSF.tif",
+            "488": "cropped_488_PSF.tif"
+        }
+    }
+},
+"decon": {
+    "n": 5,
+    "bit-depth": 16,
+    "subtract": 100.0
+}
+```
+
+### Command Line Example
+When directly using the decon module on the command line, you must specify the input and output files as well as the PSF. You should also input the number of iterations. Pixel size, etc. should be specified to avoid any conflicts with default values. The example below will deconvolve an input image stack that was previously deskewed.
+```c
+decon -n 5 -b 16 -s 100.0 -w -k /path/to/calibration/cropped_488_PSF.tif -p 0.1 -q 0.21462536238843902 -o /path/to/experiment/decon/scan_Cam1_ch0_tile0_t0000_decon.tif /path/to/experiment/deskew/scan_Cam1_ch0_tile0_t0000_deskew.tif
+```
+
+### Decon Options
+
 ```text
 decon: deconvolves an image with a PSF or PSF parameters
 usage: decon [options] path
@@ -25,5 +64,3 @@ Allowed options:
   -v [ --verbose ]                    display progress and debug information
   --version                           display the version number
 ```
-
-The `decon` function performs a Richardson-Lucy deconvolution on a deskewed TIFF image. The function requires a centered PSF kernel, the PSF z-step, and the image z-step.

docs/deskew/deskew.md

@@ -6,6 +6,43 @@ nav_order: 4
 
 # Deskewing
 
+The `deskew` function performs an affine transformation on an image stack to correct the shearing of the data during acquisition. This step is only necessary if images were acquired in stage scanning mode (see [metadata](https://aicjanelia.github.io/LLSM/pipeline/bdv_save.html#metadata) to identify an experiment's scanning mode).
+
+Images on the LLSM and MOSAIC are acquired at an angle with respect to the coverslip because of the orientation of the objectives.  To learn more about this, check out the AIC's blog post on [Understanding Objective Orientation in the LLSM](https://www.aicjanelia.org/post/understanding-objective-orientation-in-the-llsm).  The deskew module takes as an input the objective angle and the stage step size. Note that the stage step size (the amount that the stage moves between slices) is not the same as the z-step size in the final stack. Step sizes are automatically calculated by the pipeline's setting parsers, and can also be determined manually from the `Settings.txt` file (see [metadata](https://aicjanelia.github.io/LLSM/pipeline/bdv_save.html#metadata)). You may also optionally specify the fill for empty regions in the skewed regions (default 0) and the bit-depth (default 16).
+
+If you need to calculate the z-step (for example, for downstream processing of the images), use the following equation:
+```
+z-step = stage-step*sin(system-angle)
+```
+Make sure that the calculator that you use to calculate sin knows that the angle is in degrees, or convert the angle to radians (`radians = degrees*pi/180`). The relevant values for each microscope are listed in the table below.
+
+| Microscope | Angle | μm/pixel |
+| ----- | ----- | -----|
+| LLSM | 31.8° | 0.104 |
+| MOSAIC | -32.45° = 147.55° | 0.108 |
+
+# Usage
+
+### Pipeline: Configuration File
+This is an example portion of a configuration file for a MOSAIC acquisition to be deskewed.
+
+```json
+"deskew": {
+    "xy-res": 0.108,
+    "fill": 0.0,
+    "bit-depth": 16,
+    "angle": 147.55
+}
+```
+
+### Command Line Example
+When directly using the deskew module on the command line, you must specify the input and output files. The best practice would be to also explicitly input values such as the angle, pixel size, etc. to avoid any conflicts with default values. The example below will deskew an input MOSAIC image stack.
+```c
+deskew -a 147.55 -x 0.108 -f 0.0 -b 16 -w -s 0.4 -o /path/to/experiment/deskew/scan_Cam1_ch0_tile0_t0000_deskew.tif  /path/to/experiment/scan_CamA_ch0_CAM1_stack0000_488nm_0000000msec_0004732481msecAbs_000x_000y_000z_0000t.tif
+```
+
+### Deskew Options
+
 ```text
 deskew: removes skewing induced by sample scanning
 usage: deskew [options] path
@@ -23,5 +60,3 @@ Allowed options:
   -v [ --verbose ]                   display progress and debug information
   --version                          display the version number
 ```
-
-The `deskew` function performs an affine transformation on a LLSM TIFF image to correct the shearing of the data during acquisition. The step/interval parameter refers to the distance travelled by the stage for each slice. This is not the same as the z-step which is equal to `step * sin(31.8) * (PI/180.0)`.

docs/index.md

@@ -3,13 +3,34 @@ title: Home
 layout: home
 nav_order: 1
 ---
+# LLSM Processing Pipeline
 
-The Advanced Imaging Center (AIC) uses this pipeline for pre-processing of images created on our LLSM or MOSAIC systems. The pipeline consists of multiple discrete modules that are accessed via the command line using either the `llsm-pipeline` or `mosaic-pipeline` commands. The main difference in the two pipelines is the parsing of filenames and acquisition settings metadata.
+The Advanced Imaging Center (AIC) uses this pipeline for pre-processing of images created on our LLSM or MOSAIC systems. The pipeline consists of multiple discrete modules that are accessed via the command line using either the `llsm-pipeline` or `mosaic-pipeline` commands. Which modules to implement are controlled by a configuration JSON file. The main difference in the two pipelines is the parsing of filenames and acquisition settings metadata.
 
 Each of the modules compiles to a separate binary that can also be directly executed by calling it on the command line. These modules are:
 - Cropping
 - Deskewing
 - Deconvolution
 - Maximum Intensity Projection
 
-The main pipeline command and each individual module are further described in this documentation. With any command, you can also use the `-h` option to get a list of supported arguments.
+The main pipeline command and each individual module are further described in this documentation. With any command, you can also use the `-h` option to get a list of supported arguments.
+
+
+## Cheatsheet
+
+### Command Line Examples
+```
+llsm-pipeline /aic/instruments/llsm/experimentFolder/config.json
+
+mosaic-pipeline /aic/instruments/MOSAIC/experimentFolder/config.json
+```
+
+### Configuration File
+See an [example configuration file here](https://aicjanelia.github.io/LLSM/pipeline/config.html#example-configjson).
+
+### Microscope Parameters
+| Microscope | Angle | μm/pixel |
+| ----- | ----- | -----|
+| LLSM | 31.8° | 0.104 |
+| MOSAIC | -32.45° = 147.55° | 0.108 |
+

docs/mip/mip.md

@@ -6,6 +6,36 @@ nav_order: 4
 
 # Maximum Intensity Projection (MIP)
 
+The `mip` module is run together with the other processing modules and will create MIPs for any modules that are run. For example, if running both `deskew` and `decon`, MIPs will be made for both. The subfolder created by `mip` will contain subfolders indicating which input images were used to create them (e.g., a `decon` subfolder). Inside this folder will be files that end with `mip_x`, `mip_y`, and `mip_z`. The dimension in the file name is the axes over which the images have been projected.
+
+## Viewing MIPs
+
+2D MIPs are very useful for screening data, but in general, should not be used for downstream analysis (use the appropriate processed image folder of 3D images). To quickly look at MIPs in ImageJ, use the option under `File > Import > Image Sequence...`. This will open a dialog box that allow for selecting the directory of images to load in. The directory will be something like `/path/to/experiment/mip/deskew/`. By default, the importer will attempt to open all images in the folder. This begins by opening the first image in the folder, and then excluding any subsequent files that are not the same size. In the case of `mip` folders, this will lead to importing only `_x` files. To get only the files of interest, the `Filter:` dialog can be used to select files. For example, to select the z projections, input `_z` in the filter dialog.
+
+More complex filtering of imported images is also possible using regular expressions. In ImageJ, `.*` is a wildcard, and the complete regular expression must be enclosed in parentheses. For example, to load in y projections of only channel zero, use `(ch0.*_y)` in the filter dialog. When the filter dialog box is changed, the number of images to be loaded automatically changes (shown in the `Count:` dialog). If the value is `---` or otherwise does not make sense, there is likely a typo in the regular expression.
+
+
+# Usage
+
+### Pipeline: Configuration File
+In the configuration file, simply specify which axes to create (`true`) or ignore (`false`). In general, set all values to `true`.
+
+```json
+"mip": {
+    "x": true,
+    "y": true,
+    "z": true
+}
+```
+
+### Command Line Example
+When directly using the mip module on the command line, you must specify the input and output files. You should also input the axes to project over (`-x -y -z`). The following example makes MIPs in all three dimensions for a deskewed image. Specify the pixel sizes to output properly scaled projections.
+```c
+mip -x  -y  -z  -x 0.104 -q 0.21462536238843902 -o /path/to/experiment/mip/deskew/scan_Cam1_ch0_tile0_t0000_deskew_mip.tif /path/to/experiment/deskew/scan_Cam1_ch0_tile0_t0000_deskew.tif
+```
+
+### MIP Options
+
 ```text
 mip: generates a maximum intensity projection along the specified axes
 usage: mip [options] path
@@ -24,4 +54,3 @@ Allowed options:
   --version                          display the version number
 ```
 
-The `mip` function performs a maximum intensity projection on any 3D TIFF image. The function creates mips along any specified axis (i.e., `-x`, `-y`, or `-z`), and the output filenames are appended with the corresponding axis name. The `x` and `y` dimensions are scaled to match the resolution of the `z` dimension.