Add flatfield documentation

docs/crop/crop.md

@@ -1,7 +1,7 @@
 ---
 title: Cropping
 layout: default
-nav_order: 3
+nav_order: 4
 ---
 
 # Cropping

docs/decon/decon.md

@@ -1,7 +1,7 @@
 ---
 title: Deconvolution
 layout: default
-nav_order: 5
+nav_order: 6
 ---
 
 # Deconvolution

docs/deskew/deskew.md

@@ -1,7 +1,7 @@
 ---
 title: Deskewing
 layout: default
-nav_order: 4
+nav_order: 5
 ---
 
 # Deskewing

docs/flatfield/flatfield.md

@@ -0,0 +1,77 @@
+---
+title: Flatfield Correction
+layout: default
+nav_order: 3
+---
+
+# Flatfield Correction
+
+The `flatfield` module performs a flatfield correction as described in Hobson, et al.$^1$ The function outputs $I_{Corrected}$ using the following equation:
+
+$$I_{Corrected} = \frac{I_{Measured} - I_{Dark}}{I_N}$$
+
+The input $I_{Measured}$ is the acquired image, while $I_{Dark}$ and $I_N$ are calibration images. To prepare these calibration images, see [Flatfield Inputs](#flatfield-inputs) below.  The paths to these calibration images must be specified in the paths section of the configuration file. Enable flatfield correction by adding a flatfield section to the configuration file with the image bit depth specified.
+
+## Flatfield Inputs
+Several flatfield calibration images should be collected for each channel to be corrected. Similarly, several dark images (i.e., camera aperture closed) should be collected for the system. Experimental details on collecting these images can be found in Hobson, et al.$^1$ Once the sets of images are collected, they are averaged to create $I_{flatfield}$ and $I_{dark}$, respectively. The ImageJ macro described below performs this averaging while creating the required $I_N$ input image. $I_N$ is a normalized flatfield image, calculated as:
+
+$$I_N = \frac{I_{Flatfield}-I_{Dark}}{<I_{Flatfield}-I_{Dark}>}$$
+
+where <> indicates the mean intensity value of the enclosed image.
+
+### ImageJ Macro
+When averaging the collected calibration images, it is best to use ImageJ's 32-bit type for more precision. Additionally, $I_N$ **must** be saved as a 32-bit image as fractional values are necessary for the flatfield correction. To help keep the type conversions consistent, we provide an [ImageJ macro](https://github.com/aicjanelia/LLSM/blob/master/src/imagej/flatfield_inputs.ijm) that will perform all necessary calculations. This plugin takes as input all collected dark images as a multi-image tiff file (e.g., 100 dark images saved as a single tiff file) and a single tiff stack per channel of flatfield images.
+
+To use, run the `flatfield_inputs.ijm` macro. A GUI will open, as shown in the figure below. You can specify the darkfield image in multiple ways: (i) use the `Browse` button to find the stack of darkfield images, (ii) copy the full path into the box, or (iii) simply drag and drop the image file onto the darkfield line of the GUI. Repeat this process for the flatfield stacks. Any channel entries without files will be ignored. The macro will average the stacks and save a file called *DarkAverage.tif* in the same folder as the input darkfield stack. The macro will then continue to perform the $I_N$ calculation, saving one *I_N_###.tif* file per channel in the same folder, with *###* specified by the values in the Channel input line of the macro GUI. These files can then be specified in the paths section of the pipeline configuration file to run the flatfield correction module.
+
+![](FlatFieldGUI.png)
+
+## Reference
+$^1$ Hobson, C. M., Guo, M., Vishwasrao, H. D., Wu, Y., Shroff, H. and Chew, T.-L. (2022). Practical considerations for quantitative light sheet fluorescence microscopy. Nat Methods 1-12. [doi:10.1038/s41592-022-01632-x](https://doi.org/10.1038/s41592-022-01632-x)
+
+# Usage
+
+### Pipeline: Configuration File
+This is an example portion of a configuration file that sets up the inputs to the flatfield module and enables the module.
+
+```json
+"paths": {
+    "root": "/aic/instruments/llsm/pipeline-test/",
+    "flatfield": {
+        "dir": "Calibration",
+        "dark": "DarkAverage.tif",
+        "laser": {
+            "488": "I_N_488.tif",
+            "560": "I_N_560.tif"
+        }
+    }
+},
+"flatfield": {
+    "bit-depth": 16
+} 
+```
+
+### Command Line Example
+When directly using the flatfield module on the command line, you must specify the input and output files as well as the input calibration images. The best practice would be to also explicitly input values such as the pixel size to avoid any conflicts with default values. The example below will flatfield correct an input MOSAIC image stack.
+```c
+flatfield -d /path/to/experiment/Calibration/DarkAverage.tif -n /path/to/experiment/Calibration/I_N_488.tif -x 0.108 -q 0.21462536238843902 -o /path/to/experiment/flatfield/scan_Cam1_ch0_tile0_t0000_flatfield.tif -b 32 -w /path/to/experiment/scan_CamA_ch0_CAM1_stack0000_488nm_0000000msec_0004732481msecAbs_000x_000y_000z_0000t.tif
+```
+
+### Flatfield Options
+
+```text
+usage: flatfield [options] path
+
+Allowed options:
+  -h [ --help ]                    display this help message
+  -d [ --dark ] arg                dark image file path
+  -n [ --n-image ] arg             N image file path
+  -x [ --xy-rez ] arg (=-1)        x/y resolution (um/px)
+  -q [ --image-spacing ] arg (=-1) z-step size of input image
+  -o [ --output ] arg              output file path
+  -b [ --bit-depth ] arg (=16)     bit depth (8, 16, or 32) of output image
+  -w [ --overwrite ]               overwrite output if it exists
+  -v [ --verbose ]                 display progress and debug information
+  --version                        display the version number
+```
+

docs/mip/mip.md

@@ -1,12 +1,25 @@
 ---
 title: Maximum Intensity Projection
 layout: default
-nav_order: 6
+nav_order: 7
 ---
 
 # Maximum Intensity Projection (MIP)
 
-The `mip` module is run together with the other processing modules and will create MIPs for any modules that are run. For example, if running both `deskew` and `decon`, MIPs will be made for both. The subfolder created by `mip` will contain subfolders indicating which input images were used to create them (e.g., a `decon` subfolder). Inside this folder will be files that end with `mip_x`, `mip_y`, and `mip_z`. The dimension in the file name is the axes over which the images have been projected.
+The `mip` module is run together with the other processing modules and will create MIPs for key modules that are run. For example, if running both `deskew` and `decon`, MIPs will be made for both. The subfolder created by `mip` will contain subfolders indicating which input images were used to create them (e.g., a `decon` subfolder). Inside this folder will be files that end with `mip_x`, `mip_y`, and `mip_z`. The dimension in the file name is the axes over which the images have been projected.
+
+
+| Modules Enabled | MIPs created |
+| ----- | ----- |
+| mip only | input images |
+| flatfield| flatfield |
+| crop | crop |
+| crop + flatfield | crop |
+| (crop and/or flatfield or none) + deskew | deskew |
+| decon + mip | input images & decon |
+| flatfield + decon | flatfield & decon |
+| crop +/- flatfield + decon | crop & decon |
+| (crop and/or flatfield or none) + deskew + decon | deskew & decon |
 
 ## Viewing MIPs
 

docs/pipeline/bdv_save.md

@@ -80,12 +80,12 @@ Excitation Filter, Laser, Power (%), Exp(ms), Laser2, Power2 (%), Laser3, Power3
 
 
 ## Processed Files
-Each module that is run will create its own subfolder of files. Subsequent modules will look in the relevant subfolder to continue the processing. For example, if running both cropping and deskewing, the deskewing will be run on the files in the `crop` folder, not on the original raw images.
+Each module that is run will create its own subfolder of files. Subsequent modules will look in the relevant subfolder to continue the processing. For example, if running both cropping and deskewing, the deskewing will be run on the files in the `crop` folder, not on the original raw images. Modules are run in the order `flatfield > crop > deskew > decon`.
 
 ### _Maximum Intensity Projections (MIPs)_
 Maximum Intensity Projections take the maximum intensity across one axis of an image to turn a 3D image into a 2D image. This can be useful for quickly reviewing images without needing to load the full experiment into memory. See more about this in the [MIP documentation](https://aicjanelia.github.io/LLSM/mip/mip.html). In general, analysis should be done on the 3D images, and thus the `mip` folder should not be used for analysis.
 
-MIPs are created for each module that is used, so inside the `mip` folder will be additional subfolders corresponding to modules (e.g., `deskew` and `decon`). The folder name indicates which input image was used to create the MIP. MIPs can be created by taking the maximum along the x, y, or z axes, and file names will reflect this by ending with `_x`, `_y`, and `_z` respectively.
+MIPs are created for key modules that are used, so inside the `mip` folder will be additional subfolders corresponding to modules (e.g., `deskew` and `decon`). The folder name indicates which input image was used to create the MIP. MIPs can be created by taking the maximum along the x, y, or z axes, and file names will reflect this by ending with `_x`, `_y`, and `_z` respectively.
 
 ## BDV Saving
 The raw image file names contain informative metadata, but the long complex names can make converting to other file formats (e.g., Imaris) difficult. When bdv saving is enabled in the [configuration file](https://aicjanelia.github.io/LLSM/pipeline/config.html), the raw images will keep their informative names, but all processed files will have simplified names that are inspired by the [Automatic Loader in BigSticher](https://imagej.net/plugins/bigstitcher/autoloader), which runs on BigDataViewer (bdv). The simplified version keeps track of cameras, channels, tiles, and time points as simple numeric values (e.g., the specific laser line is removed). An example of the naming convention is shown below for a raw image that was run through the crop module.

docs/pipeline/config.md

@@ -34,7 +34,10 @@ PSFs are required for running the deconvolution module. In prior versions of the
 This optional section determines the naming convention for output files. It defaults to false if the section is not provided. To learn more see [File Organization](https://aicjanelia.github.io/LLSM/pipeline/bdv_save.html).
 
 ## Individual Modules
-Individual modules are requested by adding their own section to the JSON file.  The [example json file](#example-configjson) requests cropping, deskewing, deconvolution, and the generation of MIP files. More details on the parameters for each are provided in the discussion of each module, but a high-level overview is provided here.  If all modules are requested, they are run in the order of `crop > deskew > decon`, with mips created at each stage as appropriate.
+Individual modules are requested by adding their own section to the JSON file.  The [example json file](#example-configjson) requests flatfield correction, cropping, deskewing, deconvolution, and the generation of MIP files. More details on the parameters for each are provided in the discussion of each module, but a high-level overview is provided here.  If all modules are requested, they are run in the order of `flatfield > crop > deskew > decon`, with MIPs created at each stage as described in [MIP](https://aicjanelia.github.io/LLSM/mip/mip.html).
+
+### _flatfield_
+To use the flatfield correction, paths to an average darkfield image and one normalized flatfield image per channel must be included in the paths section. These inputs are described further in [Flatfield Inputs](https://aicjanelia.github.io/LLSM/flatfield/flatfield.html#flatfield-inputs). Adding the paths does not enable this module; the module is enabled by adding a flatfield section with the image bit-depth.
 
 ### _crop_
 For each side of the image that cropping is desired, the number of pixels to remove from that side is a parameter. For example, `"cropTop": 10` removes 10 pixels from the top of the image. Any sides that are not provide are assumed to be zero. Other optional parameters are described further in [Cropping]().
@@ -74,11 +77,22 @@ Each slot has 15 GB of memory. The maximal memory is used by the deconvolution m
                 "560": "560_PSF.tif",
                 "488": "488_PSF.tif"
             }
+        },
+        "flatfield": {
+            "dir": "Calibration",
+            "dark": "DarkAverage.tif",
+            "laser": {
+                "488": "I_N_488.tif",
+                "560": "I_N_560.tif"
+            }
         }
     },
     "bdv": {
         "bdv_save": true
     },
+    "flatfield": {
+        "bit-depth": 16
+    },
     "crop": {
         "cropTop": 10,
         "cropBottom": 0,