Add masks module (#47)

* notes.txt to test the branches * masks folder, notebook to test FlatCam * added dimensions to simulate PSF * MURA masks with different sizes * psf simulation notebook * inverted colormaps for MURA * new masks with inverted colormap * minor changes * PhlatCam notebooks and masks creation * FZA notebooks and masks creation * PNG masks for the website * PNG masks for the website * fixed blank images * change MLS sequence * testing new MLS sequence generator * edge detection * testing new MLS sequence generator * inp * folders for MURA masks and MLS masks * removed axis on MLS * captioning * rectangular shape * deleting images * higher dimension mask * different sizes and periods * iterative phase retrieval notebook * bigger plot titles * iterative phase retrieval attempt * near-fresnel phase retrieval * testing fresnel on fza * print shapes * mask class py file * change edges shape * mask generation test * mask_class rename * removed settings file * answering pull request * testing phase retrieval * added to gitignore * add psf computation * added notebooks to gitignore * removed notebooks from tracking * removed from tracking * add IPYNB, PNG and NPY * removed simulated PSF images * 2nd pull request comments * mapping values from [-π,π] to [0,2π] * corrected PhaseContour * adding forward propagation coherence test * added formasks folders * added celeba_mini dataset * Fix simulation examples for 3D. * Fix single file example. * Add function to set initial estimate. * Add message if reset=False * Update CHANGELOG * scripts/sim/test_single_file.py * added scripts/sim/test_dataset.py * added CodedAperture upscaling * corrected CodedAperture rows and columns * testing upscaling * added roundint to have 0s and 1s in upscaled MLS mask * squared amplitude PSF * making FZA mask binary * making FZA mask binary * 3D PSF * simplified phase contour * changing upscaled coded aperture mask into int * added cloned flatcam repo by tanjasper * inverted dimensions for FZA * from_sensor class method * testing from_sensor class method * fixed PhaseContour broadcast error * fixed first 3 tests * added from_sensor tests for phase contour and FZA * upscaling for phase contour, removed default args from from_sensor, comments * removed simulation-related files * placing mask.py in hardware module * corrected import * add requirements for mask tests * removed 4 channel requirement * added warnings about feature size for PhaseContour * sensor_dict copy in from_name * copies of sensor attributes in from_sensor * no global variables, if main condition * Fix overwriting of sensor parameters. * Refactor names, compute PSF over different wv. * Update tests and requirements. * Update CHANGELOG. * Added header, corrected docstrings * removed useless requirements * added mask * created mask rst file * FIx documentation config to render mask module. * Update mask docstrings. * added Phase Contour and FZA * corrected docstring * corrected docstring * Fix docstrings. * Add target_psf docstring. * Changed the FZA radius from px to meters * Changed FZA radius from pixels to meters * Added FlatCam authors' data for simulation * Replaced scipy.ndimage.zoom by lensless.utils.image.resize for uscpaling * minor changes * FlatCam recon and sim file * deletion (replaced by tikhonov * Corrected docstrings * fixed bayer2rgb * Created reconstruction class, adapter rgb2bayer to different patterns * Docstrings * Added conversions between RGB and 4d Bayer * removed image-related functions * changed default noise period for PhaseContour * added custom P and Q matrices * removed resize import * added .upper() to coded aperture methods * fixed assertions * single file simulation with custom masks (Tikhonov method included) * Edit docstrings. * Clean up mask example script. * Update gitignore. * Fix comment. * Update comments. * Fix settings for FZA. * changed default ADMM n_iter to 20 * separated P & Q computation function * changed default ADMM iter to 20 * Changed default FZA radius * Change file name * name change * name change * Custom mask dataset simulation * fixed MLS issue, removed comments * added reconstruction normalization * changed docstrings and config name * change default flatcam parameter to false * Add line for formatting. * Add support for float-valued feature_size. * Refactor. * Clean up mask simulation and tikhonov recon. * Formatting. * Formatting * Add docs for coded aperture reconstruction. * minor changes * minor changes (plots, docstring, etc) * minor changes (plots, docstring, etc) * implemented tikhonov reconstruction * FIx mask parsing. * Inherit from single file config, add torch support --------- Co-authored-by: Aaron Fargeon <[email protected]> Co-authored-by: Eric Bezzam <[email protected]>
LCAV · Jul 31, 2023 · 53c6113 · 53c6113
1 parent c73141b
commit 53c6113
Show file tree

Hide file tree

Showing 23 changed files with 1,597 additions and 18 deletions.
diff --git a/.github/workflows/python_no_pycsou.yml b/.github/workflows/python_no_pycsou.yml
@@ -58,4 +58,5 @@ jobs:
       run: |
         pip install -U pytest
         pip install -r recon_requirements.txt
+        pip install -r mask_requirements.txt
         pytest
diff --git a/.github/workflows/python_pycsou.yml b/.github/workflows/python_pycsou.yml
@@ -58,5 +58,6 @@ jobs:
       run: |
         pip install -U pytest
         pip install -r recon_requirements.txt
+        pip install -r mask_requirements.txt
         pip install git+https://github.com/matthieumeo/pycsou.git@v2-dev
         pytest
diff --git a/.gitignore b/.gitignore
@@ -1,6 +1,4 @@
-scripts/recon/outputs/
 .idea/
-
 *_sol.py
 scripts/admm_*
 scripts/gd_*
@@ -97,6 +95,17 @@ target/
 
 # Jupyter Notebook
 .ipynb_checkpoints
+*.ipynb
+
+# Images and NPY files
+*.png
+*.npy
+
+# Datasets
+data/celeba_mini
+
+# FlatCam reconstruction
+masks/FlatCam/flatcam-authors
 
 # IPython
 profile_default/
@@ -152,3 +161,4 @@ dmypy.json
 
 # Pyre type checker
 .pyre/
+
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
@@ -18,10 +18,12 @@ Added
 - Link and citation for JOSS.
 - Authors at top of source code files.
 - Add paramiko as dependency for remote capture and display.
+- Mask module, for CodedAperture (FlatCam), PhaseContour (PhlatCam), and FresnelZoneAperture.
 - Script for measuring arbitrary dataset (from Raspberry Pi).
 - Support for preprocessing and postprocessing, such as denoising, in ``TrainableReconstructionAlgorithm``. Both trainable and fix postprocessing can be used.
 - Utilities to load a trained DruNet model for use as postprocessing in ``TrainableReconstructionAlgorithm``.
 - Support for unrolled loading and inference in the script ``admm.py``.
+- Tikhonov reconstruction for coded aperture measurements (MLS / MURA).
 
 
 Changed
@@ -36,7 +38,8 @@ Changed
 Bugfix
 ~~~~~~
 
--  Displaying 3D reconstructions by summing values along axis would produce un-normalized values.
+- Fix overwriting of sensor parameters when downsampling.
+- Displaying 3D reconstructions by summing values along axis would produce un-normalized values.
 
 1.0.4 - (2023-06-14)
 --------------------

diff --git a/configs/mask_sim_dataset.yaml b/configs/mask_sim_dataset.yaml
@@ -0,0 +1,31 @@
+defaults:
+  - mask_sim_single
+  - _self_
+
+seed: 0
+save: True
+
+files:
+  dataset: data/celeba_mini
+  image_ext: jpg
+  n_files: 10    # null to use all
+
+simulation:
+  object_height: [0.25, 0.3]   # range for random height, or scalar
+  random_shift: False
+  grayscale: False
+
+# torch for reconstruction
+torch: False
+torch_device: 'cuda:0'
+
+recon:
+
+  algo: "tikhonov"   # "tikhonov" or "admm" or None to skip
+
+  tikhonov:
+    reg: 3e-4
+
+  admm:
+    # Recommend to not display, ok for small number of files, otherwise many windows will pop up!
+    disp_iter: null
diff --git a/configs/mask_sim_single.yaml b/configs/mask_sim_single.yaml
@@ -0,0 +1,61 @@
+hydra:
+  job:
+    chdir: True    # change to output folder
+
+
+files:
+  original: data/celeba_mini/000019.jpg
+  #original: data/original/mnist_3.png
+
+save: True
+
+simulation:
+  object_height: 0.3
+  # these distance parameters are typically fixed for a given PSF
+  scene2mask: 40e-2
+  mask2sensor: 4e-3
+  # see waveprop.devices
+  sensor: "rpi_hq"
+  snr_db: 20
+  # Downsampling for PSF
+  downsample: 8
+
+  # max val in simulated measured (quantized 8 bits)
+  max_val: 230
+
+  image_format: rgb   # rgb, grayscale, bayer_rggb, bayer_bggr, bayer_grbg, bayer_gbrg
+
+  flatcam: False   # only supported if mask.type is "MURA" or "MLS"
+
+
+mask:
+  type: "MLS"    # "MURA", "MLS", "FZA", "PhaseContour"
+
+  # Coded Aperture (MURA or MLS)
+  #flatcam_method: 'MLS'
+  n_bits: 8 # e.g. 8 for MLS, 99 for MURA
+
+  # Phase Contour
+  noise_period: [16, 16]
+  refractive_index: 1.2
+  phase_mask_iter: 10
+
+  # Fresnel Zone Aperture
+  radius: 0.32e-3
+
+
+recon:
+
+  algo: "admm"   # tikhonov or admm
+
+  tikhonov:
+    reg: 3e-4
+
+  admm:
+    n_iter: 20
+    disp_iter: 2
+    # Hyperparameters
+    mu1: 1e-6
+    mu2: 1e-5
+    mu3: 4e-5
+    tau: 0.0001
diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -7,6 +7,8 @@
 
 MOCK_MODULES = [
     "scipy",
+    "scipy.signal",
+    "scipy.linalg",
     "pycsou",
     "matplotlib",
     "matplotlib.pyplot",
@@ -28,7 +30,12 @@
     "PIL",
     "tqdm",
     "paramiko",
-    "paramiko.ssh_exception"
+    "paramiko.ssh_exception",
+    "perlin_numpy",
+    "waveprop",
+    "waveprop.fresnel",
+    "waveprop.rs",
+    "waveprop.noise",
 ]
 for mod_name in MOCK_MODULES:
     sys.modules[mod_name] = mock.Mock()

diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -25,6 +25,7 @@ Contents
    measurement
    reconstruction
    evaluation
+   mask
    sensor
    utilities
    demo

diff --git a/docs/source/mask.rst b/docs/source/mask.rst
@@ -0,0 +1,33 @@
+.. automodule:: lensless.hardware.mask
+
+
+   Abstract Mask Class
+   ~~~~~~~~~~~~~~~~~~~
+
+   .. autoclass:: lensless.hardware.mask.Mask
+      :members:
+      :special-members: __init__
+
+
+   Coded Aperture (FlatCam)
+   ~~~~~~~~~~~~~~~~~~~~~~~~
+
+   .. autoclass:: lensless.hardware.mask.CodedAperture
+      :members:
+      :special-members: __init__
+
+
+   Phase Contour (PhlatCam)
+   ~~~~~~~~~~~~~~~~~~~~~~~~
+
+   .. autoclass:: lensless.hardware.mask.PhaseContour
+      :members:
+      :special-members: __init__
+
+
+   Fresnel Zone Aperture
+   ~~~~~~~~~~~~~~~~~~~~~
+
+   .. autoclass:: lensless.hardware.mask.FresnelZoneAperture
+      :members:
+      :special-members: __init__
diff --git a/docs/source/reconstruction.rst b/docs/source/reconstruction.rst
@@ -44,6 +44,14 @@
 
    .. autofunction:: lensless.recon.admm.finite_diff_gram
 
+
+   Tikhonov (Ridge Regression)
+   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+   .. autoclass:: lensless.CodedApertureReconstruction
+      :special-members: __init__, apply
+
+
    Accelerated Proximal Gradient Descent (APGD)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

diff --git a/lensless/__init__.py b/lensless/__init__.py
@@ -21,6 +21,7 @@
     FISTA,
     GradientDescentUpdate,
 )
+from .recon.tikhonov import CodedApertureReconstruction
 from .hardware.sensor import VirtualSensor, SensorOptions
 
 try:
-Original file line number
+Diff line change
@@ Expand Up / @@ -21,6 +21,7 @@ @@
         FISTA,
         GradientDescentUpdate,
     )
+    from .recon.tikhonov import CodedApertureReconstruction
     from .hardware.sensor import VirtualSensor, SensorOptions
     try:
@@ Expand Down @@