This is an unsupervised unrolled deep network that recoconstructs the hyperspectral image from RGB images captures under two lighting conditions.
Unsupervised spectral reconstruction (SR) aims to recover the hyperspectral image (HSI) from corresponding RGB images without annotations. Existing SR methods achieve it from a single RGB image, hindered by the significant spectral distortion. Although several deep learning-based methods increase the SR accuracy by adding RGB images, their networks are always designed for other image recovery tasks, leaving huge room for improvement. To overcome this problem, we propose a novel, to our knowledge, approach that reconstructs the HSI from a pair of RGB images captured under two illuminations, significantly improving reconstruction accuracy. Specifically, an SR iterative model based on two illuminations is constructed at first. By unfolding the proximal gradient algorithm solving this SR model, an interpretable unsupervised deep network is proposed. All the modules in the proposed network have precise physical meanings, which enable our network to have superior performance and good generalization capability. Experimental results on two public datasets and our real-world images show the proposed method significantly improves both visually and quantitatively as compared with state-of-the-art methods.
Python3.8
torch 1.12,torchvision 0.13.0
Numpy,Scipy
Also, we will create a Paddle version that implements FeafusFormer in AI Studio online for free!
CAVE dataset,
Preprocessed CAVE dataset.
For any questions, feel free to email Xuheng Cao(📧[email protected]).
If you find our work useful in your research, please cite our paper 🙂
@article{article,
author = {Xuheng Cao, Yusheng Lian, Zilong Liu, Jin Li, and Kaixuan Wang},
year = {2024},
month = {04},
pages = {1993-1996},
title = {Unsupervised spectral reconstruction from RGB images under two lighting conditions},
journal = {Optics Letters},
doi = {10.1364/OL.517007}
}