STATE-OF-THE-ART QUALITY ENHANCEMENT APPROACHES FOR DEPTH IMAGES (Updating)

A list of depth quality enhancement approaches and the summary of some available codes or projects. This list is maintained by: [Lijun Zhao]. If your paper about [this topic] is not listed among them，please contact us（[email protected]）.

The summary of some available codes or projects

• Depth-Enhancement-Toolbox(Thanks to Junyi Liu(Email:[email protected]))[Code]

  • Joint Bilateral Filter[PDF] Digital photography with flash and no-flash image pairs

  • Joint Bilateral Upsampling[PDF] Joint bilateral upsampling

  • Noise-aware Filter[PDF] A noise-aware filter for real-time depth upsampling

  • Weight Mode Filter[PDF] Depth video enhancement based on weighted mode filtering

  • Anisotropic Diffusion[PDF] Guided Depth enhancement via Anisotropic Diffusion

  • Markov Random Field[PDF] An application of markov random fields to range sensing

  • Markov Random Field[PDF] Image and sparse laser fusion for dense scene reconstruction

  • Layered Bilateral Filter[PDF] Spatial-depth super resolution for range images

• Robust Color Guided Depth Map Restoration[Code]

• Color-guided Depth Recovery from RGB-D Data Using an Adaptive Auto-regressive Model[Code]

• Depth Map Super-Resolution by Deep Multi-Scale Guidance [Project] [Code]

• Deep Joint Image Filtering [Project] [Code]

• Fast Guided Global Interpolation for Depth and Motion [Project] [Code]

• Edge guided single depth image super resolution [Project] [Code]

• ATGV-Net: Accurate Depth Super-Resolution [Code]

• A Deep Primal-Dual Network for Guided Depth Super-Resolution [Code]

• Patch based synthesis for single depth image super-resolution[Project] [Code]

• Simultaneous color-depth super-resolution with conditional generative adversarial networks[Project] (including results for HR-color-image-guided depth 4X super-resolution)

• Local Activity-Driven Structural-Preserving Filtering for Noise Removal and Image Smoothing [Project] [Code]

• Iterative range-domain weighted filter for structural preserving image smoothing and de-noising [Code]

• Two-stage filtering of compressed depth images with Markov Random Field [Code]

• Candidate value-based boundary filtering for compressed depth images [Code]

A list of depth quality enhancement approaches

Depth Image Super-Resolution

CNN-based Methods

• RDN[PDF]: Residual dense network for intensity-guided depth map enhancement (Information Sciences 2019), Yifan Zuo, Qiang Wu, Yuming Fang, Yong Yang, Xiwu Shang, Bin Wang.

• PDDSR[PDF]: Perceptual deep depth super-resolution (arXiv), Oleg Voinov, Alexey Artemov, Vage Egiazarian, Alexander Notchenko, Gleb Bobrovskikh, Denis Zorin, Evgeny Burnaev.

• MRGPS[PDF]: Depth Map Super-Resolution via Multilevel Recursive Guidance and Progressive Supervision (IEEE ACCESS 2019), Bolan Yang, Xiaoting Fan, Zexun Zheng, Xiaohuan Liu, Kaiming Zhang, Jianjun Lei.

• MFR[PDF]: Multi-scale Frequency Reconstruction for Guided Depth Map Super-resolution via Deep Residual Network (IEEE Trans. on Circuits and Systems for Video Technology 2019), Yifan Zuo, Qiang Wu, Yuming Fang, Ping An, Liqin Huang, Zhifeng Chen.

• CFCNS[PDF]: Deep Color Guided Coarse-to-Fine Convolutional Network Cascade for Depth Image Super-Resolution (IEEE Trans. on Image Processing 2019), Yang Wen, Bin Sheng, Ping Li, Weiyao Lin, David Dagan Feng.

• DSDSR[PDF]: Deeply Supervised Depth Map Super-Resolution as Novel View Synthesis (IEEE Trans. on Circuits and Systems for Video Technology 2019), Xibin Song, Yuchao Dai, Xueying Qin.

• SDISR[PDF]: Single Depth Image Super-Resolution Using Convolutional Neural Networks (ICASSP 2018), Baoliang Chen, Cheolkon Jung.

• FDMSR[PDF]: Fast Depth Map Super-Resolution Using Deep Neural Network (2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)), Alisa Korinevskaya, Ilya Makarov.

• GDN[PDF]: Guided deep network for depth map super-resolution: How much can color help? (ICASSP 2017), Wentian Zhou, Xin Li, Daryl Reynolds.

• FEAG[PDF]: Depth super-resolution via fully edge-augmented guidance (IEEE Visual Communications and Image Processing 2017), Jingyu Yang, Hao Lan, Xiaolin Song, Kun Li.

• DSR[PDF]: Deep Depth Super-Resolution: Learning Depth Super-Resolution Using Deep Convolutional Neural Network (ACCV 2016), Xibin Song, Yuchao Dai, Xueying Qin.

• DMSG[PDF][Code]: Depth Map Super-Resolution by Deep Multi-Scale Guidance (ECCV 2016), Tak-Wai Hui, Chen Change Loy,Xiaoou Tang.

• DJIF[PDF][Code]: Deep Joint Image Filtering (ECCV 2016), Yijun Li, Jia-Bin Huang, Narendra Ahuja, Ming-Hsuan Yang.

• ATGV-Net[PDF][Code]: ATGV-Net: Accurate Depth Super-Resolution (ECCV 2016), Gernot Riegler, Matthias Rüther, Horst Bischof.

• DPN[PDF][Code]: A Deep Primal-Dual Network for Guided Depth Super-Resolution (BMVC 2016), Gernot Riegler, David Ferstl, Matthias Rüther, Horst Bischof.

Traditional Methods

• CIN[PDF]: Coupled Ista Network for Multi-modal Image Super-resolution (ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)), Xin Deng, Pier Luigi Dragotti.

• AGDSR[PDF]: Alternately Guided Depth Super-resolution Using Weighted Least Squares and Zero-order Reverse Filtering (ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)), Kailong Zhou, Shengtao Yu, Cheolkon Jung.

• CGDIR[PDF]: Color-Guided Depth Image Recovery with Adaptive Data Fidelity and Transferred Graph Laplacian Regularization (IEEE Trans. on Circuits and Systems for Video Technology 2019), Yongbing Zhang, Yihui Feng, Xianming Liu, Deming Zhai, Xiangyang Ji, Haoqian Wang, and Qionghai Dai

• JCIER[PDF]: Depth Super-Resolution via Joint Color-Guided Internal and External Regularizations (IEEE Trans. on Image Processing 2019), Xianming Liu, Deming Zhai, Rong Chen, Xiangyang Ji, Debin Zhao, Wen Gao.

• JG-DSR[PDF]: Joint-Feature Guided Depth Map Super-Resolution With Face Priors (IEEE Trans. on Cybernetics 2018), Shuai Yang, Jiaying Liu, Yuming Fang, Zongming Guo.

• EIE[PDF]: Explicit Edge Inconsistency Evaluation Model for Color-guided Depth Map Enhancement (IEEE Trans. on Circuits and Systems for Video Technology 2018), Yifan Zuo, Qiang Wu, Jian Zhang, Ping An.

• MSF[PDF]: Minimum spanning forest with embedded edge inconsistency measurement model for guided depth map enhancement (IEEE Trans. Image Process. 2018), Yifan Zuo, Qiang Wu, Jian Zhang, Ping An.

• MJTF[PDF]: Depth image super-resolution reconstruction based on a modified joint trilateral filter (2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)), Dongsheng Zhou, Ruyi Wang, Xin Yang, Qiang Zhang, Xiaopeng Wei.

• MRDLR[PDF]: Single depth image super-resolution with multiple residual dictionary learning and refinement (ICME 2017), Lijun Zhao, Huihui Bai, Jie Liang, Anhong Wang, Yao Zhao.

• VBW[PDF]: Variable Bandwidth Weighting for Texture Copy Artifact Suppression in Guided Depth Upsampling (IEEE Trans. on Circuits and Systems for Video Technology 2017), Wei Liu, Xiaogang Chen, Jie Yang, Qiang Wu.

• IEDU[PDF]: Intensity-guided edge-preserving depth upsampling through weighted L0 gradient minimization (Journal of Visual Communication and Image Representation 2017), Cheolkon Jung, Shengtao Yu, Joongkyu Kim.

• VSQ-SR[PDF]: Depth Map Super-Resolution Considering View Synthesis Quality (IEEE Trans. on Image Processing 2017), Jianjun Lei, Lele Li, Huanjing Yue, Feng Wu, Nam Ling, Chunping Hou.

• FGI[PDF][Code]: Fast Guided Global Interpolation for Depth and Motion (ECCV2016), Yu Li, Dongbo Min, Minh N. Do,Jiangbo Lu.

• EGSR[PDF][Code]:Edge guided single depth image super resolution (IEEE Trans. Image Process. 2016), Jun Xie, Rogerio Schmidt Feris, Ming-Ting Sun.

• JSRD[PDF]:Joint Super Resolution and Denoising From a Single Depth Image (IEEE Tran. on Multimedia 2015), Jun Xie, Rogerio Schmidt Feris, Shiaw-Shian Yu, Ming-Ting Sun.

• SVAM[PDF]: Depth map super-resolution using stereo-vision-assisted model (Neurocomputing 2015), Yuxiang Yang, Mingyu Gao, Jing Zhang, Zhengjun Zha, Zengfu Wang.

• HDMU[PDF]: High-Quality Depth Map Upsampling and Completion for RGB-D Cameras (IEEE Trans. Image Process. 2014), Jaesik Park, Hyeongwoo Kim, Yu-Wing Tai, Michael S. Brown, In So Kweon.

• PS[PDF]: Patch based synthesis for single depth image super-resolution (ECCV 2012), Oisin Mac, AodhaNeill D. F., CampbellArun Nair, Gabriel J. Brostow.

• SDSR[PDF]: Spatial-Depth Super Resolution for Range Images (CVPR 2007), Qingxiong Yang, Ruigang Yang, James Davis, David Nister.

Depth Image Restoration/Enhancement

• PDUE[PDF]: Multiscale Directional Fusion for Depth Map Super Resolution with Denoising (ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)), Dan Xu, Xiaopeng Fan, Shibo Zhang, Yang Wang, Debin Zhao, Wen Gao.

• PDUE[PDF]: Precise depth map upsampling and enhancement based on edge-preserving fusion filters (IET Computer Vision 2018), Ting-An Chang, Jar-Ferr Yang.

• RDE[PDF]: Robust depth enhancement based on texture and depth consistency (IET Computer Vision 2018), Ting-An Chang, Wei-Chen Liao, Jar-Ferr Yang.

• RGDR[PDF][Code]: Robust Color Guided Depth Map Restoration (IEEE Trans. Image Process. 2017), Wei Liu, Xiaogang Chen, Jie Yang, Qiang Wu.

• EDE[PDF]: Efficient Depth Enhancement Using a Combination of Color and Depth Information (Sensors 2017), Kyungja Lee, Yuseok Ban, Sangyoun Lee.

• PMDIE[PDF]: Probabilistic Multiview Depth Image Enhancement Using Variational Inference (IEEE Journal of Selected Topics in Signal Processing 2015), Pravin Kumar Rana, Jalil Taghia, Zhanyu Ma, Markus Flierl.

• LTPA[PDF]: Depth Image Enhancement Using Local Tangent Plane Approximations (CVPR 2015), Kiyoshi Matsuo, Yoshimitsu Aoki.

• LRMC[Project][PDF]: Depth Enhancement via Low-rank Matrix Completion (CVPR 2014), Si Lu, Xiaofeng Ren, Feng Liu.

• RMDE[PDF]: Reliability-Based Multiview Depth Enhancement Considering Interview Coherence (IEEE Trans. on Circuits and Systems for Video Technology 2014), Jinwook Choi, Dongbo Min, Kwanghoon Sohn.

• WMF[PDF]: Depth video enhancement based on weighted mode filtering (IEEE Trans. Image Process. 2012), Dongbo Min, Jiangbo Lu, Minh N. Do.

Color-Depth Image Super-Resolution

• CDcGAN[Project][PDF]: Simultaneous color-depth super-resolution with conditional generative adversarial networks (Pattern Recognition 2019), Lijun Zhao, Huihui Bai, Jie Liang, Bing Zen, Anhong Wang, Yao Zhao.

• MFSR[PDF]: Multi-frame Super-resolution for Time-of-flight Imaging (Pattern Recognition 2019), Fengqiang Li, Pablo Ruiz, Oliver Cossairt, Aggelos K. Katsaggelos.

• RSR[PDF]: Robust Super-Resolution for Mixed-Resolution Multiview Image Plus Depth Data (IEEE Tran. on Circuits and Systems for Video Technology 2015), Thomas Richter, Jürgen Seiler, Wolfgang Schnurrer, André Kaup.

• SSR[PDF]: Simultaneous Super-Resolution of Depth and Images Using a Single Camera (ECCV2013), Hee Seok Lee, Kuoung Mu Lee.

Compressed Depth Image Filtering

Filtering Methods

• CMF[PDF]: Cross-View Multi-Lateral Filter for Compressed Multi-View Depth Video (IEEE Transactions on Image Processing 2019), You Yang, Qiong Liu, Xin He, Zhen Liu.

• LADF[Free Offical PDF] [Web][Code]: Local Activity-Driven Structural-Preserving Filtering for Noise Removal and Image Smoothing (Signal Processing 2019), Lijun Zhao, Huihui Bai, Jie Liang, Anhong Wang, Bing Zeng, Yao Zhao.

• TSF[PDF][Code]: Two-stage filtering of compressed depth images with Markov Random Field (Signal Processing: Image Communication 2017), Lijun Zhao, Huihui Bai, Anhong Wang, Bing Zeng, Yao Zhao.

• IRWF[PDF][Code]: Iterative range-domain weighted filter for structural preserving image smoothing and de-noising (Multimedia Tools and Applications 2017), Lijun Zhao, Huihui Bai, Anhong Wang, Bing Zeng, Yao Zhao.

• JIGF[PDF]: Joint iterative guidance filtering for compressed depth images (Visual Communications and Image Processing (VCIP 2016), Lijun Zhao, Huihui Bai, Anhong Wang, Yao Zhao.

• CCF[PDF]: Cluster-based cross-view filtering for compressed multi-view depth maps (Proc. Vis. Commun. Image Process. 2016), Zhen Liu, Qiong Liu, You Yang, Yuchi Liu, Gangyi Jiang, Mei Yu.

• CVBF[PDF][Code]: Candidate value-based boundary filtering for compressed depth images (Electronics Letters 2015), Lijun Zhao, Anhong Wang, Bing Zeng, Yingchun Wu.

• ADTF[PDF][Code]: Adaptive depth truncation filter for MVC based compressed depth image ( Signal Processing: Image Communication 2014), Xuyuan Xu, Lai-Man Po, Terence Chun-Ho, Cheung, Kwok-Wai Cheung, Litong Feng, Chi-Wang Ting, Ka-HoNga.

• DBR[PDF]: Depth boundary reconstruction based on similarity of adjacent pixels for efficient 3-D video coding (IEEE Trans. Consum. Electron. 2013), Donghyun Kim, Seungchul Ryu, Kwanghoon Sohn.

• JTF[PDF]: Joint trilateral filtering for depth map compression (Proc. SPIE 2010), Shujie Liu, PoLin Lai, Dong Tian.

CNN-based Methods

• IFQE[PDF]: Information Fusion based Quality Enhancement for 3D Stereo Images Using CNN (EUSIPCO) 2018), Zhi Jin, Haili Luo, Lei Luo, Wenbin Zou, Xia Lil, Eckehard Steinbach.

• CNNDIF[PDF]: Convolutional Neural Network-based Depth Image Artifact (ICIP 2017), Lijun Zhao, Jie Liang, Huihui Bai, Anhong Wang, Yao Zhao.

• CNNC[PDF]: A CNN cascade for quality enhancement of compressed depth images (IEEE Visual Communications and Image Processing 2017), Zhi Jin, Lei Luo, Yi Tang, Wenbin Zou, Xia Li.