Skip to content

lcybuzz/Personal-Face-Related-Paper-Record

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

61 Commits
README.md
README.md
 
 

Repository files navigation

Face Related Paper Record

Under construction！

Table of Contents

Rank

  • Face Parsing
    • ★★★
    • ★★
      [MO-GC], [Guided by Detected]
    • ★
      [CnnRnnGan], [RNN-G], [FC-CNN], [Adaptive Receptive Fields], [Exemplar-Based], [RED]
  • Face Detection
    • ★★★
    • ★★
      [MTCNN], [SSH]
    • ★
  • Landmark Detection
    • ★★★
    • ★★
      [DAN], [LAB]
    • ★

Face Parsing

Deep Learning Methods

Hierarchical face parsing via deep learning

[Paper] Hierarchical face parsing via deep learning
[Year] CVPR 2012
[Author] Ping Luo, Xiaogang Wang, Xiaoou Tang
[Pages]
[Description]

MO-GC ★★

[Paper] Multi-Objective Convolutional Learning for Face Labeling
[Year] CVPR 2015
[Author] Sifei Liu, Jimei Yang, Chang Huang, Ming-Hsuan Yang
[Pages] https://www.sifeiliu.net/face-parsing
[Description]

  1. 模拟CRF, 提出一种用多个目标函数优化一个CNN的人脸解析方法. 一个loss针对unary label likehood, 一个loss针对pairwise label dependency;
  2. 提出一种nonparametric prior作为global regularization. 首先在脸部key point真值图像块上基于PCA建立一形状子空间, 测试时搜索与测试图像最相似的若干真值图像, 根据key point将真值图像与测试图像align，将几张aligned后的mask取平均作为prior;
  3. 在LFW和Helen上实验, 多目标函数的策略对精度有微小提升, nonparametric prior效果提升明显：

Guided by Detected ★★

[Paper] A CNN Cascade for Landmark Guided Semantic Part Segmentation
[Year] ECCV 2016 workshop
[Author] Aaron S. Jackson, Michel Valstar, Georgios Tzimiropoulos
[Pages] http://aaronsplace.co.uk/papers/jackson2016guided/index.html
[Description]

  1. 提出一种用landmarks引导part segmentation的方法, 用pose-specific信息辅助分割, 分为landmark检测和分割两步;
  2. landmark detection: 先用一个FCN预测68个高斯状的landmarks(68个输出channel,每个channel对应1个2D Gaussian)
  3. segmentation: 将detection得到的68个channel加到输入图像上, 再用1个FCN完成分割. 这个的一个key aspect是验证集上的landmark localization error加到landmark真值上去生成2D Gaussian (没看懂他的理由???)
  4. 实验部分用IoU评价, 但是没与其它方法对比, 说服力略显不足; 数据是自行从landmark数据集中生成的分割图.

CnnRnnGan ★

[Paper] End-to-end semantic face segmentation with conditional random fields as convolutional, recurrent and adversarial networks
[Year] arXiv 1703
[Author] Umut Güçlü, Yagmur Güçlütürk, Meysam Madadi, Sergio Escalera, Xavier Baró, Jordi González, Rob van Lier, Marcel van Gerven
[Pages] https://github.com/umuguc (还没开源)
[Description]

  1. 大致浏览. 本文提出了一个大杂烩, 将dilation, CRFasRNN, adversarial training整合到一end to end的框架中. 不过, 首先要检测landmark, 将landmark连接生成初始分割图, 再用landmark将输入图像和分割图与模板对齐.
  2. 效果较好, 但暂时未开源.
  3. 有一个问题没细看: 在Helen上实验时, 是分别训练了5个网络解析不同类别吗??

RNN-G ★☆

[Paper] Parsing via Recurrent Propagation
[Year] BMVC 2017
[Author] Sifei Liu, Jianping Shi, Ji Liang, Ming-Hsuan Yang
[Pages]
[Description]

  1. 提出一种coarse to fine的人脸解析方法, 第一阶段解析出皮肤, 头发和背景, 第二部解析facial components. CNN和RNN参数都较少, 因此速度很快.
  2. 第一阶段将CNN的hierarchical representation和RNN的label propagation结合起来. CNN有两个输出, 一个是feature map, 另一个是edge map. RNN考虑上下左右四个方向, 以feature map为输入, 并用edge map作为gate, 即边缘处两个node的联系应该小, 相同类别区域两个node联系应该大.
  3. 第二个阶段设计了eye/eyebrow, nose和mouth三个子网络, 根据landmark将五官crop成patch, 送入相应的网络进行解析.
  4. 本文也需要额外的landmark检测, 检测出的landmard用于将脸转正和crop五官.

FC-CNN ★☆

[Paper] Face Parsing via a Fully-Convolutional Continuous CRF Neural Network
[Year] arXiv 1708
[Author] Lei Zhou, Zhi Liu, Xiangjian He
[Pages]
[Description]

  1. 将CRF与CNN结合起来, CRF的思路应该是来源于MO-GC, 模型包括unary, pairwise和continuous CRF(C-CRF)三个子网络; 网络基于Caffe, 可以端到端训练. 未开源,性能较好.
  2. Unary net采用类似SegNet的结构. pairwise net将相邻像素的feature连接起来并用12和21的卷积得到其水平和垂直方向的相似的, 最后得到相似度矩阵.
  3. C-CRF网络首先用superpixel pooling layer将unary和pairwise网络的pixel-level feaature转化为region-level feature. 目的是保留边界信息和保证同区域标注的一致性(?). 再使用unary和pairwise的超像素特征构成目标能量函数.
  4. 介绍了一种端到端训练C-CRF的方法, 没细看.
  5. 貌似应该需要额外的方法得到超像素.

Adaptive Receptive Fields ★

[Paper] Learning Adaptive Receptive Fields for Deep Image Parsing Network
[Year] CVPR 2017
[Author]   Zhen Wei, Yao Sun, Jinqiao Wang, Hanjiang Lai, Si Liu
[Pages]
[Description]

  1. 提出学习一个参数f, 对feature map进行缩放, 从而自适应地改变感受野大小.
  2. 设计一个multi-path模型, 为打破各支路的均衡性, 使用了loss guidance, 即对某一支加大某些类的权重, 如把类别分为{eye, eyebrow}和{nose, lip, mouth}两组, 用起分别对不同支路加权. 这样能引导各个分支学习到适合分割特定目标的感受野.
  3. loss guidance的思路可以借鉴, 但从结果来看多个支路的精度反而不如单支路的...
  4. 在一个数据集学到的参数f, 应该是只适应于当前任务, 感觉不太适用于模型迁移?

RED ★

[Paper] Residual Encoder Decoder Network and Adaptive Prior for Face Parsing
[Year] AAAI 2018
[Author]   Tianchu Guo, Youngsung Kim, Hui Zhang, Deheng Qian, ByungIn Yoo, Jingtao Xu, Dongqing Zou, Jae-Joon Han, Changkyu Choi
[Pages]
[Description]

  1. 基于DeconvNet网络结构的人脸解析方法, online计算先验.

Classical Methods

Exemplar-Based ★

[Paper] Exemplar-Based Face Parsing
[Year] CVPR 2013
[Author] Brandon M. Smith, Li Zhang, Jonathan Brandt, Zhe Lin, Jianchao Yang
[Pages] http://pages.cs.wisc.edu/~lizhang/projects/face-parsing/
[Description]

  1. 粗读, 基于exemplar的人脸解析. 提供了一个基于Helen的人脸解析数据集

Face Detection

Cascade CNN

[Paper] A Convolutional Neural Network Cascade for Face Detection
[Year] CVPR 2015
[Author] Haoxiang Li, Zhe Lin, Xiaohui Shen, Jonathan Brandt, Gang Hua
[Pages] https://github.com/anson0910/CNN_face_detection (Unofficial)
[Description]

Faceness-Net

[Paper] From Facial Parts Responses to Face Detection: A Deep Learning Approach
[Year] ICCV 2015
[Author] Shuo Yang, Ping Luo, Chen Change Loy, Xiaoou Tang
[Pages] http://shuoyang1213.me/projects/Faceness/Faceness.html
[Description]

MTCNN ★★

[Paper] Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks
[Year] SPL 2016
[Author] Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, Yu Qiao
[Pages] https://kpzhang93.github.io/MTCNN_face_detection_alignment/
[Description]

  1. 以3个CNN级联的方式，完成coarse到fine的人脸检测和对齐;
  2. 三个网络分别为Prposal(P)-Net, Refine(R)-Net和Output(O)-Net, 三个网络都是结构相似的小型CNN，总体速度较快;
  3. 网络的训练包括三个task: 人脸分类(是否是人脸的二分类问题), bounding box回归, landmark定位. 三个任务是分别取样本和训练的;
  4. 提出online hard sampling mining, 在一个mini-batch中对每个sample的loss排序, 只取loss由大到小前70%的sample参与back propagation

SSH ★★

[Paper] SSH: Single Stage Headless Face Detector
[Year] ICCV 2017
[Author] Mahyar Najibi, Pouya Samangouei, Rama Chellappa, Larry S. Davis
[Pages] https://github.com/mahyarnajibi/SSH
[Description]

  1. Single stage, no head of classification network
  2. Scale-invariant by design, detect faces from various depths

Finding Tiny Faces

[Paper] Finding Tiny Faces
[Year] CVPR 2017
[Author] Peiyun Hu, Deva Ramanan
[Pages] http://www.cs.cmu.edu/~peiyunh/tiny/index.html
[Description]

Landmark Detection

CNN_FacePoint

[Paper] Deep Convolutional Network Cascade for Facial Point Detection
[Year] CVPR 2013
[Author] Yi Sun, Xiaogang Wang, Xiaoou Tang
[Pages] http://mmlab.ie.cuhk.edu.hk/archive/CNN_FacePoint.htm
[Description]

TCDCN

[Paper] Facial Landmark Detection by Deep Multi-task Learning
[Year] ECCV 2014
[Author] Zhanpeng Zhang, Ping Luo, Chen Change Loy, Xiaoou Tang
[Pages] http://mmlab.ie.cuhk.edu.hk/projects/TCDCN.html
[Description]

3DDFA

[Paper] Face Alignment Across Large Poses: A 3D Solution
[Year] CVPR 2016
[Author] Hai Zhu, Lei Zhen, Xiaoming Liu, Hailin Shi, Stan Z Li
[Pages] http://www.cbsr.ia.ac.cn/users/xiangyuzhu/projects/3DDFA/main.htm
[Description]

DAN ★★

[Paper] Deep Alignment Network: A convolutional neural network for robust face alignment
[Year] CVPRW 2017
[Author] Marek Kowalski, Jacek Naruniec, Tomasz Trzcinski
[Pages]
https://github.com/MarekKowalski/DeepAlignmentNetwork (Official, Theano)
https://github.com/zjjMaiMai/Deep-Alignment-Network-A-convolutional-neural-network-for-robust-face-alignment (TF)
https://github.com/mariolew/Deep-Alignment-Network-tensorflow (TF)
[Description]

  1. 级联的人脸对齐方法, 思路简洁, 速度快.
  2. 每个stage的输入由三部分组成: 输入图像, 上一个stage的landmark heatmap和feature, 值得注意的是, 算法使用上阶段的landmark和标准脸计算变换矩阵, 并对三个输入做了对齐, 这使得算法对大姿态人脸有较好的鲁棒性.

LAB ★★

[Paper] Look at Boundary: A Boundary-Aware Face Alignment Algorithm
[Year] CVPR 2018
[Author] Wayne Wu, Chen Qian, Shuo Yang, Quan Wang, Yici Cai, Qiang Zhou
[Pages] https://wywu.github.io/projects/LAB/LAB.html
[Description]

  1. 利用边界信息和对抗训练做人脸关键点检测, 效果很好, 对大姿态表情鲁棒性较强.
  2. 模型由Boundary-Aware关键点回归, Boundary Heatmap预测和Landmark-Based Boundary Effectiveness Discriminator三部分组成. 关键点回归采用多级boundary heatmap融合策略; 边界heatmap预测的是像素到边界的距离, 并使用了message passing机制; 对抗训练部分使用一判别器判断boundary heatmap的有效性, 并为整个网络引入一对抗loss
  3. 开源了基于caffe的测试代码. 感觉方法的训练包括很多细节, 复现性能可能有些难度.

Others

CBN ★

[Paper] Deep Cascaded Bi-Network for Face Hallucination
[Year] ECCV 2016
[Author] Shizhan Zhu, Sifei Liu, Chen Change Loy, Xiaoou Tang
[Pages] https://github.com/zhusz/ECCV16-CBN
[Description]

  1. 大致浏览, 人脸增强类似于超分辨率重建, 但利用了人脸的结构信息. 设计了一gated deep bi-network, 一个分支是common branch即普通的残差预测网络, 一个分支是high-frequency branch利用dense field得到spatial cues. 若干个bi-network串接起来, 形成一逐层上采样的网络结构.
  2. dense field之前没接触过, 可以了解一下

Applications

Face Swapping

Face Swapping

[Paper] On Face Segmentation, Face Swapping, and Face Perception
[Year] FG 2018
[Author] Yuval Nirkin, Iacopo Masi, Anh Tuan Tran, Tal Hassner, Gerard Medioni
[Pages] https://github.com/YuvalNirkin/face_swap
[Description]

3D

3DMM

[Paper] Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network
[Year] CVPR 2017
[Author] Anh Tuan Tran, Tal Hassner, Iacopo Masi, Iacopo Masi
[Pages]
https://www.openu.ac.il/home/hassner/projects/CNN3DMM/
https://github.com/anhttran/3dmm_cnn
[Description]

Datasets

Segments

Helen http://pages.cs.wisc.edu/~lizhang/projects/face-parsing/
LFW http://vis-www.cs.umass.edu/lfw/part_labels/
Mut1ny http://www.mut1ny.com/face-headsegmentation-dataset

Landmarks

LFW http://vis-www.cs.umass.edu/lfw/
Helen http://www.ifp.illinois.edu/~vuongle2/helen/
LFPW https://neerajkumar.org/databases/base/databases/lfpw/
WIDER http://mmlab.ie.cuhk.edu.hk/projects/WIDERFace/
http://www.face-rec.org/databases/
WFLW https://wywu.github.io/projects/LAB/WFLW.html (96 points)
ICME2019 https://facial-landmarks-localization-challenge.github.io/ (106 points)

Librarys

libfacedetection https://github.com/ShiqiYu/libfacedetection

Resources-Lists

https://blog.csdn.net/chenriwei2/article/details/50631212
https://github.com/betars/Face-Resources
https://blog.csdn.net/u011995719/article/details/78890333
https://www.jianshu.com/p/e4b9317a817f

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

7 stars

Watchers

0 watching

Forks

3 forks
Report repository

Releases

No releases published

Packages

No packages published