Head Pose estimation via dlib models and solvepnp

Download models

• download following models (cnn_face_detector, landmark_v1, landmark_v2)
• convert to .dat format
  • bzip2 -d FileName.bz2

Models

• Dlib mmod cnn based face detector cnn_face_detector
  • note: This model is slow using CPU but mobile gpu like NVIDIA GeForce MX250 run smooth enough in webcam
• The well trained dlib landmark 68 model cam be downloaded from landamrk_v1
  • regression tree
• Robust landmark model use cnn based face detector for large head pose variation: landmark_v2
  • regression tree

Demo

Face Detection:

• compare hog+svm(blue) and cnn(green)

python viaualize_fd.py

Landmark Estimation

• compare hog+landmark_v1(blue) and cnn+landmark_v2(green)

python visualize_lm.py

Canonical Face 3d

• visualize Canonical Face Mdoel in World Coord.

python visualize_face_in_world.py

• 58 pts only

Head Pose Estimation

• visualize headpose estimation in pixel coord.

python visualize_hp.py

• pitch bug

TODO:

• dlib_frontal_face_detection(HOG) + dlib_landmark(landmark_v1)
• dlib_cnn_face_detection(CNN) + dlib_landmark(landmark_v2)
• visualize canonical face model in world coordinate system
• head pose estimation with solvePnP
  • axis visualization
  • cube visualization
• visualize canonical face model in camera coordinate system
• camera intrinsic calibration

Reference

• dlib-project
• dlib-models
  • landmark_v1: shape_predictor_68_face_landmarks.dat.bz2
  • landmark_v2: shape_predictor_68_face_landmarks_GTX.dat.bz2
  • cnn_face_detector: mmod_human_face_detector.dat.bz2
• head-pose-estimation
  • python dlib get headpose example

Reference Papers

• dlib_cnn_face_detector
  • paper: Max-Margin Object Detection(mmod)
• dlib_landmark_v1
  • paper: One Millisecond Face Alignment with an Ensemble of Regression Trees
  • dlib hog face detector only
• dlib_landmark_v2
  • paper: Real-time face alignment: evaluation methods, training strategies and implementation optimization
  • dlib CNN/hog face detector