We detect and track faces in video, then extract features from those face tracks and try to cluster them into given number of Clusters, each Cluster representing a unique person. To get a full idea of our project, refer to the file "ppt_explaining_the_project.pdf". Results were obtained on video: https://youtu.be/A1fVcj29xhk
Running the project on a video-file involves 4 steps:
- Shot Detection:
We need the information of shots from the video. The shot-detection is done using Python.
Application used is ShotDetect by John Mathe: https://github.com/johmathe/Shotdetect ReadMe on this GitHub page describes how to use the app via command line.
This app is available for Ubuntu: sudo apt-get install ShotDetect
After running app on your video, a folder will be generated. Paste the "xml_parser.py" file (available here) in that folder and run it.
A file by name "shot_info.txt" will be generated. This file contains shot information in a way understandable by our Face Tracking code. Copy this file and paste it at the location of "main_for_shots.cpp"
- Detection and Tracking:
You need to build an OpenCV project, with Dlib Library for Facial Landmark Points Detection, available at: https://sourceforge.net/projects/dclib/files/dlib/v18.10/dlib-18.10.tar.bz2/download
We need the DLib Detector to return the Landmark Points as vector of Points. For this, replace the "full_object_detection.h" file of Dlib Library with the one given here.
Now add the "main_for_shots.cpp" file in the OpenCV project with DLib. Add your video and its "shot_info.txt" generated in step-1 at the cpp file's location.
- Feature Extraction:
Copy the "data" folder created in Step-2, and paste it at the location of the Feature Extraction cpp files. Run the .cpp files then.
- Clustering:
Is done in Python. Copy the folders generated in step3 and paste them where the Python files are.
If features are generated using "avg_img_features.cpp", run "Clustering_basic_feats.py" results will be stored in "resultsAVG/".
If features are generated using "3D_track_features.cpp", run "Clustering_adv_feats.py" results will be stored in "results/".