Home

Welcome to the SmileDetection wiki!

Its a Simple Android Project which is compatible till Android Jellybean version or previous as post that there are some changes made in OpenCV library and I couldn't update the openCV jni.

So with previous android version, just load the project in eclipse. You need to enable your Android NDK - http://developer.android.com/tools/sdk/ndk/index.html http://stackoverflow.com/questions/3469448/installing-using-the-android-ndk-in-eclipse

I did everything based on openCV- http://docs.opencv.org/modules/objdetect/doc/cascade_classification.html http://coding-robin.de/2013/07/22/train-your-own-opencv-haar-classifier.html

Setup

The First thing to do for this project is setup openCV based environment in Android. http://opencv.org/platforms/android.html http://docs.opencv.org/doc/tutorials/introduction/android_binary_package/dev_with_OCV_on_Android.html

Steps

A new Android project is being created. As soon as the android project is launched, the first thing getting called in AndroidManifest.xml Internally it then calls the FdActivity.java File as its inherits the Activity. Now once Activity is ready I need to define the way to start the Camera. Opening a Camera .So after Activity basic steps it reaches to onCreate Method-

The basic thing to know that we call requestWindowFeature which “Enable extended window features. This is a convenience for calling getWindow().requestFeature().”

BaseLoaderCallback

Now the main task is to load the OpenCV libraries .The initAsync call is made for Loads and initializes OpenCV library using OpenCV Engine service.

So now as shown above BaseLoaderCallback after loading libraries , we are supposed to enabling the view. So there is a new Api FaceDetectionView to handle it.

http://docs.opencv.org/android/service/doc/BaseLoaderCallback.html http://docs.opencv.org/java/org/opencv/android/BaseLoaderCallback.html

FaceDetection

Let me explain a brief about FaceDetectionView class. This class is a derivative of SampleCvViewBase responsible for creating Camera Instance. https://code.ros.org/trac/opencv/browser/trunk/opencv/samples/android/image-manipulations/src/org/opencv/samples/imagemanipulations/SampleCvViewBase.java?rev=6109

FaceDetectionView is responsible for loading all the classifierFiles.All files are xml based files which is being generated from Training (I already explained last time about this) http://docs.opencv.org/trunk/doc/tutorials/objdetect/cascade_classifier/cascade_classifier.html http://docs.opencv.org/doc/user_guide/ug_traincascade.html

Here we have following scenarios to achieve- • Draw a rectangle around face • Draw eyes on face • Draw Mouth if selected by user • Draw Smile if selected by user So considering above we must have a set of 4 classifier files to do the needful. • haarcascade_frontalface_alt.xml • haarcascade_eye_tree_eyeglasses.xml • haarcascade_mcs_mouth.xml • Smile.xml

Now we got the file , so now task is to load the file so that openCV api could do operations on the same. The constructor is doing the job-

Then once the data in hand we need to pass the same to CasCadeClassifier to do further. http://docs.opencv.org/modules/objdetect/doc/cascade_classification.html

How things are working

BACIS OverView

1. It loads the native OpenCV library so that we can use it using Java API.
2. Create an instance of CascadeClassifier passing it the name of the file from which the classifier is loaded.
3. Next we convert the image to a format which the Java API will accept using the Highui class. Mat is the OpenCV C++ n-dimensional dense array class.
4. Then we call the detectMultiScale method on the classifier passing it the image and MatOfRect object. After processing, the MatOfRect will have face detections.
5. We iterate over all the face detections and mark the image with rectangles.

Above Basic Cases implemented in our case

Once the user click on App and the activity triggers the Camera and even then surfaceChanged is getting called inside FaceDetectionView.

Now the call will come inside processFrame. The processFrame will be passing the VideoCapture as param. http://docs.opencv.org/java/org/opencv/highgui/VideoCapture.html then I need to retrive the frames- what is retrieve? Decodes and returns the grabbed video frame.

The methods/functions decode and return the just grabbed frame. If no frames has been grabbed (camera has been disconnected, or there are no more frames in video file), the methods return false and the functions return NULL pointer.

Note: OpenCV 1.x functions cvRetrieveFrame and cv.RetrieveFrame return image stored inside the video capturing structure. It is not allowed to modify or release the image! You can copy the frame using "cvCloneImage" and then do whatever you want with the copy.

Now the classifier will detect the object based on classifications.

Detects objects of different sizes in the input image. The detected objects are returned as a list of rectangles.

If any faces are recognized as in the logic we can detect Multiple faces as well, the Array will promt with value-

So Now face is found , then now we can check Eyes , mouth or Smile-

For More Details- http://docs.opencv.org/modules/objdetect/doc/cascade_classification.html

C++: void CascadeClassifier::detectMultiScale(const Mat& image, vector& objects, double scaleFactor=1.1, int minNeighbors=3, int flags=0, Size minSize=Size(), Size maxSize=Size()) Python: cv2.CascadeClassifier.detectMultiScale(image[, scaleFactor[, minNeighbors[, flags[, minSize[, maxSize]]]]]) ? objects Python: cv2.CascadeClassifier.detectMultiScale(image, rejectLevels, levelWeights[, scaleFactor[, minNeighbors[, flags[, minSize[, maxSize[, outputRejectLevels]]]]]]) ? objects C: CvSeq* cvHaarDetectObjects(const CvArr* image, CvHaarClassifierCascade* cascade, CvMemStorage* storage, double scale_factor=1.1, int min_neighbors=3, int flags=0, CvSize min_size=cvSize(0,0), CvSize max_size=cvSize(0,0) ) Python: cv.HaarDetectObjects(image, cascade, storage, scale_factor=1.1, min_neighbors=3, flags=0, min_size=(0, 0)) ? detectedObjects

Parameters: cascade – Haar classifier cascade (OpenCV 1.x API only). It can be loaded from XML or YAML file using Load(). When the cascade is not needed anymore, release it using cvReleaseHaarClassifierCascade(&cascade). image – Matrix of the type CV_8U containing an image where objects are detected. objects – Vector of rectangles where each rectangle contains the detected object. scaleFactor – Parameter specifying how much the image size is reduced at each image scale. minNeighbors – Parameter specifying how many neighbors each candidate rectangle should have to retain it. flags – Parameter with the same meaning for an old cascade as in the function cvHaarDetectObjects. It is not used for a new cascade. minSize – Minimum possible object size. Objects smaller than that are ignored. maxSize – Maximum possible object size. Objects larger than that are ignored. The function is parallelized with the TBB library.

Note (Python) A face detection example using cascade classifiers can be found at opencv_source_code/samples/python2/facedetect.py