The purpose of the project is to develop a mobile app that enables hands-free interaction.
Hands-free 2048 combines two hands-free interactions, sensor-based interaction and gaze-based interaction with game 2048.
People can play 2048 by tilting their phone and eye-tracking. For example, tilt the phone to the left to move the numbered tiles to left.
Move the numbered tiles to the right by looking to the right.
The eye-tracking mode depends on two open source libraries:
- OpenCV (OpenCV-4.4.0) https://opencv.org/
- Dlib (Dlib 19.21) http://dlib.net/
- Compress Dlib https://github.com/miaoerduo/dlib-face-landmark-compression
The application includes two hands-free interaction - tilt mode and eye-tracking mode.
- Tilt mode
The tilt mode derives data from the accelerometer and magnetometer, computes the orientation angles of the phone in XYZ axes
When the rotation angle of the mobile phone in a certain axis reaches the set threshold, the corresponding operation is triggered.
- Eye-tracking mode
The eye tracking mode uses haar cascade classifier in OpenCV to detect human face, uses Dlib to align landmarks and extract eye images.
Camera captures real-time eye pictures, compare it with templates to judge which direction the user is looking
- First clone the repository.
- Before importing the project into Android Studio, you need to:
- Open the
app/src/main/cpp/CMakeLists.txt - Then, replace with your path the variable "OpenCV-android-sdk"
- Open the
- Now the project is ready to be run on your Android Studio.