EasyText - Chat Application

Description and Justification

The AI features chosen for my chat application suport facial recognition, message generation, and voice-to-text transcription. Sometimes, things can get lost in translation (textlation) as it can be hard to depict another person's emotions or convey your own through text. With EasyText, users can hopefully overcome this hurdle. These enhancements aim to create a more intuitive and human-like interaction within conversations, bridging the gap between textual communication and emotional intent.

Facial Emotion Recognition - when it is challenging to convey your emotions

This feature analyzes the user's facial expressions via their webcam to determine their prevailing emotional state. The application then uses this data along with the previous messages to make an API call to OpenAI to generate a context-aware response by pretending to be the user.

Generated Responses - when it is challenging to know what to say to others

This feature allows the user to single out someone in the chat room, selected a certain number of text messages for context, and with an API call to OpenAI, a message is written for their selected target.

Voice to Text - when it can be bothersome to type how you feel

This feature allows the user to record themselves. This is temporarily turned to an audio file and with an API call to OpenAI's Whisper model, the audio is transcribed to text.

Documentation

User Manual

Enabling Facial Emotion Recognition
- Navigate to a chat room
- Click on the "Analyze Emotion" button
- Allow the app to access your webcam
- A video will appear below. Please ensure your face is visible to the webcam
- After a couple of seconds, your face will be tracked and your emotions will be detected
- Click on the video and a response will be generated for you

Enabling Generated Responses
- Navigate to a chat room
- Click on the "Generate Response" button
- Select the user you want to write the message to in the pop-up
- Select the number of previous messages in the pop-up
- Click "Done" and a response will be generated for you

Turning Voice to Text
- Navigate to a chat room
- Click on the "Record Voice" button
- Allow the app to access your microphone
- Speak into your microphone
- Click "Stop Recording" when you are finished speaking and a response will be generated for you

Design and Implementation

AI Model Choice
The integration utilizes the face-api.js library for facial emotion recognition along with OpenAI for generating responses and voice-to-text transcription.

Setup and Running Guide

Dependencies Required

Here is a list of dependencies along with their versions:
"dotenv": "^16.4.5",
"express": "^4.18.2",
"express-fileupload": "^1.5.0", "face-api.js": "^0.22.2",
"mongodb": "^6.4.0",
"openai": "^4.33.0",
"ws": "^8.16.0"
This list can be found inside package.json. Please ensure MongoDB is running on mongodb://127.0.0.1:27017

API Keys

There is one API Key for OpenAI. You will need your own if you want to use the AI features. Please create a .env file in the project's root directory and paste OPENAI_API_KEY="your API key" within it.

Running the Application

- Clone the repository
- Install dependencies in the root directory of the project: npm install
- Create a .env folder in the root directory of the project
- Set the OpenAI API Key in .env as OPENAI_API_KEY="your API key" (replace with actual key)
- Run the command npx webpack --config webpack.config.js if changes made to app.js
- Start the server by typing node server in your console
- Open the client application in your browser (http://localhost:3000/)