I have built this using a model with two microservices: a Flask server for backend operations and a Streamlit server for frontend operations. The Flask server handles data processing and sentiment analysis, while the Streamlit server provides a user-friendly interface for interacting with the application.
- Install Dependencies :
pip install -r requirements.txt
- Start the Server :
streamlit run index.py
-
User Input : The user provides a Twitter username through the Streamlit interface.
-
Flask Endpoint : The username is sent to the Flask server endpoint
/get_sentimentanalysis_overall
. -
Scraper Module : The Flask server calls the
scraper.main(username)
function to scrape the latest tweets from the provided username. -
Data Processing : The scraped tweets are processed using various utility functions in
utils.py
. -
Sentiment Analysis : The processed tweets are analyzed for sentiment using a pre-trained model.
-
Response : The average sentiment score and corresponding sentiment label (Positive, Neutral, Negative) are returned to the Streamlit interface.
Endpoints :
-
GET
/
: Returns the homepage. -
POST
/get_sentimentanalysis_overall
: Accepts a Twitter username and returns the average sentiment score and label.
-
server.py : Main entry point for the Flask server.
-
utils.py : Contains utility functions for data cleaning and sentiment analysis.
-
scraper.py : Handles scraping tweets from Twitter.
-
valuetofeeling(i)
: Converts a sentiment score to a textual representation. -
cleanREGEX(raw)
: Cleans raw text using regex to remove HTML tags and special characters. -
deEmojify(x)
: Removes emojis from the text. -
remove_punct(text)
: Removes punctuation and numbers from the text. -
lower_case(df)
: Converts text to lowercase. -
cleaner(df)
: Applies a series of cleaning functions to the dataframe. -
sentimentanalyze(tweet)
: Analyzes the sentiment of a tweet. -
load_saved_artifacts()
: Loads the pre-trained model and vectorizer from disk.