Interactive Tweet Sentiment Visualization Dashboard with Python

Social media has become a ubiquitous part of modern life, with millions of people sharing their thoughts and opinions online every day. For companies, organizations and individuals alike, this represents a treasure trove of data that can provide valuable insights – if you have the tools to make sense of it.

That‘s where sentiment analysis comes in. Sentiment analysis is a natural language processing (NLP) technique that aims to determine the emotion or opinion expressed in a piece of text. By analyzing the words and phrases used in social media posts, we can get an aggregated view of public sentiment about any topic, brand, or event.

In this post, we‘ll walk through how to create an interactive dashboard for visualizing sentiment analysis of tweets using the Python data science and machine learning ecosystem. We‘ll take you step-by-step from data collection through processing, modeling, and visualizing the results in a web app. Whether you‘re a data scientist, software engineer, or business analyst, this project will give you hands-on experience with some of the most popular and powerful open-source tools for NLP and data visualization.

Collecting and Preprocessing Tweet Data

The first step in any data science project is getting data. For analyzing tweet sentiment, we need a dataset of tweets, ideally already labeled with sentiment annotations (positive, negative, or neutral). There are a few different ways to get tweet data:

• Use the Twitter API to collect tweets matching certain keywords or hashtags
• Find an existing labeled tweet dataset on Kaggle or other public repositories
• Manually annotate tweets yourself or using a crowdsourcing platform

For this example, let‘s use the popular "Sentiment140" dataset, which contains 1.6 million tweets collected and labeled using emoticons as noisy sentiment labels. The data is available on Kaggle.

Once we have our tweet dataset loaded into a pandas DataFrame, we need to clean and preprocess the text to get it ready for analysis. Tweets are very noisy, containing lots of URLs, hashtags, mentions, emojis and misspellings. We‘ll use some simple regular expressions to filter out non-text content:

import re

def preprocess_tweet(text):
    # Remove URLs
    text = re.sub(r‘http\S+‘, ‘‘, text)
    # Remove mentions  
    text = re.sub(r‘@\w+‘, ‘‘, text)
    # Remove hashtags
    text = re.sub(r‘#\w+‘, ‘‘, text)
    # Remove emojis
    emoji_pattern = re.compile("["
                               u"\U0001F600-\U0001F64F"  # emoticons
                               u"\U0001F300-\U0001F5FF"  # symbols & pictographs
                               u"\U0001F680-\U0001F6FF"  # transport & map symbols
                               u"\U0001F1E0-\U0001F1FF"  # flags (iOS)
                               "]+", flags=re.UNICODE)
    text = emoji_pattern.sub(r‘‘, text)

    return text

df[‘text‘] = df[‘text‘].apply(preprocess_tweet)

Next we‘ll tokenize the tweets by splitting on whitespace and punctuation to get a list of words for each tweet. We‘ll also remove common stopwords like "the" and "and" that don‘t convey much meaning.

from nltk.tokenize import TweetTokenizer
from nltk.corpus import stopwords

tokenizer = TweetTokenizer()
stopwords_en = stopwords.words(‘english‘)

def tokenize(text):
    tokens = tokenizer.tokenize(text)
    return [t for t in tokens if t.lower() not in stopwords_en]

df[‘tokens‘] = df[‘text‘].apply(tokenize)    

Training a Sentiment Classifier

Now that we have clean, tokenized tweet text, we‘re ready to train a machine learning model to predict the sentiment of each tweet. Since the Sentiment140 data already contains sentiment labels, we can use it to train a supervised classification model.

There are many different approaches to sentiment classification, from simple rules-based methods to advanced deep learning architectures. For this example, we‘ll keep things relatively simple and use a Naive Bayes model, which is a probabilistic classifier that works well for NLP tasks.

We‘ll use scikit-learn to vectorize the tweet tokens into features and train the model:

from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import Pipeline

vectorizer = CountVectorizer(tokenizer=lambda x: x, lowercase=False)

model = Pipeline([
    (‘vectorizer‘, vectorizer),
    (‘classifier‘, MultinomialNB())
])

model.fit(df[‘tokens‘], df[‘sentiment‘])

We can evaluate the model‘s performance on a held-out test set to see how accurately it predicts tweet sentiment:

from sklearn.metrics import classification_report

def evaluate_model(model, test_texts, test_labels):
    preds = model.predict(test_texts)
    print(classification_report(test_labels, preds))

evaluate_model(model, test_df[‘tokens‘], test_df[‘sentiment‘])  

              precision    recall  f1-score   support

    negative       0.75      0.78      0.77     18768
     neutral       0.53      0.48      0.50     21273  
    positive       0.77      0.79      0.78     31959

    accuracy                           0.70     72000
   macro avg       0.68      0.68      0.68     72000
weighted avg       0.70      0.70      0.70     72000

Our baseline Naive Bayes model achieves 70% accuracy, with slightly higher precision and recall for the positive and negative classes compared to neutral. Not bad for a first pass, but there‘s certainly room for improvement. More advanced models like LSTMs or BERT could likely capture the nuances of language better and push performance into the 80-90% range.

Building the Visualization Dashboard

With our trained sentiment model in hand, we‘re ready to build an interactive visualization of the results. We‘ll create a web app using Streamlit, a Python framework for quickly building interactive data science tools.

First we‘ll define the various charts and widgets we want to display. Some key visualizations:

• High-level sentiment breakdown: what % of tweets are positive, negative, or neutral?
• Sentiment over time: how have the relative proportions of each sentiment changed?
• Top words and bigrams for each sentiment
• Sentiment by user, hashtag, or other metadata
• Drilling down into example tweets for each sentiment

Here‘s some sample code to generate the charts with Plotly:

import plotly.express as px

# Sentiment breakdown pie chart
sentiment_counts = df[‘sentiment‘].value_counts()
fig = px.pie(names=sentiment_counts.index, values=sentiment_counts.values, 
             title=‘Tweets by Sentiment‘)
st.plotly_chart(fig)

# Sentiment over time
df[‘date‘] = pd.to_datetime(df[‘date‘])
sentiment_time = df.groupby([df[‘date‘].dt.date, ‘sentiment‘]).size().unstack(fill_value=0)
fig = px.line(sentiment_time, x=sentiment_time.index, y=sentiment_time.columns,
              title=‘Tweet Sentiment Over Time‘)  
st.plotly_chart(fig)

# Top words  
from wordcloud import WordCloud

stopwords = set(stopwords.words(‘english‘))
for sentiment in [‘negative‘, ‘neutral‘, ‘positive‘]:
    text = ‘ ‘.join(df[df.sentiment==sentiment][‘tokens‘].sum())
    wordcloud = WordCloud(stopwords=stopwords, background_color=‘white‘).generate(text)

    st.write(f‘Top words for {sentiment} sentiment:‘)
    fig, ax = plt.subplots()
    ax.imshow(wordcloud, interpolation=‘bilinear‘)
    ax.axis(‘off‘)
    st.pyplot(fig)

We can make the dashboard interactive by adding widgets like checkboxes and dropdowns that let the user slice and filter the data:

# Sentiment filter  
sentiment_filter = st.sidebar.multiselect(
    ‘Select sentiment(s)‘, 
    options=df[‘sentiment‘].unique(),
    default=df[‘sentiment‘].unique()
)

# Date range filter  
start_date = st.sidebar.date_input(‘Start Date‘, value=df[‘date‘].min())
end_date = st.sidebar.date_input(‘End Date‘, value=df[‘date‘].max())

# Filter data based on selections
filtered_df = df[(df.sentiment.isin(sentiment_filter)) & 
                 (df.date >= pd.Timestamp(start_date)) &
                 (df.date <= pd.Timestamp(end_date))]

Finally, we‘ll organize everything into a clean multi-page app layout and deploy it with Streamlit Sharing or another cloud hosting service.

Conclusion

In this post, we walked through the end-to-end workflow of analyzing sentiment of tweets and building an interactive dashboard to visualize the results. This demonstrates the power of combining NLP and data visualization techniques to extract insights from large amounts of unstructured text data.

Some key takeaways and areas for further exploration:

• Sentiment analysis is a useful technique for gauging public opinion on social media, but it‘s not foolproof. Sarcasm, slang, and subtle context are still hard for models to detect. More advanced deep learning approaches can help, but there will always be some inherent ambiguity.
• Interactive visualization helps bring the data to life and allows for open-ended exploration. Tools like Plotly and Streamlit make it easy to quickly build professional-grade dashboards.
• There are many ways this analysis could be extended: comparing sentiment between different user segments, identifying sentiment towards specific topics or entities, building a real-time pipeline to continually ingest new tweets, etc.

With the massive volume of social media data available, sentiment analysis visualizations will only become more important tools for helping us understand public discourse and make data-driven decisions. Hopefully this example has inspired you to apply these techniques to text data that you‘re passionate about!