822 lines
26 KiB
Plaintext
822 lines
26 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"collapsed": true
|
|
},
|
|
"source": [
|
|
"___\n",
|
|
"\n",
|
|
"<a href='http://www.pieriandata.com'> <img src='../Pierian_Data_Logo.png' /></a>\n",
|
|
"___"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Sentiment Analysis\n",
|
|
"Now that we've seen word vectors we can start to investigate sentiment analysis. The goal is to find commonalities between documents, with the understanding that similarly *combined* vectors should correspond to similar sentiments.\n",
|
|
"\n",
|
|
"While the scope of sentiment analysis is very broad, we will focus our work in two ways.\n",
|
|
"\n",
|
|
"### 1. Polarity classification\n",
|
|
"We won't try to determine if a sentence is objective or subjective, fact or opinion. Rather, we care only if the text expresses a *positive*, *negative* or *neutral* opinion.\n",
|
|
"### 2. Document level scope\n",
|
|
"We'll also try to aggregate all of the sentences in a document or paragraph, to arrive at an overall opinion.\n",
|
|
"### 3. Coarse analysis\n",
|
|
"We won't try to perform a fine-grained analysis that would determine the degree of positivity/negativity. That is, we're not trying to guess how many stars a reviewer awarded, just whether the review was positive or negative."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Broad Steps:\n",
|
|
"* First, consider the text being analyzed. A model trained on paragraph-long movie reviews might not be effective on tweets. Make sure to use an appropriate model for the task at hand.\n",
|
|
"* Next, decide the type of analysis to perform. In the previous section on text classification we used a bag-of-words technique that considered only single tokens, or *unigrams*. Some rudimentary sentiment analysis models go one step further, and consider two-word combinations, or *bigrams*. In this section, we'd like to work with complete sentences, and for this we're going to import a trained NLTK lexicon called *VADER*."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## NLTK's VADER module\n",
|
|
"VADER is an NLTK module that provides sentiment scores based on words used (\"completely\" boosts a score, while \"slightly\" reduces it), on capitalization & punctuation (\"GREAT!!!\" is stronger than \"great.\"), and negations (words like \"isn't\" and \"doesn't\" affect the outcome).\n",
|
|
"<br>To view the source code visit https://www.nltk.org/_modules/nltk/sentiment/vader.html"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Download the VADER lexicon.** You only need to do this once."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[nltk_data] Downloading package vader_lexicon to\n",
|
|
"[nltk_data] C:\\Users\\Mike\\AppData\\Roaming\\nltk_data...\n",
|
|
"[nltk_data] Package vader_lexicon is already up-to-date!\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"True"
|
|
]
|
|
},
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"import nltk\n",
|
|
"nltk.download('vader_lexicon')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"<div class=\"alert alert-danger\">NOTE: At the time of this writing there's a <a href='https://github.com/nltk/nltk/issues/2053'>known issue</a> with SentimentIntensityAnalyzer that raises a harmless warning on loading<br>\n",
|
|
"<tt><font color=black> UserWarning: The twython library has not been installed.<br> Some functionality from the twitter package will not be available.</tt>\n",
|
|
"\n",
|
|
"This is due to be fixed in an upcoming NLTK release. For now, if you want to avoid it you can (optionally) install the NLTK twitter library with<br>\n",
|
|
"<tt><font color=black> conda install nltk[twitter]</tt><br>or<br>\n",
|
|
"<tt><font color=black> pip3 install -U nltk[twitter]</tt></div>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from nltk.sentiment.vader import SentimentIntensityAnalyzer\n",
|
|
"\n",
|
|
"sid = SentimentIntensityAnalyzer()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"VADER's `SentimentIntensityAnalyzer()` takes in a string and returns a dictionary of scores in each of four categories:\n",
|
|
"* negative\n",
|
|
"* neutral\n",
|
|
"* positive\n",
|
|
"* compound *(computed by normalizing the scores above)*"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"{'neg': 0.0, 'neu': 0.508, 'pos': 0.492, 'compound': 0.4404}"
|
|
]
|
|
},
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"a = 'This was a good movie.'\n",
|
|
"sid.polarity_scores(a)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"{'neg': 0.0, 'neu': 0.425, 'pos': 0.575, 'compound': 0.8877}"
|
|
]
|
|
},
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"a = 'This was the best, most awesome movie EVER MADE!!!'\n",
|
|
"sid.polarity_scores(a)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"{'neg': 0.477, 'neu': 0.523, 'pos': 0.0, 'compound': -0.8074}"
|
|
]
|
|
},
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"a = 'This was the worst film to ever disgrace the screen.'\n",
|
|
"sid.polarity_scores(a)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Use VADER to analyze Amazon Reviews\n",
|
|
"For this exercise we're going to apply `SentimentIntensityAnalyzer` to a dataset of 10,000 Amazon reviews. Like our movie reviews datasets, these are labeled as either \"pos\" or \"neg\". At the end we'll determine the accuracy of our sentiment analysis with VADER."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>label</th>\n",
|
|
" <th>review</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>Stuning even for the non-gamer: This sound tra...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>1</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>The best soundtrack ever to anything.: I'm rea...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>2</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>Amazing!: This soundtrack is my favorite music...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>3</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>Excellent Soundtrack: I truly like this soundt...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>4</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>Remember, Pull Your Jaw Off The Floor After He...</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>"
|
|
],
|
|
"text/plain": [
|
|
" label review\n",
|
|
"0 pos Stuning even for the non-gamer: This sound tra...\n",
|
|
"1 pos The best soundtrack ever to anything.: I'm rea...\n",
|
|
"2 pos Amazing!: This soundtrack is my favorite music...\n",
|
|
"3 pos Excellent Soundtrack: I truly like this soundt...\n",
|
|
"4 pos Remember, Pull Your Jaw Off The Floor After He..."
|
|
]
|
|
},
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"import numpy as np\n",
|
|
"import pandas as pd\n",
|
|
"\n",
|
|
"df = pd.read_csv('../TextFiles/amazonreviews.tsv', sep='\\t')\n",
|
|
"df.head()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"neg 5097\n",
|
|
"pos 4903\n",
|
|
"Name: label, dtype: int64"
|
|
]
|
|
},
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"df['label'].value_counts()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Clean the data (optional):\n",
|
|
"Recall that our moviereviews.tsv file contained empty records. Let's check to see if any exist in amazonreviews.tsv."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# REMOVE NaN VALUES AND EMPTY STRINGS:\n",
|
|
"df.dropna(inplace=True)\n",
|
|
"\n",
|
|
"blanks = [] # start with an empty list\n",
|
|
"\n",
|
|
"for i,lb,rv in df.itertuples(): # iterate over the DataFrame\n",
|
|
" if type(rv)==str: # avoid NaN values\n",
|
|
" if rv.isspace(): # test 'review' for whitespace\n",
|
|
" blanks.append(i) # add matching index numbers to the list\n",
|
|
"\n",
|
|
"df.drop(blanks, inplace=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"neg 5097\n",
|
|
"pos 4903\n",
|
|
"Name: label, dtype: int64"
|
|
]
|
|
},
|
|
"execution_count": 9,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"df['label'].value_counts()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"In this case there were no empty records. Good!"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Let's run the first review through VADER"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"{'neg': 0.088, 'neu': 0.669, 'pos': 0.243, 'compound': 0.9454}"
|
|
]
|
|
},
|
|
"execution_count": 10,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"sid.polarity_scores(df.loc[0]['review'])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"'pos'"
|
|
]
|
|
},
|
|
"execution_count": 11,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"df.loc[0]['label']"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Great! Our first review was labeled \"positive\", and earned a positive compound score."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Adding Scores and Labels to the DataFrame\n",
|
|
"In this next section we'll add columns to the original DataFrame to store polarity_score dictionaries, extracted compound scores, and new \"pos/neg\" labels derived from the compound score. We'll use this last column to perform an accuracy test."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>label</th>\n",
|
|
" <th>review</th>\n",
|
|
" <th>scores</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>Stuning even for the non-gamer: This sound tra...</td>\n",
|
|
" <td>{'neg': 0.088, 'neu': 0.669, 'pos': 0.243, 'co...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>1</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>The best soundtrack ever to anything.: I'm rea...</td>\n",
|
|
" <td>{'neg': 0.018, 'neu': 0.837, 'pos': 0.145, 'co...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>2</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>Amazing!: This soundtrack is my favorite music...</td>\n",
|
|
" <td>{'neg': 0.04, 'neu': 0.692, 'pos': 0.268, 'com...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>3</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>Excellent Soundtrack: I truly like this soundt...</td>\n",
|
|
" <td>{'neg': 0.09, 'neu': 0.615, 'pos': 0.295, 'com...</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>4</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>Remember, Pull Your Jaw Off The Floor After He...</td>\n",
|
|
" <td>{'neg': 0.0, 'neu': 0.746, 'pos': 0.254, 'comp...</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>"
|
|
],
|
|
"text/plain": [
|
|
" label review \\\n",
|
|
"0 pos Stuning even for the non-gamer: This sound tra... \n",
|
|
"1 pos The best soundtrack ever to anything.: I'm rea... \n",
|
|
"2 pos Amazing!: This soundtrack is my favorite music... \n",
|
|
"3 pos Excellent Soundtrack: I truly like this soundt... \n",
|
|
"4 pos Remember, Pull Your Jaw Off The Floor After He... \n",
|
|
"\n",
|
|
" scores \n",
|
|
"0 {'neg': 0.088, 'neu': 0.669, 'pos': 0.243, 'co... \n",
|
|
"1 {'neg': 0.018, 'neu': 0.837, 'pos': 0.145, 'co... \n",
|
|
"2 {'neg': 0.04, 'neu': 0.692, 'pos': 0.268, 'com... \n",
|
|
"3 {'neg': 0.09, 'neu': 0.615, 'pos': 0.295, 'com... \n",
|
|
"4 {'neg': 0.0, 'neu': 0.746, 'pos': 0.254, 'comp... "
|
|
]
|
|
},
|
|
"execution_count": 12,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"df['scores'] = df['review'].apply(lambda review: sid.polarity_scores(review))\n",
|
|
"\n",
|
|
"df.head()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>label</th>\n",
|
|
" <th>review</th>\n",
|
|
" <th>scores</th>\n",
|
|
" <th>compound</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>Stuning even for the non-gamer: This sound tra...</td>\n",
|
|
" <td>{'neg': 0.088, 'neu': 0.669, 'pos': 0.243, 'co...</td>\n",
|
|
" <td>0.9454</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>1</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>The best soundtrack ever to anything.: I'm rea...</td>\n",
|
|
" <td>{'neg': 0.018, 'neu': 0.837, 'pos': 0.145, 'co...</td>\n",
|
|
" <td>0.8957</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>2</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>Amazing!: This soundtrack is my favorite music...</td>\n",
|
|
" <td>{'neg': 0.04, 'neu': 0.692, 'pos': 0.268, 'com...</td>\n",
|
|
" <td>0.9858</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>3</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>Excellent Soundtrack: I truly like this soundt...</td>\n",
|
|
" <td>{'neg': 0.09, 'neu': 0.615, 'pos': 0.295, 'com...</td>\n",
|
|
" <td>0.9814</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>4</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>Remember, Pull Your Jaw Off The Floor After He...</td>\n",
|
|
" <td>{'neg': 0.0, 'neu': 0.746, 'pos': 0.254, 'comp...</td>\n",
|
|
" <td>0.9781</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>"
|
|
],
|
|
"text/plain": [
|
|
" label review \\\n",
|
|
"0 pos Stuning even for the non-gamer: This sound tra... \n",
|
|
"1 pos The best soundtrack ever to anything.: I'm rea... \n",
|
|
"2 pos Amazing!: This soundtrack is my favorite music... \n",
|
|
"3 pos Excellent Soundtrack: I truly like this soundt... \n",
|
|
"4 pos Remember, Pull Your Jaw Off The Floor After He... \n",
|
|
"\n",
|
|
" scores compound \n",
|
|
"0 {'neg': 0.088, 'neu': 0.669, 'pos': 0.243, 'co... 0.9454 \n",
|
|
"1 {'neg': 0.018, 'neu': 0.837, 'pos': 0.145, 'co... 0.8957 \n",
|
|
"2 {'neg': 0.04, 'neu': 0.692, 'pos': 0.268, 'com... 0.9858 \n",
|
|
"3 {'neg': 0.09, 'neu': 0.615, 'pos': 0.295, 'com... 0.9814 \n",
|
|
"4 {'neg': 0.0, 'neu': 0.746, 'pos': 0.254, 'comp... 0.9781 "
|
|
]
|
|
},
|
|
"execution_count": 13,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"df['compound'] = df['scores'].apply(lambda score_dict: score_dict['compound'])\n",
|
|
"\n",
|
|
"df.head()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/html": [
|
|
"<div>\n",
|
|
"<style scoped>\n",
|
|
" .dataframe tbody tr th:only-of-type {\n",
|
|
" vertical-align: middle;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe tbody tr th {\n",
|
|
" vertical-align: top;\n",
|
|
" }\n",
|
|
"\n",
|
|
" .dataframe thead th {\n",
|
|
" text-align: right;\n",
|
|
" }\n",
|
|
"</style>\n",
|
|
"<table border=\"1\" class=\"dataframe\">\n",
|
|
" <thead>\n",
|
|
" <tr style=\"text-align: right;\">\n",
|
|
" <th></th>\n",
|
|
" <th>label</th>\n",
|
|
" <th>review</th>\n",
|
|
" <th>scores</th>\n",
|
|
" <th>compound</th>\n",
|
|
" <th>comp_score</th>\n",
|
|
" </tr>\n",
|
|
" </thead>\n",
|
|
" <tbody>\n",
|
|
" <tr>\n",
|
|
" <th>0</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>Stuning even for the non-gamer: This sound tra...</td>\n",
|
|
" <td>{'neg': 0.088, 'neu': 0.669, 'pos': 0.243, 'co...</td>\n",
|
|
" <td>0.9454</td>\n",
|
|
" <td>pos</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>1</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>The best soundtrack ever to anything.: I'm rea...</td>\n",
|
|
" <td>{'neg': 0.018, 'neu': 0.837, 'pos': 0.145, 'co...</td>\n",
|
|
" <td>0.8957</td>\n",
|
|
" <td>pos</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>2</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>Amazing!: This soundtrack is my favorite music...</td>\n",
|
|
" <td>{'neg': 0.04, 'neu': 0.692, 'pos': 0.268, 'com...</td>\n",
|
|
" <td>0.9858</td>\n",
|
|
" <td>pos</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>3</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>Excellent Soundtrack: I truly like this soundt...</td>\n",
|
|
" <td>{'neg': 0.09, 'neu': 0.615, 'pos': 0.295, 'com...</td>\n",
|
|
" <td>0.9814</td>\n",
|
|
" <td>pos</td>\n",
|
|
" </tr>\n",
|
|
" <tr>\n",
|
|
" <th>4</th>\n",
|
|
" <td>pos</td>\n",
|
|
" <td>Remember, Pull Your Jaw Off The Floor After He...</td>\n",
|
|
" <td>{'neg': 0.0, 'neu': 0.746, 'pos': 0.254, 'comp...</td>\n",
|
|
" <td>0.9781</td>\n",
|
|
" <td>pos</td>\n",
|
|
" </tr>\n",
|
|
" </tbody>\n",
|
|
"</table>\n",
|
|
"</div>"
|
|
],
|
|
"text/plain": [
|
|
" label review \\\n",
|
|
"0 pos Stuning even for the non-gamer: This sound tra... \n",
|
|
"1 pos The best soundtrack ever to anything.: I'm rea... \n",
|
|
"2 pos Amazing!: This soundtrack is my favorite music... \n",
|
|
"3 pos Excellent Soundtrack: I truly like this soundt... \n",
|
|
"4 pos Remember, Pull Your Jaw Off The Floor After He... \n",
|
|
"\n",
|
|
" scores compound comp_score \n",
|
|
"0 {'neg': 0.088, 'neu': 0.669, 'pos': 0.243, 'co... 0.9454 pos \n",
|
|
"1 {'neg': 0.018, 'neu': 0.837, 'pos': 0.145, 'co... 0.8957 pos \n",
|
|
"2 {'neg': 0.04, 'neu': 0.692, 'pos': 0.268, 'com... 0.9858 pos \n",
|
|
"3 {'neg': 0.09, 'neu': 0.615, 'pos': 0.295, 'com... 0.9814 pos \n",
|
|
"4 {'neg': 0.0, 'neu': 0.746, 'pos': 0.254, 'comp... 0.9781 pos "
|
|
]
|
|
},
|
|
"execution_count": 14,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"df['comp_score'] = df['compound'].apply(lambda c: 'pos' if c >=0 else 'neg')\n",
|
|
"\n",
|
|
"df.head()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Report on Accuracy\n",
|
|
"Finally, we'll use scikit-learn to determine how close VADER came to our original 10,000 labels."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 15,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from sklearn.metrics import accuracy_score,classification_report,confusion_matrix"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 16,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"0.7091"
|
|
]
|
|
},
|
|
"execution_count": 16,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"accuracy_score(df['label'],df['comp_score'])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 17,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
" precision recall f1-score support\n",
|
|
"\n",
|
|
" neg 0.86 0.51 0.64 5097\n",
|
|
" pos 0.64 0.91 0.75 4903\n",
|
|
"\n",
|
|
" micro avg 0.71 0.71 0.71 10000\n",
|
|
" macro avg 0.75 0.71 0.70 10000\n",
|
|
"weighted avg 0.75 0.71 0.70 10000\n",
|
|
"\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(classification_report(df['label'],df['comp_score']))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 18,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[[2623 2474]\n",
|
|
" [ 435 4468]]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(confusion_matrix(df['label'],df['comp_score']))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"This tells us that VADER correctly identified an Amazon review as \"positive\" or \"negative\" roughly 71% of the time.\n",
|
|
"## Up Next: Sentiment Analysis Project"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.6.7"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|