A Combined Approach of Sentimental Analysis Using Machine Learning Techniques

A great deal of research has been conducted in the field of text classification and sentiment analysis. One of the fundamental problems in sentiment analysis is to categorize sentiment score. The task is to determine whether a fraction of text is negative or positive. The aspect or entity level, the document level, and the sentence level are the three different forms of sentiment polarity classification. The entity type is concerned with what is the opinion of people about their likes and dislikes. The document level perceives the valence, i.e., negative or positive sentiment, for the whole text, whereas the sentiment categorization for each sentence comes under sentence level.

Many scholars from all over the world have conducted studies using semi-supervised, unsupervised, and supervised machine learning techniques. A sentiment analysis methodology was suggested by Bhatt et al. [6] for evaluation of iPhone5 reviews on amazon. The technique combines several pre-processing methods for reducing noise in data such as punctuation, numbers, and HTML tags. The features are identified with the help of part-of-speech (POS) tagger, and a rule-based procedure is used for reviews categorization. Haddi et al. [7] investigate the impact of text pre-processing on reviews of online movie. To remove noisy data, many preprocessing procedures such as stemming, html tag removal, and data cleaning are performed. The chi-square technique for feature selection is used to eliminate the irrelevant features. The SVM is applied on reviews to categorize them into negative or positive. Wassan et al. [8] proposed a new approach which focuses on sentimental components of the item's characteristics. Customer reviews on Amazon have been presented and evaluated. They derived dataset from data world center, and detect opinion rates and extract meaningful information such as negativity or positivity, and performs pre-processing activities on datasets such as tokenization, stone-coating, stop-words deletion, and boxing. They focus to examine the data's aspect level to understand consumer preferences and continuing to develop their behavior accordingly.

Srujan et al. [9] first eliminate noise from the reviews of amazon book by using various preprocessing techniques such as URL removal and HTML tag, whitespace, punctuation, stemming, and special character removal. Term frequency- inverse document frequency(tf-idf) are utilized for feature selection to represent the preprocessed data. They compare the accuracy of various classifiers such as K-Nearest Neighbor (KNN), Decision Tree (DT), Support Vector Machine (SVM), Random Forest (RF), and Naive Bayes (NB) and the time taken by each classifier and also find the sentimental score of different books. Shrestha and Nasoz [10] assess the consistency of reviews from Amazon.com to their correlating ratings using sentiment analysis. Sometimes there is a discrepancy between the review and the rating what consumers submit on e-commerce websites. They employed deep learning sentiment analysis on product review from amazon.com to identify reviews with mismatched ratings. Using paragraph vector, the product reviews were transitioned to vectors, which were later used to train a recurrent neural network with gated recurrent unit. They consider both the product information and the semantic relationship of the review text. Using the trained model, they created a web service application that predicts the rating score for a submitted review and provides feedback to the reviewer if there is a difference in the submitted rating score and the predicted rating score. Nandal et al. [11] proposed a novel approach for aspect level sentiment detection, which emphasis on the features of item. The task was trialled on Amazon customer reviews, at which aspect terms were identified first for each review. The preprocessing operations were performed by the system on the dataset such as tokenization, stemming, stop-word removal, and casing, to retrieve meaningful insights before assigning a rank to its classification in positivity or negativity. Chen et al. [12] perform review embedding on the IMDB and Yelp datasets using a one-layer convolutional neural network (CNN). That CNN generates 300-dimensional vectors from reviews of varying length. They padded shorter reviews with zero vectors to accommodate varying lengths of reviews. Filters with widths of 3 and 5 performed one-dimensional convolution on the word embeddings, yielding multiple feature maps. By using max-overtime pooling in the pooling layer, only useful features were captured. Multiple filter outputs were then concatenated to form a 300-dimensional vector. To train the network over K-classes, the Softmax function was used as an activation function. The Softmax function was used as an activation function to train the network over K-classes. They then employed a Recurrent Neural Network (RNN) to learn the temporal information and capture both user and product information, reporting cutting-edge results on the Yelp and IMDB datasets. Their research is based on the assumption that an item which gets positive feedback initially is more likely to get positive feedback in the future, and vice versa.

We, in this work, presents a method for elimination of noise and feature extraction using TF-IDF from amazon and IMDB datasets by combining them and performs comparative analysis on the accuracy of various classifiers and their sentiment scores. The methodology and classifiers used in our experiment are discussed in the following section.

This section contains a detailed description of evaluation metrics, and a discussion of the results obtained.

4.1 Evaluationmetrics

Evaluation metrics are critical for measuring performance of classification. Accuracy is the commonly used metric for performance checking. A classifier's accuracy on a specified test dataset is the proportion of those datasets classified correctly by the classifier. And, as the accuracy measure is not sufficient in the text mining analysis to make a valid judgment, we also implement some other metrics to evaluate classification performance [21]. F-measure, recall, and precision are three fundamental measures which are generally used. Before delving into various measures, there are a few terms we should become acquainted with:

• TP (True Positive) shows the number of data points that were classified correctly.

• FP (False Positive) shows the number of misclassified correct data.

• FN (False Negative) shows the number of incorrect data that has been classified as positive.

• TN (True Negative) represents the number of incorrect data classified as negative.

Accuracy- Accuracy indicates how oftenly the classifier appears to make correct prediction. The accuracy ratio is calculated by dividing the number of correct predictions to the total number of predictions.

Accuracy $=\frac{\text { Correct Prediction }}{\text { Total prediction }}$              (1)

Recall- Recall calculates a classifier's sensitivity. More recall means lesser false negatives. The recall ratio is the number of correctly classified occurrences divided by the total number of predicted occurrances. This can be illustrated as:

$Recall(R)=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$           (2)

Precision- The exactness of a classifier can be measured by its precision. A low level of precision will result in a greater number of false positives, whereas a high level of precision will result in fewer of these errors. The term "precision" (P) is defined as:

$Precision(P)=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$             (3)

F-Measure- A single metric called F-measure (the weighted harmonic mean of recall and precision) is formed when precision and recall are combined. It is defined as follows:

$F-measure=\frac{2 \mathrm{PR}}{\mathrm{P}+\mathrm{R}}$           (4)

4.2 Experimental results

In the study, we consider two classes of problems, negative and positive. We have taken reviews of three different sites, namely amazon, IMDB, and Yelp. Each of these review includes the user's rating and text reviews. Based on their ratings, these reviews are divided into two categories: negative and positive. Reviews with ratings of 4 or 5 are considered positive, while reviews with ratings of 1 and 2 are considered negative. The set is divided in 80:20 proportions for training and testing. In this experiment, various machine learning algorithms like Support Vector Machine Classifier (SVC), K Nearest Neighbor (KNN), Naive Bayes, and Random Forest are used. Several other types of preprocessing operations, such as data cleaning, the elimination of whitespace, and stemming, are carried out on the dataset. The train set and the test set are both transformed into individual term document matrices (TDM), and then the frequency of occurrence of each word is counted. The TF-IDF is utilized so that the data can be represented. After that, the TDM is used as an input for a number of different classifiers. The confusion matrix of the given classifiers is shown below in Figure 3.

3.png

3b.png

Figure 3. Confusion matrix of (a) KNN, (b) SVM, (c) Naive Bayes, and (d) Random Forest

Experimental result of different classifier is shown in Table 1. From Table 1, We can notice that Random Forest outperforms the other methods in the two-class classification problem. The highest accuracy obtained from these results was 78.96%. The TDM is made up of feature vector values known as points. The categorization accuracy significantly improved by constructing an ensemble of trees and allowing them to vote on the most popular class.

Table 1. Experimental result of classifiers on combined dataset