A Hybrid Model Integrating Adaboost Approach for Sentimental Analysis of Airline Tweets

To aquire a better understanding the opinions of passenger’s sentimental analysis has a significant impact for many researchers. In this study we have worked for appropriate prediction of comments in the area of airline industry where updating in services is now the main significant and important task. Many people cannot give their feedback directly so they share their opinions either in short messages, emojis, special symbols which are known as tweets. Nowadays social network travelling websites are the fast growing internet platform where people can post their comments or tiny texts called comments and also give rating for any airline according to their experience and another people can read the tweets and take decision to select appropriate airline without physically interference with related airline service. Additionally, as a past study application for our SAF (sentiment_analysis_framework), proposed a new cross_disciplinary field of geopolitics [1]. We have gathered tweets from different travelling web sources for six airlines then applied various preprocessing techniques to process the accurate feature data. We made use of “extract sentiment” operator of rapid miner tool to calculate sentiment score values and applied condition statement using “generate attribute” operator for prediction of two categories of tweets either pos or neg tweet. The experiment was done using self-written score condition code in rapidminer “text processing” operator and partitioned the dataset in testing and training using K-fold cross validation technique with stratified sampling to to support the model. The initial classifiers Nave Bayes, RF, RT, SVM, Logistic Regression, Decision Stump, and K-NN are utilized in this investigation paper to resolve the airline tweet sentimental analysis problem while gaining maximum accuracy for datasets and accurate prediction of pos and neg prediction value. We needed to make a robust model called HMIAA (Hybrid model integrated Adaboost approach), which is a set of initial classifiers together SVM and a forward learning ensemble model GBT integrated with the Adaboost meta-algorithm, which is itself a combination of many other ML algorithms to boost performance.

The proposed system consists of different seven phases such as Data Acquisition, Feature Construction (Text Preprocessing), Sentiment Score or Polarity calculation, Feature Extraction, Feature Selection, Splitting of Dataset, Sentimental Classification using base classifiers and proposed HMIAA (Hybrid model integrated Adaboost approach). The proposed methodology flow diagram is depicted in Figure 1 where the overall work has divided into seven phases from data collection to sentimental polarity prediction where SVM and GBT performs better result but to enhance accuracy of these classifiers we integrate both with Adaboost by tuning different parameter values of ensemble classifier and classifiers so it is possible to build a powerful model.

3.1 Data acquisition (Data collection)

This is the initial phase of our research framework where we have gathered six different datasets as tweets related to Air Asia, Air India, Spice jet, Indigo, Go Air and Vistara airlines using various travelling online web sources such as Tripadvisior, Skytrax, Makemytrip, Trustpiolet, Mouth shut etc. through the Webharvy data fetcher tool. For our research paper, we wanted to gather information or data from various Indian airlines and extract the precise sentiments of tweets for each and every airline company and in addition to develop a modelthat provides a proper recommendation of airline among the various airlines. We used the Rapid Miner machine learning tool for optimizing the sets of data sets.

1.png

Figure 1. Proposed methodology framework of airline tweet SC (Sentimental Classification) using HMIAA approach

3.2 Data preprocessing

This is the second phase of our research which is also called feature construction phase where we preprocess or transform the tweets which is initially in the raw form into an understandable format so we can suitably import for LMs. Because tweets contain a lot of noise and ambiguous symbols, it is essential to pre-process data in tweets such as:

·The following symbols have replaced by blank space using “Replace” operator in rapid miner-[-!"#$%&'()*+,./:;<=>?@\[\\\]_`{|}~].

The bulk of the distinct symbols called “Regular Expressions” supplied bythe passengers along with their comments were meaningless, so we used the Rapid Miner operator to replace them.

·Emoticons have replaced by its equivalent meaning so these can serve as useful feature to detect sentiments.

·For the removal of stop words “filter stopwords” and “filter tokens” (by length) filter operator have used.

·Stemming has done through stemming algorithms stem (snowball and porter) operators to reduce each word to its root word.

3.3 Sentiment score or polarity calculation

This is the third phase of our research, in which We employed the Extract Sentiment operator to calculate sentiment_score utilizing the Text attribute of a related dataset, followed by generated attribute set reduction and Transformation polarity categories for pos and neg classes according on the Score ≥ 0 and Score<0 conditions, and finally used the Write Excel operator to review and save both the “Text” and “Sentiment filed” in filtered datasets.

3.4 Feature extraction

3.4.1 Word2Vec

A W2V model is alike to a dictionary or hash map. Each word in the train-corpus has a word vector in this dictionary. The training corpus is exported to an example set using this method. Additional calculations, such as lexical detection, can be carried out using the generated set. A TF-IDF vectorization technique is used to convert processed text into numeric feature vectors, which are then fed into computers for ML applications.The BoW approach is straightforward and efficient; however it does not distinguish between frequent and unusual terms. This BoW vectorization problem is solved by TF-IDF.

3.4.2 N-Gram (Range max_length="2")

This operator constructs term n-Grams from tokens in a document. A series of consecutive tokens of length n is defined as an n-Gram. This operator generates the term n-Grams, which is made up of all series of consecutive tokens of length n.

3.4.3 POS tags -filter tokens

To filter tokens, the POS tag types that have been defined are used. In the PENN system for English POS tags and the STTS system for German POS tags, the possible POS tags are determined by a regular expression of types. For instance, all nouns and adjectives would remain in the sentence JJ. *|N. *. The regular phrase that was used to match the POS tags (STTS for German and PENN for English).

3.5 Feature selection (Removing a feature based on a filter)

3.5.1 Filter 1. Info Gain (IG)

The information gain and entropy of all attributes are computed. The entropy of the property with the lowest value is chosen for splitting. This method favors attributes with a great number of possible values.

3.5.2 Filter 2. Gain Info (GI)

It's a type of IG. It modifies the information gain for each attribute for the purpose of allow for the breadth and uniformity of the attribute values. Following this procedure, Actual Feature Vectors and the Final Feature Set were used. Info Gain and Gain Info both are the filter method to decide the relevance of an attribute and the attributes with higher weight are considered more relevant. In our research for polarity orcomment feature positive attribute has the highest info gain among another attributes.

3.6 Splitting dataset

In this phase we have followed the following procedures:

·Dividing the set of training into two disjoint sets.

·On the I^st section, train 67% of numerous base learners.

·On II^nd section, test 33%S of the initial learners.

·To practice a higher-level learner, use the predictions from step 3 moreover the right inputs and answers as outputs.

3.7 Sentimental classification using base classifiers

The bulk of the times, the classifiers have already been used as a foundation for analyzing of sentiments. In our investigation, we used the following base classifiers:

3.7.1 (RF) Random forest

The RF technique produces a set of randomly generated trees. The random trees are created in the same manner that a tree is formed using the Random Tree operator. A specific number of random tree models are merged to the resulting forest model. The number_of_trees parameter specifies the number_of_trees required. For all of the random trees, the model that emerges is a voting model. Pruning is the removal of leaf nodes that don't make a participation in the tree's distinguishing ability. The parameter the RF classifier values are tuned as: number of trees=100, criterion=”Gain_Ratio/Information_Gain, Max_depth=10, applying pruning checked and confidence=0.1.

3.7.2 Logistic regression (LR)

According to Stefan Rueping's internal Java implementation of the myKLR in rapid miner. This learning method may applied to both regression and classification, and it features a simple algorithm that yields good results for a broad range of learning situations. Loss functions that are linear, quadratic, or even asymmetric can be handled with mySVM. The parameter_values for the LR classifier are tuned as: solver=”AUTO”, missing values handling =”meanlmeasures” , standardize checked and MaxIteration=0. When the dependent_variable (target) is categorical, logistic_regression is utilised. In our research We've employed the default “AUTO” value of “solver” parameter with this model but in this line “Loss functions that are linear, quadratic, or even asymmetric can be handled with mySVM” we have written the combined role of LR and SVM model.

3.7.3 Naïve bayes (NB)

On the resulting Example Set, the NB classification model is utilized (i.e. the testing data_set reduced dimensions). The existence or the lack of a specific feature of a class (i.e. attribute) has no influence on the absence or presence of other feature, on the basis of NB classifier. The NB classifier has the pros of estimating the means and the variables' variances required for using a classification oflittle volume of data for training. Because they create a couple of "naive" the perceptions of the data, the NB algorithms are termed as such. The NB classifier is highly adaptable; it necessitates a few parameters concerning the number of characteristics (characters/predictors) in a learning problem [11]. The parameter values of NB classifier are tuned as: Laplace correction checked.

3.7.4 (K-NN) K_nearest_neighbor

Among the most prominent fundamental MLAs is the K-NN algorithm: an illustration is categorized as an illustration assigned to the group most prevalent among its k_nearest_neighbor by a majority vote of its neighbors (k is a PI, typically small).If k=1, the example is just put into the class of the next-closest neighbour. In regression, the same method may be employed by simply averaging the label value for the example's k NN values. The parameter_ values of the K-NN classifier are tuned as: k=5, weighted vote checked, measure types=”Mixed Measures” and mixed measure=”MixedEuclideanDistance”.

3.7.5 (SVM) Support_vector_machine

This type of learning has a rapid algorithm, may be used for classification and regression, and produces good results for a variety of learning difficulties. SVM techniques are designed to ensure that dot products can be easily computed in relation to the variables in the original location by defining them in pertaining toa kernel function K(x,y) that is tailored to the task. Kernel type (selection) [12-14]. A kernel function is of the kind determined by this parameter. The Dot, radial, polynomial, neural, anova, epachnenikov, gaussian combination, and multiquadric kernels are among many that are supported. As Kernel type, we specifically used dot and polynomial parameter values. The LR classifier parameter values are tuned as: The possible view of the prediction equations for SVM classification is: $\mathrm{h}(\mathrm{xi})=\operatorname{sign}\left(\sum_{i=1}^s \alpha_j y_j K\left(x_j, x_i\right)+b\right), \mathrm{K}(\mathbf{v}, \mathbf{v}^{\prime} \mathbf{})=\exp$ “( $\left\|v-v^{\prime \prime}\right\|^{\wedge} 2 /\left(2 \gamma^{\wedge} 2\right)”$.

The forecast of Xi is vector, and Xj is the support_vector, that is an amount of the instruction data in example. K is a k function that accepts two vector input and output a number.

3.7.6 Random tree (RT)

This operator creates decision trees using nominal and numerical data. DTs are simple-to-understand categorization systems with a huge power. The R_Tree operator works similarly to Quinlan's CART or C4.5, with the exception that it chooses a R attributes subset before using them. The subset size is determined by the subset _ratio_parameter and prepruning is a type of pruning that takes place while the tree continues to grow [15-20]. As opposed to that, post-pruning is completed once the tree has been produced. The parameter criterion (selection) defines the criterion that will be put to use divide characteristics. The RT classifier's parameter values have defined as follows: criterion = “Gain_Ratio, Information_Gain”, max_depth = 10 and confidence = 0.1.

3.7.7 Decision stump (DS)

When only one split is implemented in a DT, the Decision Stump operator is utilized. The tree's structure can be applied to categorise previously undiscovered cases [21-24]. Each branch (to another decision tree) represents a potential attribute value, whereas the decision node represents an attribute test. The parameter values of DS classifier are tuned as: criterion=Gain_Ratio and minimal_leaf_size=1.

3.8 Proposed Hybrid model integrated Adaboost approach (HMIAA)

To make a more robust model, we frequently mix the best classifiers. This improves the model's performance. By changing various algorithmic parameters to distinct values, the proposed/created model integrates Adaboost, GBT (Gradient Boost Tree), and SVM by tweaking different algorithmic parameters to distinct values.

·AdaBoost, standing for Adaptive Boosting, is a meta-algorithm that may used for boost the effectiveness of many different LMs. It is a nested operator, which means there exists a sub process. In the sub process, a learner is needed, that is an operator that takes an Example Set as input and produces a model. This operator seeks to generate a better model using the learner provided in its subprocess.AdaBoost creates and calls newly weak classifier in every single t = 1... , T rounds. A weighted distribution D (t), representing the significance of examples in the set of data for categorization, is updated for each call. Although, it is possible less alert to the over fitting problem in some circumstances than traditional learning methods. We set the maximum number of AdaBoost algorithm iterations to boost the performance value. Number_of_iterations = "10 to 30" has been fine-tuned in this algorithm's Iteration parameter to improve efficiency.

·A GBT is a model that combines classification and regression_trees. Both are ensemble approaches that use forward learning to assume outcomes by steadily improving estimations. Boosting is a versatile nonlinear regression method that aids in improving tree accuracy. Ensemble techniques integrate multiple models to produce better predictive performance than the individual models. To put it another way, an ensemble is a means of combining multiple weak learners to make a powerful learner. Because evaluating an ensemble's forecast usually necessitates more processing than evaluating a single model's prediction, ensembles can be viewed as a means to compensate for bad learning algorithms by performing a significant amount of additional work. Applying weak classification algorithms sequentially to gradually changing data produces a sequence of decision trees, resulting in a collection of weak prediction models. Whileincreasing tree accuracy increases speed and human interpretability, it diminishes speed and human interpretability. To overcome these challenges, the GB approach generalizes tree boosting. The operator sets up a one-node local H2O_cluster and performs the programme on it. Despite the fact that just one node is used, the process is parallel. Adjusting the Settings/Preferences/General/Count of threads parameter can change the degree in terms of parallelism. It uses the number of threads recommended by the system by default. Only one cluster instance is launched, and it still exists operational until Rapid Miner Studio is stopped. This output node provides the Gradient_Boosted_ classification or regression_model. It is possible to forecast the label attribute on previously unidentified pieces of data using this classification or regression model. The following parameters were changed to increase the efficiency of the GBT model: The total_number_of _trees =20, Maximal depth=5, Number of bins = 50, Distribution_method=”AUTO/Bernoulli/multinomial.

The algorithm below explains how to extract opinions expressed in tweets and create Sentiment Scores using the "Extract Sentiment" and "Generate Attribute" operators.

There is an algorithm for proposed model which describe the sentiment score calculation using VADER (Valence Aware Dictionary for Sentiment Reasoning) model which analyses the text sentiment that is sensitive to both polarity (Positive/Negative) and intensity of emotion and also applied TF-IDF vector creation method in manner to extract features. We have used our self condition for polarity of sentiments using extract sentiment operator as if Score ≥ 0 then positive tweets and if Score<0 then negative tweets. After this process we have to applied different classifiers for actual positive prediction and negative prediction of tweets.

Algorithm I

Algorithm to calculate Sentiment Score by Extracting Sentiment operator for base and ensemble models-

Function Calculate Sentiment Score (Retrive_Tweet, Test_Tweet, Sentiment_Score, Sentiment)

1. Input:Retrive_Tweet
2. Output:Sentiment_Score, Sentiment
3. for eachRetrive_Tweet in Test_Tweetdo

// after obtaining the score value, predict the sentiment_of_tweets by extracting the words and delivering a word-vector operator using the TF-IDF Vector Creation method.

4. If Sentiment_Score ≥ 0 then
5.           Sentiment=”Positive”;
6.          Else
7.          Sentiment=”Negative”;

End

5.1 Datasets

The suggested approach was tested on the total_number of 41,464 tweets collected for six different Indian airlines, which are listed below:

5.1.1 Air India

There are 20,000 records in this collection of training tweets, containing 6,776 pos labeled tweets and 13,222 neg labeled tweets.

5.1.2 Spicejet

This DS contains 3,551 training tweets, 1,304 of which include labeled pos and 2,246 of which include labeled neg.

5.1.3 Air Asia India

This DS contains 1,248 training tweets, 1,304 of which include labeled pos and 2,246 of which include labeled neg.

5.1.4 Go Air

This DS contains 5,262 training tweets, 1,203 of which include tagged pos and 4,058 of which include labeled neg.

5.1.5 Indigo

This DS contains 9,554 training tweets, 3,309 of which include labeled pos and 6,244 of which include labeled neg.

5.1.6 Vistara

This DS contains 1,849 training tweets, 1,119 of which include tagged pos and 649 of which include labeled neg.

5.2 Results analysis & discussion

In below Table 1 the performance measures of general classifiers and our proposed HMIAA approach have shown where the maximum accuracy 96.84% achieved for actual prediction of both positively and negatively tweets for SA. The general classifier SVM and forward learning algorithm GBT have performed well in comparison to other classifiers so we integrate both classifiers with Adaboost ensemble model for better results accuracy by tuning various parameters. The optimum level of accuracy has been achieved for Go Air airline, hence this airline may be recommended to passengers.

In the Table 1 we have shown the base classifier's accuracy S_V_M along with forward learning algorithm Gradient Boosted Technique because these classifiers have given the maximum accuracy and after that we’ve implemented Adaboost Meta classifier integrated using SVM and GBT which is basically used for weak learners to increase efficiency more to design a robust model.

7.png

Figure 7. Performance evaluation chart of different proposed HMIAA for six datasets (a) Vistara (b) Spicejet (c) Air Asia India (d) Air India (e) Go Air (f) Indigo

As seen in above Figure 7, all the performance measures such as acc, precision, f_measure and recall of suggested Hybrid_Model Integrated with Adaboost Approach (HMIAA) viewed the entire set of six airline tweets such as Vistara, Spicejet, Air India, GO Air, Indigo, Air Asia.

Due to the various numbers of tweets from each airline through social media travel sites, the data fetcher tool has compiled maximum numbers of comments for all of the airlines and according to a passenger's comment, every airline is different from the others in form of comment count.

The total_number of pos and neg predictions split up by the total number of predictions is a statistic that represents a model's performance as classification accuracy.

Precision=$\frac{\text{True_Positive_Sentiment}}{\text{True_Positive_Sentiment+False_Positive_Sentiment}}$

It's the total_number of genuine positives across divided by the sum of real positives and false positives.

Recall=$\frac{\text{True_Positive_Sentiment}}{\text{True_Positive_Sentiment+False_Negetive_Sentiment}}$

It's calculated by dividing the number of true positives from the addition of number of true positives and false negatives.

F-measure=$\frac{\text{2*Precision*Recall}}{\text{Precsion+Recall}}$

It generates a single score that accounts for both preci and reca concerns in a number.

The Figure 8 below shows the AUC curves have calculated because if the model has a probabilistic scoring system where scores above a certain threshold are identified as positive, then the elements of the confusion_matrix will depend on the threshold. The space beneath the curvature (AUC) is calculated simply adjusting the threshold and plotting a point (x, y) for each threshold value:

• True positive rate=(True positive predictions)/(Number of positive Instances)=TPP/(TPP+FNP) on the Y-axis

• X-axis - false positive rate=(False positive predictions)/(Number of negative Instances)=FPP/(FPP+TNP) =FPP/(FPP+TNP).

To determine the curve's undersurface area, the predictions are sorted by score from highest to lowest, and the graph is displayed Example by Example (AUC). If two or more Examples possess the same score, the ranking is unclear. The area in the typical form of AUC is computed using the average_of_AUC (optimistic) and AUC (realistic) (pessimistic).

Based on sentiments and classification, Indigo is the second-highest recommended airline; as a result, the spatial discreteness of ROC is quite high but lower than that of Go Air.

8a.png

(a) Vistara

8b.png

(b) Spice Jet

8c.png

(c) Air Asia

8d.png

(d) Air India

8e.png

(e) Go Air

8f.png

(f) Indigo

Figure 8. AUC for evaluating the acc of proposed model HMIAA for six datasets (a) Vistara (b) Spicejet (c) Air Asia (d) Air India (e) Go Air (f) Indigo