Enhanced Sarcasm Detection in Telugu Dialogue Systems Using Self Attention-Based RNN and Gated Recurrent Unit Models

The intricacies of sarcasm are further compounded in low-resource languages, where existing models and datasets predominantly cater to English and a few other widely spoken languages. Examine that while there has been substantial progress in sarcasm detection in English, the same cannot be said for languages like Hindi, Telugu, and Tamil, which are rich in morphology and often employ unique linguistic features [3, 4]. The scarcity of annotated datasets in these languages presents a significant barrier to developing effective sarcasm detection algorithms. For instance, Telugu, the second most spoken language in India, has a rich literary heritage and a growing presence on social media, yet it remains underrepresented in the field of natural language processing (NLP) [4]. The challenge is exacerbated in languages that are morphologically rich, such as Telugu, where the structure and form of words can significantly alter meaning and sentiment interpretation. To address this gap, recent studies have focused on creating annotated corpora specifically for Telugu, utilizing conversational data from popular comedy shows. This approach not only provides a rich source of sarcastic dialogue but also aligns with the cultural context in which sarcasm is frequently employed. By analyzing these conversational patterns, researchers have developed algorithms that leverage hyperbolic features—such as interjections and intensifiers—to improve sarcasm detection accuracy [1]. The application of deep learning models, particularly attention-based Recurrent Neural Networks and Graph RNN, has shown promising results, with the latter achieving higher accuracy in sarcasm detection tasks [5]. The detection of sarcasm in Telugu presents a unique set of challenges and opportunities. By focusing on the development of tailored datasets and advanced deep learning models, investigators aim to enhance the accuracy of sentiment analysis systems in low-resource languages. This work not only contributes to the field of NLP but also fosters a deeper understanding of the linguistic nuances that characterize human communication in diverse cultural contexts. various methodologies and previous research outlined in study [2].

The prevalence of sarcasm in everyday communication highlights its importance in sentiment analysis, particularly in social media contexts where informal language and cultural nuances play a significant role. Sarcasm is often intentional and can serve various communicative functions, including humor, criticism, or social bonding. Absence of vocal variation and facial cues in written text makes it difficult for algorithms to detect sarcasm reliably. Existing sarcasm detection models have predominantly focused on English, leaving a significant gap in research for low-resource languages, including Hindi, Telugu, Tamil, and others [6]. This gap is particularly concerning given the increasing use of regional languages in digital communication, where sarcasm is frequently employed. In the context of Indian languages, Telugu stands out due to its rich morphological structure and cultural significance. It ranks as the second most widely spoken language in India, and its speakers increasingly engage in social media communication in their native tongue. This shift necessitates the development of automated sentiment analysis tools that can effectively handle sarcasm in Telugu. The morphological richness of Telugu presents unique challenges for researchers, as traditional sentiment analysis techniques may not be directly applicable [7, 8]. The existing literature indicates that while there have been efforts to address sarcasm detection in other languages, the specific challenges posed by Telugu remain largely unexamined.

To address this gap, recent studies have focused on collecting and annotating corpora of Telugu conversational sentences, particularly those derived from comedy shows where sarcasm is prevalent. These datasets typically consist of exchanges that feature a question followed by a sarcastic response, providing a rich source of data for training sarcasm detection models. The development of algorithms that leverage hyperbolic features—such as interjections, intensifiers, and punctuation—has shown promise in enhancing the accuracy of sarcasm detection in Telugu [9, 10]. The application of deep learning models, including (ABRNNs) and RNNs, has further advanced the field, demonstrating improved performance in recognizing sarcastic utterances compared to traditional methods [11].

The significance of sarcasm detection extends beyond mere sentiment analysis; it has implications for various applications, including brand management, customer feedback analysis, and social media monitoring. As sarcasm can distort the perceived sentiment of a message, accurately identifying it is crucial for businesses and organizations seeking to understand public opinion and sentiment accurately. Moreover, the ability to detect sarcasm can enhance the performance of dialogue systems and conversational agents, allowing them to respond more appropriately to user inputs [12, 13]. In the detection of sarcasm in Telugu and other low-resource languages presents both challenges and opportunities for researchers in the field of natural language processing. The development of specialized algorithms and the collection of annotated datasets are essential steps toward improving sarcasm detection capabilities. As the digital landscape continues to evolve, the need for effective sentiment analysis tools that can navigate the complexities of sarcasm will only grow, underscoring the importance of this research area [14, 15].

2.1 Sarcasm detection

Linguistic characteristics are essential for identifying irony and sarcasm in written language, particularly in the context of sentiment analysis. In the realm of sarcasm detection, various lexical and syntactic features have been employed to enhance the accuracy of identifying sarcastic expressions. For instance, a semi-supervised approach has been employed to identify sarcasm in tweets and Amazon product reviews, utilizing two key lexical features: pattern and punctuation-based features. These features contribute to the development of a weighted (KNN) categorization representation for effective sarcasm detection [16, 17]. In addition to lexical features, numerous other linguistic indicators have been explored to identify sarcasm in tweets. These include attributes obtained from linguistic analysis and word frequency, as well as WordNet affect, which provides a semantic framework for understanding emotional connotations. Pragmatic features, and replies, have also been integrated into sarcasm detection frameworks, emphasizing the multifaceted nature of sarcasm as a communicative phenomenon [18]. A well-constructed lexicon-based approach has been proposed, based on the premise that sarcastic tweets often present a contrast between a positive sentiment and a negative situation. This approach utilizes unigram, bigram, and trigram features for lexicon generation, enhancing the model's ability to detect sarcasm effectively.

Hyperbolic features, such as intensifiers and interjections, have been identified as particularly effective in sarcasm detection. For example, the utterance "fantastic weather when it rains" is more readily recognized as sarcastic compared to a non-hyperbolic statement. The presence of hyperbolic language serves as a strong indicator of sarcasm, allowing for more accurate interpretation of the speaker's intent. Furthermore, parsing techniques have been employed to segment tweets into phrases, facilitating the generation of lexicon files that support sarcasm identification in Twitter data. Beyond lexical and syntactic features, the behavioral traits of Twitter users have emerged as valuable indicators for sarcasm detection. The contextual nature of sarcasm necessitates a shared understanding between the speaker and the audience, which can be informed by the user's past behavior and sentiment expressed in previous tweets. This behavioral context has been integrated into sarcasm detection frameworks, drawing on theories drawn from behavioural and psychological research to develop a comprehensive modeling strategy [19]. Moreover, a system has been developed to identify sarcastic tweets in the context of predicting electoral outcomes by analyzing public sentiment on Twitter. This system utilizes various features, including exclamation marks, question marks, hashtags, emoticons, adjectives, and verbs, to ascertain the sarcastic polarity of tweets through a supervised deep learning approach [2]. The incorporation of researchers early tweets provides additional context for sarcasm detection, enhancing the model's predictive capabilities [1]. In the detection of sarcasm in text, particularly in languages like Telugu, requires a comprehensive approach that integrates lexical, syntactic, and behavioral features. The development of advanced frameworks that consider these diverse features is essential for improving the accuracy of sarcasm detection in dialogue systems. This research contributes to the ongoing efforts to enhance sentiment analysis capabilities, particularly in low-resource languages, by leveraging deep learning models such as attention-based RNN and graph RNN to analyze the complexities of sarcastic expressions in Telugu conversational data [20, 21].

2.2 Irony exposure in LRL

The initial work on irony exposure in LRL was conducted on Indonesian social media data [22]. This dataset is physically collected from Twitter and introduced two additional features for irony detection, beyond standard emotion analysis: negativity information and count of interjection words. The study also utilized a transformed version of SentiWordNet for emotion categorization. Thelwall et al. [23] highlighted the vast no. of casual messages posted daily on social networks, blogs, and discussion forums. Existing algorithms have been designed to identify both sentiment and sentiment strength, aiding in understanding emotional expression in informal communication and detecting inappropriate or anomalous affective utterances, which could indicate potentially harmful behavior. A set of characters precisely aimed at detecting irony in social media was developed, along with a novel (MSELA) to address the class imbalance problem in English and Chinese judgments [24]. Additionally, a system is proposed for detecting sarcastic sentences in Hindi using support [25], focusing on features such as emoticons and punctuation marks. To the best of our knowledge, no prior research has been conducted on sarcasm detection in the Telugu language.

4.1 Statistical evaluation metrics for deep learning models

To assess the performance of the proposed deep learning models, SA-RNN and GRU—three statistical evaluation metrics were utilized: Precision, Recall, and F-Score. These metrics provide a comprehensive understanding of how well the models performed in detecting sarcasm.

Precision measures the proportion of correctly identified relevant instances out of all instances predicted as relevant. It indicates how accurate the model is in classifying sarcastic sentences. Recall quantifies the ability of the model to identify all relevant instances, showing how well the model captures sarcastic sentences from the entire dataset. F-Score is the harmonic mean of Precision and Recall, offering a balanced measure that considers both accuracy and completeness in the model's predictions.

The formulas for these metrics are as follows:

Precision= $T_P / T_P+F_P$

Recall = $\frac{T_P}{T_P+F_N}$

F-Score = 2 × Precision × Recall / Precision + Recall

where:

T_P= (True Positive) refers to the number of correctly classified sarcastic sentences.

F_P = (False Positive) refers to the number of non-sarcastic sentences incorrectly classified as sarcastic.

F_N = (False Negative) refers to the sarcastic sentences that were incorrectly classified as non-sarcastic. In this study, the SA-RNN outperformed the GRU, achieving an accuracy of 96% compared to GRU’s 94%. Both models utilize hybrid mode word embeddings to capture semantic and syntactic nuances, improving the identification of sarcasm in Telugu dialogues.

4.2 Analysis in the deep learning approaches

In this study, we analyzed the effectiveness of two deep learning models—SA-RNN and GRU—for sarcasm detection in Telugu conversation sentences. The features used for training were extracted from the annotated corpus, leveraging word embeddings such as GloVe specific to the special tokens to capture semantic and syntactic nuances.

Interjections were regarded as distinct tokens during tokenization, ensuring that the model accurately captured the embeddings for these words.

In certain instances, incorporated into the vocabulary, and their embeddings were either pre-trained during the model's training phase. Interjections were addressed independently in certain sections of the model, while punctuation marks were regarded as tokens in both the GloVe and IndicBERT embeddings. Both sets of embeddings were amalgamated using a concatenation approach. The integrated embeddings were subsequently processed by the model to identify sarcasm.

The training and testing splits were varied across multiple trials, starting from a 80-20 split to evaluate the performance of the models.

The performance of the SA-RNN and GRU models was tested individually, as well as with combined features. Hyper parameters of the model shown in Table 8. The results are averaged over 5 trials for each split ratio. The highest accuracy of 96% was achieved by the SA-RNN model with an 80-20 training-to-testing ratio, outperforming the GRU model, which reported an accuracy of 94%. Both models showed consistent performance across the different split ratios, with minimal fluctuations in accuracy.

Table 8. Hyperparameter values

Models	Hyper Parameter Values
SA-RNN and GRU	Batch Size	256
	Epochs	100
	Hidden Layers	16
	Kernal size	5
	Filters	128
	Dropout, Recurrent dropout	0.2
	Learning Rate	0.001
	Activation Function	Relu,tanh
	Dense Layer (Activation Function)	Softmax
	Optimizer	Adam

2.png

Figure 2. Accuracy of different classifiers of (80-20)

3.png

Figure 3. Accuracy of combined algorithm of different classifiers

In these trials, AdaBoost (AB) achieved the highest accuracy of 93% in the second trial, demonstrating its superior performance compared to other classifiers shown in Figure 2.

The experimental results for the SA-RNN model are shown in Figure 3, indicating that it consistently outperformed the GRU model across different trials. In contrast, GRU, although competitive, demonstrated slightly lower accuracy and exhibited minor variations, particularly in certain split ratios. A detailed breakdown of the 5 trials with the 80-20 split ratio is shown in Tables 9 and 10, highlighting the superior performance of the SA-RNN. Figures 4 and 5 illustrate the performance of the GRU model, where the highest accuracy of 94% is observed. However, the GRU model's performance was slightly less stable compared to the SA-RNN.

Table 9. 5 trials of the classifiers with a (80-20) split

Trial	NB	SVM	DT	KNN	RF
1	0.89	0.88	0.89	0.85	0.9
2	0.89	0.87	0.84	0.85	0.88
3	0.87	0.88	0.84	0.81	0.88
4	0.84	0.83	0.82	0.82	0.86
5	0.84	0.83	0.8	0.81	0.84

Notes: NB: Naive Bayes; SVM: Support Vector Machine; DT: Decision Tree; KNN: K-Nearest Neighbors; RF: Random Forest; AB: AdaBoost

Table 10. 5 trails on SA-RNN and GRU (80-20 split-up)

Trail	SA-RNN	GRU
1	0.96	0.94
2	0.94	0.92
3	0.94	0.92
4	0.9	0.91
5	0.91	0.94

4.png

Figure 4. Performance comparision of SA-RNN and GRU

5.png

Figure 5. Evalution metric comparision on SA-RNN and GRU

4.3 Experimental evaluation

Experiments is conducted using two deep learning models, SA-RNN and GRU, for sarcasm detection on a test set of 1380 Telugu conversational sentences. The performance of both models was evaluated using a confusion matrix, precision, recall, and F-score. The results of the experimental evaluation for both models are presented in Table 11, with a breakdown of true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). The combined performance of both models is shown, as well as individual results for each model. Training and validation accuracy shown in Figure 6.

Further evaluation metrics including accuracy, precision, recall, and F-score are provided in Table 12. The SA-RNN model achieved an accuracy of 96%, outperforming the GRU model, which attained an accuracy of 94%. The comprehensive performance of all classifiers is illustrated in Figure 7 and in Table 13. The RNN model showed higher precision, recall, and F-score compared to GRU.

Table 11. Confusion matrix for SA-based RNN and GRU models

Model	TP	FP	TN	FN
SA-RNN	1239	2	98	41
GRU	1204	4	96	76

6.png

Figure 6. Training performance on models

Table 12. Evaluation metrics for SA-based RNN and GRU models

The results demonstrate that the SA-RNN model, enhanced by GloVe word embeddings, performed better across all metrics. While the GRU model was competitive, it fell slightly behind in both precision and recall. The use of GloVe embeddings played a crucial role in enhancing the models' ability to detect sarcasm in Telugu conversational sentences.

7.png

Figure 7. Combined classifier results across 5 trails

Table 13. 5 trials of the classifiers with combined results