A Hybrid Semantic Enrichment Approach for Multi-Label Toxic Speech Detection

The rapid advancement of digital technology has significantly facilitated the production and dissemination of user-generated content, making it faster, more accessible, and cost-effective. Alongside these advantages, this development has also contributed to the widespread proliferation of toxic speech, which can be easily shared across social media platforms with minimal effort and often under the guise of anonymity. The uncontrolled spread of harmful content poses serious risks, including escalating social conflicts, fostering discrimination, and inciting real-world harm [1]. Due to these risks, effective mechanisms to regulate and detect toxic speech have become an urgent necessity.

Among various social media platforms, Twitter (X) serves as a major space for real-time public discourse, allowing users to freely share opinions, criticisms, and emotions. While this openness fosters engagement, it also makes X media social a potential hotspot for toxic speech [2]. The ability to detect and mitigate toxic speech is crucial for preventing cybercrimes, minimizing harm in online communities, and fostering a healthier digital environment [3, 4]. However, identifying toxic speech is particularly challenging due to its diverse expressions, contextual dependencies, and linguistic variations. Toxic speech is often multi-dimensional, involving multiple overlapping categories such as targeting specific individuals or groups, addressing sensitive topics like race, religion, or gender, and varying in intensity from mild to severe hostility [5, 6].

One of the key challenges in toxic speech detection lies in its semantic complexity. Toxic expressions are not always overt but can be conveyed through sarcasm, implicit meaning, metaphors, and coded language, making detection difficult. Moreover, the brevity of social media posts, especially on platforms like X media social that impose character limits, further complicates the ability of models to capture the necessary context [7, 8]. Traditional methods such as Latent Semantic Analysis (LSA) have been employed to extract patterns from short texts [9], but these methods face notable limitations in handling synonyms, understanding word order, and maintaining contextual coherence [10]. Recent studies have attempted to improve these limitations through contextualized variants of LSA, such as LSA-BERT [11], and LSA with transformer-based embeddings [12, 13], yet their performance in dynamic social media environments remains limited.

To improve toxic speech classification, various approaches have been proposed, including knowledge-based semantic enrichment techniques and word embedding representations. Semantic enrichment methods have emerged as more effective solutions for resolving word ambiguity [14] and enhancing semantic comprehension [15]. These methods improve semantic analysis by integrating knowledge-based approaches, word embedding representations, and hybrid techniques [16, 17]. Knowledge-based methods, such as WordNet-based thesauruses, ConceptNet-based ontologies, and specialized lexicons like Kateglo, provide structured linguistic knowledge to resolve word ambiguities and improve interpretation accuracy [12-14]. However, these methods often struggle to handle evolving language trends, newly introduced slang, and domain-specific terminology [18]. Recent evaluations have also shown that rigid ontologies and rule-based knowledge systems lack flexibility when applied to informal, multilingual, and sarcastic content in toxic speech detection tasks [19-21].

On the other hand, word embedding techniques like Word2Vec, GloVe, and BERT have been widely used to model semantic relationships between words [16, 17]. While these methods capture contextual similarities, they often lack explicit semantic structures, making them prone to misclassification in cases of subtle meaning variations. For example, embedding-based models may assign high similarity scores to semantically unrelated terms due to shared contexts, introducing noise in the classification process [22].

To overcome semantic analysis limitations such as contextual bias, semantic restrictions, and challenges in multi-label classification, hybrid approaches integrate multiple methodologies to achieve a more comprehensive understanding. These methods have demonstrated effectiveness in enhancing semantic analysis performance [23] and improving classification accuracy [24], particularly for multi-label datasets. Classification accuracy serves as a critical metric in evaluating machine learning models. Traditional machine learning techniques often struggle to capture the complexities of label dependencies in toxic speech classification. For instance, models such as Support Vector Machine (SVM) and Random Forest Decision Tree (RFDT) rely heavily on manually engineered features and are generally less effective in capturing intricate label relationships. Even deep learning approaches such as Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM) networks face challenges as they process data sequentially and lack the parallelization capabilities necessary for real-time toxic speech detection [25]. Recent benchmarks have shown that LSTM-based models still suffer from latency issues and underperform compared to CNNs [26]. LSTM-based processes data sequentially but is not effective in parallel, while CNN can process data in parallel and is faster in training [6].

By addressing the existing challenges in toxic speech classification, this study introduces a hybrid feature extraction approach based on Semantic Enrichment and Convolutional Neural Networks (CNN) to improve multi-label toxic speech classification. The model integrates various natural language processing (NLP) techniques such as back translation, word sense disambiguation, text expansion, and semantic similarity measurement to provide deeper contextual understanding and richer feature representations. By enhancing input texts with additional semantic context, this approach improves classification accuracy across multiple labels, significantly enhancing model performance compared to traditional approaches.

This study makes several important contributions. First, it proposes a novel hybrid feature extraction model that leverages semantic enrichment techniques to enhance multi-label classification performance. By integrating structured linguistic knowledge with deep learning architectures, the model improves the ability to detect contextual variations and complex toxic speech patterns [27]. Second, the study demonstrates that the HSE-CNN approach outperforms traditional models such as SVM, RFDT, BiGRU, and BiLSTM, particularly in addressing challenges related to ambiguity, brevity, and multi-label dependencies in toxic speech classification. Third, this research provides a detailed analysis of the impact of each semantic enrichment technique, offering insights into how different linguistic enhancements contribute to model performance. These findings are valuable for advancing AI-driven content moderation systems and improving the reliability of automated toxic speech detection in social media environments.

By tackling the inherent challenges in toxic speech classification, this study contributes to the development of robust, context-aware models that can be applied in various content moderation applications, policy enforcement strategies, and social media monitoring tools. The integration of semantic enrichment with deep learning represents a promising direction for enhancing the interpretability and effectiveness of AI-based toxic speech detection systems.

In this study, we introduce a Hybrid Semantic Enrichment and Neural Network based on CNN (HSE-CNN) method to enhance semantic understanding and improve the performance of toxic speech classification in multi-label datasets. The methodology involves several preprocessing steps followed by semantic expansion. We employ techniques such as back-translation, word sense disambiguation, text expansion, and semantic similarity to create a robust, hybrid approach. Vector representations are generated using IndoBERT, and deep learning models such as CNN, BiGRU, and BiLSTM are employed for classification to identify the optimal model for toxic speech detection. The overall schematic of this study is illustrated in Figure 1.

1.png

Figure 1. Working model of toxic detection

3.1 Data

The dataset used in this study consists of 13,169 tweets from X media social [31], annotated with 12 multi-label categories relevant to toxic speeches. Thirty linguists conducted the annotation to ensure the validity of the labels. Multi-label classification introduces unique challenges such as class imbalance, high dimensionality, and label interdependencies. For instance, a tweet may simultaneously be labeled as both Abuse and Toxic speech if it contains offensive language or threats. To address these complexities, the labels are organized into hierarchical groupings based on three key aspects: target (individual or group), category (religion, race, physical, gender, and offensive), and intensity level (weak, moderate, or strong). This hierarchical structure provides a systematic and interpretable approach to multi-label annotation.

Multi-label data arises when a sample belongs to more than one class, making classification challenging as the model must recognize multiple relevant labels. This issue is especially pronounced in cases of class imbalance, high dimensionality, and label interactions. For example, a text could be labeled both as Abuse and Toxic speech if it contains offensive language or threats. To handle this, labels are organized hierarchically by target, category, and intensity. Each label is assigned an output layer corresponding to its characteristics, allowing for a comprehensive understanding of the hierarchy of toxic speech labels.

3.2 Hybrid semantic enrichment

To improve text comprehension, this study employs HSE through four stages: back-translation, word sense disambiguation, text expansion, and semantic similarity, which are explained and illustrated in Figure 2.

Based on Figure 2, the HSE process begins with identifying the target word in a sentence, such as "lit" in "That party was lit!". The target word is then analyzed using the Lesk Algorithm in the word sense disambiguation (WSD) stage to determine the most appropriate synset from lexical sources like WordNet and Kateglo. The disambiguation process identifies relevant synonyms for "lit", such as "amazing", "very enjoyable", and "great". Next, a text expansion strategy is applied by incorporating these synonyms into the original sentence, resulting in "That party was lit, amazing, great, very enjoyable". To further enhance semantic understanding, each word in the sentence is represented as a vector using BERT embeddings. For example, "amazing" has a vector representation of [0.5, 0.7], "very enjoyable" [0.3, 0.8], and "great" [-0.6, 0.4]. Finally, semantic similarity is measured using Cosine Similarity, which evaluates contextual relationships between words. For instance, ("amazing", "damned") = 0.95 indicates a strong semantic association, whereas ("very enjoyable", "abominable") = 0.02 suggests a weak relationship. This process enables NLP models to better understand word context, resolve linguistic ambiguities, and enhance performance in semantic analysis and text classification tasks.

3.2.1 Back-translation

In this stage, the text is translated into a different language and then back to the original to enhance comprehension and refine sentence structure. Using the Google Translate API, the text is first translated into another language and then returned to its original form. This process captures a variety of expressions and nuances, enriching the semantic content of the text. The back-translation follows the equation, as shown in Eq. (1).

T' = Back-translation (T)     (1)

where, T is the original text, and T' is the resulting back-translated text. For example, "That party was lit!" is translated into German as “Die Party war der Hammer!” and then back to English as “The party was the hammer!”, which reveals linguistic variations and deepens semantic understanding.

3.2.2 Word sense disambiguation

Word sense disambiguation is used to resolve ambiguities in word meanings based on context. This is achieved by utilizing WordNet and the Lesk algorithm to identify the correct meaning of a word in a given context, as shown in Eq. (2):

$synset(t, C)=\operatorname{Lesk}(t, C, WordNet)$     (2)

2.png

Figure 2. Example of the HSE process

where, t represents the word, C represents the context, and synset(t, C) represents the resulting word sense. This process helps clarify ambiguities, providing more accurate interpretations of words. For example, the word “lit” in “That party was lit!” could mean “illuminated” or “very enjoyable.” The Lesk algorithm clarifies that in the context of a party, “lit” means “amazing,” yielding a clearer statement: "That party was amazing!".

3.2.3 Text expansion

Text expansion is a technique used to enrich the original input text by adding semantically related words to improve contextual understanding. In this study, we expand the input text by appending relevant synonyms to selected keywords. Synonym candidates are initially retrieved from lexical databases such as WordNet and Kateglo. However, to ensure that only contextually appropriate synonyms are included, we implement a two-stage filtering process.

First, we apply a semantic similarity threshold using IndoBERT pre-trained embeddings, retaining only those synonyms whose cosine similarity with the original word exceeds 0.7. This threshold ensures that added words maintain close semantic proximity to the original context.

Second, we apply part-of-speech (POS) tagging and contextual matching. Only synonyms that share the same POS tag and appear in similar syntactic and semantic contexts are selected. This helps avoid noise from grammatically or semantically incompatible additions.

• Rule 1: S ← Synonyms(target_word) from {WordNet ∪ Kateglo}.
• Rule 2: S' ← {w ∈ S | POS(w) = POS(target_word)}
• Rule 3: S'' ← {w ∈ S' | cosine_similarity(IndoBERT(w), IndoBERT(target_word)) ≥ 0.7}
• Rule 4: S_final ← {w ∈ S'' | context_match(w, context_window) = TRUE}

      Example: Synonym Selection Process for the word "berbahaya (dangerous)":

• Step 1: Retrieve candidate synonyms from lexical databases such as WordNet Bahasa and Kateglo. Result: ["mengancam (threatening)", "mematikan (deadly)", "menyeramkan (scary)", "tidak aman (unsafe)"]
• Step 2: Filter the retrieved candidates to include only those that match the target word’s POS (e.g., adjectives). Result after filtering: ["threatening ", " deadly ", " scary "]
• Step 3: Compute cosine similarity using pre-trained IndoBERT embeddings. Only synonyms with a similarity score ≥ 0.70 are retained.

1. " threatening " → 0.82
2. " deadly " → 0.79
3. " scary " → 0.65

• Step 4: The final set of synonyms used for text expansion: ["threatening ", " deadly "].

3.2.4 Semantic similarity

To evaluate the semantic similarity between the original and expanded texts, we use cosine similarity and pre-trained BERT. This ensures that the expanded text maintains semantic coherence. Semantic similarity between words or phrases in the text is computed using Eq. (3).

$\operatorname{Sim}(v 1, v 2)=\frac{v 1 . v 2}{\|v 1\|\|v 2\|}$     (3)

where, v1 and v2 are embedding vectors of two words generated by BERT.

3.3 Classification model

The toxic speech classification process consists of two main stages: data partitioning and model development. Data partitioning involves five types of preprocessed and semantically enriched data: data without semantic enhancements, back-translation data, text expansion and disambiguation data, text expansion data alone, and hybrid semantic enrichment data. The data is split into 80% for training and validation, and 20% for testing, using k-fold cross-validation with k = 5 to optimize data usage. Model development and evaluation involve the use of IndoBERT embeddings and a CNN-based text classification model. This model is compared with others, including Bidirectional Gated Recurrent Unit (BiGRU) and Bidirectional Long Short-Term Memory (BiLSTM), to assess their effectiveness in toxic speech classification.

The text classification model utilized is a 1D CNN, known for its effectiveness in text classification tasks. The process involves constructing and training the model, followed by validation and testing. Hyperparameters such as the number of convolutional layers, kernel sizes, and activation functions are tuned for optimal model performance. Validation is carried out using parameters such as Batch Size, Epoch, and Learning Rate.

The CNN architecture includes several layers: input, embedding, Convolution Layer 1, pooling layer, Convolution Layer 2, pooling layer, fully connected layer, dropout layer, and output layer. This architecture is adapted from previous research [25, 29], to extract features from word or character sequences and reduce feature dimensions. The model utilizes a one-dimensional convolutional layer with TensorFlow, featuring 128 kernel filters of size 3 (Conv1D (128, 3)), as shown in Figure 3.

3.png

Figure 3. Architecture of the CNN model

3.4 Performance metric

The performance of the proposed method is evaluated using several metrics: precision, recall, F1-score, accuracy, and AUC (Area Under Curve). These metrics are calculated using Eqs. (4)-(7):

F1_Score $=2 * \frac{\text { Precision } * \text { Recall }}{\text { Precision }+ \text { Recall }}$     (4)

Accuracy $=\frac{T P+F N}{T P+F N+T N+F P}$     (5)

Recall $=\frac{T P}{T P+F N}$     (6)

Precision $=\frac{T P}{T P+F P}$     (7)

where, TP, FN, TN, and FP correspond to true positive, false negative, true negative, and false positive, respectively. In addition, the AUC (Area Under Curve) metric is used to evaluate how well the model distinguishes between positive and negative classes (Eqs. (8) and (9)). It is calculated by plotting the Receiver Operating Characteristic (ROC) curve, which shows the relationship between True Positive Rate (TPR) and False Positive Rate (FPR):

$T P R=\frac{T P}{(T P+F N)}$     (8)

$F P R=\frac{F P}{(F P+T N)}$     (9)

These metrics provide comprehensive insights into the model's performance and its ability to accurately classify data.

The development of a toxic speech classification model has been successfully carried out using a hybrid semantic enrichment strategy combined with a Convolutional Neural Network (HSE-CNN). The model leverages the lexical database WordNet for semantic feature engineering, enabling the grouping of words into synsets and capturing lexical relationships such as synonyms, antonyms, hyponyms, and meronyms. This semantic enhancement has proven essential for improving classification performance, especially in handling the contextual complexity of Indonesian multi-label toxic speech data. The proposed model achieved a notable performance of 93% accuracy using the HSE-CNN approach.

Before modeling, semantic analysis of unstructured text data from X (formerly Twitter) was conducted, emphasizing preprocessing steps such as case folding, normalization, and back-translation. These steps ensure the structural integrity of input sentences and help the model better interpret context and meaning. Additionally, contextual embeddings such as BERT were used to represent semantic information at the sentence level. Label definitions were adapted from prior studies on toxic speech in Indonesian language contexts [32], simplifying from 12 to 9 labels for better label balance and clearer categorization.

Hyperparameter tuning was performed using K-Fold Cross-Validation, resulting in optimal settings: a learning rate of 0.001, batch size of 16, and 30 epochs. The HSE-CNN achieved 76% precision, 84% recall, 80% F1-score, 93% accuracy, and 91% AUC, surpassing other semantic enrichment baselines such as back translation (F1-score 64%, accuracy 86%), semantic text expansion (F1-score 60%, accuracy 85%), and expansion with disambiguation (F1-score 69%, accuracy 88%).

Table 4. Comparative analysis of toxic detection model with state of art

To further assess model performance, additional experiments were conducted with BiGRU and BiLSTM models using the same settings. Results showed CNN consistently outperformed both, with a 4% and 5% accuracy margin, respectively, reaffirming CNN’s strengths in processing parallel inputs and extracting local features.

The comparative performance of HSE-CNN with other state-of-the-art models is shown in Table 4.

Previous models such as BiGRU + IndoBERT [43] and SVM [32] were either limited to binary classification or showed lower accuracy for multi-label classification. While IndoBERT alone [46] achieved strong results, the integration of semantic enrichment in HSE-CNN provides a substantial improvement. Studies like [29, 44] also employed translation and classifier chains, with limited effectiveness for multi-label scenarios, confirming the advantage of the proposed HSE-CNN approach.

Although the proposed HSE-CNN approach demonstrates high performance under experimental conditions, several practical limitations must be acknowledged for real-world deployment, particularly in social media environments:

• Computational Efficiency: The semantic enrichment steps (including back translation, text expansion, and disambiguation), introduce significant computational overhead. These steps are not optimized for real-time processing and may hinder deployment in large-scale social media monitoring systems.
• Real-Time Application: While CNN itself is efficient during inference, the entire HSE-CNN pipeline is not currently suitable for real-time toxicity detection. For operational deployment, it may require optimization strategies such as model quantization, knowledge distillation, or switching to lightweight transformer variants (e.g., DistilBERT, TinyBERT).
• Scalability: Real-world applications on platforms like X (Twitter) require attention to API latency, memory efficiency, and throughput. The current approach, although effective, needs streamlining of preprocessing and enrichment stages to meet performance demands at scale.
• Language Evolution: Toxic language in social media is constantly evolving. The static training of the current model may result in decreased performance over time unless it is periodically retrained or integrated into continual learning frameworks.

Future work will address these challenges by focusing on the optimization of semantic enrichment modules, real-time inference efficiency, and adaptive learning techniques. These enhancements are necessary to bridge the gap between research and deployment in live moderation systems on social media platforms.