Software Requirement Specification Compatibility Design Method Through Use Case Diagram Artifacts Using Text Mining on Akuonline Learning Application

This section describes all the basic concepts related to processing text data for software documentation, weighting, similarity, and also how to validate research results.

2.1 SRS

SRS is a document that describes in detail the requirements in software development created from the results of the Requirements Elicitation process between Software Developers and Clients [1, 9]. From the Requirement Elicitation activities, it can be used as a reference in software development. This reference can be a set of Functional Requirement (FR) and Non-Functional Requirement (NFR) statements.

If observed in an FR group, then there is the use of words in expressing software development needs, ranging from essential to optional needs; usually, the characteristics of the words have to, can, and may [10, 11].

In this study, several related works have been cited to provide the theoretical basis and methodology used. The following is a further analysis of the similarities and differences between these works and current research:

A text-mining approach [1] was employed to evaluate the similarity between Requirement Elicitation and Requirement Specification results, using Cosine Similarity as the primary measurement tool. This work provides an initial contribution to the application of text analysis in software requirements. However, the approach did not address aspects of statistical validity and reliability. The current study builds upon this foundation by integrating Gwet's AC1 method, resulting in a more in-depth and academically rigorous analysis. This study highlights the need for methodological enhancement in prior research that focused solely on textual similarity.

Various elicitation techniques and challenges in mobile application development were identified, with an emphasis on the importance of SRS quality through a practical approach [9]. Although the study offers valuable insights into the communication process between developers and users, its approach remains descriptive and does not explore the quantitative analysis of requirement suitability. The current study addresses these limitations by systematically applying Text Mining techniques to measure the similarity and validity of requirements. This synthesis illustrates how a data-driven approach can enhance elicitation outcomes that were previously limited to qualitative insights.

While existing approaches have examined input/output changes in functional requirement test cases [10], they fail to assess semantic alignment between requirements and implementation. Our enhanced methodology builds upon this foundation by introducing textual similarity analysis (TF-IDF with Cosine Similarity) and statistical validation (Gwet's AC1) to provide more comprehensive verification. The critique of previous work lies in its limited attention to semantic dimensions and the absence of validity measures in assessing requirement compatibility, which this study addresses as a central focus.

The classification approach for system and software requirements [11] enhances their understanding and management. While this approach offers a solid conceptual framework, classification alone is not sufficient to ensure alignment between requirements and technical implementation. The current study integrates this classification approach with Text Mining-based analysis to quantitatively assess the similarity and validity of requirements. In doing so, it not only synthesizes the conceptual foundation but also critiques its limitations in delivering a more objective and measurable evaluation mechanism based on data.

2.2 UCD

Artifact UCD is made based on FR from the results of Requirement Elicitation. Each FR should be made into one Use Case in UCD [6, 12]. So the number of FR will be the same as the Use Case. In this UCD, all actors' involvement in a Use Case can be observed. After designing the UCD, a description is needed to explain the process that occurs for each Use Case. For this reason, a Use Case Description (scenario) must be made for each Use Case so that each step in a Use Case can be explained in detail through one Step Performed [6, 13, 14].

Prior work [6] developed an application to validate textual content and derive Functional and Non-Functional Requirements from documentation. While this aligns with the current research in its use of text analysis for requirement validation, the present work expands the scope by emphasizing semantic similarity and integrating Text Mining techniques to enhance the depth and accuracy of the validation process.

The development of an automated Use Case extractor using an XML Parser shares a common objective with this study—streamlining the modeling of requirements [12]. However, instead of relying solely on structural automation, the current approach leverages Text Mining to assess both the semantic alignment and the reliability of the extracted requirements, offering a more nuanced and data-driven evaluation.

The use of text processing software to map architectural requirements to quality attributes demonstrates the potential of textual analysis in requirement engineering [13]. Building upon this foundation, the present study incorporates similarity metrics to not only validate but also quantify the alignment between requirements and system attributes, thereby reinforcing the analytical rigor and practical relevance.

The automation of process performance analysis using textual descriptions and event logs [14] highlights the utility of text-based data in operational evaluation. Extending this concept, the current research applies Text Mining to examine requirement compatibility, placing greater emphasis on semantic similarity and statistical validation to support more informed design decisions.

2.3 Text preprocessing

The initial step that must be done in the Natural Language Processing method is Text Preprocessing. This stage must be carried out in order to prepare unstructured text data to be processed. Text Preprocessing activities implemented in this study include [15, 16]:

(1) Case Folding for converting to lowercase.

(2) Tokenization in breaking sentences into words.

(3) Stopwords removal in order to eliminate words that have no meaning in the Text Mining process.

(4) Stemming/Lemmatization to make the root word.

Text preprocessing techniques were used to assess the similarity between Requirement Elicitation and Requirement Specification in the CINEMALOKA application [15]. This text Preprocessing aligns with the current research in terms of methodology, particularly in leveraging preprocessing steps to analyze requirement texts. What sets the present study apart is its incorporation of statistical validation—specifically Gwet’s AC1—to ensure the reliability and accuracy of similarity measurements, thereby extending the analytical depth beyond surface-level comparisons.

The role of text preprocessing in sentiment analysis has been explored using techniques including case folding, tokenization, stopwords removal, and stemming/lemmatization [16]. While employing similar text preprocessing methods, this study applies them to evaluate textual similarity within Software Requirement Specifications (SRS), contrasting with prior sentiment analysis applications focused on emotional tone and opinion mining [16]. Our work shifts the analytical purpose toward semantic alignment and requirements validation.

2.4 Method for weighting and similarity

Term Frequency–Inverse Document Frequency is a method used as a weighting factor for text mining. The product of the weighting of Term Frequency (TF) with Inverse Document Frequency (IDF) of a term/words can be done by using the following formula [17, 18]:

    $t{{f}_{i}}=\frac{fre{{q}_{i}}\left( dj \right)}{\mathop{\sum }_{i=1}^{k}fre{{q}_{i}}\left( dj \right)}$                         (1)

$id{{f}_{i}}=\log \frac{\left| D \right|}{\left| \left\{ d:{{t}_{i}}\in d \right\} \right|}$                         (2)

    ${{\left( tf-idf \right)}_{ij}}=t{{f}_{i}}\left( {{d}_{j}} \right)*id{{f}_{i}}$                         (3)

Term/words that often appear are topics that are sought in a text mining activity. Furthermore, to find the similarity of the results of TF-IDF can use calculations using Cosine Similarity through the use of the following formula [19-21]:

$\begin{gathered}\operatorname{Sim}=\cos (a, b)=\frac{a \cdot b}{\|a\| \cdot\|b\|} =\frac{a_1 b_1+\cdots+a_n b_n}{\sqrt{a_1^2+\cdots+a_n^2}+\sqrt{b_1^2+\cdots+b_n^2}}\end{gathered}$                         (4)

Textual data from Non-Functional Requirements has been processed to evaluate similarity between Requirement Citations and Deployment Diagrams, with proposed improvements for SRS documentation [17]. While the methodology shares common ground with the current study through the use of TF-IDF and Cosine Similarity, this research advances the approach by incorporating statistical validation techniques to ensure both reliability and analytical depth.

While employing computational methods similar to those used in patient support forum analysis (TF-IDF and Cosine Similarity) [18], this study applies them in a significantly different context. The current research adapts these techniques to the domain of Software Requirement Specifications, focusing on semantic alignment and validation of technical documentation rather than user-generated content.

The identification of causal loop variables for systems thinking has been explored using text mining approaches [19]. While sharing methodological similarities, the current study applies text mining to evaluate requirement similarity and validity in software engineering, demonstrating a more targeted application of these techniques.

Cosine Similarity measures have shown adaptability across domains, including their application to image fuzzy sets [20]. Building on this mathematical foundation, our work specifically applies the metric to textual data in SRS, focusing on semantic precision and compatibility assessment rather than visual pattern recognition.

Sentence similarity detection using Cosine Similarity has been investigated for the Malayalam language [21]. While employing the same technical approach, this study adapts the method to evaluate textual congruence between software requirements and their modeled representations, thereby advancing system specification validation.

2.5 Validity and reliability

Gwet's AC1 method is used to test validity and reliability based on the results of the agreement index between two experts/experts who tested a recommended situation [22-24]. Table 1 is the format of the questionnaire results between expert-1 and expert-2. Variations A, B, C, and D are the number of answers based on filling out the questionnaire.

Table 1. Elicitation statement results [1, 22-24]

Furthermore, the formula used to calculate it is as follows:

   $P=\frac{A+D}{N}$                         (5)

${{P}_{1}}=\frac{\left( {{A}_{1}}+{{B}_{1}} \right)/2}{N}$                         (6)

$e\left( Y \right)=2{{P}_{1}}\left( 1-{{P}_{1}} \right)$                         (7)

$AC1=\frac{P-e\left( Y \right)}{1-e\left( Y \right)}$                         (8)

Table 2. Gwet’s AC1 value index [1, 23, 24]

For the measurement index of the value of the AC1 statistical calculation, we can refer to the Aggregation Coefficient, which can be seen in Table 2.

Comparative analysis between Cohen's Kappa and Gwet's AC1 for inter-rater reliability assessment has been conducted in personality disorder evaluations [22]. While employing the same statistical methodology, this study adapts Gwet's AC1 for validating reliability in Software Requirement Specifications (SRS), demonstrating its cross-disciplinary applicability beyond clinical settings.

Inter-rater reliability variance under high-agreement conditions has been extensively analyzed [23]. Building on this statistical foundation, our research specifically applies Gwet's AC1 to assess requirement compatibility consistency in SRS documentation, providing a domain-specific implementation distinct from general agreement scenarios.

Comprehensive methodologies for inter-rater agreement measurement, including Gwet's AC1, have been well-documented across diverse contexts [24]. This study specializes these principles for textual compatibility evaluation in software engineering, particularly for SRS validation, showcasing how general reliability frameworks can be effectively tailored to technical documentation analysis.

By analyzing these similarities and differences, the current study highlights innovations in using various Text Mining techniques and similarity analysis to improve the quality and reliability of SRS documentation. This study also emphasizes the importance of validity and reliability of results through comprehensive statistical measurements.

This section presents a process flow to describe all activities carried out in the research proposal. Based on the datasets that are used as references, the following are the steps and methods implemented to achieve the goals and results (Figure 1), namely:

ping_mu_jie_tu_2025-07-04_142510.png

Figure 1. Methodology

(1) It begins with the Requirement Specification data identified previously, based on the results of the Requirement Elicitation.

(2) Perform extraction based on the results of the Requirement Specification on the FR section only.

(3) Based on the UCD artifact, then the Use Case Description extraction is carried out.

(4) Furthermore, based on all the Use Case Descriptions, the Step Performed can be identified, which will be used as datasets.

(5) In the Text Pre-Processing activity, all FR and Step Performed extraction results are carried out.

(6) The results of the Text Preprocessing will be processed to determine compatibility, which includes the Weight Calculation (TF-IDF) calculation stage, then look for the Cosine Similarity value, and the last stage is to carry out its validity/reliability (Gwet's AC1). Advantages of AC1 Gwet, namely:

Overcoming high agreement bias: AC1 Gwet is more accurate in conditions where the level of inter-rater agreement is very high, avoiding underestimation that often occurs in Cohen's Kappa.

Stability: AC1 Gwet provides more stable and consistent results in various data distribution conditions.

Ease of interpretation: AC1 Gwet values can be easily interpreted using clear agreement categories, such as "Fair agreement", "Moderate agreement", and "Almost perfect".

By using AC1 Gwet, this study can ensure that the validity and reliability of the SRS compatibility measurement results through the UCD artifact are more accurate and reliable.

This research has succeeded in extracting data from SRS by doing Text Preprocessing and similarity to identify the compatibility of FR with Step Performed. The following are four things that can be concluded as the core of this research activity, namely:

(1) Based on the SRS named "Akuonline Learning Application," it has been identified that the Requirement Specification in the form of 11 FRs and 9 NFRs has been identified. The Requirement Specifications match FR (11) and the Use Case Diagram. In this diagram, there are 11 Use Case Descriptions whose number is the same as the number of Use Cases. Step Performed in the Use Case Description is used as a source for compatibility

(2) Through Python NLTK and based on the description text of an FR and Step Performed, this activity resulted in Case Folding, Tokenization, Stopwords Removal, and Stemming, which were applied to 22 documents (d1 to d22).

(3) Based on the results of the Cosine Similarity calculation by comparing the FR matrix and the Step performed matrix, the validation results through the Python version of Gwet's AC1 reliability measurement = 0.299, while for the questionnaire version = 0.379. There is a difference of 0.08.

(4) Referring to the Interpretation of Gwet's AC1 Value, the validity and reliability values are included in the Kappa Index = "Fair agreement" category.

Future work activities for further development are conducting semantic textual similarity activities applied to the SRS documentation case study. Gwet's AC1 Validity and Reliability will be used for validation and will be conducted on experts (people who understand UML) through questionnaire activities. To improve the effectiveness of the compatibility design method, semantic information can be incorporated into the similarity calculation with the following approaches: use of machine learning models, word embedding, use of sentence transformers, semantic similarity calculation, and integration with text preprocessing. By incorporating semantic information into the similarity calculation, future research can improve the effectiveness of the compatibility design method, allowing for deeper and more accurate identification of similarities between software requirements.