Usability Evaluation of “Ceritoke Bae,” an Android-Based Psycho-Counseling Application for Women with Infertility: A System Usability Scale Study

Infertility is a crisis and chronic condition of human reproductive health that affects aspects of life accompanied by fertility, and psychological and social problems [1, 2]. The World Health Organization indicates that infertility cases worldwide range from 10% to 15% [1]. The prevalence of world data is that 70 million couples experience infertility, an incident of approximately 2 million new infertile couples each year with an increase in the number of cases [3, 4]. Infertile cases in Indonesia are 60 million couples with a prevalence of 15-20% [5]. The Total Fertility Rate (TFR) of Indonesia is significantly below the target growth rate of 1%. The infertility rate in South Sumatra in 2020, at 2.6, has not yet reached the national target. Meanwhile, infertility cases of 10% total 8.6 million residents [6]. Data from the National Infertility Association outlines psychological problems experienced by women with infertile conditions ranging from prolonged stress, excessive worry, despair, helplessness, suicidal ideation and depression, which are mostly experienced by women [7-9]. Psychological pressure on the condition of women and infertile couples will certainly have an impact on the emergence of emotional problems ranging from mild to severe stress, feelings of isolation, guilt, loss of self-control, fear, anxiety to depression, the longer the duration of treatment, the greater the risk of psychological problems [10]. Infertility is classified as a stressful experience, due to various feelings of sadness, anxiety, fear, feelings of loss of sexual identity, low self-esteem, poor self-concept and loss of self-control. If left untreated, it can have a negative impact on the quality of life and the risk of suicide arises [11, 12]. Psychological problems are a woman's experience which is the effect of gender inequality that occurs, pressure from various sources will be felt in infertility conditions so that it has an impact on social relationship partners [13]. In several study anxiety was the first thing felt by infertility patients [7, 14]. The philosophy inherent in Indonesian culture is that children are present in marriage as a symbol of happiness, success, and fertility. This condition certainly causes its psychological problems, especially for wives [15, 16].

Public perception if they do not have children, the wife is most blamed, considered disabled, embarrassing, and a bringer of bad luck. Around 40% of the inability to have children is caused by the wife [17]. Research shows that women experience psychological problems, namely severe stress, feelings of isolation, guilt, loss of self-control, anxiety, fear, and depression, the longer the duration of treatment, the greater the risk of psychological problems [18]. In a qualitative study [19], the results of interviews with 17 infertile women showed that they faced various forms of stigma that could lead to a decrease in self-esteem, even so, women used various defense mechanisms and tried to balance between a sense of empowerment and pressure. As many as 56.6% of women undergoing treatment have an evaluation of increased depression problems [20].

Infertility conditions are classified as stressful experiences, due to various feelings of sadness, anxiety, fear, feelings of loss of sexual identity, low self-esteem, poor self-concept, and loss of self-control, if left untreated it hurts quality of life, and the risk of suicide appears [21, 22]. The number of infertility treatment searches is increasing, while diagnosis and therapy also place depression problems as a heavy burden [23]. Cases of anxiety and depression in infertile couples can be a cause/effect. The above conditions require health professionals to provide psychological services. The psychological counseling approach is appropriate and highly recommended by mental health professionals [11]. The qualitative research conducting interviews with 8 infertile women, it was found that they experienced poor coping mechanisms, while stress that could not be managed well could trigger higher levels of infertility because it resulted in unstable hormones [24].

The experiences, feelings, and opinions of infertile women play an important role in the treatment phase, because it is necessary for them to feel physically, psychologically, and socially healthy, even if they fail, they try every method they hear [25]. Infertility counseling is recommended with a multidisciplinary approach to addressing psychological problems of infertility facing stigma [18]. The results of the study discussed that counseling provides the right solution to reduce stress, anxiety, and depression in infertile women responding to various pressures [26]. Meanwhile, the negative stigma of infertile status is common in society as a source of stressor/as a source of support, this illustrates the need for in-depth identification of sources of strength from aspects of life, and culture [27]. Previous studies have adopted cultural elements (culturally adapted interventions) in infertility psychoeducation counseling to contribute to overcoming depression in infertile women. Cultural beliefs provide support in emotional management reducing depression in women is significant [23].

Infertility counseling is recommended as an integral part of a multidisciplinary approach in addressing various psychological problems for infertile couples, whether undergoing a treatment program or not yet interested in treatment, only to face social stigma from society and family [18]. Many studies have discussed that counseling provides the right solution in reducing the problems of stress, anxiety and depression in women with infertile conditions who have to respond to judgments from friends, family and society [28]. The research results discuss that counseling provides the right solution to reduce stress, anxiety and depression in infertile women responding to various pressures [29, 30]. Nursing and midwifery interventions are a necessity, and they are expected to provide solutions from the perspective of counselors. The importance of appropriate intervention solutions through specialized psycho-counseling measures to provide emotional support is paramount. Furthermore, appropriate intervention needs through counseling services will significantly assist in optimally integrating the role and function of infertility counselors with technology [26, 31]. Cyber counseling media is a counseling strategy that is commonly carried out easily, practically, confidentially, by utilizing technology in the form of computers, laptops, androids [32]. This is also supported by the main role of technology in reproductive health services, especially fertility problems, which provide the right solutions, disseminate more information and store data safely and with wide reach [30, 33].

The current era of advancement and technology has increased the demand for digital health solutions, as people seek more convenient and accessible ways to manage their health. The potential for alternative services offered through digital transformation enhances the quality of healthcare ecosystems and ultimately increases public satisfaction [34]. Psychological problems require healthcare professionals to provide psychological services. A psychological counseling approach is appropriate and highly recommended by mental health professionals [35, 36]. Previous research identifies several mobile applications designed to address psychological problems among infertile couples, including FertiStrong, MediEmo, Internet-Based Mind/Body for Infertility, Virtual Network for Connectedness and Shared Experience, FertiCalm, FertiSTAT, and PreLife. However, a critical examination of these applications reveals significant limitations: they are predominantly developed in Western cultural contexts, delivered exclusively in English, and lack cultural adaptation for non-Western populations. None of these existing mHealth solutions integrate local languages, culturally specific counseling approaches, or address the unique socio-cultural dynamics of infertility in developing countries such as Indonesia, where communal stigma and cultural expectations around childbearing profoundly shape the psychological experience of infertile women [31, 37-39].

Advances in communication technology have changed the way people search for and find information. This is especially true for information related to healthcare services. Therefore, healthcare providers must be able to update their education, health promotion, and counseling strategies to disseminate information from conventional printed materials such as pamphlets and flipcharts to more interactive and up-to-date materials such as mobile applications [40]. Mobile applications have the advantage of being widely available after being developed across various platforms. The usability of a mobile application plays a crucial role in determining its effectiveness in increasing knowledge and awareness of healthcare services. An application must not only be user-friendly but also engaging [41]. Usability is defined as the extent to which a product can be used by a specific user to achieve specific goals effectively, efficiently, while maintaining data security and providing user satisfaction in terms of affordability and speed of a service, particularly healthcare services [42]. The System Usability Scale (SUS) is one of the most widely used questionnaires to assess the usability of a system or product. This scale has been used in numerous surveys to determine the usability of various user interfaces, such as standard operating system-based software interfaces, web pages, mobile applications, and network equipment. Based on the identified gap in culturally adapted digital psycho-counseling tools for Indonesian women with infertility, the aim of this study is to evaluate the usability of the “Ceritoke Bae” Android-based application—specifically designed to incorporate local Palembang language elements and culturally sensitive counseling content—using the SUS, thereby contributing to the emerging body of mHealth research in reproductive health within Indonesian populations.

Ceritoke Bae application is an Android-based application that provides educational and psycho-counseling services in real time, equipped with problem detection features and changes in psychological problem scores that are generally experienced by women with infertility.

Figure 2 shows the landing page of the Ceritoke Bae app. This is the first screen that appears upon opening the app, displaying an illustration of a pregnant woman. This illustration visually indicates that one of the app's primary target users is pregnant women.

The application used is Ceritoke Bae, this application offers several mental health support features, including educational materials in the form of articles, learning modules, assessment features such as Stress Detection and Surveys, and reports to monitor user conditions. Figure 3 shows the main menu and some of the application's content. This system provides features for managing application content, including assessment question management, video tutorial management, and user data management. The image shows an example of a panel for managing each of these features. This process consists of several sequential steps the user goes through, starting with completing the Early Detection questionnaire, watching the narrative and tutorial videos, and completing the Final Test. This series of features concludes with the Results page, which displays a score comparison to demonstrate the user's progress after going through the intervention process within the app.

Figure 2. Ceritoke Bae application landing page

Figure 3. The main menu and material feature display Ceritoke Bae application

3.1 The result of System Usability Scale score assessment

Upon the finalization of the research instrument, the questionnaire distribution process was subsequently set in motion. This particular study garnered the active participation of 60 respondents. The initial sampling frame comprised 150 women with infertility who were registered at the study site; however, after applying inclusion and exclusion criteria (including willingness to participate, active smartphone usage, and accessibility during the data collection period), the eligible target population was reduced to 120 participants. The minimum requisite sample size was then determined through the application of Slovin’s formula to this eligible population of 120, yielding a minimum of 55 respondents. To enhance representativeness, 60 respondents were ultimately recruited. From a methodological standpoint, the selection of a larger sample size is generally considered more judicious when the research objective is to comprehensively delineate the characteristics inherent within each constituent element of a given population.

$n=\frac{N}{1+N e^2}$     (1)

where, n represents the minimum sample size, N denotes the total population, and e refers to the margin of error expressed as a proportion.

In this study, the eligible target population amounted to 120 (after screening from the initial sampling frame of 150), with an error tolerance level set at 10%. Based on these parameters, the minimum required sample size was calculated as approximately 55 respondents; however, 60 respondents were recruited to ensure adequate representation. Once the questionnaires were successfully collected, the SUS score calculation process was carried out using the precisely established formula. Specifically, the formula for calculating the SUS score for each respondent can be formulated as follows:

$\begin{aligned} \text { SUS Score }= & ((R 1-1)+(5-R 2)+(R 3-1)+(5-R 4)+(R 5-1)+(5-R 6)+(R 7-1)+(5-R 8) +(R 9-1)+(5-R 10) * 2.5)\end{aligned}$     (2)

Table 5 presents the results of the SUS assessment, indicating a mean SUS score of 76.08 for the Ceritoke Bae application. This score falls within Grade B (“Good”) and within the “Acceptable” range according to established SUS benchmarking frameworks. Notably, the individual SUS scores exhibited considerable variation, ranging from a minimum of 37.5 to a maximum of 92.5. This substantial range (55 points) suggests heterogeneous user experiences, with some respondents finding the application highly intuitive while others encountered significant usability challenges. The lower scores (below 50) may indicate difficulties related to navigation complexity, content comprehension, or interface design elements that require targeted improvement. Conversely, the cluster of higher scores (above 80) suggests that, for the majority of users, the application’s design effectively supports its intended psycho-counseling functions. Furthermore, an analysis of the individual SUS items reveals that positively worded statements (items 1, 3, 5, 7, 9) related to learnability and perceived ease of use received consistently higher ratings, while negatively worded statements (items 2, 4, 6, 8, 10) concerning system complexity and need for technical support showed greater variance, indicating that a subset of users perceived the application as more complex than desired. The practical implication of the “B/Good” rating is that, while the application is suitable for initial deployment and promotion among target users, iterative design improvements—particularly addressing interface simplification and user onboarding are recommended to elevate the usability score toward the “Excellent” threshold (above 80.3).

Table 5. System Usability Scale calculation results

3.2 Instrument validity test result

The validity of the questionnaire instrument was rigorously assessed utilizing the SPSS software, based on the responses acquired from 60 participating individuals. The outcomes of this validity assessment are systematically presented in Table 6. Employing Pearson's correlation (two-tailed) at a 5% level of significance, the criteria for establishing validity stipulated that the calculated r-value (r_count) for each item must exceed the critical r-table value of 0.254. As delineated in Table 6, the r-values for all ten questionnaire items demonstrably surpassed this critical threshold (r_count > r_table), thereby affirming the statistical validity of each item. In this context, Qi denotes an individual question item, where, strictly for illustrative purposes, Q1 refers to question item 1.

Table 6. Description of validity test result

3.3 Instrument reliability test result

The reliability of the questionnaire was assessed using Cronbach's Alpha, with values exceeding 0.7 indicating adequate reliability. SPSS analysis revealed a Cronbach's Alpha coefficient of 0.884 for the ten questionnaire items. As this value surpasses the threshold of 0.7, the questionnaire is deemed reliable.

Table 7 shows the results of the Cronbach’s alpha reliability test, indicating a coefficient value of 0.884 across 10 items, which falls into the reliable category.

Table 7. Cronbach alpha reliability test result

3.4 Sus score analysis test

The SUS functions as a comprehensive, global metric evaluating users' subjective perceptions of a system's reusability across critical dimensions: effectiveness, efficiency, and satisfaction. This score intrinsically reflects the The SUS score of 76.08 achieved by the Ceritoke Bae application positions it within the “Good” (Grade B) and “Acceptable” categories according to Bangor et al.’s adjective rating scale, with a threshold of 71.1 typically delineating acceptable usability. This score is comparable to the “DiagNurse” mobile application (SUS = 83.3–84.0) developed using the same ADDIE framework, though slightly lower, which may reflect differences in user population characteristics and application complexity. When compared with other Indonesian-developed applications such as the Frostid application (SUS = 72.75) and the Gamification Tari Rakyat app (SUS = 68), the Ceritoke Bae application demonstrates superior usability performance, likely attributable to its culturally adapted content and local language integration. The considerable score variation (37.5–92.5) observed among respondents warrants careful examination. Lower SUS scores may be associated with several factors: varying levels of smartphone proficiency among respondents, interface design elements that certain user demographics found less intuitive, or the inherent complexity of navigating psycho-counseling content through a mobile platform. The higher scores suggest that users who were more technologically literate and those who found the local language content relatable experienced significantly better usability. This finding aligns with user-centered design principles emphasizing that cultural and linguistic relevance directly influences perceived usability. The expert-recommended addition of a real-time chatbox feature represents a concrete improvement direction that could address the interactive counseling needs identified during alpha testing. Such a feature would enable immediate professional support, potentially increasing user engagement and therapeutic effectiveness. Furthermore, the high Cronbach’s Alpha (0.884) confirms that the SUS instrument performed reliably in this Indonesian population context, supporting the validity of the usability assessment. For practical deployment and promotion, the “B/Good” rating indicates the application meets the minimum threshold for clinical recommendation; however, targeted improvements in interface simplification, user onboarding tutorials, and enhanced interactive features are recommended to elevate usability toward the “Excellent” threshold (above 80.3) in future iterations. As shown in Table 8, the percentile range, grade, adjective ratings, and acceptability categories describe the SUS score.

Table 8. Percentile range, grade, adjectives and acceptable categories describes System Usability Scale score

The calculated SUS score places the application within the "good" usability category, corresponding to a "B" grade. This qualitative classification indicates positive user acceptance but suggests that the application has not yet achieved exceptionally high levels of satisfaction. Such a scenario could potentially impede rapid adoption among the target user demographic. Minor enhancements to the application's design or user experience (UX) could effectively convert passively satisfied users into active promoters, thereby increasing the likelihood of independent recommendations. Soliciting direct feedback from these 'passive' users on specific improvements that would earn their endorsement, followed by a thorough analysis of their responses, presents a viable strategic solution.

The mean SUS score of 76.08 achieved by the Ceritoke Bae application falls within the "Good" (Grade B) acceptability range according to Nowicki et al. [43], positioning it above the global average SUS benchmark of 68 [44]. When compared with similar health-related applications developed using the ADDIE model, this score is moderately lower than the DiagNurse mobile application, which achieved SUS scores of 83.3 and 84.0 for nursing students and practicing nurses respectively [43]. However, the Ceritoke Bae score surpasses the Frostid application for stunting detection which obtained a SUS score of 72.75 [48], and substantially exceeds the Gamification Tari Rakyat educational application which scored 68 [47]. This comparative positioning suggests that while the application demonstrates satisfactory usability, there remains opportunity for improvement to reach the "Excellent" threshold.

The substantial variation in individual SUS scores, ranging from 37.5 to 92.5 (a 55-point spread as shown in Table 5), warrants careful examination. Several factors may explain this heterogeneity. First, differences in participants' smartphone proficiency and prior experience with health applications likely influenced their perceived usability. Second, the interface design elements, particularly navigation complexity and information architecture, may have been more intuitive for some users than others. Third, the cultural and linguistic adaptation of the application using local Palembang language, while representing a key innovation of this study, may have been perceived differently by participants with varying levels of comfort with technology-mediated counseling. Analysis of specific SUS dimensions reveals that positively worded items related to learnability and ease of use received higher ratings, while negatively worded items concerning system complexity and the need for technical support showed greater response variance, indicating that perceived complexity remains an area requiring attention.

The Cronbach's Alpha coefficient of 0.884 demonstrates excellent internal consistency of the SUS instrument in this Indonesian population context, exceeding the commonly accepted threshold of 0.70 for reliability [46]. This finding validates the applicability of the SUS questionnaire for evaluating mHealth applications among Indonesian women with infertility. Expert evaluation during alpha testing further identified the need for a real-time chatbox feature to enhance immediate counselor-client interaction, which aligns with current trends in digital mental health services emphasizing synchronous communication capabilities. Based on these findings, future iterations of the Ceritoke Bae application should prioritize three key improvements: (1) simplifying the user interface to reduce perceived complexity, (2) implementing the recommended real-time chat functionality, and (3) expanding culturally adapted content to address the diverse psychological needs of infertile women across different Indonesian cultural contexts [26, 31, 43, 49].