Fivi Rahmatus Sofiyah*
| Ami Dilham | Andrew Satria Lubis | Hayatunnufus | Jonathan Liviera Marpaung | Diapari Lubis
© 2024 The authors. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
Customer relationship management is complicated by a lack of understanding of customer attributes and needs. Transformational and inventive techniques are needed to improve customer service. WhatsApp-based AI Chatbots can improve academic service efficiency and quality. The platform's usability and interaction with customer management systems are the main reasons. By adopting the AI Chatbot, clients and internal users will receive consistent and correct responses. Qualitative interviews and survey analysis are used to collect data for the study. Qualitative interviews are utilized for qualitative analysis, while survey analysis validates research survey data. The report reveals 83.8% customer satisfaction and 75.3% customer loyalty. The study found that the chatbot's ability to minimize service time greatly affected customer satisfaction. Due to their consistency and speed, chatbots in customer care have improved consumer satisfaction. Customers like chatbots' speed and efficiency, which allow them to get help immediately. The chatbot's ability to handle common questions and tasks frees up staff to handle more complex and urgent issues, improving service.
chatbot, artificial intelligence, modelling, assistant virtual, customer relationship management
Organizations often face challenges that can hinder the successful implementation and effectiveness of customer relationship management (CRM) systems. A key difficulty in managing customer relationships lies in the lack of deep understanding of consumer behavior, preferences, and needs. Without a thorough grasp of these aspects, delivering a personalized and meaningful experience becomes difficult. CRM's dependence on accurate and up-to-date customer data is also problematic, as inconsistent or incomplete data can lead to faulty analysis and poor decision-making. Integrating complex systems within large organizations, which involve multiple departments, adds another layer of difficulty. Specifically, merging CRM systems with other business functions like sales, finance, or customer service can be time-consuming and challenging. Insufficient integration might hinder data visibility and operational efficiency. If the company culture does not support these initiatives, employees may use the CRM less. Failure to secure and protect client data could lead to unauthorized access and breaches [1, 2]. Effective customer relationship management (CRM) relies heavily on strong collaboration across different organizational units, such as sales, marketing, and customer support. Inadequate communication between departments can compromise the effectiveness of CRM, emphasizing the importance of fostering collaboration. Efficient interdepartmental communication and collaboration are crucial for CRM systems to deliver a seamless and integrated customer experience. The lack of regular updates and improvements emphasizes the necessity to maintain and upgrade the CRM system to meet company and customer needs. Without enough resources for these changes, the CRM system may become outdated and ineffective [3]. When considering the integration of an AI Chatbot into secretarial services, it is important to evaluate it from different perspectives. This includes looking at operational efficiency, service quality, and the satisfaction of both customers and internal users [4]. Implementing efficient and effective technologies can automate secretaries' everyday responsibilities like answering questions, delivering basic information, maintaining schedules, and handling electronic communications. For Smutny and Schreiberova, chatbots are AI systems that demonstrate human-computer interaction [5]. Advanced AI technology allows chatbots to communicate like humans. Chatbots allow companies to consistently and accurately respond to customer and internal user inquiries. AI allows chatbots to act as personal assistants, booking flights and making restaurant reservations, among other jobs [6-8].
Chatbots on WhatsApp, Line, Telegram, and Slack enable simple, customizable access. Successful AI Chatbot implementation should improve customer and employee satisfaction [9]. A fast and accurate AI Chatbot can reduce client wait times and solve common issues. Company reputation improves when clients and employees feel valued and supported. Companies can provide 24/7 customer service with AI Chatbots instead of secretaries [10]. Companies with clients or branches in multiple time zones need this. Customers and employees will feel supported after hours.
Chatbot use in customer service has revealed several noteworthy findings, including a significant increase in service efficiency due to speedy responses, which customers value [11]. Chatbots also handle frequently requested queries and repetitive duties, freeing up people to handle more difficult concerns. Chatbots improve client happiness by providing timely service and easy access. However, research shows that a chatbot's natural language understanding and response considerably affect consumer satisfaction. To leverage chatbots' customer service benefits, natural language processing technology must be continuously improved [6].
Client and internal user data can be collected by AI chatbots. Analysis of this data can reveal patterns, needs, and concerns. Management and secretarial teams can use such insights to make data-driven decisions to improve service quality. According to Sucupira Furtado et al. [12], technology can improve living. Information on the internet, especially social media, promotes health education. Based on chatbot customer satisfaction research, Padangsidimpuan is creating an intelligent AI chatbot. Help customers and communities in real time.
1.1 Chatbot satisfaction and acceptance
Previous studies have shown that the primary factor influencing chatbot acceptance is user satisfaction. A positive experience of satisfaction reflects a favorable attitude toward the performance of chatbots and information systems [13]. User happiness can be influenced by chatbot aspects such simplicity of use, interoperability, and information quality and completeness [14]. Users appreciation of chatbot services can also improve their experience. Research shows that individuals are happier with chatbots that offer improved engagement, human-like traits, anthropomorphic qualities, intimacy, and empowerment [15]. Chatbots' practical, entertaining, technological, and social benefits might boost user happiness, but perceived privacy risks may lower it [16]. Despite previous research on chatbot experiences, a thorough study of chatbot satisfaction variables is needed. Existing research has concentrated on certain areas of chatbot use, but there is a paucity of holistic study that addresses user opinions on many aspects of chatbot performance [17]. This study considers chatbots task-oriented information systems.
To ensure user pleasure and chatbot adoption, three quality traits must be prioritized. Poor chatbot system quality can cause user issues, poor decision-making, and diminished motivation to utilize the chatbot. A high-quality chatbot's information quality affects how users evaluate and trust the system [18]. Chatbots may disappoint users if their service quality is poor. Some studies have shown that these three perceived quality characteristics improve chatbot satisfaction, although more research is needed to understand the factors that contribute to each [19]. Also, the original quality dimensions and scales may not apply to a task-oriented chatbot. Chatbot technology is new, therefore assessing its quality may need considering factors that are different from the initial intent. However, past research has included perceived quality criteria in operational process evaluations. This study examines quality dimension-specific aspects to improve comprehension [11, 20, 21].
This mixed-method study employed online questionnaires and focus groups. Mixed-methods research on enjoyment and usage intents deepens understanding and creates a new framework [22]. Focus group interviews were done to determine the major factors affecting usage intentions using the Chatbot success model and privacy emphasis. The research focused on privacy, especially in new technology. Qualitative data can disclose unanticipated issues, hence this strategy was chosen. Detail was provided by two focus group interviews with academic service providers and student administrative service users. Chatbot technology, cognitive needs, emotional responses, and privacy concerns were discussed in these interviews. 400 people used a task-oriented chatbot in the study. Get scholarly mail using a WhatsApp chatbot [23-25].
2.1 Chatbot smart administration
This project aims to construct a smart administrative AI chatbot. WhatsApp integration boosts this chatbot's productivity. The chatbot will securely save client data in a cloud-based database for 24/7 real-time storage. This AI Chatbot provides real-time student academic administration support to clients. It aims to improve the manual administrative management method, which is limited to working hours. Figure 1 depicts administrative administration.
Figure 1. An illustration of administrative management
Figure 1 illustrates the administrative management process, emphasizing the difference in service duration. While manual services are limited to a 7-hour workday, the AI Chatbot is capable of delivering real-time client support 24/7. This extended service availability significantly influences the management of customer relationships within the organization.
2.2 Chatbot modeling
Implementing chatbot modeling for student academic administration requires integrating WhatsApp with a chatbot-based strategy and adding a comprehensive database of academic administrative operations [9, 26, 27]. The database should include course registration, academic leave requests, transcript applications, and other procedures. Chatbots must be trained on a labeled chat dataset to understand and respond to student requests. Chatbots use LSTM or Transformer algorithms to interpret and follow conversations. Secure APIs from university academic database systems let chatbots verify student academic records and adjust enrollment status [14, 22, 28, 29]. When chatbots cannot comprehend or understand requests, they should redirect to human agents for help. The modeling approach helps academic services reduce inefficiencies like excessive proposal file printing and long wait times by meeting students' demands. Mail processing requires only basic student data, simplifying academic file management.
Students benefit greatly from chatbot services, according to the satisfaction study. However, chatbots still fail to respond due to call code or trigger word anomalies during testing. WhatsApp chatbots use several technologies to enable real-time user-system interactions [29]. The WhatsApp API integration server receives the user's first WhatsApp message. Intermediary server sends message to chatbot engine. NLP is used to build these chatbots to understand and respond to human speech. Chatbots evaluate new messages, detect user intent, and respond using AI models and machine learning techniques. After sending the results to the WhatsApp API integration server, WhatsApp users receive them in real time [30]. A chatbot can check order status, schedule appointments, and recommend products by connecting to a database or other backend systems. Continuous learning helps conversational bots understand and reply to many queries and requests. Many integrations provide security features to protect user data. Customers can simply interact with company services via WhatsApp without speaking to a human person unless they need help. Researchers enhanced the chatbot architecture to streamline and improve service delivery in this study. Chatbots have been studied in healthcare, corporate marketing, and e-commerce. Users must download and install platform-specific apps to use most chatbots.
2.3 Mathematical approach on structural equation modeling
Partial Least Squares (PLS) is a statistical technique employed to design predictive models and investigate the connections between independent variables (X) and dependent variables (Y) using latent structures. This approach is frequently employed in research to evaluate models that involve hidden variables and intricate connections, which are commonly encountered in evaluations of consumer happiness and loyalty. X is a set of independent variables that characterizes the attributes of chatbots that influence pleasure. The features encompassed are responsiveness, precision of information, and user interface design. These features are evaluated by the analysis of numerous observed indicators. The vector represents the dependent variables that indicate client loyalty, such as the intention to reuse and referral. This latent construction is derived from the observed indicators through a calculation process. The relationship between the latent satisfaction ($\eta$) structure and customer loyalty (X) can be expressed by the following structural equation:
$\xi=\beta \eta+\varepsilon$ （1）
The path coefficient $\beta$ represents the magnitude and direction of the influence of pleasure on loyalty, whereas $\varepsilon$ refers to the error term. Each latent structure is measured through observed indicators, which can be modeled with the equation:
$x=\lambda_x \eta+\delta_x, \quad y=\lambda_y \xi+\delta_y$ (2)
where, x and y are observed indicators for satisfaction and loyalty, $\lambda_x$ and $\lambda_y$ are the measuring load that connects the indicator to its construction, $\delta_x$ and $\delta_y$ is the measurement error. PLS employs iterative algorithms to decrease prediction errors of dependent variables, typically using a method known,
The purpose of evaluating the measurement model in SmartPLS analysis is to verify the validity and reliability of the structure as evaluated by its indicators. This examination evaluates composite reliability (CR) to measure internal consistency, average variance extracted (AVE) to determine convergence validity, and validity discriminatory testing to verify the distinctiveness of different constructs. Through conducting these evaluations, researchers may verify the reliability and validity of the measurement model prior to analyzing the structural model. This ensures that the overall analysis results are more credible and corrects a PLS-Path algorithm.
The Determination Coefficient (R2) in Partial Least Squares (PLS) analysis is calculated to evaluate the extent to which an independent variable in a structural model can account for the variation in the dependent variable. R2 quantifies the predictive power of the model by indicating the percentage of variability in the dependent variable that can be accounted for by the independent variables. A higher R2 value implies a greater ability of the model to explain variability in the data, suggesting that the model has strong predictive capability. Within the framework of Partial Least Squares (PLS), this aids researchers in comprehending the efficacy of underlying structures in forecasting desired results, while also evaluating the overall excellence and appropriateness of models.
$R^2=1-\frac{\sum\left(y_i-\widehat{y_i}\right)^2}{\sum\left(y_i-\overline{y_i}\right)^2}$ (3)
The predictive relevance (Q2) value is calculated in PLS analysis to evaluate the model's capacity to forecast data that is not incorporated into the model estimates. Q2 employs blindfolding techniques to assess the predictive strength of a model, resulting in a value that indicates the model's ability to accurately predict indicators on endogenous constructions. The model's predictability is enhanced by a positive Q2 value, whereas a Q2 value that is negative or nearly zero suggests a low predictability. Researchers can enhance their confidence in the validity of models in practical applications by verifying that models not only align with existing data but also have the capacity to generalize and make precise predictions for new data. This is achieved by tallying Q2.
$Q^2=1-\frac{\sum\left(\hat{y}_{i, \text { omit }}-y_i\right)^2}{\sum\left(y_i-\overline{y_i}\right)^2}$ (4)
The moderation test in questionnaire analysis using SmartPLS is designed to ascertain whether the relationship between two latent variables is influenced by an additional variable known as the moderator variable. The moderation test involves the addition of the interaction between the independent variable and the moderator variable to the model to determine the extent to which the presence of the moderator variable influences the effect of the independent variable on the dependent variable. To accomplish this, the path coefficient of the interaction is inserted and tested for significance. The moderator variable moderates the relationship between the independent variable and the dependent variable if the interactions are significant. Moderation tests offer a more profound understanding of the dynamics of relationships between variables in models, thereby assisting researchers in comprehending the contexts or conditions under which these relationships may be more or less robust.
$\eta=\beta_1 \xi+\beta_2 M+\beta_3(\xi \cdot M)+\zeta$ (5)
where, M is the moderator variable.
2.4 Research hypotheses
Upon analyzing the interview transcripts, distinct themes and subthemes were identified. Interviews revealed that three specific concerns related to quality and privacy have the potential to impact user satisfaction and willingness to employ task-oriented chatbots. User Satisfaction (US) and loyalty (UL) for chatbots can be influenced by three factors: System Quality (SQ), Information Quality (IQ), and Service Quality (VQ). System Quality encompasses flexibility, availability, and reaction speed. Information Quality refers to relevance and completeness. Service Quality includes enjoyment, reliability, and empathy.
H1: SQ will positively impact US with chatbots.
H2: SQ will positively impact UL with chatbots.
H3: IQ will positively impact US with chatbots.
H4: IQ will positively impact UL with chatbots.
H5: VQ will positively impact US with chatbots.
H6: VQ will positively impact UL with chatbots.
H7: US will positively impact UL with chatbots.
H8: US will mediate the effect of SQ on UL.
H9: US will mediate the effect of IQ on UL.
H10: US will mediate the effect of VQ on UL.
The study’s hypotheses are illustrated in Figure 2, showing the relationships between the latent variables connected through a mediator variable.
Figure 2. Chatbot modeling on cloud
A sample of 400 students was surveyed online in order to analyze qualitative data. We employed WhatsApp chatbots to conduct surveys. Unanimous agreement is required for the survey. The survey consists of three parts. Following the study of the introduction, participants analyzed chatbots that are designed for specific tasks. The screening query asked whether or not they utilized this chatbot. Inexperienced participants were excluded from the study and their data was eliminated. They then reported the frequency of employment of task-oriented chatbots. Additionally, queries regarding the assessment of variables based on theory were also responded to. User satisfaction was evaluated using a semantic differential scale, whilst all other criteria were evaluated using a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Ultimately, the researchers gathered data on age, gender, education, and monthly income.
3.1 Experimental simulation
The sample was measured using a SmartPLS 4.0 structural model after investigation. Table 1 shows the distribution of AI Chatbot users and CRM enhancement survey participants.
Table 1. Data collecting
|
Measure |
Type |
Frequency |
Percentage |
|
Gender |
Male |
170 |
42.5% |
|
Female |
230 |
57.5% |
|
|
Age |
18-23 |
400 |
100% |
|
24-29 |
0 |
0% |
|
|
Study Program |
D3 Secretary |
119 |
29.75% |
|
D3 Finance |
93 |
23.25% |
|
|
D3 Statistics |
75 |
18.75% |
|
|
D3 Computer Engineering |
113 |
28.25% |
3.2 Structural model
Our questionnaire analysis parameter estimates forecast SmartPLS 4. This software weights test indicators with PLS. We then employ bootstrapping with 5000 subsamples and 0.05 two-way test significance. After bootstrapping, the structural model is blindfolded to evaluate reliability, eliminating 7 samples per iteration. Results of structural model testing shown in Figure 3. The factors' strong and plausible link is shown in Figure 3.
Figure 3. Chatbot workflow diagram
Figure 3 is a flowchart that outlines the data analysis process using SmartPLS 4.0. The process starts with the 'User Input Interface' and progresses to 'Response Generation,' followed by collecting 'Respondent Opinions via Google Form.' The gathered data is then analyzed using SmartPLS 4.0, which includes steps such as the bootstrapping resampling procedure, effect size evaluation, blindfolding procedure, and collinearity assessment. The analysis results are interpreted by assessing their significance through measurement model evaluation, construct validity, and Average Variance Extracted (AVE). Ultimately, these interpreted results contribute to 'Questionnaire Measurement.' The flowchart provides a visual representation of the sequential steps from data collection to analysis and interpretation.
According to Table 2, satisfaction mediates loyalty and the three latent variables. Service quality and satisfaction are statistically linked at 62%. Lowest loyalty correlation: system quality (54%). These variables are often closely related, therefore a smaller correlation does not indicate no association.
Table 2. Latent variable correlation
|
IQ |
UL |
US |
VQ |
SQ |
|
|
IQ |
1.00 |
0.57 |
0.60 |
0.59 |
0.58 |
|
UL |
0.59 |
1.00 |
0.57 |
0.58 |
0.54 |
|
US |
0.62 |
0.57 |
1.00 |
0.62 |
0.59 |
|
VQ |
0.61 |
0.58 |
0.62 |
1.00 |
0.61 |
|
SQ |
0.60 |
0.54 |
0.59 |
0.61 |
1.00 |
Table 3. The effect of satisfaction as a mediator of loyalty
|
Model |
Direct Effects |
Indirect Effects |
Total Effects |
|
IQ -> US -> UL |
0.09 |
0.07 |
0.30 |
|
SQ ->US -> UL |
0.05 |
0.06 |
0.34 |
|
VQ -> US -> UL |
0.08 |
0.03 |
0.05 |
A satisfaction mediator facilitates mediation between endogenous and exogenous variables to improve latent variable effectiveness. Table 3 shows that the satisfaction variable mediates the association between exogenous and endogenous variables with a t-test result greater than 0.02. This supports the idea that contentment mediates these variables. Figure 4 shows test sample distribution results.
This Figure 4 is a structural equation model illustrating the relationships between different quality factors and their impacts on satisfaction and loyalty. The model includes three primary factors: System Quality, Information Quality, and Service Quality, represented by blue circles. Each of these factors is connected to specific indicators (A1-A6 for System Quality, B1-B2 for Information Quality, and C1-C5 for Service Quality) shown in yellow rectangles. These quality factors influence Satisfaction (Y1-Y4) and Loyalty (Z1-Z2), also shown as blue circles with their respective indicators in yellow rectangles. Arrows indicate the direction of influence, showing how System Quality, Information Quality, and Service Quality directly affect Satisfaction, which in turn impacts Loyalty. This model visually demonstrates the hypothesized pathways through which quality dimensions contribute to overall user satisfaction and loyalty.
Figure 4. Structural model of research
Figure 5. Inner model path coefficients and t values
|
(a) Path coefficients IQ - US |
(b) Path coefficients VQ - US |
|
(c) Path coefficients SQ - US |
(d) Path coefficients US - UL |
Figure 6. Path coefficients pada latent variable
Figure 5 is a SEM showing how System Quality, Information Quality, and Service Quality affect Satisfaction and Loyalty. System Quality, Information Quality, and Service Quality are shown by blue circles and connected to specific indicators (A1-A6, B1-B2, C1-C5) in yellow rectangles. All indicators have a loading of 0.000, suggesting they are standardized values or placeholders. System Quality (0.173, t=3.475), Information Quality (0.433, t=8.932), and Service Quality (0.355, t=6.987) significantly affect Satisfaction. Satisfaction strongly affects Loyalty (0.232, t=2.964). System and Service Quality both affect Loyalty with coefficients of 0.331 (t=3.311) and 0.356 (t=3.861). All route coefficients and t-values show that quality dimensions significantly affect consumer happiness and loyalty.
The path coefficient approach in Structural Equation Modeling (SEM) is ideal for analyzing chatbot client satisfaction and loyalty. This method lets us carefully measure how client happiness affects loyalty. Customer pleasure from chatbots is the independent variable, while customer loyalty is the dependent variable. These variables' route coefficient shows how much satisfaction affects loyalty (refer to Figure 6). Customer loyalty grows with chatbot capability and reactivity, according to a favorable association. This report shows corporations how optimizing chatbots enhances customer pleasure and loyalty. The hypothesis test sample distribution shows the relationship between histogram density and normal distribution across two variables. The hypothesis results are in Table 4.
Table 4. Hypothesis test results
|
Hypotheses |
Path Coefficients |
Supported |
|
H1: SQ will positively impact US with chatbots. |
0.14 |
Yes |
|
H2: SQ will positively impact UL with chatbots. |
-0.008 |
No |
|
H3: IQ will positively impact US with chatbots. |
0.27 |
Yes |
|
H4: IQ will positively impact UL with chatbots. |
0.25 |
Yes |
|
H5: VQ will positively impact US with chatbots. |
0.05 |
Yes |
|
H6: VQ will positively impact UL with chatbots. |
0.26 |
Yes |
|
H7: US will positively impact UL with chatbots. |
0.18 |
Yes |
|
H8: US will mediate the effect of SQ on UL. |
0.04 |
Yes |
|
H9: US will mediate the effect of IQ on UL. |
0.007 |
Yes |
|
H10: US will mediate the effect of VQ on UL. |
0.011 |
Yes |
Table 4 summarizes the results of a study examining the relationships between system quality (SQ), information quality (IQ), and visual quality (VQ) on user satisfaction (US) and usage likelihood (UL) of chatbots. The results indicate that both IQ and VQ have a positive significant impact on US and UL, whereas SQ has a positive effect on US but not on UL. US itself positively impacts UL, and it also mediates the effects of SQ, IQ, and VQ on UL. These findings suggest that while system quality alone may not directly influence usage likelihood, satisfaction plays a critical role in enhancing the effects of these quality dimensions on the likelihood of continued chatbot use.
The structural model suggests that Chatbot AI in customer relationship management services may increase customer loyalty. The analysis shows 83.8% consumer satisfaction and 75.3% client loyalty. These findings demonstrate chatbots' usefulness in customer service, particularly in reducing service time, which boosts satisfaction. AI Chatbot, developed on WhatsApp, has a simple UI and convenient functionality. Customer satisfaction has increased thanks to chatbots' constant and rapid responses. Customers like chatbots because they can get help quickly without waiting. The chatbot's ability to handle ordinary requests and routine activities frees up human staff to handle more difficult and urgent situations, enhancing service.
The research was funded by the Research Institute of the Universitas Sumatera Utara (Grant No.: 286/UN5.2.4.1/PPM/2024).
[1] Hsu, C.L., Lin, J.C.C. (2023). Understanding the user satisfaction and loyalty of customer service chatbots. Journal of Retailing and Consumer Services, 71: 103211. https://doi.org/10.1016/j.jretconser.2022.103211
[2] Agnihotri, R., Afshar Bakeshloo, K., Mani, S. (2023). Social media analytics for business-to-business marketing. Industrial Marketing Management, 115: 110-126. https://doi.org/10.1016/j.indmarman.2023.09.012
[3] Kecht, C., Egger, A., Kratsch, W., Röglinger, M. (2023). Quantifying chatbots’ ability to learn business processes. Information Systems, 113: 102176. https://doi.org/10.1016/j.is.2023.102176
[4] Sowa, K., Przegalinska, A., Ciechanowski, L. (2021). Cobots in knowledge work: Human - AI collaboration in managerial professions. Journal of Business Research, 125: 135-142. https://doi.org/10.1016/j.jbusres.2020.11.038
[5] Smutny, P., Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook Messenger. Computers & Education, 151: 103862. https://doi.org/10.1016/j.compedu.2020.103862
[6] Selamat, M.A., Windasari, N.A. (2021). Chatbot for SMEs: Integrating customer and business owner perspectives. Technology in Society, 66: 101685. https://doi.org/10.1016/j.techsoc.2021.101685
[7] Illescas-Manzano, M.D., López, N.V., González, N.A., Rodríguez, C.C. (2021). Implementation of chatbot in online commerce, and open innovation. Journal of Open Innovation: Technology, Market, and Complexity, 7(2): 125. https://doi.org/10.3390/joitmc7020125
[8] Wei, R., Pardo, C. (2022). Artificial intelligence and SMEs: How can B2B SMEs leverage AI platforms to integrate AI technologies? Industrial Marketing Management, 107: 466-483. https://doi.org/10.1016/j.indmarman.2022.10.008
[9] Motulsky, A., Bosson-Rieutort, D., Usher, S., et al. (2023). Evaluation of a national e-booking system for medical consultation in primary care in a universal health system. Health Policy, 131: 104759. https://doi.org/10.1016/j.healthpol.2023.104759
[10] Ashfaq, M., Yun, J., Yu, S., Loureiro, S.M.C. (2020). I, Chatbot: Modeling the determinants of users’ satisfaction and continuance intention of AI-powered service agents. Telematics and Informatics, 54: 101473. https://doi.org/10.1016/j.tele.2020.101473
[11] Kumar, P., Sharma, S.K., Dutot, V. (2023). Artificial intelligence (AI)-enabled CRM capability in healthcare: The impact on service innovation. International Journal of Information Management, 69: 102598. https://doi.org/10.1016/j.ijinfomgt.2022.102598
[12] Sucupira Furtado, L., da Silva, T.L.C., Ferreira, M.G.F., de Macedo, J.A.F., de Melo Lima Cavalcanti Moreira, J.K. (2023). A framework for digital transformation towards smart governance: Using big data tools to target SDGs in Ceará, Brazil. Journal of Urban Management, 12(1): 74-87. https://doi.org/10.1016/j.jum.2023.01.003
[13] Nagarajan, G. (2023). Artificial intelligence (AI) in banking industry and customers perspective. Journal of Urban Management, 10(1): 1378-1381.
[14] Jenneboer, L., Herrando, C., Constantinides, E. (2022). The impact of chatbots on customer loyalty: A systematic literature review. Journal of Theoretical and Applied Electronic Commerce Research, 17(1): 212-229. https://doi.org/10.3390/jtaer17010011
[15] Lee, S.K., Kavya, P., Lasser, S.C. (2021). Social interactions and relationships with an intelligent virtual agent. International Journal of Human-Computer Studies, 150: 102608. https://doi.org/10.1016/j.ijhcs.2021.102608
[16] Tulus, Marpaung, T.J., Marpaung, J.L. (2023). Computational analysis for dam stability against water flow pressure. Journal of Physics: Conference Series, 2421: 012013. https://doi.org/10.1088/1742-6596/2421/1/012013
[17] Yang, Z., Feng, J. (2023). Explainable multi-task convolutional neural network framework for electronic petition tag recommendation. Electronic Commerce Research and Applications, 59: 101263. https://doi.org/10.1016/j.elerap.2023.101263
[18] Jo, H. (2022). Impact of information security on continuance intention of artificial intelligence assistant. Procedia Computer Science, 204: 768-774. https://doi.org/10.1016/j.procs.2022.08.093
[19] Farrow, E. (2022). Determining the human to AI workforce ratio - Exploring future organizational scenarios and the implications for anticipatory workforce planning. Technology in Society, 68: 101879. https://doi.org/10.1016/j.techsoc.2022.101879
[20] Ng, D.T.K., Leung, J.K.L., Chu, S.K.W., Qiao, M.S. (2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2: 100041. https://doi.org/10.1016/j.caeai.2021.100041
[21] Pallathadka, H., Ramirez-Asis, E.H., Loli-Poma, T.P., Kaliyaperumal, K., Ventayen, R.J.M., Naved, M. (2023). Applications of artificial intelligence in business management, e-commerce and finance. Materials Today: Proceedings, 80: 2610-2613. https://doi.org/10.1016/j.matpr.2021.06.419
[22] Premant, S.N., Arun, A. (2019). A qualitative study of artificial intelligence application framework in human resource management. Journal of Xi’an University of Architecture & Technology, 11(12): 1193-1209.
[23] Febriyantoro, M.T., Suleman, D., Ariawan, J., Hakim, L. (2023). Assessing the impact of strategic and ethical entrepreneurship competence, and network competence on the sustainable growth of SMES in south Tangerang city, Indonesia. International Journal of Sustainable Development & Planning, 18(12): 3973-3981. https://doi.org/10.18280/ijsdp.181228
[24] Handoko, B.L., Indrawati, D.S., Zulkarnaen, S.R.P. (2024). Embracing AI in auditing: An examination of auditor readiness through the TRAM framework. International Journal of Computational Methods and Experimental Measurements, 12(1): 53-60. https://doi.org/10.18280/ijcmem.120106
[25] Abdullah, W.M.Z.B.W., Zainudin, W.N.R.A.B., Ismail, S.B., Zia-Ul-Haq, H.M. (2022). The impact of microfinance services on Malaysian B40 households’ socioeconomic performance: A moderated mediation analysis. International Journal of Sustainable Development and Planning, 17(6): 1983-1996. https://doi.org/10.18280/ijsdp.170634
[26] Zhao, J., Patrick Rau, P.L. (2020). Merging and synchronizing corporate and personal voice agents: Comparison of voice agents acting as a secretary and a housekeeper. Computers in Human Behavior, 108: 106334. https://doi.org/10.1016/j.chb.2020.106334
[27] Rehmat, M.A., Hassan, M.A., Khalid, M.H., Dilawar, M. (2022). Next level of hospitalization through smart ICU. Intelligent Systems with Applications, 14: 200080. https://doi.org/10.1016/j.iswa.2022.200080
[28] Asante, I.O., Jiang, Y., Hossin, A.M., Luo, X. (2023). Optimization of consumer engagement with artificial intelligence elements on electronic commerce platforms. Journal of Electronic Commerce Research, 24(1): 7-28. http://www.jecr.org/sites/default/files/2023vol24no1_Paper2.pdf.
[29] Erwin, Hasibuan, C.D., Siahaan, D.A.S., Manurung, A., Marpaung, J.L. (2024). Stability analysis of spread of infectious diseases COVID-19 using SEIAR-V1V2Q model for asymptomatic condition with Runge-Kutta order 4. Mathematical Modelling of Engineering Problems, 11(5): 1348-1354. https://doi.org/10.18280/mmep.110526
[30] Tulus, Marpaung, J.L., Marpaung, T.J., Suriati. (2020). Computational analysis of heat transfer in three types of motorcycle exhaust materials. Journal of Physics: Conference Series, 1542: 012034. https://doi.org/10.1088/1742-6596/1542/1/012034
