Would Lecturers Use AI-Based Software to Write Scientific Article? A Quantitative Approach in Indonesia

Artificial Intelligence-based (AI-based) software is used in various sectors and domains to help human work, including education [1]. An example of AI-based software usage is student failure detection based on the learning management system (LMS) data track record [1, 2]. It actualized in the ChatGPT and Bard (now Gemini) software [3, 4], which students widely use to complement the teacher's role in discussing subject matter [2]. Students also use the software to write content in scientific articles to be submitted to journals or conference proceedings. Moreover, there are also several AI-based software to help the process of writing scientific articles [5, 6], for example, correcting English grammar [7, 8], searching articles for literature reviews [3], and converting voice into text. Students commonly can leverage AI-based software [9], considering their limited writing experience and the need to get high grades.

AI-based software usage can be controversial and debatable if the user is a lecturer. People have a perception that lecturers should have more adequate competence and experience. It has been a subjective prestige for lecturers to be more independent in writing articles without the help of software. It looked at the AI-based software usage to assist writing is considered a degradation of abilities, irresponsible, and unethical. Moreover, Shen et al. [10] mentioned a study in 2023 that claimed that reviewers identified 63% of fake abstracts created by ChatGPT. It signalized limited human ability to detect whether lecturers wrote scientific articles. Also, Dashti et al. [11] emphasized that scientific writing is only valid when performed manually by researchers (including lecturers). Separately, Zou and Huang reveal doctoral students’ opinion that stated these disadvantages since using ChatGPT: breeding laziness, weakening higher-order thinking, impairing writing ability, and demotivating writing learners [12]. Extremely, overuse of AI-based software may lead plagiarism and a lack of accurateness [7].

However, many lecturers think AI-based software is only helpful because they have performed research essence processes. AI-based software can support lecturers with limited English proficiency and translating references [13, 14]. Hence, writing processes are negotiable to be assisted by AI-based software. In addition, AI-based software usage supports technology's general principle to assist people in doing anything more efficiently [5]. AI-based software usage has become a trending technology that should increase productivity. Specifically, Zou and Huang told several benefits from ChatGPT to facilitate the brainstorming, outlining, and thinking development efficiently [12].

These pros and cons underlie this research to explore the factors influencing lecturers using AI-based software for writing scientific articles. This research took the specific case of writing scientific articles because this is currently a trending and debatable topic among academics. AI-based software is generally considered commonplace in society, but not for writing scientific articles among lecturers. Lecturers must be an example for students when writing quality scientific articles so that the behavior of using AI-based software is questionable. Specifically, several people think AI-based software for writing scientific articles is plagiarism [7].

To address these pros and cons, this research offers a solution by identifying factors influencing lecturers' intentions to use AI-based software. Factor identification reflects a landscape that describes the software adoption process and reveals what causes context-specific use for scientific writing among lecturers. Moreover, factor identification also becomes a reflection for stakeholders to determine policies relevant to the landscape. The pros and cons can be addressed long-term by determining the allowed and prohibited cases following the current behavior.

This research can contribute theoretically and practically. After identifying influencing factors, its results can contribute theoretically to knowledge about software adoption based on the positivist paradigm and contemporary cases. In addition, the identification results can provide input to universities and lecturer professional associations to formulate clear regulations for using AI-based software.

This research consists of several chapters. Section 1 portrays the introduction of this research with the background, problem boundaries, and objectives. Section 2 narrates literature reviews that outline fundamental theories to support the research and hypotheses generation process, while Section 3 reveals the methodology used to obtain research data. Sections 4 and 5 unveil the testing results and discussions, respectively. Finally, Section 6 contains the conclusion following the research objectives, while Section 7 promotes the relevant suggestions for future research.

This research implemented classification from Saunders et al. [35] with each layer to define categorization from Philosophies to the Time-Horizon type. In Philosophies, it was Positivism because this research constructed participants’ belief and perception [36]. The knowledge were elaborated from the postulated hypotheses at the beginning to be tested in the end of the research as a signal of the Deductive approach, while this research also applied a Survey as a strategy. Figure 2 visualizes its research categorization.

Following its goals as stated in the Introduction, this research explored the causative factors through a survey strategy. It covers identifying the driving and inhibiting factors for people when they stop using software in an office environment. This research tailored the quantitative approach in these main phases: (1) Instrument generation from related theories in literature review, (2) Primary data collection (including respondent recruitment and data capturing), and (3) Results processing and interpretation. Using the codification technique, it performed the content analysis method to elaborate primary data into identified patterns. Each respondent's statement was classified into the same categories and extracted to perceive causative factors by comparing, contrasting, and synthesizing.

Following its goals, this research identified the potential factors through hypotheses formulated in Section 2. It actualized the hypotheses testing in a quantitative approach using PLS-SEM due to its reliability in verifying suspected relationships among variables, especially for individual perception. Moreover, PLS-SEM utilization for hypotheses has been implemented successfully in other AI-based software or application adoption, such as Pillai and Sivathanu [32] in 2020, Chatterjee et al. [21] in 2021, Gansser et al. [29] in 2021, and Phuoc [25] in 2022. Moreover, Pillai et al. and Chatterjee et al. performed TAM quantitatively while Gansser actualized UTAUT. These experiences indicate PLS-SEM's ability to emphasize objective interpretation following statistical principles, especially when this research adapting TAM and UTAUT. PLS-SEM utilization facilitate this research to objectively determine whether a factor empirically influences to lecturer’s intention to use AI-based software.

This research tailored the quantitative approach in these main phases: (1) Research instrument production following the formulated hypotheses; (2) Primary data collection using an online questionnaire; and (3) Data testing with the PLS-SEM procedures and result interpretation.

3.1 Instrument production

This research composed the instruments following related theories referred to entire variables in the hypothesis framework (see Figure 1). The instruments were packaged in a questionnaire with two main parts: demography profiling and perception measurement using the Likert scale. This research has 37 instruments from seven variables. Before releasing the questionnaire to the respondents, this research hosted a pivot testing to ensure its readability. Table 1 exposes the produced instruments with their references.

2.png

Figure 2. Research classification following its paradigm, adapted from Saunders et al. [35]

Table 1. List of instrument

Table 2. Respondents’ profile

3.2 Data collection

This research collected primary data from lecturers in Indonesian higher education. They were recruited through convenience/accidental sampling during August 2023. The sample was selected based on the availability and willingness of lecturers in Indonesia to participate in this research without any intervention to deliberately determine the gender proportion of writing experience and scientific field. Voluntarism is essential considering the relatively large number of questions that must be filled in and the characteristics of PLS-SEM, which require relatively low data adequacy compared to CB-SEM. The demographic composition in Table 2 occurs organically. However, the author argues that the sample size of 110 with the proportion of each attribute is representative by considering the involvement of lecturers in various provinces, the diversity of the frequency of productivity in writing scientific articles, and the diversity of scientific fields.

After inputting their profile, they used a five-point Likert scale to express their perception following the instruments. Since its hypotheses framework had six paths, this research targeted 60 persons as a minimum sample due to considering Hair et al. [39] formulation. Initially, this research got 112 respondents who completed the questionnaire filling. After data checking, 110 was valid and indicated this research qualification was beyond the targeted minimum sample number. Based on the respondent’s profile (Table 2), this research has successfully captured a sample from diverse backgrounds. Moreover, this research captured Grammarly as their most favorite AI-based software for scientific writing (85.14%), followed by ChatGPT as consulting service (55.14%).

5.1 Theoretical implications

Based on quantitative calculations, this research found that two variables influence lecturers' intentions to use AI-based software in writing scientific articles. The two variables are quality (QLT) and security (SEC). Proving that the Quality variable (QLT) is a contributor indicates that software quality theory and its components are valid empirically. The results of this research align with several previous studies relating to contributing factors.

Software quality has a wide range of definitions, including aspects of benefit, efficiency, effectiveness, accuracy, and usability [26]. The beneficial aspects proven in this study were in line with Salvagno's statement in the study by Salvagno et al. [7], while the effectiveness and accuracy sharpened Bringula's statement in the study by Bringula [22]. Meanwhile, the usability aspect has the same results as Xu et al. [23] and Nolay et al. [24]. It confirms that aspects of usefulness and reusability play a significant role in software adoption, especially AI-based software. Usefulness signalizes software ability to fulfill users’ needs and requirements. Software ability to fulfill them correlated strongly with competition among similar software to achieve its product continuity. Hence, AI-based software will enrich its scope following future enhanced requirements. Specifically for usability, AI-based software currently relies on an interface that offers interaction using natural language to increase the users experience when communicating with the software and its users. It encourages the increasing role of the ability aspect as part of software quality.

Security was another contributing factor that denotes lecturers had strong attention to obtain information security protection, especially personal data protection. This finding strengthens the previous research that performed by Yu [38], Bringula [22], and Xu et al. [23]. They had mentioned information security as crucial determinant when academician (teachers and students) to use ChatGPT. They put information leakage as threat that reduce academicians’ intention. However, this research converted it into software’s ability (and its operator capability) to protect the users’ information security. By converting it, this research attempts to capture users’ direct concern and requirements about information security as shown in SEC.01 until SEC.03 (see Table 1). Although take different perspective, it generated similar landscape with previous research.

On the other hand, the four rejected factors (Motivation, Subjective Norms, Ability, and Support) have explanations and arguments as interpretations of statistical test results. As the rejected variable, the Ability shows that there is no significant difference in influence between people who have scientific writing skills and those who do not. Both characteristics have a relatively high intention of utilizing AI-based software in scientific writing. It indicates that AI-based software is open to various groups with various skill levels. Even when people already have scientific writing skills, they feel it is okay to continue using AI-based software because it delivers the expected benefits. This research suspects that AI-based software provides excellent benefits. This significant benefit is the recommendations on techniques for writing scientific articles, which lecturers still need even though they have had many writing experiences.

Another rejected variable is Support (SUP). This research suspects that many indicators in the questionnaire do not construct a sufficient correlation pattern to use AI-based software for writing scientific articles. For example, at the beginning of this research, suspects that financial support was suspected as a contributing factor for purchasing AI-based software licenses. However, this indicator must be proven by considering the relatively cheap price and many similar products. Moreover, AI-based software with a free package provides sufficient benefits. Therefore, this research showed a blurry difference between those who subscribe and those who use the free version. Other indicators related to the SUP variable are digital literacy skills and technical abilities to operate the software. This research argues that AI-based software has good user experience and usability (easy and practical). Thus, digital literacy and the technical ability to operate software do not influence intentions.

Another interesting phenomenon is the non-acceptance of the Subjective Norm variable in lecturers' intentions when writing scientific articles using AI-based software. It proves that currently, lecturers are not affected by the regulations set by institutions regarding using AI-based software to assist in writing scientific articles. However, it could be influenced by the fact that the institution does not set regulations regarding its use. Without regulations, lecturers do not have a legal reference for using or not using them. AI-based software usage relies on the lecturer's rationalization and ethics in distinguishing between permitted and prohibited. In the future, attitudes and intensity may change when regulations allow or prohibit it. Interestingly, the lecturers saw that the surrounding environment (fellow lecturers or the community) also had no influence. This research suspects that this is due to the use of AI-based software, which has already become widespread, so lecturers ignore the attitudes of the surrounding environment when deciding to use AI-based software.

Another variable whose correlation with Intention is rejected is Motivation (MOT). It showed that the three indicators of Motivation (personal demands for productivity, institutional demands for productivity, and personal demands for producing quality work) are not automatically directly proportional to the intention to utilize AI-based software in writing scientific articles. This situation happened because of the diversity of lecturers' characters in Indonesia based on their inclination towards institutional duties. Some lecturers are interested in research, so they set targets for publication. However, some lecturers were interested in teaching or organizational assignments, so they did not have publication productivity targets except to fulfill the minimum annual publication quantity standards. This situation has the impact of not forming a positive correlation between lecturer motivation and intention to use AI-based software. This research suspects that in the future, it will be necessary to group lecturer characteristics (as mentioned previously) to ensure the influence of specific motivation only on lecturers who have passion for the research.

Finally, this research also advocates the academic institution to formulate and legalize a guideline for AI-based software. It helps lecturers to distinguish which activities that can be supported by AI-based software or not. It also gives legal certainty as narrated also by Gupta and Bhaskar [40] in 2020. Separately in 2023, Zou and Huang [12] and Salvagno et al. [7] agree it can balance productivity and fairness in academic publication.

5.2 Practical implications

First implication was related with the acceptance of H1 (QLT$\rightarrow$INT). In implementing AI systems, the software must still have the quality to fulfill user needs and/or objectives. Specifically, its indicators were valid and increased the lecturer's behavior to utilize AI. They empirically characterized software quality attributes that influence its use by lecturers. In writing typed scientific articles in the future, similar applications must pay attention to these four things as part of the quality of the software built to attract user intentions and maintain business continuity. Therefore, this finding implies that software developers in the AI industry should prioritize product quality attributes to satisfy users since software quality has a fundamental impact on ensuring user’s intention at the present and continuity to use in the future.

Another accepted variable is Security, a hot issue in software development and artificial intelligence. Recently, society has become increasingly aware that security in using software, especially those based on artificial intelligence, is mandatory. The public demands guarantees that the use of software has adequate information security protection. This premise is proven when using AI-based software to write scientific articles. Using AI-based software to write scientific articles does not use personal data, so recognizing its behavior does not cause privacy violations. However, the use of software that uses an account causes the potential for misuse of personal data in the account by the application owner. Moreover, using an account with a Single Sign-On system or API access to several general accounts (for example, Google) can potentially cause data leaks that need to be anticipated. Therefore, software developers should also ensure information security compliance with requirements and regulations, especially personal data protection. Academic institutions should also build academicians' awareness of information security to balance AI-based software’s benefits with related risks.

Future research should expand the respondents' participation to improve profile diversity. With more diverse respondents, this research can escalate representativeness and feasibility. This research also suggests more comprehensive qualitative feedback from experts, such as psychologists and philosophers, through focus group discussions to escalate the findings' validity. Their participation can enrich the interpretation of the adoption process from a multi-disciplinary. Also, this research also promotes comparison studies about AI-based software adoption by segregating respondents into two or more classes, such as digital literacy or research experience. It enables insight into specific segments for formulating the necessary regulations.

Another recommended research topic is the classification and evaluation of various types of AI-based software in writing scientific articles following the ethical corridor. Considering the increasingly powerful features and services of AI-based software in writing scientific articles. For example, using it for grammar correction still gets positive sentiments, while idea generation until manuscript finalization gets negative sentiments. Also, several educational institutions have created guidelines for using AI to write scientific articles. This reaction phenomenon has become exciting by emphasizing its influence on productivity and ethical compliance.