Francka S. Lee*
| Henny Hartono | Johanes F. Andry | Aziza Chakir | Jessica
© 2024 The authors. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
E-Learning is widely used in distance education, particularly in vocational high schools (VHS), where it supports students in accessing materials and engaging in interactive learning without face-to-face interaction. These portals are crucial for developing both practical and theoretical skills. However, the effectiveness and usability of E-Learning systems in VHS are critical concerns. Given the unique needs of VHS students, these platforms must efficiently deliver content and support skill development through interactive modules. Ensuring the quality of E-Learning systems is essential for meeting the educational goals of vocational programs. This study evaluates the VHS E-Learning portal using the ISO 9126 standard, a recognised framework for software quality assessment that includes functionality, usability, reliability, and efficiency. Despite the broad adoption of E-Learning, there is a research gap specifically focused on quality assurance for VHS portals. This study addresses that gap, offering insights and recommendations to enhance the effectiveness of E-Learning in vocational education, ensuring these portals meet the high standards necessary for vocational training success.
E-Learning, vocational high school, systems, software, education, ISO 9126
The development of technology nowadays is growing rapidly and inseparable from daily activities that are in lineal with human needs to carry out activities in everyday life [1]. In the sphere of education recently, there have been many changes and revolutions, especially regarding the media that can be used to carry out educational activities [2]. Many educational places or schools want to take advantage of technological developments, especially in making it easier for teachers and students to carry out teaching and learning activities [3]. The teaching staff must have prepared the best thing to educate their students, especially in the world of VHS education because these students will enter the world of work in accordance with the previously taken fields [4]. Hence, this E-Learning portal must be beneficial and reliable for VHS because it is correlated to the students’ learning process [5]. E-Learning portals can be made with many choices of platforms such as mobile-based software, and web-based software [6]. For testing purposes, a web-based portal was carried out since many people have been familiar with it. Further, reusability and quality assessment were opted to determine the quality of a web [7]. and were expected to improve organisations in education [8, 9]. Remote learning such as using school or campus portals has been widely observed in recent times, but it is more about the development of technological features in this E-Learning portal. E-Learning is data-intensive, and user-driven, and may increase over time due to multiple cultures, efficiency issues, and adaptation. Hence, this study is conducted to highlight and facilitate the increasing need for new methods that maximize usability and bring satisfaction to users [10]. Yet, researching the quality of software is something that has been done for a long time in the science of software engineering. Quality software, especially web-based is the result of analysis [11]. Observation and practice show that an assessment of the quality of a software element is generally a very broad and quite complicated destination [12]. The structure or standard utilized in testing the E-Learning portal is based on six quality characteristics, which are ISO 9126 standards for software systems: functionality, reliability, usability, efficiency, maintainability, and portability [13]. The quality of web-based software can be evaluated measurably by using predefined methods and by conducting tests. Hence, such methods can be considered for developing web products from the perspective of the user in this case the student [10].
Quality standards from ISO 9126 are used as the basis for the quality model designed. This model can take data from observation (testing) then it is concluded whether the E-Learning system model is ideal or not. The set of criteria is then linked with the characteristics and sub-characteristics of the ISO 9126 quality model to further specify the quality measurement matrix. Despite the fact that the use of E-Learning is increasing, there are various complaints about the quality of E-Learning systems. A number of E-Learning is regarded as having poor performance, poor usability, and a high level of complexity in customizing some E-Learning. Therefore, measuring the E-Learning system using the ISO 9126 quality standard is preferable since it is an international quality standard that has been known for its reliability to be applied to various types of applications. The application of a survey (testing) can show the value of the quality of each E-Learning and recommendations that can be used in further development of E-Learning [14]. Furthermore, the software maker can achieve a goal when the software used by the client or user can run well without problems.
The large number of users who use the portal for schools has made VHS implement this web-based software or web portal. The problem with implementing this software is the performance of the software used whether it is good or not, especially in the current situation which changes face-to-face classes into online classes. The negative impact may be seen on the school web portal, which can make it difficult for students in the learning process and may reduce student’s learning intentions.
Problems can be identified from the content or functionality provided that is functioning properly or not. The problems can be found clearly if there are errors or bugs, not working as expected and other problems through the interview and observation process. It can assess the performance of this web-based software, so the author can tell what needs to be changed to suit the wishes of the user, namely vocational high school students. Therefore, the purpose of the authors chose to test the performance of a web-based software belonging to this vocational high school portal because students need to develop or hone their knowledge in accordance with their chosen major, testing this software, can ensure that students can learn from a distance without any significant obstacles, and get the advantages of obtaining the computerized system.
Previous researches [2, 14, 15] conducted research on the assessment of the quality or performance of an E-Learning service, for reference [2] specialized its research for the campus scope, while reference [16] was in the scope of E-Learning in general and for reference [12] in the scope of competency-based training and education held by the state. Based on this research, the author got the idea to research the performance of web-based software, namely a portal (E-Learning) for vocational high school student learning. The theoretical benefit of this research is that readers can understand and know the results of the software tested using ISO 9126, and the benefit of this research for the organization is that it can implement it according to the needs of the user, in this case, it is students from vocational high schools.
The research methodology adopted in this study follows a systematic approach, starting with a comprehensive literature review and followed by a detailed analysis of the E-Learning portal used in vocational high schools (VHS). This section outlines the steps taken to gather, analyze, and evaluate the data, ensuring a thorough assessment of the E-Learning portal's quality based on the ISO 9126 standard. These show in Figure 1.
Figure 1. Diagram method of research [2]
2.1 Literature review and data collection
The first step in this research involved a detailed literature review to identify existing studies on E-Learning quality assessment. The data was gathered from credible sources such as academic journals, E-books, and verified websites, which provided a solid foundation for understanding the current state of research in this field [17]. The literature review aimed to identify gaps in the existing research, particularly concerning the application of quality standards like ISO 9126 in vocational high school settings. By synthesizing findings from previous studies, the review highlighted the need for more targeted research on the specific requirements and challenges of E-Learning systems in vocational education.
(1) ISO/IEC 9126
ISO/IEC 9126 is a comprehensive standard developed to ensure the quality of software products, providing a structured approach to assessing various aspects of software performance [7, 16, 17]. This standard, established by the International Organization for Standardization's Sub Committee on Software and Systems Engineering, is recognized for its detailed quality model that surpasses other quality frameworks in both scope and applicability [18-20].
The ISO 9126 standard is sectioned into four key components: quality demonstrate, inner measurements, outside measurements, and quality measurements [21, 22]. These components collectively address the coherence, unwavering quality, and operational life span of program, guaranteeing that it meets thorough quality benchmarks. The demonstrate is especially esteemed for its six fundamental characteristics—functionality, unwavering quality, convenience, effectiveness, viability, and portability—each of which is encourage subdivided into more granular sub-characteristics [15].
Despite its robustness, ISO 9126 has been subject to critique and evolution in the context of modern software environments. Several studies have pointed out that while ISO 9126 provides a solid foundation, it may require updates to accommodate the dynamic nature of software development and emerging technologies, such as those used in E-Learning platforms. For instance, some researchers argue that the standard's emphasis on traditional software metrics may overlook the unique requirements of web-based applications, where user experience and real-time adaptability are crucial [23]. This highlights a potential gap in the standard's application to E-Learning, suggesting a need for further research to adapt ISO 9126 criteria specifically to the E-Learning context.
(2) E-Learning
E-Learning has gotten to be an urgent instrument in cutting edge instruction, giving a adaptable and open stage for instructing and learning over different instructive levels [24]. The multiplication of web gets to and web-based frameworks has empowered E-Learning to advance into a basic instructive asset, bolstered by intelligently innovations that encourage learning anytime, anyplace [25]. E-Learning portals serve multiple functions, including assignment submission, material access, schedule management, and communication of important school announcements, making them integral to the educational experience. However, the rapid adoption of E-Learning has also spurred interest in assessing the quality of these systems. Existing research has explored various frameworks and models to evaluate the effectiveness of E-Learning platforms, with a significant focus on usability, functionality, and the overall user experience [26]. For example, the generative learning framework proposed in 2001 emphasizes creating meaningful learning experiences through well-designed E-Learning features that cater to both student and instructor needs [25, 26].
While these studies have contributed valuable insights, there is a notable gap in the literature concerning the comprehensive assessment of E-Learning quality using standardized metrics like those provided by ISO 9126. Most existing research tends to focus on individual aspects of E-Learning, such as user satisfaction or specific functional capabilities, without fully integrating these into a cohesive quality assessment framework [27, 28]. This gap suggests a need for more holistic studies that apply established software quality standards to E-Learning systems, thereby providing a more rigorous evaluation of their overall quality. Moreover, the specific challenges of E-Learning—such as the need for real-time feedback [29], the integration of multimedia content, and the facilitation of interactive learning experiences—are not always adequately addressed by traditional quality models. This underscores the importance of adapting existing frameworks like ISO 9126 to better align with the unique demands of E-Learning platforms. Future research should focus on developing and validating new or modified metrics that reflect the specific requirements of E-Learning, ensuring that these platforms meet the high standards expected in modern education.
2.2 Component analysis of the E-Learning portal
The second stage of the research involved a detailed analysis of the components and services provided by the E-Learning portal. This analysis was conducted through direct observation and interaction with the portal to assess its functionality, usability, and overall performance as a learning tool for VHS students [15]. The focus was on evaluating how well the portal supports the learning and teaching processes, with particular attention to its ability to deliver content effectively and facilitate student engagement.
2.3 Functional testing and analysis
Following the component analysis, the third stage involved testing the functionality of the E-Learning portal. This step was crucial for identifying any issues or limitations in the portal's performance. Tests were conducted on various functions, such as content delivery, user interface, and interactive features, to determine their effectiveness in supporting the educational goals of the vocational high school. The results of these tests provided valuable insights into the portal's strengths and areas that require improvement. The research focused on evaluating the quality of the E-Learning portal using the ISO 9126 standard. This standard provides a comprehensive framework for assessing software quality across six key characteristics: functionality, reliability, usability, efficiency, maintainability, and portability. The analysis was conducted by comparing the portal's performance against these criteria to determine its overall quality and suitability for vocational education.
2.4 Analysis and recommendations
The final stage of the research involved synthesizing the findings from the previous stages to provide a comprehensive assessment of the E-Learning portal. The analysis was conducted using the ISO 9126 framework to ensure that the portal meets the necessary quality standards for effective deployment in vocational high schools. Based on this analysis, recommendations were made for future improvements to the portal, with the goal of enhancing its functionality and usability to better support the learning needs of VHS students.
This research methodology ensures a thorough and systematic evaluation of the E-Learning portal, providing valuable insights into its quality and effectiveness. By addressing the gaps identified in the literature review, this study contributes to the ongoing development of high-quality E-Learning systems tailored to the specific needs of vocational education.
In this segment, we display the strategies and comes about of testing conducted on the school's E-Learning site. We utilized dark box testing, centering on the program's usefulness without requiring to get it its inside structure. The objective was to confirm whether the application meets the desired guidelines as per the ISO 9126 quality show, especially in terms of usefulness and convenience. The testing distinguished and classified program mistakes into three sorts: dialect blunders, run-time blunders, and logic mistakes. Program blunders that will happen are classified into 3 sorts, specifically:
(1) Language errors
These are syntax errors that occur during the coding process, where program code does not conform to required standards. These errors are usually easy to detect and correct, as the browser or compiler provides error messages indicating the location and cause of the error.
(2) Run-time errors
These errors occur when the program is executed and cause the program to stop prematurely. They arise from unfulfilled conditions during execution. Like language errors, these are also relatively easy to diagnose and fix, as the browser provides error messages with specific details.
(3) Logic errors
These are more challenging to identify as they do not trigger error messages. The program may run and produce results, but the results are incorrect. Logic errors are typically detected by comparing the system's output with known correct results using test data.
Examples of testing black box testing
(1) Testing of input data
a) Testing mechanism
This test examined the data input process into the database, particularly the system's ability to handle incorrect data types during input. A successful data storage process triggers a confirmation message indicating that the data has been saved. The student registration input was the primary component tested.
b) The results obtained, the data input process functioned smoothly, except when duplicate codes were entered, which caused the system to generate an error message.
(2) Testing of edit and update data
a) Testing mechanism, the test evaluated the edit feature and the ability to save changes. This feature was designed for ease of use, with all components integrated into a single form. If the update process was successful, the changes were reflected in the updated list, as shown in Figure 2.
Figure 2. Appearance of student data
b) The results obtained the edit and update functions performed smoothly, delivering the expected outcomes.
(3) Clear data test
a) Testing mechanism, this test assessed the system’s ability to handle data deletion processes, including how it manages errors during deletion. The system's responsiveness in providing confirmation messages before data deletion was also tested. The deletion tests focused on student, teacher, and admin data can be seen in Figure 3.
Figure 3. Delete confirmation display
b) Results obtained all deletion components worked as intended, with the system correctly prompting for confirmation before deletion and successfully removing the data, as indicated by a confirmation message.
All functions of the school's E-Learning website were further tested using Selenium IDE. The results showed that the website’s functions operated as expected, with no detected errors. The tests confirmed that the system runs smoothly and meets the intended usability and functionality requirements.
The results of this interface are intended to make it easier for admins to run the VHS E-Learning information system application. In terms of creating an interface, using a method that is indeed the facility that is already available. In making links, only distinguish between the names of the links that are needed and those that are available. Discussion of interfaces is also carried out by following methodological steps such as observation, and analysis in accordance with the service, of course based on ISO 9126.
a) Index page, this index page is the main page of the school E-Learning web information system interface. This page is the starting page that will be used to use this application. everyone can access the content or content of the E-Learning web information system, to access every feature and content in this web, anyone can access it without having to log in, because all content on this web is information about schools and except for students or teachers who want to enter more features, please log in or register as students to take classes that have been registered and hope to study the material, quizzes and take exams to see their scores.
b) Index Page 2
Based on Figures 4 and 5, this is the second page of the system interface VHS E-Learning information.
Figure 4. Index Page 2 (Part 1)
Figure 5. Index Page 2 (Part 2)
This page is a main page that will contain several link pages which are divided into menus that can be accessed by everyone to view the web contents such as news portals, school galleries or school achievements history.
Behind that 2 menus can be accessed by students and teachers with different views.
c) Student menu page
Based on Figure 6, this page students can access several menus including your class menu, where in this class menu students can see the students in the class with the user and find out who the homeroom teacher and class leader are in the class. Then in the subject’s menu, students can see what subjects they find during a year of study, along with who the teacher is teaching and a description of these subjects.
The subject, students can access the material menu with the aim that students can continue the independent learning process by downloading the material that has been prepared in the hope that students do not stick with what the teacher says.
Figure 6. Student Menu Page
There is a menu of assignments and quizzes from each lesson, usually these assignments and quizzes are accompanied by processing time and grades at the end after students work on their assignments and quizzes.
At the end there is a value menu, where students can see the value of the assignment and the quiz. All of these menus’ students can access it whenever and wherever students need it.
4.1 Selenium testing on E-Learning VHS
The testing phase, several website features were tried using selenium, some of these features were the login feature, the open material feature for students, the material download feature, to the task progress feature and the test quiz were created in the name of testing1.
The results all went well, there were no problems, there were no errors in this 1st test. Then authors conducted the test under the name testing 2 with several other features that were tested, including the feature of opening news for website visitors, the menu opening school galleries, and the menu opening the history of vocational schools. The results are also in line with expectations but there are some bugs, where the map display or school location does not appear, indeed in the selenium testing application there is no error or bug notification, but the author is aware of this.
4.2 ISO 9126 approach and dataset considerations
The following is the table result with the ISO 9126 approach.
The third testing phase, the author tested the features that worked in the admin and the teacher’s section, all the menus went well with no errors. and testing of this Vocational High School E-Learning website is complete.
Table 1. ISO 9126 based approach (1)
|
Characteristics |
Module Set |
Parameter |
|
Functionality |
Suitability |
Data Input, Process, Output Function |
|
Accurateness |
Accuracy of data processing and displaying data |
|
|
Interoperability |
The ability of software components to interact with components |
|
|
Other components or systems |
||
|
Compliance |
The software that has been made must comply with the prevailing laws and regulations |
|
|
Security |
Save data security |
|
|
Reliability |
Maturity |
The maturity model |
|
Fault tolerance |
Errors in usage |
|
|
Recoverability |
Data repair |
Table 2. ISO 9126 based approach (2)
|
Characteristics |
Module Set |
Parameter |
|
Usability |
Understand ability |
The features in this software are easy for users to understand |
|
Learnability |
How to install the configuration method |
|
|
Operability |
Operation: Open Help Exit |
|
|
Attractiveness |
User interface Forms appear |
|
|
Efficiency |
Time behavior |
The length of the data transfer process |
|
Resource behavior |
Memory and data storage capacity is not large |
|
|
Maintainability |
Analyzes ability |
Cause Analysis if something goes wrong |
|
Changeability |
Feature changes Upgrade to the next version |
|
|
Stability |
Stability capability |
|
|
Testability |
Verification ability |
|
|
Portability |
Adaptability |
Opportunity to adapt to different systems |
|
Install ability |
Ease and speed at install time |
Based on Tables 1 and 2, is a table for the ISO 9126 approach and with the existing parameters the appropriate questions are made so that they can produce appropriate or inappropriate answers. For the set of modules that have the characteristics of the functionality, that is, "compliance" alone is not appropriate. Then the usability characteristics in the "learnability" module set are still not suitable. Furthermore, it is also obtained inappropriate in the maintainability characteristics, namely the analyze ability and change ability module sets. In addition to those previously mentioned, all the characteristics and sub-characteristics (set of modules) in E-Learning are sufficiently suitable for their use in the learning process.
The evaluation of the Vocational High School (VHS) E-Learning system was conducted using a public dataset, which played a critical role in assessing the system against the ISO 9126 quality model. This model evaluates software across multiple characteristics such as functionality, usability, reliability, and more, ensuring a comprehensive understanding of the system's strengths and weaknesses.
(1) Dataset representativeness and applicability
The dataset employed in this evaluation was selected due to its relevance and availability, representing typical user interactions and system behaviors in an educational setting. However, the representativeness of this dataset is crucial to the validity of the research findings. To ensure the dataset accurately reflects the target environment, several factors were considered:
Selection Criteria: The dataset was chosen based on its alignment with the core functionalities of the VHS E-Learning system, including user logins, course material access, quiz completions, and data input activities. This ensures that the dataset covers a broad spectrum of typical E-Learning operations, making it a suitable candidate for evaluating system performance.
Coverage: The dataset spans a diverse range of user interactions across different roles (students, teachers, and administrators), which provides a comprehensive view of how the system performs under varying conditions. This diversity is crucial for assessing the system's adaptability and reliability across different use cases.
Potential Biases: While the dataset is comprehensive, there is a possibility of inherent biases that could impact the results. For instance, if the dataset predominantly includes interactions from a particular user group (e.g., students from a specific grade level), the findings may not fully generalize to other user groups or educational contexts. To mitigate this, the analysis took into account the distribution of data across different user roles and activities, ensuring a more balanced evaluation.
(2) Implications of dataset bias
The dataset's representativeness directly influences the generalizability of the research findings. If the dataset is not fully representative of the broader user base or typical E-Learning scenarios, the conclusions drawn about the system's functionality, usability, and other ISO 9126 characteristics might be limited in scope. Therefore, future research should aim to include datasets from multiple sources or contexts to enhance the reliability and generalizability of the findings.
Based on the analysis using the public dataset, the VHS E-Learning system demonstrated satisfactory performance across most ISO 9126 characteristics. However, specific areas such as "compliance" under functionality, "learnability" under usability, and "analyze ability" and "change ability" under maintainability showed room for improvement. These findings indicate that while the system is generally effective for its intended use, targeted enhancements in these areas could further optimize its performance and user satisfaction.
The evaluation of the VHS E-Learning system, using black box testing, Selenium IDE, and the ISO 9126 quality model, provides a comprehensive overview of the system's performance, highlighting both its strengths and weaknesses with implications for practical application. Functionally, the system meets its core requirements, performing well in key areas such as data input, editing, updating, and deletion, as confirmed by Selenium IDE tests. This indicates that the system is capable of supporting essential administrative tasks with reliability, which is crucial for maintaining the integrity of educational data. The user-friendly interface, particularly beneficial for students and teachers, supports independent learning by making educational materials and quizzes accessible at any time. This usability suggests that the system could enhance learning experiences by providing flexible access to resources, thereby promoting self-paced education [12].
However, the system also exhibits notable weaknesses, particularly in compliance with applicable regulations and standards, which could compromise data security and privacy—critical concerns in an educational setting. The lack of full compliance may not only expose the system to potential legal risks but also affect users' trust, which is vital for its widespread adoption. Additionally, the system's learnability is challenged by a complex installation and configuration process, potentially hindering adoption by less tech-savvy users, which could limit the system's reach and effectiveness in diverse educational environments. The maintainability of the system is also a concern, with issues in analyzability and changeability that may impede future updates or modifications, increasing the risk of introducing new errors and complicating long-term system upkeep [10].
To enhance the system, the study suggests a thorough review to ensure compliance with the latest legal standards, including the implementation of stronger encryption and privacy measures. This would not only protect user data but also align the system with best practices in educational technology. Improving the system's learnability by simplifying the installation process and providing detailed guides or automated setup tools is also recommended, as this could lower the barriers to entry and encourage broader adoption. Furthermore, enhancing maintainability through a modular design approach and regular code reviews, along with automated testing, could facilitate easier updates and mitigate potential issues early, ensuring that the system can evolve alongside educational needs [29].
The study also acknowledges limitations that could affect the generalizability and applicability of the results. The testing focused on external behaviors without fully assessing the internal code structure, potentially overlooking hidden inefficiencies or vulnerabilities. The use of a public dataset, while relevant, may not fully capture the diversity of real-world educational environments, limiting the results' generalizability. Additionally, the evaluation centered on the system's current state without considering future developments or changes in user needs, which could necessitate significant updates to maintain effectiveness. The potential for bias in user feedback, influenced by familiarity with the system or reluctance to criticize, is also noted.
To address these limitations and expand the discussion, it is crucial to explore the practical implications of the quality assessment results. For instance, the system's usability strengths could lead to increased student engagement and improved learning outcomes, while its compliance weaknesses might necessitate institutional policy changes or additional user training [30]. Future research should incorporate white-box testing, use more diverse datasets, and conduct longitudinal studies to provide a more comprehensive and actionable evaluation. By addressing the identified weaknesses and considering the study's limitations, the VHS E-Learning system can evolve to better meet the needs of its users and remain an effective tool in the educational landscape [31].
The research on the VHS E-Learning web system highlights its effectiveness in improving the teaching and learning process, particularly during the pandemic. Developed using PhpMyAdmin and MySQL, this web-based system integrates several key features provided by the admin board, offering advantages such as the ability to present information quickly, accurately, and relevantly, significant time savings in data management, and reduced repetitive tasks that enhance service efficiency and promotional outreach. However, to provide actionable insights, it is crucial to clarify the current quality status of the VHS E-Learning system in vocational high schools. The evaluation reveals that while the system meets basic functional requirements, it necessitates improvements in compliance and security, as it does not fully adhere to current data protection regulations, posing risks to user privacy. Additionally, users encounter challenges with the installation and configuration process, which can hinder adoption among less tech-savvy individuals; thus, simplifying this process and providing comprehensive guides are recommended to enhance the user experience. Furthermore, addressing issues related to maintainability—specifically in analyzability and changeability—is essential to ensure the system can adapt to future technological advancements and user needs. The implications of these findings suggest that future development efforts should prioritize these identified weaknesses, with regular updates to incorporate user feedback and technological advancements, ensuring the system remains relevant and effective. Ultimately, by addressing these areas for improvement, the VHS E-Learning system can significantly enhance the educational experience in vocational schools, fostering a more effective learning environment while minimizing the risk of data loss for students and teachers and improving overall data storage efficiency.
The manuscript’s authors express appreciation and gratitude to our university specially to thank Center for Research and Community Service or Pusat Penelitian dan Pengabdian Masyarakat (P3M) University of Bunda Mulia, Jakarta, Indonesia for support for supporting this research.
[1] Amulya, A., Pratap, K.V.N.R., Padma, T.M., Kalyan, V.S., Srikanth, P. (2019). The effect of technology face-to-face communication. International Journal of Research in Engineering, Science and Management, 2(12): 255-257.
[2] Stefani, A., Vassiliadis, B., Xenos, M. (2006). On the quality assessment of advanced E-Learning services. Interactive Technology and Smart Education, 3(3): 237-250. https://doi.org/10.1108/17415650680001449
[3] Lee, F.S., Chakir, A., Nathanael, R., Andry, J.F. (2020). Architecture information system in electrical distribution company using TOGAF. International Journal of Advanced Trends in Computer Science and Engineering, 9(5): 7149-7156. https://doi.org/10.30534/ijatcse/2020/38952020
[4] Mustikawati, D.L., Khumaedi, M., Anis, S. (2018). Analysis of the process of implementation of work practices industry vocational high school students in Purwodadi. Journal of Vocational and Career Education, 3(1): 37-44. https://doi.org/10.15294/jvce.v3i1.15240
[5] Yengin, İ., Karahoca, D., Karahoca, A., Yücel, A. (2010). Roles of teachers in E-Learning: How to engage students & how to get free E-Learning and the future. Procedia-Social and Behavioral Sciences, 2(2): 5775-5787. https://doi.org/10.1016/j.sbspro.2010.03.942
[6] Ying, Y., Paulina, P., Mulyadani, Y.R., Lim, K.M.O. (2021). Impact of E-Learning on learning mandarin in elementary schools during Covid-19. CommIT Journal, 15(1): 19-24. https://doi.org/10.21512/commit.v15i1.6643
[7] Suwawi, D.D.J., Darwiyanto, E., Rochmani, M. (2015). Evaluation of academic website using ISO/IEC 9126. In 2015 3rd International Conference on Information and Communication Technology (ICoICT), Nusa Dua, Bali, Indonesia, pp. 222-227. https://doi.org/10.1109/ICoICT.2015.7231426
[8] Usoro, A., Abid, A. (2008). Conceptualising quality E-Learning in higher education. E-Learning and Digital Media, 5(1): 75-88. https://doi.org/10.2304/elea.2008.5.1.75
[9] Vakkalanka, S., Prasadu, R., Sasank, V.V.S., Surekha, A. (2020). A framework for evaluating the quality of academic websites. In: Raju, K., Govardhan, A., Rani, B., Sridevi, R., Murty, M. (eds) Proceedings of the Third International Conference on Computational Intelligence and Informatics. Advances in Intelligent Systems and Computing, vol 1090. Springer, Singapore. https://doi.org/10.1007/978-981-15-1480-7_44.
[10] Budiman, E., Wati, M., Angelina Widians, J., Puspitasari, N., Firdaus, M.B., Alameka, F. (2018). ISO/IEC 9126 quality model for evaluation of student academic portal. Universitas Mulawarman Repository "REPO-MAN", 5(5): 16-18, https://repository.unmul.ac.id/handle/123456789/4925.
[11] Munassar, N.M.A., Govardhan, A. (2010). A comparison between five models of software engineering. International Journal of Computer Science Issues, 7(5): 94-101.
[12] Andry, J.F., Suroso, J.S., Bernanda, D.Y. (2018). Improving quality of SMEs information system solution with ISO 9126. Journal of Theoretical and Applied Information Technology, 96(14): 4610-4620.
[13] Al Nawaiseh, A.J., Helmy, Y., Khalil, E. (2020). A new software quality model for academic information systems ‘case study E-Learning system,’. International Journal of Scientific & Technology Research, 9(1): 3822-3833.
[14] Khairudin, N., Khairudin, R., Hamid, M.N.A., Hancock, P., McGill, T., Zamani, Z. A. (2016). The importance of human capital perspective in the learning management system (LMS) decision making process at universities. Jurnal Psikologi Malaysia, 30(2): 102-113.
[15] Mahmood, T., Hafeez, K. (2013). Performance assessment of an E-Learning software system for sustainability. International Journal of Quality and Service Sciences, 5(2): 208-229. https://doi.org/10.1108/IJQSS-04-2013-0022
[16] Farid, S., Qadir, M., Ahmed, M.U., Khattak, M.D. (2018). Critical success factors of E-Learning systems: A quality perspective. Pakistan Journal of Distance and Online Learning, 4(1): 1-20.
[17] Tarifi, N. (2021). A critical review of theoretical aspects of strategic planning and firm performance. Open Journal of Business and Management, 9(4): 1980-1996. https://doi.org/10.4236/ojbm.2021.94107
[18] Sanjaya, H., Andry, J. F. (2021). Quality assurance of project management information system with ISO 9126. Bulletin of Social Informatics Theory and Application, 5(2): 82-87. https://doi.org/10.31763/businta.v5i2.174
[19] Andry, J.F., Hadiyanto, Gunawan, V. (2023). Refining the ISO 9126 model for enhanced decision support system evaluation in the manufacturing industry. Ingénierie des Systèmes d’Information, 28(5): 1309-1315. https://doi.org/10.18280/isi.280519
[20] Djouab, R., Bari, M. (2016). An ISO 9126 based quality model for the E-Learning systems. International Journal of Information and Education Technology, 6(5): 370-375. https://doi.org/10.7763/ijiet.2016.v6.716
[21] ISO/IEC JTC 1 N 16434. (2023). JTC 1/SC 7 final business plan. ISO/IEC. https://www.iso.org/committee/45086.html.
[22] Al-Qutaish, R.E., Al-Fawareh, H.J. (2006). Mapping between ISO 9126 on software product quality metrics and ISO 12207 on software life cycle processes. In the 2006 International Arab Conference on Information Technology (ACIT'06), Yarmouk University, Irbid, Jordan.
[23] Miguel, J. P., Mauricio, D., Rodríguez, G. (2014). A review of software quality models for the evaluation of software products. International Journal of Software Engineering & Applications, 5(6): 31-54.
[24] Abou El-Seoud, M.S., Taj-Eddin, I.A., Seddiek, N., El-Khouly, M.M., Nosseir, A. (2014). E-Learning and students' motivation: A research study on the effect of E-Learning on higher education. International Journal of Emerging Technologies in Learning, 9(4): 20-26. https://doi.org/10.3991/ijet.v9i4.3465
[25] Onyema, E.M., Deborah, E.C., Alsayed, A.O., Noorulhasan, Q., Sanober, S. (2019). Online discussion forum as a tool for interactive learning and communication. International Journal of Recent Technology and Engineering, 8(4): 4852-4859. https://doi.org/10.35940/ijrte.d8062.118419
[26] Herrington, J., Oliver, R. (2000). An instructional design framework for authentic learning environments. Educational Technology Research and Development, 48(3): 23-48. https://doi.org/10.1007/BF02319856
[27] Inglis, A., Joosten, V., Ling, P. (2003). Delivering Digitally: Managing the Transition to the New Knowledge Media. Routledge. https://www.taylorfrancis.com/books/mono/10.4324/9780203417201/delivering-digitally-alastair-inglis-vera-joosten-peter-ling.
[28] Farid, S., Ahmad, R., Alam, M., Akbar, A., Chang, V. (2018). A sustainable quality assessment model for the information delivery in E-Learning systems. Information Discovery and Delivery, 46(1): 1-25. https://doi.org/10.1108/IDD-11-2016-0047
[29] Weli, W., Sjarief, J. (2022). E-Learning acceptance model in a pandemic period with an expansion to the quality of work life and information technology self-efficacy aspects. CommIT Journal, 16(2): 117-127. https://doi.org/10.21512/commit.v16i2.8145
[30] Aprilia, K., Lee, F.S., Geasela, Y.M., Everlin, S., Tampinongkol, F.F. (2024). Design of enterprise architecture using zachman framework at private school in center Jakarta. Journal of Computer Science, 20(9): 1059-1068. https://doi.org/10.3844/jcssp.2024.1059.1068
[31] Jordan, Andry, J.F., Adikara, F., Geasela, Y.M., Lee, F.S. (2024). Implementation of information system architecture using TOGAF and Ward Peppard analysis for high school. Journal of Computer Science, 20(6): 690-699. https://doi.org/10.3844/jcssp.2024.690.699
