OPEN ACCESS
Usability is a major concern within adaptive user interfaces. It presents a combination of different attributes. The impact of each usability attribute may vary from one layer to another during the usability evaluation of adaptive user interfaces. On that basis, one question that arises is: “What are the priority levels of usability criteria that need to be assessed in individual layers and in the whole adaptive system?” This paper presents possible directions to address this question by identifying the priority level of usability criteria to be assessed in the adaptive user interfaces of information systems, considering the ISO/IEC 25040 standard. The priority level is calculated using a multi-criteria decision analysis method, namely the Analytic Hierarchy Process. The proposed approach provides guidance for evaluators to better evaluate adaptive user interfaces. An adaptive information system in the field of transport is presented in order to validate and illustrate our approach.
adaptive user interface, multi-criteria decision analysis method, ISO/IEC 25040 standard, usability criteria, layered evaluation
We thank Google for the royalty-free images and Google Maps illustrations that have been used in the prototype presented in this research.
Alshammari M., Anane R., Hendley R. (2015). Design and usability evaluation of adaptive elearning systems based on learner knowledge and learning style. Human-Computer Interaction - INTERACT. 9297, p. 584-591.
Alshammari M., Anane R., Hendley R. (2016). Usability and Effectiveness Evaluation of Adaptivity in E-Learning Systems. The 34th Annual ACM Conference on Human Factors in Computing Systems, San Jose, USA, p. 2984-2991.
Aydogan E. K., Delice, E. K., Papajorgji, P. (2013). An effective approach for evaluating usability of Web sites. Enterprise business modeling, optimization techniques, and flexible information systems- Hershey : Business Science Reference, ISBN 978-1-4666-3946-1, p. 97-107.
Bastien C., Scapin D. (2001). Évaluation des systèmes d’information et Critères Ergonomiques. in Environnements évolués et évaluation de l’IHM, Interaction hommemachine pour les SI, Hermès, Paris, p. 53-80.
Benyon D. (1993). Adaptive systems: a solution to usability problems. User Model. Useradapt. Interact. vol. 3, p. 65-87.
Bevan N. (1995). Measuring usability as quality of use. Software Quality Journal, vol. 4, n° 2, p. 115-130.
Brossard A., Abed M., Kolski C. (2007). Modélisation conceptuelle des IHM : Une approche globale s’appuyant sur les processus métier. Ingénierie des Systèmes d’Information (ISI),vol. 12, p. 69-108.
Brusilovsky P., Karagiannidis C., Sampson, D. (2001). The benefits of layered evaluation of adaptive applications and services. Proceedings of Empirical Evaluation of Adaptive Systems workshop associated with UM’01, S. Weibelzahl, D. N. Chin, G. Weber (Eds.), Sonthofen, Germany, p. 1-8.
Brusilovsky P., Karagiannidis C., Sampson D. (2004). Layered evaluation of adaptive learning systems. International Journal of Continuing Engineering Education and and Lifelong Learning, Burke R. (2002). Hybrid recommender, vol. 14, n° 4/5, p. 402-421.
Brusilovsky P., Farzan R., Ahn J. (2006). Layered evaluation of adaptive search. In: Workshop on Evaluating Exploratory Search Systems at SIGIR06, Seattle, WA, p. 11-13.
Dagdeviren M., Yavuz S., Kilinc N. (2009). Weapon selection using the AHP and TOPSIS methods under fuzzy environment. Expert Systems with Applications vol. 36, p. 81438151.
Delice E.K., Gungor Z. (2009). The usability analysis with heuristic evaluation and analytic hierarchy process. International Journal of Industrial Ergonomics, vol. 39, n° 6, p. 934-939.
Dhouib A., Trabelsi A., Kolski C., Neji M. (2016a). A classification and comparison of usability evaluation methods for interactive adaptive systems. 9th International Conference on Human System Interactions, Portsmouth, UK, p. 246-251.
Dhouib A., Trablesi A., Kolski C., Neji M. (2016b). An approach for the selection of evaluation methods for interactive adaptive systems using analytic hierarchy process. IEEE Tenth International Conference on Research Challenges in Information Science, 1-3 June, Grenoble, France, p. 1-10.
Dhouib A., Trablesi A., Kolski C., Neji M. (2017). EvalCHOICE: A Decision Support Approach for the Usability Evaluation of Interactive Adaptive Systems. International Conference on Knowledge Based and Intelligent Information and Engineering Systems, KES2017, 6-8 September 2017, Marseille, France.
Ezzedine H., Bonte T., Kolski C., Tahon C. (2008). Integration of traffic management and traveller information systems: basic principles and case study in intermodal transport system management. International Journal of Computers, Communications & Control, vol. 3, p. 281-294.
Gena C. (2005). Methods and techniques for the evaluation of user-adaptive systems. The knowledge engineering review, vol. 20, p. 1-37.
Gena C., Weibelzahl S. (2007). Usability engineering for the adaptive web. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) The Adaptive Web: Methods and Strategies of Web Personalization, p. 720-762.
Höök K. (2000). Steps to take before intelligent user interfaces become real. Interact. Comput. vol.12, n° 4, p. 409-426.
Hoo M.H., Jaafar A. (2013). An AHP-Based Approach in the Early Design Evaluation via Usability Goals. Advances in Visual Informatics. IVIC 2013. Zaman H.B., Robinson P., Olivier P., Shih T.K., Velastin S. (Eds), Lecture Notes in Computer Science, vol. 8237.Springer, Cham
ISO/IEC 14598-1 (1999). Information technology – Software product evaluation – Part 1:General overview, Geneva: International Organization for Standardization & International Electrotechnical Commission, April.
ISO 9241-110 (2006). Ergonomics of human-system interaction – Part 110: dialogue principles. International Organisation for Standardization, Geneve.
ISO/IEC 25040 (2011). Systems and software engineering. Systems and software Quality Requirements and Evaluation (SQuaRE) - Evaluation process.
Jameson A. (2003). Adaptive Interfaces and Agents. The Human-Computer Interaction Handbook, Lawrence Erlbaum Associates, New Jersey, p. 316-318.
Jameson A. (2005). User modeling meets usability goals. In: 10th International Conference on User Modeling, Edinburgh, UK. LNAI, Springer, Berlin. vol. 3538, p. 1-3.
Jameson A. (2008). Fundamentals, evolving technologies, and emerging applications. Adaptive Interfaces and Agents. In Sears A., Jacko J. (eds.), The human-computer interaction handbook, 2nd ed., Erlbaum, Mahwah, NJ, p. 433-458.
Jameson A. (2009). Understanding and dealing with usability side effects of intelligent processing. AI Mag. vol. 30, n° 4, p. 23-40.
Jitendra K., Nirjhar R. (2011). Analytic Hierarchy Process for a power transmission industry to vendor selection decisions. International Journal of Computer Applications, p. 26-30.
Karagiannidis C., Sampson D. (2000). Layered evaluation of adaptive applications and services. In: 1st International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems, Trento, Italy. Springer, Berlin, vol. 1892, p. 343-346.
Karagiannidis C., Sampson D., Cardinali F. (2001). Integrating adaptive educational content into different courses and curricula. Educational Technology and Society, vol. 4, n° 3.
Kolski C. (2011). Human-computer interactions in transport. ISTE Ltd and John Wiley & Sons Inc.
Kolski C., Ezzedine H., Gervais M., Oliveira K., Seffah A. (2012). Evaluation des SI, besoins en méthodes et outils provenant de l’ergonomie et des IHM. Actes XXXe Congrès INFORSID, INFormatique des ORganisations et Systèmes d’Information et de Décision (29-31 mai), Montpellier, mai, ISBN 2-906855-27-8, p. 395-410.
Manouselis N., Karagiannidis C., Sampson D.G. (2014). Layered evaluation for data discovery and recommendation systems: an initial set of principles. IEEE 14th International Conference on Advanced Learning Technologies, p. 518-519.
Mitta D.A. (1993). An application of the analytical hierarchy process: A rank-ordering of computer interfaces. Human Factors, vol. 35, n° 1, p. 141-157.
Mulwa C., Lawless S., Sharp M., Wade V. (2011). The evaluation of adaptive and user adaptive systems: A review. International Journal of Knowledge and Web Intelligence (IJKWI), vol. 2, n° 2/3, p.138-156.
Nielsen J. (1993). Usability Engineering. Academic Press, London.
Paramythis A., Totter A., Stephanidis C. (2001). A modular approach to the evaluation of interactive adaptive systems. In Proc. 1st Workshop on Empirical Evaluation of Adaptive Systems (UM’01), Sonthofen, Germany, p. 9-24.
Paramythis A., Weibelzahl, S. (2005). A decomposition model for the layered evaluation of interactive adaptive systems. In: 10th International Conference on User Modeling, Edinburgh, UK. LNCS, vol. 3538, p. 438-442.
Paramythis A., Weibelzahl S., Masthoff, J. (2010). Layered evaluation of interactive adaptive systems: framework and formative methods. User Modeling and User-Adapted Interaction. vol. 20, p. 383-453.
Park K. S., Lim C.H. (1999). A structured methodology for comparative evaluation of user interface designs using usability criteria and measures. International Journal of Industrial Ergonomics, vol. 23, n° 5, p. 379-389.
Saaty T.L. (1980). The analytic hierarchy process. McGraw-Hill International.
Saaty T.L. (2008). Decision making with the analytic hierarchy process. International Journal of Services Sciences, p. 83-98.
Smits D., De Bra P. (2011). GALE: a highly extensible adaptive hypermedia engine. In Proceedings of the 22nd ACM conference on Hypertext and Hypermedia, HT 11, New York, NY, USA, p. 63-72.
Soui M., Abed M., Kolski C., Ghédira K. (2012). Evaluation by simulation to optimise information systems’ personalisation quality in logistics, International Journal of Production Research, vol. 50, n° 13, p. 1-15.
Tarpin-Bernard F., Marfisi-Schottman I., Habieb-Mammar H. (2009). AnAmeter: The first steps to evaluating adaptation. In: 6th Workshop on User-Centred Design and Evaluation of Adaptive Systems at UMAP 2009, CEUR, Trento, Italy, p. 11-20.
Totterdell P., Boyle E. (1990). The evaluation of adaptive systems. Adaptive user interfaces, Browne D., Totterdell P., Norman M. (Eds.), Academic Press, London, p. 161-194.
Tobar C.M. (2003). Yet another evaluation framework. In:Workshop on Empirical Evaluation of Adaptive Systems at UM 2003, Johnstown, PA, p. 15-24.
Van Velsen L., Van der Geest T., Klaassen R., Steehouder M. (2008). User-centered evaluation of adaptive and adaptable systems: a literature review. The Knowledge Engineering Review, vol. 23, p. 261-281.
Vanderdonckt J. (1994). Guide ergonomique de la présentation des applications hautement interactives, Presses Universitaires de Namur.
Weibelzahl S. (2001). Evaluation of adaptive systems. In: 8th International Conference on User Modeling. LNCS, Springer, Berlin. vol. 2109, p. 292-294.
Weibelzahl S., Lauer C. U. (2001). Framework for the evaluation of adaptive CBR-systems. Experience Management as Reuse of Knowledge. Proceedings of the 9th German Workshop on Case Based Reasoning, I. Vollrath, S. Schmitt, U. Reimer (Eds.), p. 254-263.
Zen M., Vanderdonckt J. (2014). Towards an evaluation of graphical user interfaces aesthetics based on metrics. IEEE Eighth International Conference on Research Challenges in Information Science (RCIS), p. 1-12.