Resumen de User experience evaluation on university virtual learning through sentiment analysis
The use of new technologies and the number of users of university online learning systems have spread around the world in the last decades, showing a further increase with the spread of the Covid-19 pandemic since 2020. Additionally, ISO 9241-210:2019 sets international quality standards for designing human-computer interaction products, services, and systems that meet usability, accessibility, and User eXperience (UX) requirements. Therefore, the concept of UX has become very important as a quality requirement. For several authors, UX is a multidimensional concept that includes the motivations, feelings, and needs of end users. On the other hand, the United Nations' (UN) Sustainable Development Goal (SDG) 4 for 2030 aims to ensure inclusive, equitable, quality education for all globally. In this sense, in order to design interfaces and learning experiences in university environments that respect all quality specifications, it is necessary to evaluate the user experience of these environments automatically and accurately beforehand. Thus, the main objective of this thesis is to identify the most relevant specific characteristics in the user experience of university e-learning environments that allow specific and automatic analysis of the students' feelings in order to lay the foundations for the design of user-centered e-learning platforms. To this end, the study proposes to analyse the needs and feelings of online university students with digital, advanced, and efficient artificial intelligence methods. Therefore, this project investigates the application of machine learning models of sentiment analysis for the evaluation of user experience. These artificial intelligence techniques have been applied to the responses received from more than 2,000 university students surveyed from postgraduate online studies and massive open online courses (MOOCs). The results present the basis of a model that allows ontologically classifying categories or aspects of university online education and knowing the users' polarity of feeling about their e-learning experience in an automatic way. In this way, it has been possible to find out the students' opinions in an automated way with regard to key categories of digital teaching. In addition, student comments have been classified into several UX e-learning or UXEL dimensions. Also, it has been identified the polarity of sentiment for each dimension. To sum up, this work has generated four major contributions to the scientific community. Firstly, an adaptation of the validated questionnaire UEQ-S integrated and adapted to three e-learning platforms for specific postgraduate courses at the Universitat de València and at the Universidad Rey Juan Carlos; and for MOOCs at the Universitat Politècnica de València. Secondly, this thesis has generated an innovative application of sentiment analysis and machine learning methods through natural language processing for the evaluation of the user experience of university online students. Therefore, this method provides the analysis of learners opinions and classifies them according to their polarity in positive, negative or neutral. And thirdly, this scientific work brings out a proprietary ontology of aspects for the virtual learning experience associated with UX dimensions. This ontology used with sentiment analysis tools, allows classifying the polarity of student opinions (positive, neutral, negative) by key categories of e-learning (VLE, Teacher, Student, Sound, Image, Material, Exercise, Evaluation and Communication) and group the comments in three dimensions UX e-learning or UXEL (VLE, Social Connections, and Learning Resources and Tools). Finally, these contributions will help to evaluate in an automatic and accurate way several university e-learning environments in order to design user-centered virtual learning experiences more personalised and inclusive for all which suit quality standards and meet UN SDG 4 for 2030.
