Abstract
Generative Artificial Intelligences have experienced significant evolution in recent years and, especially, in 2022. One of the AIs that has caused the most stir in the academic world is ChatGPT, which provides an interface that greatly simplifies the productive use of a large language model. These language models are capable of analyzing and generating texts with great speed and quality. These capabilities could have a relevant impact on teaching-learning methodologies and also on evaluation methods. In order to analyze the possible impact of ChatGPT on evaluation methods, in this work, ChatGPT’s ability to solve exams from 15 Software Engineering courses in a Computer Engineering degree has been tested. The results show that ChatGPT could have a clear impact on evaluation methods; as it is capable of passing a significant number of questions and problems of different nature in multiple courses. As a main contribution, a detailed study of the results by question and problem typology is provided, which allows establishing recommendations to be taken into account in the design of evaluation methods.
Roberto Rodriguez-Echeverria
INTIA Director and Associate Professor
Associate Professor at the University of Extremadura. Software passionate, Deep learner, MTB rider and father of 2.
José M. Conejero
Associate Professor
Associate Professor at the University of Extremadura. Research interests include model-driven development, data science, and machine learning.
Alvaro E. Prieto
Associate Professor
Associate Professor and Director of Electronic Administration at the University of Extremadura.