JENUI 2023: best paper

On November 30, 2022, OpenAI introduced ChatGPT. Like most people, it caught us by surprise 🤯 with its incredible capabilities. From the very first moment we wondered what impact it would have on the different aspects of our daily lives: in Computer Engineering, in research and in teaching. From an idea proposed by our colleague Roberto, we decided to focus on the latter.

The result of several months of effort is our paper “Impact of ChatGPT on the evaluation methods of a Computer Engineering degree”, which we submitted to JENUI 2023 and for which we were awarded the best paper prize at the conference. In this edition, dominated by ChatGPT (something that has kept some speakers awake at night), the paper “Compete or not: analysis of Wooclap’s competition mode based on performance and audio processing”, by our colleagues from the University of Murcia, was also awarded.

Here you can see Roberto presenting our experience to the rest of the conference attendees (about 10 minutes plus the Q&A session):

As soon as the proceedings are published we will share them so you can read the paper and give us your opinion. In the meantime, here is the abstract:

Generative Artificial Intelligences have experienced significant evolution in recent years and, especially, in 2022. One of the AIs that has caused the most stir in the academic world is ChatGPT, which provides an interface that greatly simplifies the productive use of a large language model. These language models are capable of analyzing and generating texts with great speed and quality. These capabilities could have a relevant impact on teaching-learning methodologies and also on evaluation methods. In order to analyze the possible impact of ChatGPT on evaluation methods, in this work, ChatGPT’s ability to solve exams from 15 Software Engineering courses in a Computer Engineering degree has been tested. The results show that ChatGPT could have a clear impact on evaluation methods; as it is capable of passing a significant number of questions and problems of different nature in multiple courses. As a main contribution, a detailed study of the results by question and problem typology is provided, which allows establishing recommendations to be taken into account in the design of evaluation methods.

Interested in the topic? Want to collaborate? Get in touch with us and let’s talk.

Update: You can read the cited paper right here.