How to evaluate students in times of ChatGPT


We have been carrying artificial intelligence in our pocket for a few years and we interact with it many times without being aware of it. In education it has also been used for a long time through tutoring and evaluation systems .

With the arrival and popularization of ChatGPT in its version 3.5, we have been more aware than ever of its advances. In recent months, a large number of generative artificial intelligence tools have proliferated. We can find applications that create text, music, audio, video, websites, images, logos, concept maps, travel itineraries, teaching units, etc.

New information and ease of use

This technology allows machines to simulate certain capacities that people have, such as thinking, solving problems, learning, and even being creative and generating information.

The European Parliament defines it as “the ability of a machine to present the same capacities as human beings, such as reasoning, learning, creativity and the ability to plan”.

In other words, it is a data-trained technology that is capable of acquiring information from its environment in order to learn, using strategies to make sense of the information, and thus being able to act in ways that seem rational and autonomous.

What is truly revolutionary about these tools is their ability to create new information and, at the same time, their ease of use. Although they can make mistakes and experience hallucinations , they are extremely interesting.

Examples of this are ChatGPT , which surprises us with its answers, or LuzIA , a Spanish development that answers questions, summarizes texts, transcribes audio or creates images. These technologies continue to evolve rapidly and will undoubtedly surprise us with more usefulness in the near future.

Educational impact and challenges

International organizations such as the European Commission in its 2020 White Paper , UNESCO in its guide on AI and education and in its analysis on ethics or the experts who participated in the Horizon report have recognized the importance of AI in education.

These international reports highlight the impact that it will have and raise various dilemmas, on the one hand about its ethical use and on the other, about how to integrate it into society and education. Teachers can use it to plan, to produce content and as an assistant in tutoring tasks, but perhaps the issue that worries the most is evaluation.

Evaluate in another way

Students have always sought information and have used all possible resources to complete academic tasks and pass assessments. But the AI ​​challenges traditional copy detection and raises new questions about how to assess students: It’s hard to detect whether an assignment has been done with Chat GPT or other generative AI tools. We currently lack robust technology capable of accurately and effectively detecting the information generated by these systems.

It is no longer enough to ask the student to analyze, compare, solve a problem or reflect, because the AI ​​will know how to do it too and just as well … or even better!

For decades, information technology and educational technology have been trying to find assessment systems that facilitate automatic corrections and that involve something more than setting up objective multiple choice tests. We need to consider what value we as teachers bring to the teaching-learning process. It is essential to understand that assessment is not just about grading and that it goes beyond collecting assignments.

There are no defined formulas or solutions that can help us without first reconsidering the entire educational process and all the factors that condition it, but we can propose some ideas and strategies that help to face the evaluation in the world of generative AI:

  1. Value the process more than the product . Teachers must become aware of the importance of the learning process itself, beyond the final product they deliver. And one way to address the educational challenge of using AI tools is to promote the importance of that process by explaining the tools they have used and how. We must give value to this process in the evaluation we do, thus combining qualifications typical of a summative evaluation with others of formative evaluation.
  2. Collaboration and interactive strategies can be promoted in the classroom , designing activities that incorporate AI tools. Teachers will have to supervise the work process and ask questions that stimulate higher capacities such as critical thinking, problem solving or reasoning, but in situations of interaction with teachers: asking for justifications, analyzing processes and results, proposing alternative solutions and continually generating new questions related to the topic. The classroom must be turned into an interactive space for the construction of learning.
  3. Creative assessment strategies can be used , beyond the traditional written exam: proficiency tests based on practical skills, oral presentations, debates, role plays, resolution of practical cases in the classroom, simulations, competitions, posters, student fairs, collaborative experiences with other universities…

With imagination we have to think of innovative strategies that undoubtedly represent a challenge for teachers of the 21st century, strategies that truly assess competencies and with which AI is our ally.

In December we will publish in RiiTE a monographic issue on AI in education, which will surely serve as an inspiration for educational innovation supported by AI. It is an exciting path for the coming years and we must prepare for it.

Author Bios: M. Paz Prendes Espinosa is a Professor of Educational Technology, Maria del Mar Sanchez Vera is. Professor of the Department of Didactics and School Organization. Member of the Educational Technology Research Group and Victor Gonzalez Calatayud is a Contracted Professor of Didactics and School Organization all at the University of Murcia