Setting a good exam that leads to fair results is demanding, but do we really need to have so many questions? Isn’t it possible, by using modern methods, to achieve at least as unequivocal results with fewer questions?
A recent article about the good results achieved by second-year students during the pandemic, gave us a foretaste of possible new exam formats. The big question is now whether the exam methods used during the pandemic can be further developed in order to make written exams shorter, with fewer questions and of equal, if not better, quality. Schauber and his English colleague Matt Homer at the University of Leeds are now in the process of studying just that.
Exams must be adapted to suit their purpose
– We are working on a study which challenges the conventional way of thinking: that exams have to be long to be good. We claim that the purpose of each exam should be the deciding factor and that the train of thought should suit this purpose. In order to underpin our argument, we are going to create a simulation based on results from previous, and of course anonymized, exams taken at the Medical Faculty.
Advanced exam questions are better than many
– For those involved in setting and marking exams, it is important to decide how many observations are necessary in order to constitute a good exam. It is much simpler and cheaper to develop twenty thoroughly formulated case scenarios, where the questions are so advanced that the students have to reason their way to the answers, than to set a hundred questions. Perhaps the emphasis on quantity can be to the detriment of quality.
The current practice is that questions leading to bad results, where for example very few students give the correct answer to questions that the vast majority should get right, are evaluated as a weakness in the exam paper. If such unintended consequences are found, the question will not be marked by the examiner.
Alternative ways of measuring knowledge
Schauber also believes that it could be interesting to investigate other ways of measuring students’ knowledge.
–The students could sit a larger number of smaller tests during the course of the term instead of one big exam at the end. This may be one way of taking a critical look at the current way of assessing student’s knowledge. One long exam may actually be more about endurance than actual knowledge, says Stefan Schauber, who is a psychologist with a post at the Unit for Health Sciences Education.
He emphasizes that the data handling in the study is SIKT-approved.