Thurstone's comparative judgement The job of an exam is to judge the students using the right evidence
The notion that we need numbers in exams was developed in the
University of Cambridge in the late 18th century, as a tool to avoid
unfairness: but it is not necessary. The requirement is that we find
some way to judge the students' performances in order to create the
scale we need, and marking items to add up their scores is just the
way we seem to have chosen to do this. Louis Thurstone proposed, in
1927, a method for constructing a scale based on direct comparisons of
pairs of objects.
We introduced this method to educational assessment
in 1993, and it has since become the standard experimental method for
comparing standards in England.
We are now piloting the method for
examinations, in collaboration with Goldsmith's College, University of
London. Early pilots are very promising, and an internet-based system
has been developed to collect the judgements from judges. Because
judges compare students' work directly in terms of quality, we believe
the method may prove more valid than marks-based assessment
See also: Bramley, T(2007) Paired comparison methods. In Newton, P, Baird, J, Patrick, H, Goldstein, H, Timms, P and Wood, A (Eds) Techniques for monitoring the comparability of examination standards. London, QCA.