Comparison of Jury Ranking Systems
Read the PDF Comparison of Jury Ranking Systems to understand the math behind several different jury ranking systems that are commonly used.
Below is an excerpt from PDF Comparison of Jury Ranking Systems
Jurors tend to use the middle range of numbers when scoring work because each piece viewed is rarely the best or worst that the juror has ever seen. In other words, most of the work is average. No matter what ranking system is used, the jurors should be encouraged to use the full range of numbers to rank the images.
Juries seldom use only the 1 – 4 range (i.e. 1, 2, 3, 4) because fewer choices usually result in too many ties.
On the other hand, a ten number range (i.e. 1 – 10) offers too many choices, which tends to slow down decision-making.
Sometimes it is suggested to use a system that removes the middle number from a 1 – 5 system (e.g. eliminating the “3”) and to use only 1, 2, 4, 5 to “force” a selection outside of average. However, using fewer score choices increases the number of possible ties (regardless of the number of jurors). Mathematically, there is no difference between 1, 2, 3, 4 and 1, 2, 4, 5 since the number of sums (outcomes) is identical.
There are two advantages of using the 1 - 7 system over the 1 – 5 system.
Secondly, difficult negotiations among jurors can be kept to a minimum when designating awards because with fewer ties, it is more evident which pieces actually receive the highest scores