OLYMPIADS IN INFORMATICS, 2009, Vol. 3, 3-16
© Institute of Mathematics and Informatics,
ISSN 1822-7732
Using Item Response Theory to Rate (not only) Programmers
Michal FORISEK
Department of Informatics, Faculty of Mathematics, Physics and Informatics Comenius University Mlynská dolina, 842 48 Bratislava, Slovakia E-mail: forisek@dcs.fmph.uniba.sk
Abstract
We show how Item Response Theory (IRT) can be used to define a new type of rating system, one that is especially suitable for programming competitions (and other types of competitions where difficulty of competitions varies between rounds). We show some useful theoretical properties of this rating system, including the ability to argue about hardness of past competition tasks, and about the precision of contestants' skill estimates. We also define an objective method of comparing different rating systems. In the final section of the paper we apply our methods on real competition data.
Keywords:
item response theory, ranking, rating, programming competitions
To preview full
article text in PDF format click here
You
could obtain free Acrobat Reader from Adobe
Copyright © Olympiads in Informatics, Vilnius University Institute of Mathematics and Informatics, 2009
