An optimized outlier detection algorithm for jury-based grading of engineering design projects

• Autores: Mary Kathryn Thompson, Christina Espensen, Line Harder Clemmensen
• Localización: The International journal of engineering education, ISSN-e 0949-149X, Vol. 32, no. 1 (Parte A), 2016, págs. 172-174
• Idioma: inglés
• Texto completo no disponible (Saber más ...)
• Resumen

  • This work characterizes and optimizes an outlier detection algorithm to identify potentially invalid scores produced by jurymembers while grading engineering design projects. The paper describes the original algorithm and the associatedadjudication process in detail. The impact of the various conditions in the algorithm on the false positive and false negativerates is explored. A response surface design is performed to optimize the algorithm using a data set from Fall 2010. Finally,the results are tested against a data set from Fall 2011. It is shown that all elements of the original algorithm (the base ruleand the three additional conditions) play a role in the algorithm’s performance and should be included in the algorithm.Because there is significant interaction between the base rule and the additional conditions, many acceptable combinationsthat balance the FPR and FNR can be found, but no true optimum seems to exist. The performance of the bestoptimizations and the original algorithm are similar. Therefore, it should be possible to choose new coefficient values forjury populations in other cultures and contexts logically and empirically without a full optimization as long as thealgorithm assumptions are valid and the limitations for its use are well understood.