Resumen de Estimation of teacher practices based on text transcripts of teacher speech using a support vector machine algorithm.
Roberto Araya, Francisco Plana, Pablo Dartnell, Jorge Soto-Andrade, Gina Luci, Elena Salinas, Marylen Araya
Teacher practice is normally assessed by observers who watch classes or videos of classes. Here, we analyse an alternative strategy that uses text transcripts and a support vector machine classifier. For each one of the 710 videos of mathematics classes from the 2005 Chilean National Teacher Assessment Programme, a single 4-minute slice was selected randomly and manually transcribed. The texts were then rated by two mathematicians and two teachers on the basis of the presence/absence of eight categories of contents and 12 categories of teacher practices. Then for each rater and category a classifier was built using a support vector machine algorithm trained with his/her ratings on a subset of texts. The agreements between the classifiers and the corresponding raters computed for texts belonging to an independent subset were better than the agreements between the raters. Next, we trained classifiers by using ratings created by raters who watched the videos rather than analysing the transcripts. Again, the agreements between classifiers and the raters were satisfactory. Given that class assessment can be an extremely slow, error prone and tedious process, the proposed automatic classifier scheme could be a promising alternative for monitoring classes and building online performance support system of teacher practices. Practitioner Notes What is already known about this topic High importance of monitoring teacher practices in order to improve teaching., Monitoring is currently performed by trained raters, but it is an infrequent, tedious and laborious task, prone to biases and errors., What this paper adds A vector support machine algorithm to build automatic raters with high agreement with human experts., A systematic method to quantify contents actually covered and strategies used in class., Tool that can be implemented in order to give continuous and immediate feedback to the teacher., Implications for practice and/or policy Computer methods can be used to construct personal and confident auto-support performance support system for improving teacher practices, avoiding the natural fear for evaluation by third parties., Computerised methods can be strategic tools to help improve quality of teaching. [ABSTRACT FROM AUTHOR]
