Resumen de Assessment of conceptual knowledge using a component-based concept map scoring program
Mary Katherine Watson, Elise Barrella, Joshua Pelkey
Conceptual understanding is an important prerequisite for engineering competence. Concept maps, which capture thecontent and structure of knowledge, can be used to assess conceptual knowledge, although cumbersome scoring methodslimit their use. A literature review was conducted to summarize concept map scoring methods and automated scoringprograms. While quantitative, component-based methods prevailed in the literature, no program was available toautomate this method. Thus, the goal of this project was to present and evaluate a component-based computer programfor scoring concept maps. The program automates application of the traditional scoring method in which number ofconcepts, highest hierarchy, and number of cross-links are counted as indicators of knowledge breadth, depth, andconnectedness, respectively. A sample of concept maps (n = 78) was scored by two judges and the computer program. Highagreement (Krippendorff’s alpha > 0.80) between manual and automated scores was observed for number of concepts andnumber of cross-links. Although less than acceptable agreement between manual and automated scores was observed forhighest hierarchy, the two measures of knowledge depth were highly correlated (Spearman’s rho > 0.5). Ultimately, thecomputer program’s measure of knowledge depth was termed longest path, while judges’ measure of knowledge depth wastermed longest hierarchy. Overall, the computer program can be used to rapidly, precisely, and reliably score concept mapsto aid in assessment of conceptual knowledge.
