Resumen de Evaluating the Content and Response Process Validity of Data from the Chemical Concepts Inventory
Data produced by psychometric instruments are often used to inform understanding about a certain population’s knowledge of ideas or perspectives about specific topics. Concept inventories are an example of psychometric instruments used to probe students’ content knowledge within a defined framework. Concept inventories have been used as a measure of student understanding in science education for over two decades. The Chemical Concepts Inventory (CCI), published in 2002, is one of the most often referenced inventories in chemical education. However, not until the past year has detailed psychometric evidence been published about data from the instrument. Although this prior study focused on quantitative evidence to support the CCI as a whole, there are still no qualitative studies published to investigate the reasoning behind students’ responses. Therefore, the purpose of this study was to examine the response process validity of the CCI in order to understand the nature of student responses. In addition to evaluating individual items, this study examined the content validity of the CCI as a whole instrument. To do this, a survey was employed to understand expert judgments about the content being covered on the CCI. This information was used to group items by content prior to investigating student responses. It was found that a majority of the content on the CCI was limited to only two concepts (phase change and conservation of mass–matter). Student reasoning for their responses to items from these groupings was examined through think-aloud interviews. Both support for and threats to the response process validity for several items was observed. The results from this qualitative investigation can be used as a guide to improving the current CCI items, writing new items, and for the development of other concept inventories.
