Confidence versus performance as an indicator of the presence of alternative conceptions and inadequate problem-solving skills in mechanics

Abstract

This study investigated the use of performance-confidence relationships to signal the presence of alternative conceptions and inadequate problem-solving skills in mechanics. A group of 33 students entering physics at a South African university participated in the project. The test instrument consisted of 20 items derived from existing standardized tests from literature, each of which was followed by a self-reported measure of confidence of students in the correctness of their answers. Data collected for this study included students’ responses to multiple-choice questions and open-ended explanations for their chosen answers. Fixed response physics and confidence data were logarithmically transformed according to the Rasch model to linear measures of performance and confidence. The free response explanations were carefully analysed for accuracy of conceptual understanding. Comparison of these results with raw score data and transformed measures of performance and confidence allowed a re-evaluation of the model developed by Hasan, Bagayoko and Kelley in 1999 for the detection of alternative conceptions in mechanics. Application of this model to raw score data leads to inaccurate conclusions. However, application of the Hasan hypothesis to transformed measures of performance and confidence resulted in the accurate identification of items plagued by alternative conceptions. This approach also holds promise for the differentiation between over-confidence due to alternative conceptions or due to inadequate problem-solving skills. It could become a valuable tool for instructional design in mechanics.