Accuracy of chemistry performance evaluation of BSc four-year programme students : a case study

Abstract

The ability to make realistic judgements of one’s performance is a demonstration of the possession of strong metacognitive skills. Metacognition involves the monitoring of one’s progress during learning, and the ability to modify learning strategies for increased effectiveness. Poor-performing students are at risk because they generally exhibit high levels of overconfidence when evaluating their performance, and may fail to adjust their learning strategies in time. This study aims to explore the accuracy with which students in the BSc Four-year programme (BFYP) of the University of Pretoria evaluate their performance in a stoichiometry test, as well as the influence of teaching on test performance and on accuracy of performance evaluation. The factors that students rely on when making performance evaluations as well as shifts in the reliance on these factors after teaching are explored. Finally, the study examines the relationship between bias in performance evaluation and the self-protection, self-enhancement motivational factors and gender. Data were collected by means of a three-tier stoichiometry test instrument, administered as pre- and posttest, as well as a questionnaire administered simultaneously with the pretests to a sample of 91 students. Each test item comprised a stoichiometry question, a confidence rating and a free-response explanation for the choice of confidence rating. The confidence rating was interpreted as an indication of expected performance. The test instrument allowed for the investigation of bias in performance evaluation in the pre- and posttests, the exploration of factors that students rely on when making performance evaluations and how the reliance on these factors shifted in the posttests. The questionnaires were used to collect data on self-enhancement, self-protection and gender. The study shows that the majority of the students were overconfident in the evaluation of their performance in both the pre- and posttests. Performance improved significantly in the posttest but accuracy of performance evaluation did not. Students were categorised as overconfident (OC), realistic (R) or under-confident (UC) based on the difference between actual and expected performance. Five subgroups were defined on the basis of accuracy of performance evaluation in the pre- and posttests. The five subgroups, labelled first by their pretest and then their posttest category, were the OC-OC (50 students), OC-R (13 students), R-R (11 students), R-OC (15 students) and the R-UC (2 students) subgroups. The results indicated no significant difference between the pre-knowledge and ability of the students in the four main subgroups. The students differed significantly in terms of performance in the posttest, their pre- and posttest average confidence scores and in performance gain. A significant difference was not found with regard to performance in the CMY 143 end of semester examination. These findings confirmed that we were dealing with four discrete subgroups with different characteristics. The OC-R subgroup achieved the highest learning gain by a significant margin. Moderate learning gains were demonstrated by the R-R and OC-OC subgroups and the R-OC subgroup did not achieve any learning gain at all. Careful analysis of qualitative data revealed that accuracy in the evaluation of posttest performance was associated with both a reduction in the prevalence of vague subjective judgments and with higher performance gain. Similarly, an increase in the tendency to base metacognitive monitoring on vague global judgments of performance in the posttest was associated with reduced accuracy of self-evaluation and lower learning gain. The tendency by the four performance evaluation subgroups to self-enhance or self-protect was not found to be statistically different. P-values greater than 0.05 in the pre- and posttests indicated that males and females were not significantly different in their accuracy of performance evaluation. The study suggests that an element of bias in performance evaluation may be beneficial to learning. Inaccuracy in self-evaluation in the pretest did not hamper learning for both the OC-OC and OC-R subgroups. Students who were over-optimistic about their performance in the pretest may have been less intimidated by the challenges of the new content material than those who were better calibrated (R-R and R-OC subgroups). Students who remained overconfident in the posttest, i.e. in the OC-OC subgroup did not gain from the learning experience as much as those who entered overconfident but became better calibrated. Those who entered tentatively as realists and then, with a little exposure, became unrealistic in their performance evaluation were shown to be the most vulnerable based on their lack of learning gain. Furthermore, increasing content knowledge alone may not be enough to raise the metacognitive ability of students. Finally, chemistry educators should be aware that students often make vague subjective judgements of performance even on a topic like stoichiometry, which requires predominantly procedural knowledge and formal reasoning. Our study has shown that this deficiency, when associated with poor accuracy of self-evaluation, may hamper learning gain. Copyright