The effect of Bloom's taxonomy level and other factors on the performance of final year veterinary students in theoretical assessment

Abstract

Objective computer-based assessment was used for the final comprehensive theoretical day one competency examination of veterinary students at the University of Pretoria. This examination consisted of different question types in 11 veterinary disciplines across 6 domestic animal species. Questions were scrutinised by an expert panel. The questions were also categorised according to the six levels of Bloom’s revised taxonomy. Questions from all the cognitive levels were included in each discipline, to confirm that the students’ competence was assessed with the emphasis on application of information and higher order thinking skills (levels 3-6). With retrospective quality assurance, difficulty score was determined as the mean score achieved by students for each question. Discrimination score was determined (for single choice questions only) as the high-low discrimination ability of the question using Questionmark Perception. In the examination of November 2016 (215 questions and completed by 125 students), Bloom’s level 1 to 6 represented 11%, 27%, 29%, 23%, 7% and 3% respectively. Short answer-, single choice-, multiple response- and matching type questions represented 19%, 38%, 35% and 8% respectively of the maximum score for the examination. Data were entered into a statistical analysis programme (NCSS 2007, NCSS, Kaysville, UT, USA). Following descriptive statistics to determine univariable associations, multiple regression analysis was performed to establish the independent predictors (P < 0.05) of difficulty and discrimination score of questions. The difficulty score (mean 0.61; SD 0.26) and discrimination score (mean 0.23; SD 0.16) of single choice questions were negatively correlated; however this association was not significant (P=0.18). Question type, Bloom’s level and discipline, but neither species nor maximum score of the question, were independently associated with difficulty scores of all questions (P<0.05). It was concluded that balance between different disciplines and species represented in final year veterinary examinations can be improved by changing the weighting of different Bloom’s levels or question types in species and/or discipline categories.