Objective assessment of practical skills in finalist veterinary students

Abstract

Objective Structured Clinical Examinations (OSCEs) were used to assess practical skills as part of the final examination of veterinary students at the University of Pretoria. A list of 40 practical skills to be assessed was set up by the examination committee in consultation with two external moderators. Examiner training on skills assessment was provided in workshops, following which rubrics for OSCEs were developed through a four-stage revision process. Ten OSCEs were assigned to each of four discipline-based panels, and made available to students two months before the examination. Students were randomly allocated to an examination order in pairs, who then drew cards so that each student attended two panels: either Small Animals or Equines, and either Production Animals or Pathology and Veterinary Public Health. Students who failed one panel were offered an ancillary examination in that panel. Grades for the OSCEs were calculated as the proportion of correctly marked steps adjusted according to the subjective global rating score of the examiner and a critical error penalty. Theoretical knowledge was assessed separately, using Computer-Based Assessment. A total of 276 panels were examined with 20 failures and a mean grade of 85.5% (95% CI 83.3% - 87.6%), independent of whether it was a student’s first, second or ancillary panel (P > 0.37). Subjective examiner scores significantly decreased grades in all panels except Pathology and VPH where subjective scores were higher than in all other panels (P < 0.01), whereas objective scores were only higher than the Production Animal and Small Animal panels. Practical examination grades were not correlated with CBA grades (coefficient 0.08, P = 0.38). It was concluded that a range of practical skills can be assessed accurately and independently of theoretical knowledge using only two OSCE panels per student.