Assessing undergraduate dental students' exodontia competencies by employing a novel continuous assessment instrument

Abstract

Background Universities are obliged to ensure that health professions graduates are competent to render safe and effective treatment. Unfortunately, empirical evidence of competence development is lacking, especially at undergraduate level. Hence, this study aimed to provide empirical evidence of dental students’ development of independence in exodontia (tooth extractions) following task-level feedback in relation to achievable targets. Summary of work This entails the implementation of a novel workplace-based assessment instrument in the Module: Oro-facial Surgery, School of Dentistry, University of Pretoria. Thirteen trained clinical supervisors guided by the primary researcher assessed 28280 tooth extractions performed by fourth and fifth year dental students (2014-2016). Quarterly task-level feedback was provided to students on their ability to independently perform tooth extractions using “Independence ratios” (IR=extractions performed without assistance/total number of extractions) as key performance indicator. A customised Level of Difficulty Index (LDI) (<2=easier than standard; 2=standard; >2=more difficult than standard) was used to control for difficulty level attained. Based on 2014 data, minimum targets of 80% and 90% independence were, respectively set for fourth and fifth year students. Feedback aimed to increase the number of students who achieved targets. Only very low performing students were subjected to targeted intervention. Remaining students were left on their own to progress. Dijksterhuis’ model of progressive independence and Zimmerman’s model of self-regulated learning, served as conceptual models for analysis of structured student narrations (BChD IV 2015-2017) to evaluate the instrument. Summary of results Respective mean IRs (SD), (Ranges: IR; LDI; EC) for the 2014 (n=42), 2015 (n=51) and 2016 (n=62) BChD IV cohorts were 85% (SD: 6%) (Ranges: IR=73-95%; LDI=1.9-3.3; EC=57-232), 85% (SD: 5%) (Ranges: IR=71- 97%; LDI=1.8-3.1; EC=65-261) and 88% (SD: 4%) (Ranges: IR=74-94%; LDI=1.6-3.9; EC=55-140). The 2014 BChD V cohort (n=58) eventually achieved a mean IR of 90% (SD: 5%) (Ranges: IR=79-100%; LDI=2.2 - 4.4; EC=27-168). For the respective 2015 (n=37) and 2016 (n=51) BChD V cohorts these performance indicators increased to mean IRs of 92% (SD: 4%) (Ranges: IR=78-98%; LDI=1.8-4.7; EC=65- 219) and 94% (SD: 3%) (Ranges: IR=86-100%; LDI=1.6-4.3; EC=65-150). Students (BChD IV 2014) who achieved 80% independence increased from 60% after mid-year feedback to 81% at the end of the year. The 2015 and 2016 cohorts respectively improved from 67% to 86% and 56% to 97%. BChD V 2014 students who achieved 90% independence, increased from 40% after mid-year feedback to 48% at the end of the year. The 2015 and 2016 cohorts’ improvements were 57% to 65% and 82% to 88%, respectively. Assessment differences were generally small among assessors and could be explained by operational circumstances. The qualitative analysis using the Dijksterhuis model revealed that trainee factors, supervisor factors, the professional activity and the working environment impacted significantly on assessment results. Another analysis using the Zimmerman model revealed that most students were self-regulated learners who set independence targets for themselves. Only a few students showed limited progression. Discussion & Conclusions IR appears to be a useful indicator of clinical competence as gradients of increased independence were illustrated over time. IR measurement was sensitive enough to distinguish between low and high performing students. Target introduction in 2015/16 coincided with increased independent practice compared to 2014 baseline data, suggesting a catalytic effect of assessment. Achievable objectives should accompany task-level feedback to facilitate competence development. Methods employed may be transferable to other disciplines.