Application of ROC curve analysis to FAMACHA© evaluation of haemonchosis on two sheep farms in South Africa

Abstract

Test sensitivity and specificity for the FAMACHA© clinical test for anaemia due to haemonchosis have previously been shown to be adequate in differentiating between heavily/less heavily infected sheep, but these properties give no objective guidance for setting the optimum threshold at which anthelmintic treatment should occur. The aim of this work was to use Receiver Operating Characteristic curves (ROC) to evaluate the diagnostic accuracy of FAMACHA© testing by estimating the area under the ROC curve, and to use Two-graph ROC curves to decrease subjectivity in selecting treatment thresholds on two farms with contrasting management. Test diagnostic accuracy, and thus discriminating power as determined by the area under the ROC curves, ranged from “moderate to good” on the first farm, and from “moderate to high” on the second farm for haematocrit (the Gold Standard for the test) cut-offs of ≤22 % and ≤19 % on both farms respectively. Accuracy of classification between haematocrit cut-offs was not significantly different within farms, but did differ significantly between farms, with test accuracy being highest on the second farm at both haematocrit cut-offs (p< 0.05). The results also showed the suitability of the two-graph ROC curve approach for discriminating not only between different levels of accuracy of evaluators, but also to give an indication of the so-called ROC cut point (i.e. the desired threshold level) at which animals should be treated for a given level of risk of loss. The approach appears to have the potential not only to validate the diagnostic accuracy of the test across the complete testing range (i.e. all FAMACHA© categories from 1 – 5), but also to compensate for such inaccuracy by allowing objective adjustment of the threshold treatment level according to the output of the two-graph ROC method.