Predictive validity and reliability of an incapacity and legal state algorithm applied to purposively developed narratives depicting mood disorders

Abstract

The challenges in assessing whether psychiatric treatment should be provided on voluntary, assisted or involuntary legal basis, prompted the development of an assessment algorithm that may aid clinicians. It comprises a part that assesses incapacity to give informed consent to treatment, care or rehabilitation. It also captures the patient’s willingness to receive this, the risk posed to the patient’s health or safety, financial interests or reputation, and risks of serious harm to self or others. Through various decision paths, the algorithm yields one of four legal states: voluntary, assisted, involuntary or that treatment, care or rehabilitation should be declined. This study examined the predictive validity and the reliability of this algorithm. It was applied 4 052 times to 135 clinical case narratives by 317 research participants. The legal states yielded by the algorithm matched highly statistically significantly with the gold standard (Chi-squared = 6 963; df = 12; p < 0.001). It was accurate in yielding the correct legal state for the voluntary, assisted, in-voluntary and decline categories in respectively 94%, 92%, 88% and 86% of the clinical case narratives. For internal reliability, a correspondence model accounted for 99.8% of the variance by which the decision paths clustered together fittingly so with each of the legal states. Inter-rater reliability testing showed a moderate degree of agreement among participants on the suitable legal state (Krippendorff’s alpha = 0.66). These results suggest the algorithm is valid and reliable, which warrant a subsequent randomised controlled study on whether it is more effective in clinical practice than standard assessments.