Revisiting validity in cross-cultural psychometric test development : a systems-informed shift towards qualitative research designs

Abstract

The validity of clinical psychological tests remains a challenging issue, especially when the tests are applied in cultural groups other than those for which they were originally developed. Before the appearance of the International Test Commission (ITC) Guidelines, the commonly used cross-cultural test adaptation approaches often neglected intergroup differences in cultural values and jeopardised construct validity. The ITC Guidelines on adapting tests have shaped and streamlined current clinical cross-cultural test adaptation practice by promoting the equivalence of constructs and the elimination of bias. However, the emphasis appears to have remained on quantitative methods and the meaning behind the concepts is often still neglected. The culturally complex South African context has yielded commendable examples of clinical cross-cultural psychometric test development, but there has not been sufficient progress. In this paper we explore the potential benefits of applying systemic thinking in the field of clinical cross-cultural psychometric test development. We apply systemic concepts - viz. circularity, relationality, neutrality, and a concern with process issues - to suggest that qualitative research designs might be used more frequently. Multidisciplinary, multicultural test-developing teams that include members of the target cultural group could consult and collaborate with the target groups before embarking on test-adaptation or test-development activities. Teamwork would help to ensure that the meaning of the relevant concept or concepts is captured in a valid way for each cultural group. The emphasis might be on building theory and generating hypotheses, in order to pursue a deeper understanding of the constructs under investigation. Further exploration and application of the suggested procedures could form a focus for future research.