The stability of test design : measuring differences in performance across several administrations of an academic literacy test

Abstract

This study explores the stability or consistency across several administrations of a test designed to determine academic literacy levels. The reliability of two versions (one English, the other Afrikaans) of such a test used for placement purposes at three South African universities will first be analysed. Secondly, the number of potential misclassifications on the test is assessed (ie the extent to which the test does not measure fairly). Thirdly, the differences among the results of the various administrations of the test are explored, with a view to establishing whether such differences are both significant and relevant. AFRIKAANS: Hierdie bydrae verken die stabiliteit of konsistensie van 'n toets van akademiese geletterdheid oor verskeie aanwendings heen. Eerstens word analises aangebied van die betroubaarheid van twee weergawes (die een in Engels, die ander in Afrikaans) van toetse wat aan drie Suid-Afrikaanse universiteite vir plasingsdoeleindes gebruik word. Tweedens analiseer ons die getal potensiële misklassifikasies, dit wil sê die mate waarin die toets nie billik meet nie. Derdens ondersoek ons verskille in die resultate in afsonderlike aanwendings van die toets, en vra spesifiek of sulke verskille beduidend sowel as relevant is.