The use of Rasch Measurement Theory to address measurement and analysis challenges in social science research

Abstract

High quality assessment instruments, in conjunction with best practice for data processing, analysis and reporting are essential for the monitoring of academic school achievement. In this thesis, Rasch Measurement Theory (RMT) as the primary method, addressed issues related to the monitoring of academic achievement. Rasch theory makes use of logistic regression models, which calibrate instruments by calculating item and person fit. The main study monitored the academic achievement of 3 697 Grades 8 to 11 learners at seven independent high schools in South Africa over a three- year period. Monitoring was done via specifically designed assessment instruments for Mathematics, Science and English Language. The main research question asked: How does the application of Rasch models address measurement problems in the processing, analysis and reporting of educational monitoring results? The thesis comprises three articles (presented as chapters and seen as sub-projects), and investigates challenges arising from the monitoring project. Measurement challenges addressed includes how to impute Missing Not At Random Data (Article 1), how to evaluate anchor items and reframe results (Article 2) and create proficiency bands (Article 3). Recommendations from the articles consist of using Rasch measures as predictors for imputation models, applying the Rasch models for evaluating anchor items and reframing test re-test results and the use of Rasch Item Maps for reporting criterion-referenced results. The thesis concludes by recommending that psychometric theory and application be taught in social science courses for the development of high quality instruments and the strengthening of measurement within the human sciences.