Examining PIRLS 2021 : differential item functioning across English and isiZulu language groups

Abstract

In today's interconnected and rapidly evolving world, reading literacy stands as a fundamental skill that empowers individuals to thrive intellectually, socially and economically. This study aimed to shed light on the factors that might have a bearing in reading literacy, with a specific focus on isiZulu, using insights derived from the Progress in International Reading Literacy Study (PIRLS) data. The PIRLS trend assessments have revealed that South Africa is experiencing a ‘reading crisis’ in that all South African Grade 4 learners’ achievement falls below the International Benchmark. This study investigated measurement invariance as a possible factor that might have contributed to the difference in learner achievement in isiZulu when compared with English scores. IsiZulu learners came across words such as “i-Hammerhead shark” in the PIRLS achievement booklets that could not be translated into isiZulu by professional translators. It is challenging to expect Grade 4 learners to comprehend and answer items derived from these words if they are difficult to translate or unavailable in isiZulu. To ascertain equivalence, item difficulty was examined to ascertain whether the level of difficulty was the same for the two groups. Functional equivalence was examined to ascertain the behaviour of the items in the two groups. Using Rasch Measurement Theory (RMT), Differential Item Functioning (DIF) was conducted to determine biases towards the learners who completed the PIRLS 2021 assessment in isiZulu. Descriptive statistics revealed that there is a significant difference between the English and isiZulu mean scores. Raw scores and Rasch Analysis displayed some items in the passage lacked metric and functional equivalence.