Abstract:
The Progress in International Reading Literacy Study (PIRLS) is a large-scale reading comprehension assessment, which assesses Grade 4 learners’ reading literacy achievement. The findings from the last cycle of PIRLS 2016 indicated that South African Grade 4 and 5 learners performed poorly in reading comprehension. This finding confirms the previous cycles’ results where South African learners achieved the lowest results across the participating countries. Approximately eight out of ten Grade 4 learners cannot read for meaning in any of the tested languages. Due to the poor results in PIRLS, the President of South Africa stated that every ten-year old child should be able to read for meaning, thus cementing reading literacy as a national aim. The aim of this mixed methods research was to determine whether the PIRLS Literacy 2016 and PIRLS 2016 limited release texts are equivalent across languages, specifically English, Afrikaans and isiZulu.
Four research sub-questions were explored to assist in addressing the main research question posed by this study: To what extent are the PIRLS 2016 released texts in English, Afrikaans and isiZulu, in Grade 4 and Grade 5 equivalent? As this study took the form of a sequential explanatory mixed methods approach, the first phase investigated the South African Grade 4 and 5 results by firstly looking at descriptive statistics, such as percentages and means. After the initial exploration of the data, I conducted Rasch analyses to determine whether the items from the limited release texts showed measurement invariance – in other words, whether the items behaved differently for different groups of learners. As part of the Rasch analyses, individual item-fit statistics and differential item functioning (DIF) were conducted using RUMM2030. In phase two, the limited release texts were analysed by experts who attended workshops and completed open-ended questionnaires regarding the equivalence of the identified texts. The qualitative phase was conducted in order to complement and extend on the quantitative findings of phase one.
The findings revealed that the limited release texts, with their accompanying items, were not equivalent across the different languages. However, by looking at the items that displayed DIF, there is not a clear pattern as the items did not universally favour one language nor did the texts discriminate universally against a particular language. An in-depth look at the texts and items themselves revealed that the Flowers on the Roof text is considered the poorest translation into Afrikaans and isiZulu. Overall, all the texts were considered to be appropriate for South African learners as the texts made use of rich vocabulary and introduced the learners to new ideas and concepts. Thus, this study offers new insights into the equivalence of the PIRLS assessments as well as possible reasons for the non-equivalence for each of the limited release texts. Based on the findings of this study, recommendations and further research are provided.
