An investigation of the 2012 Annual National Assessment Grade 6 mathematics instrument

Abstract

The aim of this study was to investigate the quality of the Annual National Assessment (ANA) Grade 6 mathematics instrument including its design, with reference to a single education district. The main question that was investigated was: To what extent does the 2012 Annual National Assessment Grade 6 mathematics assessment instrument provide meaningful information for making appropriate interpretations on district level? The conceptual framework underpinning this study was drawn from the Queensland Studies Authorities Assessment Policy document. The research comprised a secondary analysis design applying mixed methods using the scripts of 546 learners in one district from 5 schools selected to represent a range of achievement. A content analysis of the instrument was undertaken, followed by a statistical item analysis applying the Rasch measurement model. These analytical methods were utilised to determine the quality of the ANA Grade 6 mathematics instrument. Content validity, construct validity and reliability was investigated in order to evaluate inferences that were made and actions that were taken based upon the mathematics performance of learners in Grade 6 in the Gauteng North District (GND) in the year 2012. The investigation revealed that construct validity and content validity were largely achieved, as items were appropriately aligned to the 2012 ANA Grade 6 mathematics curriculum. However errors in mathematics and language formulation detracted from the validity of the instrument. In the case of some items, lack of clarity may have confused learners. As far as reliability is concerned the investigation revealed that the instrument had a reasonable person separation index, a measure of both item and person reliability. However, these conclusions are based on a relatively small sample from only one district and therefore has somewhat limited applicability but is nevertheless of educational consequence.