A systems approach towards a framework for utilising evaluative knowledge in the national policy system

Abstract

Despite commitments to establishing a developmental state, evidence indicates that the array of policies implemented since 1994 has not adequately addressed the needs of South Africans, 30 years after the end of apartheid. Significant deprivations persist, highlighting the need to re-evaluate current policy configurations. Evidence can be used to understand the causes of implementation failures and how they can be mitigated by identifying factors that enhance the effective use of evidence in policy formulation. However, despite the production of evidence, persistent barriers appear to hinder its use and subsequently impede its impact on policy formulation and implementation, with resultant poor policy outcomes that fuel citizen dissatisfaction and often fester into service delivery protests. Moreover, several gaps exist in the evidence use literature, motivating the current study and informing the development of a more comprehensive understanding of evidence-based practices. Several studies, particularly those that rely on literature reviews and syntheses of existing literature, lack empirical grounding. Although they identify contextual factors that influence evidence use, the absence of primary data limits insight into the lived experiences of those involved in the evidence use process, thus restricting the potential to generate new perspectives. Those who draw insights from officials provide only limited process-oriented perspectives, largely omitting an in-depth exploration of how evidence use unfolds in practice. Almost all reviewed studies offer a limited examination of the "how" and "why" of factors that influence evidence use. To address these gaps, this study investigated factors that influence the use of evidence within the South African National Policy System. The aim was to develop an evidence use framework to strengthen evidence-based policymaking in South Africa in general and the Social Cluster in particular. The ontological foundations of the study approach are rooted in the integrative philosophical traditions of pragmatism and interactionism, which recognise that knowing is created through ongoing reflective inquiry and action that takes place through experience and interaction within an organisation. The study's focus on utilising evidence in national policy is theoretically oriented towards the social accountability root of evaluation theory. It draws on an integrative theoretical framework that builds on two additional theories that explain how evidence is used within an organisational context. These are organisational learning theory (OLT) and the knowledge-to-action framework. The relevance of OLT for this study is its focus on how evidence is acquired, interpreted, and shared in ways that can influence collective decision-making processes. The study employed a grounded theory approach with a qualitative cross-sectional multiple-case study research design, that is, officials from four departments in the Social Cluster. The population for this study was all departments of the national government, as they were constituted in 2021, and as such, excludes the provincial and local government levels. A further delimitation was to restrict the selection of the four cases to the Social Cluster. These two delimitations support the depth-over-breadth objective of qualitative research designs. By combining this research approach and a context-rich exploration of lived experiences with the development of a grounded theory, this study examined both the "how" and the "why" of evidence use. This emphasis on process and experiential detail forms a key distinction between the present study and previous research. The findings of this study surfaced several key areas where improvements can be made in evaluation and evidence-based decision-making. Based on the analysis of the coding paradigm and the grounded theory narrative, the action-outcome framework of evidence use was developed that draws from 12 evidence-use themes. This comprehensive coverage allows a thorough understanding and systematic implementation of evidence-based practices at various stages of policy development and implementation. The emphasis on context in the evidence use framework recognises that politics, culture, and socioeconomics play a significant role in whether evidence is used. The framework ensures that policymakers know how evidence use factors influence decision-making in different contexts, thereby avoiding a one-size-fits-all solution to factors that might hinder evidence use. This makes its application more contextually relevant and impactful. The framework contributes to existing knowledge and theory in the field of evaluation and public policy by advancing the social accountability root of evaluation theory while providing practical guidelines for policymakers to improve the use of evidence in policymaking. Therefore, the action-outcomes evidence-use framework supports the development of more efficient and effective policies, ultimately contributing to the development of policies that can tackle South Africa's socioeconomic challenges.