Evaluating a human-machine student intervention framework in higher education from legacy data

Abstract

This study aims to answer the question: “Can education be abstracted in a framework and conceptually studied for a student intervention process?” To do so, it designs and tests a student intervention framework that uses a systems approach and complexity theory to learn from different student contexts and recommend and apply systemic interventions for students. The framework is based on the assumption that education is a complex system that involves multiple interacting elements, such as students, teachers, curriculum, policies, and resources. The specific interventions and their impact on student success are not assessed in this work, as they depend on the expertise in the education domain. Rather, this work focuses on the framework that can be used to apply and evaluate different contexts and interventions. The study applies complexity theory and systemic intervention theory to understand the lens and the methods for studying the system. It also explains the education system in South Africa as the context of the study. One of the main challenges of the study is data sharing and data handling. To address this challenge, the study generates synthetic data from openly available tabular data and evaluates its conditional interdependence using different machine learning classification tasks. Then, it applies the same methods to a real-world education dataset from the University of the Free State. The study contextualizes the student intervention framework as a multi-armed bandit (MAB) stateless reinforcement learning problem and tests its performance and viability using probabilistic models. The results show that the probabilistic models yield the best results with the minimum required fine-tuning, and that they scale well to the real-world dataset. The results also indicate that the framework is viable for student intervention recommendations within the context of contextual bandits. The study concludes with a discussion of the implications and limitations of the framework, and suggests areas for future research.Hierdie studie het ten doel om die vraag te beantwoord: “Kan onderwys in 'n raamwerk geabstraheer en konseptueel bestudeer word vir 'n studente-intervensieproses?” Om dit te doen, ontwerp en toets dit 'n studente-intervensieraamwerk wat 'n stelselbenadering en kompleksiteitsteorie gebruik om van verskillende studentekontekste te leer en sistemiese intervensies vir studente aan te beveel en toe te pas. Die raamwerk is gebaseer op die aanname dat onderwys 'n komplekse stelsel is wat verskeie interaktiewe elemente behels, soos studente, onderwysers, kurrikulum, beleid en hulpbronne. Die spesifieke intervensies en hul impak op studentesukses word nie in hierdie werk beoordeel nie, aangesien dit afhang van die kundigheid in die onderwysdomein. Inteendeel, hierdie werk fokus op die raamwerk wat gebruik kan word om verskillende kontekste en intervensies toe te pas en te evalueer. Die studie pas kompleksiteitsteorie en sistemiese intervensieteorie toe om die lens en die metodes vir die bestudering van die stelsel te verstaan. Dit verduidelik ook die onderwysstelsel in Suid-Afrika as die konteks van die studie. Een van die hoofuitdagings van die studie is datadeling en -hantering. Om hierdie uitdaging aan te spreek, genereer die studie sintetiese data uit oop beskikbare tabeldata en evalueer dit die voorwaardelike interafhanklikheid met behulp van verskillende masjienleer klassifikasietake. Dan pas dit dieselfde metodes toe op 'n werklike onderwysdatastel van die Universiteit van die Vrystaat. Die studie kontekstualiseer die studente-intervensieraamwerk as 'n multi-arm bandiet (MAB) staatlose versterkingsleerprobleem, en toets sy prestasie en lewensvatbaarheid met behulp van waarskynlikheidsmodelle. Die resultate toon dat die waarskynlikheidsmodelle die beste resultate lewer met die minimum vereiste fynafstelling, en dat hulle goed skaal na die werklike datastel. Die resultate dui ook daarop dat die raamwerk lewensvatbaar is vir studente-intervensieaanbevelings binne die konteks van kontekstuele bandiete. Die studie sluit af met 'n bespreking van die implikasies en beperkings van die raamwerk, en stel gebiede vir toekomstige navorsing voor.Thuto e e dira go araba potso: “A thuto e ka abstrakwa mo sebokeng le go ithutelwa ka tsela ya konseptuele go dira tsamaiso ya go tsaya karolo ya baithuti?” Go dira jalo, e dira le go leka seboka sa go tsaya karolo ya baithuti se se dirisang tsela ya ditiragalo le teori ya boiteko go ithuta go tswa mo dikonteksteng tsa baithuti tse di farologaneng le go neela baithuti ditlamelo tsa ditiragalo le go di dirisa. Seboka se se theilweng mo go bonaleng gore thuto ke tiragalo e e boitekang e e akaretsang ditiragalo tse di farologaneng, jaaka baithuti, barutabana, kurikulamo, melao, le dikgwebo. Ditlamelo tse di kgethegileng le maatla a a nang le one mo go atlegeng ga baithuti ga a lekolwa mo tiro eno, ka gore e a itshetlela boitseanape mo lefelong la thuto. Go feta fa, tiro eno e fokotsa mo sebokeng se se ka dirisiwang go dirisa le go leka dikontekste tse di farologaneng le ditlamelo. Thuto e e dirisa teori ya boiteko le teori ya go tsaya karolo ya ditiragalo go tlhalosa lenaneo le metotso ya go ithuta tiragalo. E tlhalosa le tiragalo ya thuto mo Aforika Borwa jaaka kontekste ya thuto. E nngwe ya ditshwanelo tse di kwa godimo tsa thuto ke go abelana le go laola data. Go rarabolola ditshwanelo tse, thuto e e dira data e e dirilweng go tswa mo data ya tabulere e e sa tlhokomelwang le go e leka go tswa mo go bonaleng gore e na le maemo a a farologaneng go dirisa metotso ya go ithuta ya meesene e e farologaneng ya go kgaoganya. Ka morago, e dirisa metotso e e tsamaisanang le data ya thuto ya nnete mo Aforika Borwa mo Yunibesithing ya Freistata. Thuto e e dira go bonala seboka sa go tsaya karolo ya baithuti jaaka mathata a a nang le diatla tse di farologaneng (MAB) a a sa nang le maemo a go ithuta go rarabolola. Dipoelo tsa ditlhotlhomiso di bontsha gore metotso ya go tlhoka go itse pele e e neela dipoelo tse di kwa godimo, le go tlhoka go fetola sentle. Gape, dipoelo di arogana sentle le data ya nnete. Dipoelo di bontsha le gore seboka se se a tsamaya go neela baithuti ditlamelo tsa ditiragalo mo konteksteng ya diatla tse di farologaneng. Thuto e e tlhagisa ka poledisano ya maemo le ditshwanelo tsa seboka, le go neela dikarolo tsa dithuto tse di tlang.