A tutorial for integrating generative AI in mixed methods data analysis

Abstract

The current article used real data to demonstrate the analysis and synthesis of Mixed Methods Research (MMR) data with generative Artificial Intelligence (Gen AI). I explore how reliable and valid Gen AI data outputs are and how to improve their use. The current content is geared towards enhancing methodological application regardless of field or discipline and includes access to a prompt library and examples of using outputs. The demonstration data used emanated from a study done in South Africa, with a quantitative sample size of 969 first-year engineering students and, for the qualitative part, 14 first-year students. In the current article, I compare my original analysis to ChatGPT results. Generative AI as a mind tool is best used with human insight, and I found this to be especially true when coding qualitative data. ChatGPT produced generic codes if asked to do inductive coding, and the results improved when training the Gen AI on human examples, which led to moderate and significant correlations between human and machine coding. The quantitative analysis was accurate for the descriptive statistics, but the researcher had to use best judgment to select the correct inferential analysis. Quantitative and qualitative analysis should be conducted separately in generative AI before asking the Chatbot for help with mixed methods results. In the current paper, I give guidelines and a tutorial on how to use chatbots in an ethically responsible and scientifically sound manner for research in social and human sciences.