A Bayesian perspective on intervention research : using prior information in the development of social and health programs

Abstract

OBJECTIVE : By presenting a simulation study that compares Bayesian and classical frequentist approaches to research design, this paper describes and demonstrates a Bayesian perspective on intervention research. METHOD : Using hypothetical pilot-study data where an effect size of 0.2 had been observed, we designed a 2-arm trial intended to compare an intervention with a control condition (e.g., usual services). We determined the trial sample size by a power analysis with a Type I error probability of 2.5% (1-sided) at 80% power. Following a Monte-Carlo computational algorithm, we simulated 1 million outcomes for this study and then compared the performance of the Bayesian perspective with the performance of the frequentist analytic perspective. Treatment effectiveness was assessed using a frequentist t-test and an empirical Bayesian t-test. Statistical power was calculated as the criterion for comparison of the 2 approaches to analysis. RESULTS : In the simulations, the classical frequentist t-test yielded 80% power as designed. However, the Bayesian approach yielded 92% power. CONCLUSION : Holding sample size constant, a Bayesian analytic approach can improve power in intervention research. A Bayesian approach may also permit smaller samples holding power constant. Using a Bayesian analytic perspective could reduce design demands in the developmental experimentation that typifies intervention research.