Can we augment web responses with telephonic responses to a graduate destination survey?

Abstract

The 2010 Western Cape graduate destination survey utilised a sequential mixedmode design in which an initial web survey was augmented with an equivalent telephonic survey. This article examines mode effect in the Western Cape survey in terms of overall effect size and the bearing it had on the main outcome of the study. Standardised residuals and Cramér’s V are used to determine mode effect across two scenarios, a full sample vs. a subsample, and using two categorical questions with different numbers of response categories. Overall effect size appears to be small in the first question, but increases noticeably together with non-responses in the second question that has many more response categories. Web responses to alumni or graduate destination surveys can perhaps be augmented with telephonic responses if necessary, provided response categories are kept to a minimum, and interviewers are trained properly and monitored for possible interviewer misbehaviour. The benefit of obtaining larger samples should then also outweigh the benefit of using web surveys alone.