Usability and validity of a dairy intake screener as a web-based mobile application for South African adults

Abstract

Background Paper-based dietary assessment tools such as food frequency questionnaires (FFQs) and especially dietary screeners are making way for versions that use technology. Amidst low intakes of dairy and dairy-related nutrients in South Africa and to increase public awareness thereof, this research aimed to develop and evaluate the usability of an application (app), namely the Dairy Diary, to screen for dairy intake in higher income South African adults. Thereafter, the screener was evaluated in terms of test-retest reliability and comparative validity. Methods Development and usability: In a consultative process, a dairy intake screener (Dairy Diary) was developed as an eight-item quantitative FFQ with four types of commonly consumed local dairy product, namely milk, maas (fermented milk), yoghurt, and cheese. For each dairy product, the usual frequency of consumption and portion size per eating occasion were scored (product serving score; PSS) and summed, resulting in a daily serving score (DSS) as a continuous variable with three risk classes, namely < 1 serving daily; 1 to < 2 servings daily; ≥ 2 servings daily. Digitalisation included product- and portion-specific graphics with linkage to risk class-relevant preliminary dairy-related guidance as part of a web-based mobile app. For the evaluation of the usability, the 26-item user-friendly end-user version of the Mobile Application Rating Scale (uMARS) was used in an online cross-sectional survey (Qualtrics, April 2020) of conveniently sampled adult respondents. Items were scored on a five-point Likert-type scale resulting in three scores made up of six subscales. Descriptive statistics summarised the findings, with mean scores ≥ 3.0 considered acceptable. Reliability and validity: In a diagnostic accuracy study, purposefully recruited undergraduate dietetics/nutrition student volunteers from three South African universities completed three non-consecutive days of weighed food records (reference standard) within a seven-day period (comparative validity), followed by two administrations, two weeks apart, of the screener (index test) (reliability). Dairy intake from the food records was converted to be comparable to dairy intake in the screener. For the reliability and validity assessment, statistical analyses included mean differences, paired t-tests and Pearson rank correlations for continuous data, and Kappa statistic for categorical data. For test-retest reliability, McNemar’s test for symmetry was performed on categorised DSSs. For the validity assessment, agreement between the DSSs of the first administration of the Dairy Diary and mean DSSs of the three food records was verified with Bland–Altman plots. Sensitivity , specificity , predictive values, odds ratios and receiver operating characteristics (ROC) were used to quantify the diagnostic ability of the categorised DSSs of the Dairy Diary. Results Development and usability: From 1 102 respondents, 703 (64%; 81% female; mean age 29.8 ± 11.0 years) were retained for analysis. The uMARS mean app quality score (objective) (3.9 ± 0.85), app subjective quality score (3.5 ± 0.77), app-specific score (3.6 ± 0.94), and the additional question on the e-portion (4.3 ± 0.78) exceeded minimum acceptability. For the subscales, the mean score for aesthetics was the highest (4.4 ± 0.82), followed by information (4.3 ± 0.90) and functionality (4.0 ± 1.33). Engagement scored the lowest (3.0 ± 1.55). Reliability and validity: Participants included a purposefully recruited sample of 79 (100% female; mean age: 21.6 ± 3.8 years). For reliability, mean PSSs and DSSs did not differ significantly (P > 0.05) between the screener administrations. Mean PSSs were strongly correlated: milk (r = 0.69; P < 0.001), maas (fermented milk) (r = 0.72; P < 0.001), yoghurt (r = 0.71; P < 0.001), cheese (r = 0.74; P < 0.001). For DSSs, Kappa was moderate (κ = 0.45; P < 0.001). Non-agreeing responses suggest symmetry (P = 0.334). For validity, the PSSs of the screener and food records were moderately correlated [milk (r = 0.30; P = 0.0129), yoghurt (r = 0.38; P < 0.001), cheese (r = 0.38; P < 0.001)], with κ = 0.31 (P = 0.006) for DSS. Bland–Altman analyses showed acceptable agreement for DSSs (bias: −0.49; 95%CI: -0.7 to −0.3). Categorised DSSs had high sensitivity (81.4%) and positive predictive value (93.4%), yet low specificity (55.6%) and negative predictive value (27.8%). The area under the ROC curve (0.7) was acceptable. Conclusion The Dairy Diary is a user-friendly screener for dairy intake, with high aesthetic appeal and low engagement. Furthermore, it is test-retest reliable and has moderate potential to be a comparatively valid tool to screen for dairy intake of groups of higher income South Africans.