The development of a scale to measure perceptions of the advanced automated aircraft training climate

Abstract

Commercial air travel is regarded as the safest mode of transportation known to humankind; however, every year people lose their lives from aircraft accidents and incidents. In addition, the financial impact of an air disaster can destroy an airline organisation. Studies have found that in adverse events involving highly advanced aircraft employing complex automation, human factor issues, and particularly pilot training, continue to play a significant causal role. Special attention should therefore be paid to the training of airline pilots, who are ultimately the last line of defence in aircraft operations. Airline pilots’ perceptions of the training climate associated with advanced aircraft can be a pervasive and powerful determinant of training outcomes and eventual flight deck behaviour. The study undertook to develop a valid and reliable instrument to measure airline pilots’ perceptions of the training climate associated with advanced aircraft equipped with highly complex automation. The goal was to construct a questionnaire by operationalizing an unobserved hypothesised construct (perceptions of the advanced automated aircraft training climate) based on three levels of analysis (the microsphere, mesosphere and macrosphere). The study also attempted to explore the statistical relationship between the demographic variables of the respondents and the latent factors of the construct. In order to meet the research objectives, the study began with a thorough review of the current literature on the topic to develop a systems model of the main construct under investigation. The review included a critique of the theory on organisational climate, learning, training and education, of historical data on aircraft automation, of human factors, and of aircraft accident investigation principles and case studies. The objectives of the research were fulfilled by strictly observing a positivist paradigm, and engaging in a quantitative exploration, triangulating methods with data captured from a purposive sample of the target population. The empirical study was completed in four phases. Firstly, the research construct was operationalized and the items in the proposed questionnaire validated by a panel of subject matter experts using Lawshe’s (1975) content validity ratio (CVR) technique. Inter-rater bias was assessed using Cochran’s Q test. This application resulted in the retention of 42 items. Secondly, factor analysis and item analysis was performed on the responses of the respondents for the development of the final 33 item measurement instrument. Thirdly, to explore the relationship between the demographic variables and the latent factors of the main construct, an appropriate non-parametric family of statistics was selected to gain a deeper understanding of the phenomena associated with the data. Finally, a logistic regression analysis that included specific demographic variables was performed for the development of a model to predict a pilot’s perception of the training climate associated with advanced automated aircraft. A non-probability purposive sample of 17 subject matter experts and 229 qualified South African airline pilots was used to accomplish the goals of the study. The underlying structure of the advanced Automated Aircraft Training Climate Questionnaire (AATC-Q) was derived from the results of a Principal Axis Factor (PAF) analysis using a promax (Kappa-4) rotation. The number of factors extracted from the data set was based on a modified version of Horn’s (1965) parallel analysis, namely the Monte Carlo simulation algorithm designed by O’Connor (2000). Three core factors explained most of the underlying variability in the main construct. The first factor was a composite at the macro and meso levels of analysis, whilst the second and third factors became fragmented at the micro level of analysis. These three factors were then labelled Organisational Professionalism, Intrinsic Motivation and Individual Control of Training Outcomes. The quality and rigour of the derived scale were demonstrated by its content and construct validity. Overall, satisfactory results from computing Cronbach’s coefficient alpha showed that the measurement scale was also reliable. The effect of the demographic variables on airline pilots’ perceptions of the advanced automated aircraft training climate was determined by computing relationships and comparing the responses from different categorised subsets with one another, by means of a non-parametric MANOVA and non-parametric analysis of variance. The results of these tests revealed that Flight Deck Position, Size of the Airline, Computer Literacy and Flight Experience had a significant effect on a pilot’s perception of the training climate. Results from a logistic regression model indicated that the interaction between pilots’ experiences and their perceived level of computer literacy (on a sigmoid curve), their actual experience in advanced aircraft, and their preferences for route and simulator training, were related to whether a pilot perceived the advanced aircraft training climate as favourable or not. The overall percentage of cases for which the dependent variable was correctly predicted by the regression model was computed at 63.8%. This study represents a vital step toward an understanding of the dimensionality of the learning, education and training for, and the actual operation of, highly advanced commercial aircraft, which employ complex automation. The results provide sufficient empirical evidence to suggest that the research findings may be of particular interest to aviation psychologists, aviation safety practitioners, and airlines engaged in training pilots to operate advanced aircraft.