Written by By Paul O’Connor, Justin Campbell, Jennifer Newon, John Melton, Eduardo Salas, & Katherine A. Wilson. Published in The International Journal of Aviation Psychology.
Reference:
O’Connor, P., Campbell, J., Newon, J., Melton, J., Salas, E., & Wilson, K. A. (2008). Crew resource management training effectiveness: A meta-analysis and some critical needs. The international journal of aviation psychology, 18(4), 353-368.
Source: University of Galway, Ireland Research Repository
This paper, written in 2008, is a meta-analysis that systematically reviewed published evaluations of CRM training effectiveness in order to quantify the impact of CRM on key outcomes such as learner reactions, attitudes, knowledge, and behaviours. The purpose was to move beyond individual case reports and anecdotal claims by statistically analysing results from multiple studies to determine whether CRM training produces measurable benefits and to identify methodological gaps in CRM evaluation research.
The authors identified 16 empirical CRM evaluation studies that met predefined inclusion criteria and extracted data on four outcome metrics: reactions (learner satisfaction), attitudes (toward CRM principles), knowledge (cognitive gains), and behaviours (observable team behaviours). The meta-analysis aggregated effect sizes across studies to estimate overall effectiveness.
The findings were generally encouraging: participants across the studies reported positive reactions to CRM (mean ~4/5 on Likert scales), large improvements in attitudes and behaviours, and a medium effect on knowledge following training. This suggests that CRM training tends to influence how crews think about teamwork and how they behave in team contexts more strongly than it increases declarative knowledge of human factors concepts.
However, the authors also highlighted important limitations and critical needs: inconsistent data reporting made it difficult to calculate precise effect sizes; few studies used standardised measures connected to operational performance; and there was little evidence linking CRM training directly to safety outcomes or mishap rates, a gap common in training research.
For CRM training in aviation, this meta-analysis is significant because it provided one of the first quantitative syntheses showing that CRM has positive impacts on key human factors outcomes, validated CRM’s general effectiveness, and called for more rigorous, multi-level evaluation methods that link training design to real operational performance and safety metrics.