Weaver, Sallie J. MS; Lyons, Rebecca; DiazGranados, Deborah MS; Rosen, Michael A. PhD; Salas, Eduardo PhD; Oglesby, James; Augenstein, Jeffrey S. MD, PhD; Birnbach, David J. MD, MPH; Robinson, Donald MD, LTC; King, Heidi B. MS
Undoubtedly, providing quality health care today is a team-based effort. The question is, how do providers achieve a high level of team performance? The new era in health care demands optimized team interaction, and integrating team training throughout both initial educational experiences and in continuing education is one evidence-based tool for doing so.
Meta-analytic investigations1–4 of team training spanning a range of organizational contexts indicate that such programs can have meaningful effects on important team processes and outcomes. For example, Salas and colleagues1 have demonstrated, across 93 effect sizes representing 2,650 teams, that nearly 20% of the variance in team processes (ρ = 0.44) and outcomes (ρ = 0.39) can be attributed to team members' participation in team training. Furthermore, team training is equally as effective for teams who do not work together on a regular basis (i.e., ad hoc teams, ρ = 0.44) as it is for teams that do (i.e., intact teams, ρ = 0.48).
Considering the impact of team training in other high-risk areas such as aviation,5,6 health care educators and practitioners have also begun to adopt similar practices. A meta-analysis of 80 health care teams supports team training as a mechanism for improving medical team effectiveness1; however, previous reviews7,8 also caution that the quality of evidence reported for health care team training limits generalizability and the ability to draw meaningful conclusions from quantitative indicators alone. Thus, our review goes beyond an analysis of quantitative indicators to qualitatively examine the current structure of health care team training design, implementation, and evaluation. Qualitative methods provide insight into the structural interworkings of why, how, and when these programs are effective—that is, the anatomy of team training. The scope and qualitative nature of this review distinguish it from previous reviews,7–10 and our narrow focus on health-care-specific team training efforts also provides unique insight.1,2 Understanding what has been successful in terms of pretraining planning, design, content, instructional methodologies, and evaluation techniques provides guidance for future training development and implementation, and it also helps target future research areas. To this end, we first define teamwork and team training. We then describe the methodology of the qualitative review and content analysis. Results are structured around 11 key questions, derived from a combination of theoretical models of training design, evaluation, and effectiveness11–13 supplemented by additional features arising from current analyses. After a discussion synthesizing critical themes, we suggest several conclusions offering guidance for future team training efforts, in hope that the evidence base will continue to grow.
What is teamwork?
A team refers to two or more individuals, each with specific roles, working toward a common goal with concrete boundaries. Teams work on complex tasks requiring a dynamic exchange of resources (e.g., information), coordination of effort, and adaptation to changing situational factors.14 Teamwork is the vehicle through which such coordination occurs. It is defined in terms of the behaviors (e.g., closed-loop communication), cognitions (e.g., shared mental models), and attitudes (e.g., collective efficacy, trust) that combine to make adaptive interdependent performance possible.15
What constitutes team training in health care?
Team training is defined as a set of theoretically derived strategies and instructional methodologies designed to (1) increase the members' knowledge, skills, and attitudes (KSAs) underlying effective communication, cooperation, coordination, and leadership and (2) give team members opportunities to gain experience using these critical KSAs.13,15–17 It is not simply a “place” where employees go or necessarily a single program or intervention.15,17
Although many parallels can be drawn between health care team training and training programs in other complex team settings, there are unique factors affecting teamwork among health care teams. For example, team membership and team size are relatively dynamic, even throughout a single performance episode. Additionally, health care teams can be conceptualized across patient population (e.g., pediatric teams), disease type (e.g., stroke teams), and/or care delivery settings (e.g., primary care, prehospital care, inpatient care, long-term care).13 Although few direct comparisons of medical teams with other team types appear in the existing literature, recent meta-analyses have suggested that team training targeting medical teams exhibits similar effects to those observed for aviation teams and ad hoc teams used in laboratory-based studies.1 The remainder of this report is dedicated to reviewing published evaluations of health care team training to further explore such findings.
We conducted this review as part of a broader literature review designed to identify a comprehensive database of published studies relevant to team training. Our electronic search of Google Scholar, Science Direct, PsychINFO, EBSCOhost, Academic Search Premier, Business Source Premier, and PubMed/MEDLINE for articles published through November 2009 was conducted using multiple combinations of relevant keywords (e.g., teams, training, cross-training, TeamSTEPPS, crew resource management, etc.). We also examined reference lists from previous reviews.
Inclusion criteria required that studies (1) were published in a peer-reviewed source, (2) described the implementation of a specific team training intervention targeting clinical care providers, (3) reported training evaluation data, and (4) reported an adequate level of detail describing the training intervention and evaluation metrics. For example, our review did not include one study of crew resource management training in air medical teams18 because the study focused on previous team training experience as an individual difference variable, no actual training was implemented, and there was no way to determine the details of the previous team training programs participants had experienced.
Forty-eight studies were identified for inclusion (see Appendix 1). Except where otherwise noted, however, percentages are calculated out of a denominator of 40 studies to account for 8 studies reporting evaluation data on training programs described in previously published sources. In such instances, specific training design information was coded from previously published articles to develop the most comprehensive description of each training intervention possible. For example, evaluation data reported by Blum et al19 referenced an earlier study20 for detailed description of the training program.
Coding and content analysis
We adapted a coding framework documenting 50 pieces of information from coding schemes used in previous training meta-analyses1,4 and reviews10 to address five primary areas: (a) study background, (b) training design, (c) training features and components, (d) evaluation and learning outcomes, and (e) guidelines or lessons learned. Two doctoral students with expertise in team training within health care (S.W. and R.L.) independently coded each article. Any differences in coding were resolved through discussion till consensus. We used content analysis21,22 to extract overall themes from the coded content by comparing the frequencies of categorically or qualitatively similar responses. Extracted themes were organized according to 11 central questions (see Table 1 of this report, and Supplemental Table 1, available at http://links.lww.com/ACADMED/A26), which were derived from existing theoretical models of training design, evaluation, and effectiveness and additional factors arising in the process of content analysis.
In addressing each question, we outline our findings from the content analysis and provide a practical example from a reviewed study (In practice).
Training design and implementation
Question 1: Are diagnostic training needs analyses being conducted to guide training development and implementation?
Only eight (20%) reviewed studies indicated that a training needs analysis was conducted in some form. Needs analysis is the critical first step in identifying who, what, and how to train. Furthermore, only eight (20%) specified that training participants were given an opportunity to give input into the training design. Practically, participation in training design is one mechanism for creating staff and physician ownership, a vital component of long-term sustainment and generalization of trained skills.
In practice. Taylor and colleagues23 included participants' input as part of the training process by requiring members to develop a checklist of work steps for patient care. Checklists were used as measurement tools to check for omissions and errors. This involvement created participant ownership in both the definition of core team processes and measurement of these processes.
Question 2: Who is participating in team training?
The health care team training literature is not restricted to one type of provider. Whereas 24 (60%) of the 40 reviewed studies were dedicated to training clinicians specializing in emergency medicine, anesthesiology, surgery, or obstetrics/pediatrics, the included publications described training in at least 16 different areas of specialization. However, descriptions of targeted training sample were sometimes vague and unclear. Additionally, 18 (45%) reported that training sessions were multidisciplinary, with 15 (38%) incorporating team members beyond traditionally targeted physician and nursing professionals, such as technicians, pharmacists, and administrators.
In practice. Haller and colleagues24 implemented a multidisciplinary crisis resource management (CRM)-based training strategy targeting obstetrical teams that comprised nurses, physicians, midwives, and technicians as well as department managers from obstetrics, pediatrics, and anesthesia. Sehgal and colleagues25 specifically asked internal medicine residents, hospitalists, nurses, pharmacists, and other staff on a designated inpatient medical unit participating in multidisciplinary team training to report their reactions to the multidisciplinary training approach. Overall, participants rated the approach highly on a five-point Likert scale (M = 4.59 ± 0.68).
Question 3: Where is team training being held?
Twenty-one (53%) of the reviewed studies stated that training took place on-site at the workplace. However, 15 (38%) did not specify where training sessions were held. In terms of duration, 21 programs (53%) were designed to last less than one day, with the majority running four to six hours.
In practice. Some team training is being conducted without walls at all. Using the technology of virtual worlds and distributed teams, Youngblood and colleagues26 compared learning outcomes for medical students completing team training with a traditional high-fidelity patient simulator with those trained online as part of a four-person team working in a virtual emergency department (ED). Trainees in the virtual ED communicated via headsets and were represented in-game by avatars. Both groups completed six scenarios and participated in facilitated debriefings. Overall, both groups significantly improved their performance during simulated cases posttraining, as well as their self-reported levels of leadership confidence. No significant differences in performance were detected between the two training conditions.
Question 4: What size teams are being trained and how familiar are team members with one another?
Twenty-seven studies (68%) reviewed programs that conducted training sessions using a team-based approach, meaning that trainees actually worked in teams during training sessions. Of these 27 studies, 11 (41%) trained in teams of three to five members and 4 (15%) specified that teams were comprised of only two members. Additionally, 8 (30%) specified training intact teams (i.e., composed of members with an existing level of familiarity), whereas 5 (19%) trained ad hoc teams (i.e., teams formed for training purposes only). Team processes and patterns of interaction evolve over time given opportunities for interaction, especially in complex interaction sequences like patient care. Teams with previous experience working together demonstrate higher levels of performance early on; however, ad hoc teams quickly catch up after several interaction periods.27 This “early advantage” for familiar teams must be accounted for when evaluating team training programs.
In practice. Trainees in an emergency medicine crisis resource management program completed training in teams spanning both professions and clinical disciplines. Specifically, training occurred in teams of five to six that comprised one resident, one first responder, two nurses, and several actors playing paramedics.28
Question 5: Are learning objectives explicitly stated?
During training, clearly stated learning objectives help to focus trainee attention and can influence their motivation and effort. Seventeen (43%) of the reports explicitly stated training objectives. Clear objectives are the mechanisms through which the purpose and scope of training are operationally defined and communicated. Furthermore, they are a necessary foundation for determining which teamwork KSAs to target and for mapping curricula to these identified KSAs.
In practice. In their description of the Teamwork Training for Optimal Patient Safety (TOPS) program, Sehgal and colleagues25 concisely present each element of the TOPS training curriculum matched with specific training objectives.
Question 6: What content are team training programs in health care focusing on?
Twenty-two (55%) studies focused strictly on teamwork competencies, whereas 18 (45%) reported a combined emphasis on teamwork and taskwork (e.g., clinical technical competencies). The most commonly reported teamwork competencies were communication (34, or 85%), situational awareness (22, or 55%), leadership (19, or 48%), and role clarity (18, or 45%).
In practice. Examining changes in the attitudes toward teamwork of a sample of operating room personnel engaged in multiple simulation-based team training scenarios, Paige and colleagues29 focused entirely on teamwork competencies such as open communication, cross-monitoring, and the development of shared mental models. Conversely, Østergaard and colleagues30 integrated teamwork competencies such as leadership and communication into advanced trauma life support training.
Question 7: What instructional methods are team training programs in health care using?
The classic categorization scheme for instructional methods includes information-based methods (e.g., lecture), demonstration-based methods (e.g., behavioral modeling), and practice-based methods (e.g., role-playing, simulation). The majority of reviewed programs reported using a variety of instructional methodologies, with 33 (83%) using both information-based and practice-based methods. Twenty-seven (68%) reported using simulation-based training methods. Simulation-based team training provides opportunities for practice and feedback and can reflect a wide variety of clinical environments, mirroring the stress and time pressures of daily practice, thereby facilitating transfer of new skills into the actual work environment.10 Simulation includes more than high-priced, high-physical-fidelity patient simulators, however. Of the 27 studies that reported using simulation, 9 (33%) incorporated low-fidelity simulations such as role-playing. Although low in physical fidelity, these opportunities can be high in cognitive fidelity; that is, they stimulate trainees to engage in the same cognitive processes necessary when transferring and generalizing new skills into their daily work environment.
Only 14 (35%) of reviewed studies reported incorporating demonstration of targeted KSAs into the curriculum. This suggests that many trainees entered opportunities for practice without having seen actual behavioral models of what desired teamwork behaviors look like or how team processes manifest throughout the duration of a particular care episode.
In practice. In developing a curriculum for anesthesia crisis resource management (ACRM), Gaba and colleagues31 incorporated all three major instructional categories. ACRM incorporates didactic lecture in order to lay a foundational understanding of core CRM skills, videos demonstrating various examples of teams exhibiting these skills, and several simulation scenarios in which trainees practice applying these skills in a full replica operating room using a high-fidelity mannequin.
Question 8: Who is delivering team training in health care?
The majority of programs (24, or 60%) were designed as facilitated, instructor-led learning experiences. However, an additional 3 (8%) explicitly noted that self-paced learning activities were also included (e.g., preread materials). The person(s) facilitating training sessions was specified by 22 (55%) of reviewed studies. Of these, 15 (68%) were conducted by either in-house or consulting medical faculty or personnel, 4 (18%) were conducted by external, nonclinical consultants, and 3 (14%) reports specified that a mix of internal personnel and external consultants were used. None of the studies provided meaningful details regarding how trainers themselves were prepared to train teamwork skills or explicated the skills sets important for trainer effectiveness. It is vital that facilitators receive proper training to effectively deliver team training and conduct effective debriefings. For example, simulation-based training requires novel approaches to instruction, evaluation, and the provision of feedback.32,33 Ensuring that training facilitators are equipped to accurately and effectively implement such novel approaches is a vital component of successful training; however, little evidence-based guidance currently exists regarding “train-the-trainer” activities for team training in health care.
In practice. In their training program designed to teach nontechnical teamwork skills to surgical teams, Flin and colleagues34 included three consultant surgeons (general, orthopedic, pediatric), a consultant anesthesiologist, and two industrial psychologists specializing in safety research. Nielsen and colleagues35 indicated that trainers attend a three-day train-the-trainer session.
Performance measurement and feedback
Question 9: Are trainees receiving diagnostic feedback during training to enhance learning?
Twenty-six (65%) of reviewed studies specified that feedback was provided to participants. Of those, 19 (76%) specified that feedback was provided after the practice scenario in the form of debriefing. Also, the majority (14, or 56%) focused exclusively on process-oriented feedback (e.g., what behavioral processes trainees actually engaged in). As opposed to feedback focused on the outcomes of trainee behaviors, process-oriented feedback describes how and why certain outcomes occurred. Overall, the opportunity for team-oriented self-reflection during debriefing is the critical mechanism for learning and integration of practice experiences. Effective debriefing immediately following each practice opportunity facilitates learning, builds on prior knowledge, helps team members associate feedback with a procedure before it is forgotten over time, and increases members' motivation for improvement.14,36
Effective debriefing is based on diagnostic performance measurement. However, none of the reports described the measures of performance on which feedback was based. Although several included reviews of video recordings of the team's performance, there was no indication that measurement tools were developed to help guide training facilitators in providing objective feedback. Diagnostic measurement tools are important for feedback and training evaluation purposes. They also help debriefing facilitators by providing a means to organize and track specific behaviors occurring during practice. With such tools, facilitators do not have to rely on broad, overarching generalizations of performance.
In practice. Blum and colleagues19,20 incorporated both instructor-based and guided self-correction feedback at the end of each high-fidelity simulation scenario. Behavior-based feedback focused on guiding trainees to engage in reflection and discovery specifically focused on both individual and team information-sharing behaviors. Several studies also integrated videos of team performance, which provide more objective records of team performance and can be very useful for facilitating team self-correction.36
Question 10: How is the impact of training being evaluated?
Evaluation is a vital component of effective team training. Multilevel, focused evaluations capture the impact of training on more than simply trainee reactions—evaluating changes in KSAs, behavior, and in important patient safety indicators, as well as changes in outcomes such as staff perceptions of safety climate and burnout. Twenty-seven (68%) of the 40 reviewed studies reported multilevel evaluation. In total, 24 (60%) collected subjective reactionary evaluations from trainees. The majority of these were collected using Likert scale ratings asking trainees to rate their levels of satisfaction with the training, as well as the usability and viability of the targeted teamwork competencies. Four (10%) reported evaluating changes in declarative knowledge using some type of knowledge test, and 11 (28%) focused on pre–post changes in trainee scores on safety attitude surveys or other affective measures such as self-efficacy. The greatest percentage of studies evaluated changes in trainee behavior, 25 (63%). However, the validity of the methods used to evaluate behavior change ranged widely—from self-report measures to actual observations of team behavior. Twelve (30%) attempted to evaluate outcome-level metrics. Many of these studies integrated simulation scenarios and included metrics such as mannequin survival,36 though several gathered actual patient outcome data such as the number of patients receiving appropriate antibiotic and DVT prophylaxis prior to surgery, mortality, provider willingness to self-report errors, and burnout.37–39
In practice. One of the most significant hurdles for team training in health care has been demonstrating a statistically significant link with patient safety and quality indicators. Low base rates combined with small sample sizes complicate the ability to empirically demonstrate this link. The adverse event index methodology developed by Mann,39 Neilsen and colleagues35 offers a unique approach that involves the creation of a list of adverse events/outcomes, which are assigned weights by a panel of subject matter experts. These individual metrics can then be tracked over time and combined to form a single score, usually on a scale of 100 to 1,000. This process facilitates the variability necessary to find statistical relationships with team training.
Additionally, it is critical to report reliability estimates for all measures collected in evaluation studies. Robertson and colleagues,40 for example, dedicate a full page of their report to describing the exact measures collected, providing citations for published measures, and reporting reliability estimates for each scale for both pretraining and posttraining administrations. In addition to being the basis of good science, this information, as well as relevant effect size estimates, is vital for valid interpretation of results and for future meta-analytic work.
Question 11: When is the impact of training being evaluated?
Twenty-six (65%) of reviewed reports evaluated team training immediately or less than three months posttraining, 8 (20%) conducted evaluations at three to five months posttraining, and 11 (28%) collected evaluation data six months or more posttraining. Long-term evaluation is a vital component for assessing the long-term sustainment and generalizability of training to novel clinical problems or scenarios.
In practice. Ammentorp and colleagues41 demonstrated a time-series evaluation by collecting evaluation data immediately after the training, three months after training, and once more after six months to demonstrate the impact of team training on the self-efficacy of pediatric physicians and nurses over time. Results indicated that self-efficacy increased 37% after training and that this increase was maintained over time.
Our findings suggest several critical conclusions regarding the current state of practice of health care team training. First, team training is being designed and delivered as a multidisciplinary endeavor across various points in professional education. Quality patient care requires collaborative effort across multiple disciplines. Team training that integrates and simulates this interdependent care context offers opportunities to practice teamwork skills in a realistic setting—thus setting the stage for disintegration of disciplinary and professional silos. Furthermore, team competencies are being integrated across a broad portion of the professional education spectrum; programs have targeted residents, fellows, and faculty. Integrating teamwork competencies even earlier in the undergraduate and early graduate professional education of clinicians and future health care administrators is a critical component of instantiating a team approach to care.
Second, health care organizations are investing in team training, providing space and resources. Many facilities are making the effort to bring team training to their providers in the workplace, making participation easier. Providing space and time for team training is an overt demonstration of administrative support for the team approach to care. Management support, incentives, and opportunities to practice are vital mechanisms underlying training transfer and sustainment.
Third, most programs are being modeled on CRM principles, targeting teamwork competencies such as communication, leadership, and situational awareness. Although these tenets provide a framework for training content, team training is more than just CRM. By including a family of instructional strategies aimed at improving team-based competencies, team training can vary in focus (e.g., focused on team leader) and delivery (e.g., information-based and/or practice-based methods). Additionally, to promote the science of team training in health care, future research reports and publications on this topic should detail the specific features (e.g., content, practice, feedback) of the programs examined.
Fourth, most programs are incorporating opportunities to practice these competencies using simulation-based training methods. Trainees are being actively engaged in team-based activities intermixing information, demonstration, and practice-based approaches. Although simulation can be prohibitive in terms of cost, the existing literature highlights that cognitive fidelity, the degree to which the team training program facilitates practice of the actual cognitive processes involved in effective teamwork, is more important than physical fidelity. However, although many programs are conducting training in multidisciplinary teams, the transfer and generalization of targeted competencies may be limited if training focuses only on team-specific competencies. For care environments such as emergency medicine and surgery, where team membership can be fluid within a single care episode, team training must be designed to mirror such conditions, giving trainees the opportunity to practice teamwork skills that are transportable across a variety of team configurations.
The research to date on health care team training demonstrates great variability in the populations examined, training methods used, content targeted, evaluation methods, and other design elements. In all, this variety clearly demonstrates that there is no singular “one best way” to design, implement, or evaluate health care team training. However, programs clearly benefit when they are built on the science of team training and adult learning and are implemented within a supportive organizational environment.
In our look across the health care team training research, several primary areas for development have emerged. First, there is a critical need for researchers, quality improvement leaders, and others to provide a greater degree of detail pertaining to study design and team training processes. As Supplemental Table 1 (available at http://links.lww.com/ACADMED/A26) demonstrates, nearly all reviewed studies failed to specify important content (e.g., training facilitator was not reported in 45% of studies). Without thorough descriptions of training curriculum and specific facilitation methods, the interpretation of study outcomes and replication of results are limited. Davidoff and colleagues42 provide practical guidance for reporting quality improvement initiatives such as team training. Following these recommendations would greatly increase our overall understanding regarding the impact and processes of quality improvement.
Second, more thorough and comprehensive investigations of the links between health care team training and important, measurable outcomes, including measures of patient safety and quality of care, are needed. Despite a push toward a culture of openness and learning within health care, hesitation to report patient outcome data remains. Sound science, however, requires evidence. Therefore, it is imperative that such measures be reported in future investigations. Promisingly, many studies included in this review used multilevel evaluations, going beyond traditional reactionary measures. Innovative mechanisms for calculating and reporting measures such as the Adverse Outcome Index35,39 may help provide vital insight into the linkages between team training and patient outcomes.
In summary, we have provided an overview of the reported current state of practice for health care team training. Our findings reveal trends related to the characteristics of such programs and suggest several areas for improving our understanding of the necessities of effective team training and evidence-based applications for health care team training.
This work was supported by funding from the Department of Defense (Award Number W81XWH-05-1-0372).
All opinions expressed in this paper are those of the authors and do not necessarily reflect the official opinion or position of the University of Central Florida, the University of Miami, Ryder Trauma Center, TRICARE Management, or the Department of Defense.
1Salas E, DiazGranados D, Klein C, et al. Does team training improve performance? A meta-analysis. Hum Factors. 2008;50:903–933.
2Klein C, DiazGranados D, Salas E, et al. Does team building work? Small Group Res. 2009;40:181–222.
3O'Connor P, Campbell J, Newon J, et al. Crew resource management training: A meta-analysis and some critical needs. Int J Aviat Psychol. 2008;18:353–368.
4Salas E, Nichols DR, Driskell JE. Testing three team training strategies in intact teams: A meta-analysis. Small Group Res. 2007;38:471–488.
6National Transportation Safety Board. Table 5. Accidents, fatalities, and rates, 1988 through 2007, for U.S. air carriers operating under 14 CFR 121, scheduled and nonscheduled service (airlines). Available at: http://www.ntsb.gov/aviation/Table5.htm
. Accessed July 13, 2010.
7Buljac-Samardzic M, Dekker-van Doorn CM, van Wijngaarden JD, van Wijk KP. Interventions to improve team effectiveness: A systematic review. Health Policy. 2010;94:183–195.
8Zeltser MV, Nash DB. Approaching the evidence basis for aviation-derived teamwork training in medicine. Am J Med Qual. 2010;25:13–23.
9Sorbero ME, Farley DO, Mattke S, Lovejoy S. Outcome measures for effective teamwork in inpatient care (RAND technical report TR-462-AHRQ). Arlington, Va: RAND Corporation; March 25, 2008.
10Issenberg SB, McGaghie WC, Petrusa ER, Gordon DL, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review. Med Teach. 2005;27:10–28.
11Goldstein IL, Ford JK. Training in Organizations. 4th ed. Belmont, Calif: Wadsworth, 2002.
12Alvarez K, Salas E, Garafano CM. An integrated model of training evaluation and effectiveness. Hum Resour Dev Rev. 2004;3:385–416.
13Lemieux-Charles L, McGuire WL. What do we know about health care team effectiveness? A review of the literature. Med Care Res Rev. 2006;63:263–300.
14Swezey RW, Meltzer AL, Salas E. Some issues involved in motivating teams. In: O'Neil HF Jr, Drillings M, eds. Motivation: Theory and Research. Hillsdale, NJ: Lawrence Erlbaum Associates; 1994:141–169.
15Cannon-Bowers JA, Salas E, Tannenbaum SI, Mathieu JE. Toward theoretically-based principles of training effectiveness: A model and initial empirical investigation. Mil Psychol. 1995;7:141–164.
16Salas E, Dickinson TL, Converse SA, Tannenbaum SI. Toward an understanding of team performance and training. In: Swezey RJ, Salas E, eds. Teams: Their Training and Performance. Norwood, NJ: Ablex; 1992:3–29.
17Cannon-Bowers JA, Tannenbaum SI, Salas E, Volpe CE. Defining team competencies and establishing team training requirements. In: Guzzo R, Salas E, eds. Team Effectiveness and Decision Making in Organizations. San Francisco. Calif: Jossey Bass; 1995:333–380.
18Fisher J, Phillips E, Mather J. Does crew management training work? Air Med J. 2000;19:137–139.
19Blum RH, Raemer DB, Carroll JS, Dufresnes RL, Cooper JB. A method for measuring the effectiveness of simulation-based team training for improving communication skills. Anesth Analg. 2005;100:1375–1380.
20Blum RH, Raemer DB, Carroll JS, Sunder N, Feinstein DM, Cooper JB. Crisis resource management training for an anesthesia faculty: A new approach to continuing education. Med Educ. 2004;38:45–55.
21Neuendorf KA. The Content Analysis Guidebook. Newbury, Calif: Sage; 2002.
22Weber RP. Basic Content Analysis. 2nd ed. Newbury, Calif: Sage; 1990.
23Taylor CR, Hepworth JT, Buerhaus PI, Dittus R, Speroff T. Effect of crew resource management on diabetes care and patient outcomes in an inner-city primary care clinic. Qual Saf Health Care. 2007;16:244–247.
24Haller G, Garnerin P, Morales MA, et al. Effect of crew resource management training in a multidisciplinary obstetrical setting. Int J Qual Health Care. 2008;20:254–263.
25Sehgal NL, Fox M, Vidyarthi AR, et al. A multidisciplinary teamwork training program: The Triad for Optimal Patient Safety (TOPS) experience. J Gen Intern Med. 2008;23:2053–2057.
26Youngblood P, Harter PM, Srivastava S, Moffett S, Heinrichs WL, Dev P. Design, development, and evaluation of an online virtual emergency department for training trauma teams. Simul Healthc. 2008;3:154–160.
27Harrison DA, Mohammed S, McGrath JE, et al. Time matters in team performance: Effects of membership familiarity, entrainment, and task discontinuity on speed and quality. Pers Psychol. 2003;56:633–669.
28Reznek M, Smith-Coggins R, Howard S, et al. Emergency medicine crisis resource management (EMCRM): A pilot study of a simulation-based crisis management course for emergency medicine. Acad Emerg Med. 2003;10:386–389.
29Paige JT, Kozmenko V, Yang T, et al. Attitudinal changes resulting from repetitive training of operating room personnel using of high-fidelity simulation at the point of care. Am Surg. 2009;75:584–590.
30Østergaard HT, Østergaard D, Lippert A. Implementation of team training in medical education in Denmark. Qual Saf Health Care. 2004;13(suppl 1):i91–i95.
31Gaba DM, Howard SK, Fish KJ, Smith BE, Sowb YA. Simulation based training in anesthesia crisis resource management (ACRM): A decade of experience. Simul Gaming. 2001;32:175–193.
32Baker DP, Gustafson S, Beaubien M, Salas E, Barach P. Medical team training programs in healthcare. In: Henriksen K, Battles JB, Marks ES, Lewin DI, eds. Advances in Patient Safety: From Research to Implementation. Vol 4. Rockville, Md: AHRQ: February 2005. AHRQ publication no. 05-0021-2.
33Shapiro MJ, Morey JC, Small SD, et al. Simulation based teamwork training for emergency department staff: Does it improve clinical team performance when added to an existing didactic teamwork curriculum? Qual Saf Health Care. 2004;13:417–421.
34Flin R, Yule S, Paterson-Brown S, Maran N, Rowley D, Youngson G. Teaching surgeons about non-technical skills. Surgeon. 2007;5:86–89.
35Nielsen PE, Goldman MB, Mann S, et al. Effects of teamwork training on adverse outcomes and process of care in labor and delivery. Obstet Gynecol. 2007;109:48–55.
36DeVita MA, Schaefer J, Lutz J, Wang H, Dongilli T. Improving medical emergency team (MET) performance using a novel curriculum and a computerized human patient simulator. Qual Saf Health Care. 2005;14:326–331.
37Sax HC, Browne P, Mayewski RJ, et al. Can aviation-based team training elicit sustainable behavioral change? Arch Surg. 2009;144:1133–1137.
38Le Blanc PM, Hox JJ, Schaufeli WB, Taris TW, Peeters MC. Take care! The evaluation of a team-based burnout intervention program for oncology care providers. J Appl Psychol. 2007;92:213–227.
39Mann S, Marcus R, Sachs B. Lessons from the cockpit: How team training can reduce errors on L&D. Contemp Ob Gyn. 2006;51:34–45.
40Robertson B, Schumacher L, Gosman G, Kanfer R, Kelley M, DeVita M. Simulation-based crisis team training for multidisciplinary obstetric providers. Simul Healthc. 2009;4:77–83.
41Ammentorp J, Sabroe S, Kofoed PE, Mainz J. The effect of training in communication skills on medical doctors' and nurses' self-efficacy. A randomized controlled trial. Patient Educ Couns. 2007;66:270–277.
42Davidoff F, Batalden P, Stevens D, Ogrinc G, Mooney S. Publication guidelines for quality improvement in health care: Evolution of the SQUIRE project. Qual Saf Health Care. 2008;17(suppl II):i3–i9.
Appendix 1, continue...Image Tools