It is very well documented that teamwork and communication issues can adversely affect patient safety and quality.1,2 Researchers have observed that teamwork and communication issues can impact patient care, patient handoffs, and team performance.3 Poor teamwork and communication can impact the climate and culture of an organization,3 whereas physician well-being, nurse and physician burnout, and employee turnaround are all influenced by both climate and culture.4 Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS®) provides clinical teams with the tools to improve teamwork and communication by focusing on 4 behaviors or competencies: Leadership, Situational Monitoring, Mutual Support, and Communication. TeamSTEPPS® provides the clinician with the means to speak up to share timely, thorough information that can mean the difference between good and unfavorable patient outcomes.5
Although many organizations have reported successful outcomes as a result of TeamSTEPPS®,6–9 implementation has its own set of challenges. Some of these obstacles may be associated with or caused by lack of administrative support and ineffective instructional practices.10 A plan for implementation that does not include a plan for sustainment and reinforcement training can also hinder implementation team efforts.11 Organizations attempting to implement TeamSTEPPS® over longer periods might find that the revolving door affiliated with many organizations also prevents them from fully implementing TeamSTEPPS®. The saturation-in-training theory10,12 posits that training the greatest number of people in the shortest period can generate the greatest effect. Our goal was to train at least 90% of the department in less than 3 months so that everyone had the same TeamSTEPPS® knowledge and skills. The purpose of this pretest/post-test design pilot research was to assess the impact of the saturation-in-training model of TeamSTEPPS® implementation in an urban hospital department of pediatric academic setting.
The 4-phase, brain-based simulation framework for TeamSTEPPS®13 was designed to prepare learners to function as high-performing teams. This training is a modification of the original lecture-based course and has been implemented by one of the researchers in hospital systems worldwide. The course employed the latest, evidence-based practices and proven instructional design principles based on the 4-phase brain-based lesson plan for simulation.13,14 The inquire phase of the course assists learners with creating situational interest in the topic. The gather phase includes instruction in all 4 TeamSTEPPS® competencies, including leadership, situational monitoring, mutual support, and communication. Interprofessional teams train together in this course and apply the competencies progressing from written cases, video cases, and finally to simulation-based clinical cases in the process phase. Lastly, the apply phase assists learners with transferring the content to their clinical situations through the use of debriefing.
THEORETICAL FRAMEWORK FOR THIS INTERVENTION
The frameworks that support this intervention include Bandura’s Social Learning Theory15 and the 4-phase, brain-based simulation framework.13,14 Learners construct their knowledge and revise their existing frames of knowledge individually and with the assistance of others in 4 phases that are conducive to learning.
This pretest/posttest design pilot research used a combination of classroom simulation-based instruction and in situ simulation in a Pediatrics department in an urban academic center. The term in situ in this research refers to the natural clinical setting. Institutional research board approval was granted for this education-based intervention. All collected data was deidentified. All department members with direct patient contact responsibilities (n = 547) completed the intervention: 4-phase simulation TeamSTEPPS® training facilitated by Master Trainers using the saturation-in-training principle (Fig. 1).
We used the TeamSTEPPS® course pre- and posttest to evaluate changes in knowledge. This test was developed by 1 of the researchers and used at multiple sites worldwide. The course examination consisted of 20 multiple-choice questions assessing understanding of the 4 TeamSTEPPS® competencies. Scores were calculated based on the number of correct responses (0–20). The test was administered immediately before and after the TeamSTEPPS® course. Kirkpatrick16 observed 4 levels of educational outcomes: reaction to the experience, change in knowledge, observable change in practice, and change in organizational-level outcomes. The pre/posttest was used to assess whether there was a change in knowledge based on the intervention.
In Situ Team Performance
In situ team performance was evaluated to assess an educational outcome of an observable change in behavior. Two in situ simulations, pre- and postintervention, with 2-day and overnight shift teams, were conducted in the Pediatric inpatient unit using the validated TeamSTEPPS® Team Performance Observation Tool.17,18 This observation tool, available from the AHRQ TeamSTEPPS® program, was used to assess the clinical teams before and after the intervention. The validated TeamSTEPPS® Team Performance Observation Tool17,18 measures five areas: team structure, leadership, situational monitoring, mutual support, and communication. Each of the 25 evaluated criteria are scaled from 0 (done poorly) to 5 points (excellent).
The following research hypotheses and questions were created to learn about the impact of our organization’s TeamSTEPPS® intervention.
Research Hypotheses and Questions.
H1: Those who complete TeamSTEPPS® training will perform a team debrief after clinical events at a higher rate than those who have not completed this training.
H2: Teamwork and communication scores, as measured by the TeamSTEPPS® Team Performance Observation Tool, will be higher for those who have completed TeamSTEPPS® training compared with those who have not completed this training.
RQ1: Do clinical teams debrief following a TeamSTEPPS® intervention?
RQ2: Do clinical teams completing the TeamSTEPPS® training score higher on the TeamSTEPPS® Team Performance Observation Tool?
RQ2: Do team members use situational monitoring and mutual support following a TeamSTEPPS® intervention?
RQ3: Is there a clear leader and are team members assigned to roles and responsibilities following a TeamSTEPPS® intervention?
RQ4: Is there a change in teamwork and communication knowledge after the TeamSTEPPS® training?
Classroom Portion of the Research.
The sample for this research protocol included all personnel with direct patient care responsibilities (n = 547), including nurses, physicians, pharmacists, respiratory therapists, unit clerks, and environmental workers in the department of pediatric medicine. These numbers do not reflect the 17 TeamSTEPPS® Master Trainers who received training and assisted with the instruction for the 547 members of the department.
In Situ Team Performance Research
For the team performance assessment, we activated a mock code on the clinical floor, and the responding teams were briefed and consented to the research. A total of 4 teams, 2 teams before and after the intervention, were briefed that they would participate in in situ simulations that included a mock code. They were told that team performance would be assessed using the TeamSTEPPS® Team Performance Observation Tool and that the information would be deidentified. The participants were told that they could opt out of the research at any time. Teams included a total of 20 participants: resident physicians (n = 8), nurses (n = 11), and one respiratory therapist (n = 1). Due to shift scheduling, patient coverage concerns, and high patient census, teams often did not include the same members.
Using the saturation-in-training model,10,12 the entire department received training by Master Trainers in 27 courses facilitated over a period of eight weeks. Before implementing the 27 courses, we conducted 2 in situ simulations with two teams from the day and overnight shifts in the Pediatric inpatient unit. Two additional in situ sessions were conducted two months following the last TeamSTEPPS® session. We measured the performance of teams from both the day and overnight shifts because researchers19–21 have noted differences in performance in some clinical behaviors between the two shifts. This observation caused us to include a team from each shift, pre and postintervention so we could also assess if team performance would differ. Because the intervention was approximately 8 weeks long, there was nearly four months between the pre- and postintervention team simulations. The same pediatric simulator and cases were used (asystolic cardiac arrest) to ensure consistent conditions for this research. The same two master trainers assessed team performance using the TeamSTEPPS® Team Performance Observation Tool. There were no differences in the scoring results between the two raters. Scores for all areas were positioned on the same end of the 5-point scale. The assessors addressed minor differences through discussion.
Course Knowledge Scores were compared from pretraining to posttraining by Wilcoxon rank-sum tests. We described the team performance scores for the control and postintervention groups (day/night). Analyses were 2-sided with statistical significance evaluated at the 0.05 alpha level. Statisticians performed the analyses in R version 3.4.1 (Vienna, Austria).22
The first hypothesis is rejected (Those who complete TeamSTEPPS® training will perform a team debrief after clinical events at a higher rate than those who have not completed this training). Under the leadership competency, the second area of the TeamSTEPPS® Team Performance Observation Tool, the leader was assessed on conducting team events such as briefs, huddles, and debriefings. During the course, the facilitators emphasized the need for a debriefing immediately following the case.23 Therefore, the assessors specifically observed to see if a leader debriefed following the case. The assessors noted that while the 2 teams that did not receive the TeamSTEPPS® training did not debrief their teams following the simulation cases, only 1 team conducted a debriefing postintervention.
Our second hypothesis is accepted (Teamwork and communication scores, as measured by the TeamSTEPPS® Team Performance Observation Tool will be higher for those who have completed TeamSTEPPS® training compared with those who have not completed this training). As shown in Table 1, there were improvements in team performance, which we described in the discussion section. However, since team composition was not consistent, and the researchers did not conduct enough simulations, statistical significance could not legitimately be implied.
In the first domain of the Team Performance Observation Tool, Team Structure, the researchers found improvement in the day shift scores (2.5–3.8 points), but the overnight scores decreased from 2.8 to 2.7 points.
There was improvement in the second domain, Leadership, in both the day and overnight shifts (1.6–4.2 points; 2.6–3.6 points).
There were increases in the third area, Situational Monitoring in both the day and overnight shifts (2.3–4 points; 2.0–2.33 points).
Mutual support was one of the largest gains in improvement for both the day and overnight shifts (1.8–4.2 points; 2.3–4.0 points).
Communication scores were also improved. This last domain of the Team Performance Observation Tool resulted in increases in scores in both the day and overnight shifts (1.8–5.0; 2.8–3.0 points).
Overall points on the Team Performance Observation Tool improved more on the day shift (10–21.2 points) than on the overnight shift (12.5–15.6 points).
For the research question, do team members use situational monitoring and mutual support following a TeamSTEPPS® intervention, the answer is yes. Specific examples and the higher scores from those sections of the observation tool are described in the discussion section.
For the research question, is there a clear leader and are team members assigned to roles and responsibilities following a TeamSTEPPS® intervention, the researchers did not record any differences in team performance in these two leadership areas.
Our last research question asked if there would be a change in teamwork and communication knowledge following the training. As shown in Table 2 and Figure 2, mean and median postcourse Knowledge Scores were higher than the precourse scores. The minimum test score was higher postcourse than precourse (10 versus 0), and the median score was significantly higher postcourse compared with precourse (19 versus 16; P < 0.001)
DISCUSSION AND FINDINGS
The purpose of this research was to assess the impact of the saturation-in-training model to the 4-phase, brain-based simulation framework of TeamSTEPPS® in an urban department of pediatric medicine. More specifically, we were interested in changes in knowledge, teamwork, and communication as a result of this intervention. We noted improvements in in-situ team performance with statistically significant improvement in TeamSTEPPS® knowledge. Concerning the first research question, do clinical teams debrief following a TeamSTEPPS® intervention, the answer is no. This finding was discouraging because while the teams did it well following the simulation cases that were included in the TeamSTEPPS® course, only the postintervention day shift team debriefed following the in-situ simulation. That is, we hypothesized that teams would debrief at a higher rate and they did, but not consistently, or significantly. This finding may suggest that during the approximate 2 months following the intervention, these behaviors were not reinforced. The TeamSTEPPS® program includes a plan for initial training and reinforcement/sustainment training.11 The saturation-in-training model equipped the teams with the TeamSTEPPS® knowledge and team behaviors. However, if not reinforced by leaders, educators, and administrators, the gains can dwindle over time.10,11 Implementation is an iterative process, and direct-observation sessions and remedial or concurrent training plans are important for maintaining or strengthening TeamSTEPPS® behaviors following initial training.11
The researchers hypothesized that teamwork and communication scores, as measured by the TeamSTEPPS® Team Performance Observation Tool will be higher for those who have completed TeamSTEPPS® training compared with those who have not completed this training. The researchers did see improvement in most areas of the observation tool and vast improvements in overall performance. However, scores did not reach statistical significance because of a few factors. First, team composition was inconsistent on the 4 teams. When we studied each of the 4 teams, the unit census was high, and it was difficult to take away clinicians to perform in the in-situ simulation. Resident physicians also have to rotate as part of their clinical/academic development, and the researchers did not have a method for controlling team member schedules. Also, we only assessed 2 teams before and after the intervention. Additional teams would need to be assessed to be able to imply that the changes were not due to chance.
For the research question, do team members use situational monitoring and mutual support following a TeamSTEPPS® intervention, the answer is yes. As indicated by the higher scores on the teamwork observation checklist, teams in the postintervention group actively scanned their environment and mutually supported their team members more consistently. Some examples noted by the evaluators included recognizing the need for a step stool to aid in chest compressions or regularly taking over compressions from tiring team members.
The researchers also wanted to know if there would be a clear leader and are team members assigned to roles and responsibilities following a TeamSTEPPS® intervention. The researchers did not see any differences in team performance in these 2 areas. All teams could benefit from reinforcement training tailored toward leaders announcing their leadership role in the team and directly assigning team members. Although there was a clear leader in all cases, their role was not announced to the team. Also, team members often self-assigned roles rather than the leader assigning roles.
Addressing our last research question, is there a change in teamwork and communication knowledge, the answer is yes. There were significant improvements in TeamSTEPPS® knowledge following the TeamSTEPPS® training. Course participants scored significantly higher on the 20-question multiple-choice examination.
Our research results differ from implementation efforts at other institutions. Sheppard et al.24 reported on TeamSTEPPS® implementation at 2 institutions. They reported positive outcomes in 8 of the 10 facilities, including surveys of patient perspective of the teamwork and communication of the caregivers.
At the time of their article, the former North Shore–LIJ Health System in New York comprised 13 hospitals. Their TeamSTEPPS® implementation initiative spread out over a longer period and included outcomes such as the more frequent use of briefings and huddles. Paul et al.25 reported a 5-year voluntary TeamSTEPPS® initiative to integrate teamwork into a large network of reproductive health care organizations in the United States. They found that attendees felt the Master Trainer courses were useful in terms of gains in knowledge, but time restraints and competing initiatives were perceived as barriers to full implementation. The obstacles identified in the background section of this article are profound and very common across health care organizations. Even with evidence for the need for TeamSTEPPS® and what it can accomplish, there remain obstacles that organizations face when implementing the program.10 We attribute the success of our intervention in part to leadership buy-in. Coupled with a curriculum that allowed the participants to practice the 4 competencies using simulation, our implementation team used the saturation-in-training model to train our department in less than 8 weeks, virtually eliminating competing interests and the notion of time constraints.
LIMITATIONS AND SUGGESTIONS FOR FUTURE RESEARCH
Our research has a few limitations. Our research was based on implementation in 1 department. While our research included a large sample size for the knowledge tests, the simulations included actual responders to a code resuscitation and only 4 teams thus impairing or limiting our ability to assess whether there was a change in practice as a result of our intervention. With only 1 team preintervention and 1 team postintervention for both the day and night shifts, we could not perform a formal statistical analyses for comparisons. Since the saturation-in-training model was applied, improvements are likely to be the result of the training. However, we cannot prove causality or make any statistical inferences. Also, the code team responders in the pre/postsimulation groups were not the same in both pre/post, day and overnight shift simulations. A follow-up mixed method observational research protocol is needed to learn where behaviors were strengthened or dissipated over time and the reasons why those changes occurred.
Although TeamSTEPPS® can be an effective means of improving teamwork and communication, implementation can be challenging.10 This pilot study involving 1 department in an urban academic center showed that teamwork knowledge and performance could be improved following the intervention of TeamSTEPPS® that used the 4-phase simulation framework and saturation-in-training models. Studies such as this one are encouraging and may provide a useful implementation model for organizations planning a teamwork intervention. High-performing teams can improve patient safety and reduce medical errors, but teams need to be trained to a high standard. Even with an excellent implementation plan, it is equally important to have a good reinforcement-sustainment strategy. Although our department saw improvements in teamwork and communication in the clinical setting, more might be realized if leaders and educators throughout the department insist on seeing certain behaviors and include them in every education intervention.
The authors are grateful to the Master Trainers who assisted with the instruction and for the leaders, including the Chair, Dr. Gerald M. Loughlin who led by example to make this intervention successful. The authors recognize the editorial efforts by Dr. Richard E. McClead and the reviewers. Their excellent suggestions helped us improve this article immensely.
The authors have no financial interest to declare in relation to the content of this article.
1. Kohn LT, Corrigan JM, Donaldson M.STo Err Is Human: Building a Safer Health System. 1999.Washington, D.C.: National Academies Press;
2. Wachter RMUnderstanding Patient Safety. 2012.New York, N.Y.: McGraw-Hill;
3. The Joint Commission.Behaviors that undermine a culture of safety. Sentinel Event Alert. 2008;40:13. Retrieved Jan 12, 2018, from http://www.jointcommission.org/sentinel_event_alert_issue_40_behaviors_that_undermine_a_culture_of_safety/
4. Hipp DM, Rialon KL, Nevel K, et al.“Back to Bedside”: residents’ and fellows’ perspectives on finding meaning in work. J Grad Med Educ. 2017;9:269273.
5. Clapper TC, Kong MTeamSTEPPS®: the patient safety tool that needs to be implemented. Clin Simul Nurs. 2012;8:e367e373. doi:10.1016/j.ecns.2011.03.002.
6. He JC, Golob JF, Brajcich BC, et al.Benefit of TeamSTEPPS® rounding improvement project on infection-related monitoring. Surg Infect. 2016;17:530534. doi:10.1089/sur.2015.089.
7. Howe EEEmpowering certified nurse’s aides to improve quality of work life through a team communication program. Geriatr Nurs. 2014;35:132136.
8. Lisbon D, Allin D, Cleek C, et al.Improved knowledge, attitudes, and behaviors after implementation of TeamSTEPPS® training in an academic emergency department: a pilot report. Am J Med Qual. 2016;31:8690.
9. Thomas L, Galla CBuilding a culture of safety through team training and engagement. BMJ Qual Saf. 2013;22:425434.
10. Clapper TC, Ng GMWhy your TeamSTEPPS® program may not be working. Clin Simul Nurs. 2013;9:e287e292. doi:10.1016/j.ecns.2012.03.007.
11. Clapper TCNext steps in TeamSTEPPS®: creating a just culture with observation and simulation. Simul Gaming. 2014;45:306317. doi:10.1177/1046878114543638.
12. Clapper TCSaturation in training. Patient Saf InSight. 2013. Retrieved Jan 12, 2018, from https://www.researchgate.net/publication/257881230_Saturation_in_Training
13. Clapper TCSituational interest and instructional design: a guide for simulation facilitators. Simul Gaming. 2014;45:167182. doi:10.1177/1046878113518482.
14. Clapper TCFoisy-Doll CR, Leighton KTheoretical principles to effective simulation. Simulation Champions: Courage, Caring, and Connection. 2018.Philadelphia, Pa.: Wolters Kluwer;
15. Bandura ASocial Learning Theory. 1977.Englewood Cliffs, N.J.: Prentice Hall;
16. Kirkpatrick DLEvaluating Training Programs: The Four Levels. 1994.San Francisco, Calif.: Berrett-Koehler Publishers;
17. Maguire M-BRPsychometric testing of the TeamSTEPPS® 2.0 team performance observation tool. Doctorate of Nursing Science Dissertations. 2016. 2. Retrieved Jan 12, 2018, from http://digitalcommons.kennesaw.edu/dns_etd/2
18. Agency for Healthcare Research and Quality.Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS®), Guide to Action (AHRQ Publication No.06e0020e4). 2006.Rockville, Md.: Agency for Healthcare Research and Quality;
19. Bhanji F, Topjian AA, Nadkarni VM, et al.American Heart Association’s get with the guidelines—resuscitation investigators. Survival rates following pediatric in-hospital cardiac arrests during nights and weekends. JAMA Pediatr. 2016;171:3945. doi: 10.1001/jamapediatrics.2016.2535.
20. Peberdy MA, Ornato JP, Larkin GL, et alNational Registry of Cardiopulmonary Resuscitation Investigators. Survival from in-hospital cardiac arrest during nights and weekends. JAMA. 2008;299:785792.
21. Perman SM, Smith DA, Leary M, et al.Abstract 264: CPR quality is more variable at night than during the day in a multicenter study of in-hospital cardiac arrest. Circulation. 2010;12. Retrieved Jan 12, 2018, from http://circ.ahajournals.org/content/122/Suppl_21/A264.short
22. R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing. 2017. Vienna, Austria: R Core Team; Available at https://www.R-project.org/
23. Clapper TCProposing a new debrief checklist for TeamSTEPPS® to improve documentation and clinical debriefing. Simul Gaming. 2016;47:306317. doi: 10.1177/1046878114543638.
24. Sheppard F, Williams M, Klein VRTeamSTEPPS® and patient safety in healthcare. J Healthc Risk Manag. 2013;32:510.
25. Paul ME, Dodge LE, Intondi E, et al.Integrating TeamSTEPPS® into ambulatory reproductive health care: early successes and lessons learned. J Healthc Risk Manag. 2017;36:2536.