Actors (or confederates) play an important role in the conduct of many simulations, serving in a variety of roles and filling specific needs. When these confederates function within a research program, it is imperative that these individuals adhere to the specific role assigned in a way that differs from confederate roles in other contexts. There are actions that the confederates can and cannot do; variation from the performance standard can result in a confounding effect on research outcomes. Reliable and effective confederate training in the research context is necessary and is an important part of the research design.
As we began to plan for the large, multicenter CPR CARES (Improving Cardiopulmonary Resuscitation with a CPR Feedback Device and Refresher Simulations) study,1 we recognized a need for formal plan to train confederates, because performance variation would impact the primary outcome (chest compression quality as provided by main study subjects). In this report, we describe our training process, our method of adherence assessment, and suggest next steps regarding confederate training research.
THE CASE FOR CONFEDERATE TRAINING
A typical simulation design has a learner or learners placed in an environment in which they are asked to interact with 1 or more confederates. These confederates play key roles in providing the history, setting the tone (e.g., as an anxious vs. calm parent), distracting providers, introducing errors or conflicts, performing tasks, and providing input and support as required (e.g., as a nurse or allied health care professional). The confederates can be clinical staff or professional actors.2 Performance of confederates, although important in creating an immersive learning environment to address certain learning objectives, does not always require a high level of case-to-case consistency for educational purposes.3
Of note, we are aware of the ongoing discussion in the literature regarding the use of the term “confederate” regarding the specific role of an individual participating in a simulated event to assist participants and have knowledge of the case. In this article, we are following the well-reasoned case provided by Nestel et al4 in favor of the use of this term in this context.
To inform our training approach, we reviewed the literature on the body of work regarding confederate and standardized patient training searching PubMed (search date April 2015 using terms “confederate,” “standardized patient,” “training program,” “quality assurance,” “QA,” “assessment,” and “quality assessment”). We identified reports regarding finding, training, and using confederates for simulations to build realism5 and to enhance learning,4,6 but we found no empiric research that pertains directly to training confederates for simulation-based research. A recent review of simulation-based research discusses several possible strategies for training confederates, including the use of scenario scripts with detailed descriptions of confederate roles and the use of confederate cue cards.7 We note the clear comparisons to be drawn to the quality assurance steps taken with standardized patients when high-stakes outcomes are involved, specifically rigorous training and assessment of performance.8–10
Because confederate performance consistency was considered in a high priority for data integrity across this multicenter study, we developed a multifaceted method to train confederates for 2 different roles within the simulation research scenario.
CONFEDERATE TRAINING APPROACH
Our training program was part of a large multicenter trial, the CPR CARES study, conducted at 10 hospitals, between July 2012 and April 2014. A total of 41 confederates were trained and participated in at least 1 case. The main study designed called for 10 recruitment sites to conduct a 12-minute standardized pediatric cardiopulmonary arrest scenario in which a team leader and 2 team members resuscitated a simulated patient in pulseless arrest. The 2 team members performed CPR, whereas the following 2 confederates assisted: a respiratory therapist (RT) bagged the intubated patient and the nurse (RN) provided medication support. Neither confederate could assist with CPR. Details of the protocol are published separately.1 Each study site was required to recruit confederates at their site; to reduce recruitment barriers, we did not constrain the number of confederates to be used. To fill the role of a confederate, one had to be a physician, nurse, RT, simulation technician, and/or paramedic with previous experience in simulation-based education and/or research. Individuals were not permitted to be both a participant in the CPR CARES study and a confederate.
Given the distributed main study design and the need for consistency across sites, the initial training methods were implemented via an internet-based research platform11 with additional hardcopy documents to support this instruction. A 6-component process was developed for training, shown in Table 1. The 2 key online components included a slide set that described the CPR CARES study objectives, design, and outcomes, along with a detailed description of the confederate roles and video of the CPR scenario in which desired confederate behaviors were modeled.12 A script describing the scenario progression and confederate roles during each phase of the scenario was also provided. Pocket cue cards, designed to serve as a cognitive aid for confederates during the simulation scenario, were created with a description of desired confederate behaviors during each phase. See Figure 1 for an illustration of the RN version content.
All confederates were trained for both roles and were required to complete training at least once before participating in a main study scenario as a confederate. We encouraged confederates to review these materials immediately before their sessions. To ensure confederates were prepared to participate, all confederates participated in pilot testing of the simulated cardiopulmonary arrest scenario. We required confederates to demonstrate 100% adherence with either the RN or RT role before serving in the study. A postpilot debriefing allowed confederates to ask clarifying questions. Repeat pilot sessions were conducted as necessary until all confederates demonstrated 100% adherence with the confederate role(s).
ASSESSING THE TRAINING
To provide feedback and assess the impact of our training, we collected adherence data using a 7-item checklist scored by 1 member of the research staff at each center in real time (Fig. 2). The checklist was pilot tested (in 2 separate scenarios) at 1 site before implementation, resulting in 100% agreement in checklist scores between the 2 raters (for both pilot sessions). All confederates were surveyed regarding attitudes about the training process, using a 5-point Likert scale, at completion of the last of their sessions.
We identified a number of important findings as part of this process, both in terms of specific performance data and lessons learned for future application.
Adherence was excellent following this process. Eighty-five percent of all confederates scored perfectly on the checklist, and the median (interquartile range) score was 7 (7–7). There was no difference in performance by confederate type. In terms of study design, a control arm was not implemented because we felt strongly that untrained confederates might have performed in a manner that would confound the main study outcomes. This decision was made a priori with the recognition that this would limit inferences we could make regarding the confederate training process.
Our confederates reported a high degree of satisfaction with the process. When surveyed, confederates reported agreement (≥4 on a 5-point scale) with statements regarding their preparedness after training, the amount of time allocated and the value of the video and pilot session.
There was significant variation in how different centers recruit, both in number of total confederates (n = 2–7) and cases per confederate (n = 1–14). We did not restrict recruitment to reduce the burden on sites, because we felt that the local staff would know their resources best. It could be argued that a more restrictive process that would require confederates to commit to a large number of cases and thus reducing the need for more confederates per site would be desirable. Interestingly, the most active site, which conducted 18 cases, did so with the same 2 confederates. Another unanticipated effect of unrestricted confederate enrollment was role switching—confederates who filled different roles at different sessions. This result was likely due to our cross-training design in our curriculum; this was built-in to help ensure enough confederates. In our program, these role-switching confederates did not perform differently in terms of adherence versus those who only played 1 role. One could argue restricting roles would remove a potential confounder.
The multimodal approach was important to reach learners in a consistent manner when conducting larger-scale interventions. The Web-based content ensured that all confederates received content that was not filtered through the various sites, which could have introduced unintended variation. The process of creating the video scenario also served as a forcing function, requiring the team to have a very clear concept of the confederate role within the scenario flow.
Curricular design for training should use recommended design approaches and best practices. The use of expert modeling12–14 in the video development and providing the script and cue cards (cognitive aids)15–18 were important design decisions that were based on sound theoretical and published support.
Future training studies should use a design that places intervention effectiveness ahead of other considerations, rather than a secondary outcome in a larger investigation. Our model was complicated by confederates being nested within centers and requiring a repeated measures design (confederates serving repeatedly). These features led to the need for a more sophisticated statistical approach. When further impacted by limited variation due to high levels of adherence, statistical inference was challenging. All of these issues arise from this outcome being secondary to other outcomes. A formal study specifically designed to address the question of confederate training will not be limited in the same way.
SUMMARY AND APPLICATION
In this work, we describe our process for developing a confederate training program designed to promote a high degree of role adherence in the context of a simulation-based research study. We were motivated by our concern that poor confederate adherence to the predefined roles would have created a significant threat to the internal validity of the main study. We believe that our process illustrates a path to developing a functional training approach to be used in research programs.
In Table 1, we have provided a content map for our confederate training. The mix of modalities and content will vary from project to project. From the outset, it is important to decide which methods will address which needs, in what order, and to what extent message repetition will be required.
More generally, the process of confederate training falls into the larger context of simulation-based investigation in which researchers seek to make a validity argument in support of the data derived during their investigation.19 Depending on the study design, rigorous confederate, rater, standardized patient training, instrument and case development can be used to support a variety of validity evidence streams (eg, content, response process). Planning that includes methods to obtain validity data, as was done for confederate training in our design, should be part of planning from the outset.
Beyond the research context, unintended variation in confederate performance within assessment exercises is a potential (and perhaps underappreciated) threat to the inferences we make about our learners. Perhaps, a confederate might consistently “help out” lower performing subjects, impacting our ability to rate learners accurately. Our model of confederate training could be considered in the assessment context as well.
Because the use of confederates is often a key component to the design of simulation-based studies, we encourage the systematic reporting of confederate training and performance in publications. Recently published reporting guidelines20 for simulation-based research recommend a thorough description of confederates used in research, including the “experience (e.g., clinical, education), training (e.g., fellowship, courses), profession, and sex,” and a description of “various roles, including training, scripting, orientation, and compliance with roles.” Standardized reporting of confederate roles and training will assist reviewers and editors during the manuscript review process and also assist others to replicate the intervention described in the simulation study.
Clearly, further investigation into confederate training methods is warranted. The highest yield approach, a formal randomized controlled trial, will be challenging to conduct if nested in another study because of the intrinsic threat such a design creates on primary outcomes. The cost and effort of a trial sufficiently powered to detect important difference as an independent study may be difficult to justify. Other, more approachable questions can be considered:
- Does previous experience as a confederate impact performance in the research setting? Would experience improve adherence or potentially promote helping behaviors that could have been learned in nonresearch settings?
- What do “primary” study participants notice regarding confederate behaviors?
- How can we identify potential unwanted confederate behaviors from methods other than creating our own list of do's and dont's, which may miss important other factors?
- Is adherence better identified from multiple sources (participants, observers) rather than a single source?
- Is real-time versus video review the better method for assessment of confederate adherence to roles?
We describe a process to develop well-trained confederates, supported with a multifaceted educational program, that led to high levels of adherence to a research protocol. This process can serve as stepping off point for further investigations regarding confederate training in research programs.
The authors thank and acknowledge the contributions of the following individuals, who assisted with data collection or served as confederates for the study: Traci Robinson, RN, Elaine Gilfoyle, MD, MEd, Deanna Koot, RN, MN, Wendy Bissett, RN, Helen Catena, RN, Kevin Murray, KidSIM-ASPIRE Simulation Research Program, Alberta Children's Hospital; Amber Q. Youngblood, BSN, RN, J. Lynn Zinkan, RN, MPH, Children's of Alabama; Keven Cabrera, Columbia University College of Physicians and Surgeons; Max G. Dannecker, NREMT-I/85, Lifespan Medical Simulation Center; Laura McKenzie, RN, Stollery Children's Hospital; Jean Francois Trudel, RRT, Montreal Children's Hospital; Linda Crelinsten, RN, MA, The Arnold and Blema Steinberg McGill Medical Simulation Centre; Laura Clifford, BSc, Alice Parham, Caroline Box, RGN, RSCN, BSC (hons), Paul Sampson, BSc, MBChB, Tom Jerrom, Darshana Bhattacharjee, Annie Noble RN, BSc, Cert Ed. FHEA, Rob Kozlowski, BSc Hons, Sarah Sibley, Bristol Royal Hospital for Children; Bonnie Mobley, BSN, Ann & Robert H. Lurie Children's Hospital of Chicago; Michele Trinka, MSN, RN, CCRN, PCCN, Dayna Downing, MBA, Roxana Ploski, BS, Kay Martin, BSRC, RRT-NPS, Gail Dutcher, RRT-NPS, AE-C, Michael Williams, BSN, RN, Marcus Brown, BS, NREMT, Children's Medical Center of Dallas. No compensation was received by those cited previously for their contributions to the study.
INSPIRE Group Authors: Dawn Taylor Peterson PhD, Children's of Alabama, [email protected]; Linda L. Brown MD, MSCE, Hasbro Children's Hospital, Alpert Medical School of Brown University, [email protected]; Marjorie Lee White MD, MPPM, Med, Children's of Alabama, [email protected]; John Zhong MD, Children's Medical Center of Dallas, [email protected]; Nicola Robertson BSc, Alberta Children's Hospital, [email protected]; Jordan Duval-Arnould, MPH, CPH, DrPH[c], Johns Hopkins University School of Medicine, [email protected]; Hubert Wong MD, University of British Columbia, [email protected]; Jenny Chatfield, Alberta Children's Hospital, [email protected]; Nnenna Chime, MD, MPH, Albert Einstein College of Medicine and Children's Hospital at Montefiore, [email protected]; Yiqun Lin MD, MHSc, KidSIM-ASPIRE Simulation Research Program, Alberta Children's Hospital, University of Calgary, [email protected]; Quynh Doan MD, PhD, British Columbia Children's Hospital, University of British Columbia, [email protected]; Jonathan P. Duff MD, Stollery Children's Hospital, University of Alberta, [email protected]; Nancy M. Tofil MD, MEd, Children's of Alabama, University of Alabama at Birmingham, [email protected]; Farhan Bhanji MD, MSc (Ed), Montreal Children's Hospital, McGill University, [email protected]; David Grant, MBChB, MRCPCH, Bristol Royal Hospital for Children, University Hospitals Bristol, [email protected], Elizabeth A. Hunt, MD, MPH, PhD, Johns Hopkins University School of Medicine, [email protected]; David Kessler, MD, MSc, Columbia University College of Physicians and Surgeons, [email protected]; Elizabeth Bragg, Johns Hopkins University School of Medicine, [email protected].
1. Cheng A, Brown LL, Duff JP, et al. Improving cardiopulmonary resuscitation with a CPR feedback device and refresher simulations (CPR CARES Study): a randomized clinical trial. JAMA Pediatr
2. Pascucci RC, Weinstock PH, O'Connor BE, et al. Integrating actors into a simulation program: a primer. Simul Healthc
3. Cheng A, Donoghue A, Gilfoyle E, et al. Simulation-based crisis resource management training for pediatric critical care medicine: a review for instructors. Pediatr Crit Care Med
4. Nestel D, Mobley BL, Hunt EA, et al. Confederates in health care simulations: not as simple as it seems. Clin Simul Nurs
5. Black SA, Wolfe JHN, Darzi AW, et al. Simulated anaesthetists in high fidelity simulations for surgical training: feasibility of a training programme for actors. Med Teach
6. Sanko JS, Shekhter I, Kyle RR Jr, et al. Establishing a Convention for Acting in Healthcare Simulation: merging art and science. Simul Healthc
7. Cheng A, Auerbach M, Hunt EA, et al. Designing and conducting simulation-based research. Pediatrics
8. van Zanten M, Boulet JR, McKinley D. Using standardized patients to assess the interpersonal skills of physicians: six years' experience with a high-stakes certification examination. Health Commun
9. Adamo G. Simulated and standardized patients in OSCEs: achievements and challenges 1992–2003. Med Teach
10. Barrows HS. An overview of the uses of standardized patients for teaching and evaluating clinical skills. AAMC. Acad Med
1993;68(6):443–451 discussion 451–3.
11. Cheng A, Nadkarni V, Hunt EA, EXPRESS Investigators. A multifunctional online research portal for facilitation of simulation-based research: a report from the EXPRESS pediatric simulation research collaborative. Simul Healthc
12. LeFlore JL, Anderson M. Alternative educational models for interdisciplinary student teams. Simul Healthc
13. Kardong-Edgren S, Butt A, Macy R, et al. Expert modeling, expert/self-modeling versus lecture: a comparison of learning, retention, and transfer of rescue skills in health professions students. J Nurs Educ
14. LeFlore JL, Anderson M. Comparison of self-directed learning versus instructor-modeled learning during a simulated clinical experience. Simul Healthc
15. Gaba DM. Perioperative cognitive aids in anesthesia: what, who, how, and why bother? Anesth Analg
16. Marshall S. The use of cognitive aids during emergencies in anesthesia: a review of the literature. Anesth Analg
17. Nelson KL, Shilkofski NA, Haggerty JA, et al. The use of cognitive aids during simulated pediatric cardiopulmonary arrests. Simul Healthc
18. Cheng A, Hunt EA, Donoghue A, et al. Examining pediatric resuscitation education using simulation and scripted debriefing: a multicenter randomized trial. JAMA Pediatr
19. Kane MT. Validating the interpretations and uses of test scores. J Educ Meas
20. Cheng A, Kessler D, Mckinnon R, et al. Reporting Guidelines for Health Care Simulation Research: Extensions for the CONSORT and STROBE Statements. Simul Healthc
2016; In Press.