STRATEGIC LEADERSHIP AND MEMBERSHIP
To help build the infrastructure for our collaborative, we sought to include individuals who could provide the leadership and vision necessary to ensure our collective success. The founding members of our collaborative represent leadership from various societies or committees who play a crucial role in organizing and delivering resuscitation and simulation-based education at local, regional, national, and international levels (Table 2). Their earlier experience in helping large networks of individuals move forward with a common agenda in mind is invaluable in helping our collaborative achieve its research goals. This strong leadership foundation, combined with a clear vision and mandate for the collaborative, has allowed our group to move forward collectively without disruption from competing interests and goals.
The composition of a team, and factors such as diversity and previous collaborative experience can affect the success of a team project.42,43 In fact, research has demonstrated that expert teams consisting of individuals with diverse perspectives improve collective understanding and problem solving compared with teams of like-minded individuals.44,45 Building on this theory, other founding members of the collaborative with a broad scope of expertise were strategically recruited to help bridge the gap between research ideas and research productivity. Members with a background in pediatric nursing, neonatology, pediatric emergency medicine, and pediatric intensive care ensure that we have the medical expertise necessary to identify important clinical issues and to design protocols that are medically relevant. Experts in education and human factors provide guidance in the optimal implementation of educational philosophy and evaluation into our research studies. Simulation center directors from our various recruitment sites are critical to help develop feasible simulation scenarios and protocols and to enable video-assisted data collection during simulation sessions. Those with advanced degrees in clinical epidemiology and biostatistics are essential to the study design and data analysis process. Finally, energetic and dedicated young investigators willing to take on the primary investigator and study site principal investigator roles serve as the stimulant for new initiatives and are relied on for their enthusiasm and their commitment to bring projects to fruition.
With such a broad spectrum of individuals coming together to form a research collaborative, we anticipated the potential for competing interests and goals, as well as issues surrounding the time and commitment necessary to achieve our research aims. To help circumvent these potential problems, our leadership implemented a policy of transparency and open communication. The goals, objectives, and funding of the collaborative, as well as the workload expectation and deliverables of each member were clearly outlined through individual communication with leadership, frequent group conference calls, and face-to-face collaborative meetings. By creating an open and transparent environment, members were easily engaged in frank discussion, and any pertinent issues related to research were quickly resolved through group consensus.
Simulation-based research often requires the use of assessment tools to rate subject or team performance via videotaped sessions. The process of collecting and organizing videos, allowing for synchronous or asynchronous video review, scoring and discussion by study investigators, and simultaneously ensuring that all videos are maintained in a secure environment is a challenge that required a novel solution. With this in mind, we developed a universally applicable research website as part of a previously existing Learning Management System to help facilitate simulation-based research. Located on the Internet at www.CESEI.org, this research portal helps researchers to design their project, setup and manage data collection, and finally, allow for upload and review of external data, such as videos, images, and other assets. This research portal allowed us to streamline our video review process, saving us a significant amount of both money and time. For our first project (EXPRESS study), we were able to successfully manage data collected from more than 400 study subjects from 15 different recruitment sites using the research portal. In total, >350 videos were uploaded to the portal for review using three different assessment tools by our team of 24 video review experts from around the world. The research portal enabled seamless extraction of data via the video review process and rapid reporting of these data via immediate download in excel spreadsheet format. This research portal will form the foundation for our future projects, and we believe that its use will enable novice and expert researchers to carry out their projects in an efficient, coordinated, and timely manner, thus helping to advance the field of simulation worldwide.
CURRENT PROJECTS/ASSESSMENT TOOLS
Our first project (EXPRESS study) aims to evaluate whether scripted debriefing by novice instructors after a simulated pediatric resuscitation scenario will improve the knowledge, cognitive performance, and behavioral performance of team members when compared with more traditional, nonscripted debriefing. Our secondary aim is to evaluate the impact of high- versus low-fidelity mannequin simulation for PALS-based educational outcomes, such as knowledge, cognitive performance, and behavioral performance. As such we are using a factorial design to assess the independent and potentially combined effect of these two interventions (script vs. no script and high- vs. low-fidelity mannequin) on the outcomes described. Also, we will test whether novice instructors performing a scripted debriefing rated higher than those facilitating a nonscripted debriefing, as assessed by a debriefing assessment tool. Data collection and analysis for this research is now complete, and results of this research will be published in the near future.
Before implementation in the EXPRESS study, we recognized the need to identify and/or further develop assessment tools, which were proven to be both valid and reliable for a defined population and clinical context. Through several pilot studies and work done before the main EXPRESS study, we aimed to validate four separate assessment tools for the population and clinical context we used in this study: (1) a cognitive performance tool, (2) a behavioral assessment tool, (3) a multiple choice test for PALS-based objectives, and (4) the debriefing assessment for simulation in healthcare tool. We believe that this work will help to solidify the role of these tools as part of future simulation-based research and educational projects. Our current research effort includes eight projects led by diverse teams of investigators; these projects are summarized in Table 3.
RESEARCH PROCESS—PROBLEMS AND CHALLENGES MITIGATED
During the EXPRESS study, our team encountered several issues in study design that were unique to the implementation of multicenter, simulation-based research. To ensure standardization across all 15 recruitment sites, we had to carefully consider the following issues: what type of simulator do these sites have? How can we ensure that the resuscitation scenario is run in the same fashion from site to site? How do we control for environmental factors? How do we ensure that we have similar quality of data from each site, ie, standardized video angles, audio quality, similar length of scenarios? We sought to mitigate these issues through healthy discussion and heated debate.
To address these challenges, we ensured that all recruitment sites had access to the same model infant simulator. Next, the research scenario was designed and standardized to include cues for the research assistant to provide at various predetermined intervals during the scenario (eg, providing the team with capillary refill time). The scenario was carefully controlled by ensuring that progression from one phase to the next was dependent on a standardized time in each phase or by a specific intervention that was executed by the team. The research environment was also controlled—all teams had a standard list of equipment made available to them, and the crash cart was located in the exact same spot and distance relative to the patient bed during the simulated research scenarios. These efforts were critical to reduce variability inherent in simulation environments. Piloting the simulation scenarios before enrolling subjects was an essential step in identifying elements that had not been adequately described in our protocol to ensure standardization. We then edited the protocols in an iterative fashion until they were similarly interpreted and followed across diverse institutions before going live with the study.
Another major challenge was how to optimize the reliability and use of our preselected assessment tools. In tackling this problem, we instituted a number of important measures: standardization of camera angles, multiple camera angles for video review, review by videotape and not in real time (thus allowing our reviewers time to review and pause the scenario to capture proper data and minimize errors), and most importantly, rater training and calibration sessions for each of the assessment tools before implementation in the study.
As with most multicenter research studies, there were a variety of other motivational factors (eg, clinical and academic workload, funding, personal lives) influencing individual investigators' execution of the research protocol that were out of the control of our leadership team. To guarantee compliance with the research protocol, each recruitment site signed a subsite agreement with the coordinating research site (BC Children's Hospital). In addition, two backup recruitment sites were identified a priori, both of which were eventually asked to participate partway through the research project when subject recruitment was going poorly at other sites.
MANUSCRIPT OVERSIGHT COMMITTEE
To ensure transparency and avoid conflict of interest during the writing process, a manuscript oversight committee was formed consisting of three senior members of EXPRESS. The committee was responsible for ensuring that appropriate academic credit was allocated to contributors of the various projects in a consistent, fair, and logical manner. In particular, the committee attempted to prioritize young investigators as contributing authors (A.C., E.A.H., and A.D.) when appropriate. They were also tasked with the responsibility of facilitating identification and management of conflicts of interest, if they were to arise. Finally, this committee assisted in the enforcement of timelines for analysis, presentation, and publication of data. This guided approach helped to ensure consistent productivity from our young investigators and has led to the presentation of numerous research abstracts at various conferences, as well as the preparation and recent submission of several manuscripts for publication (Table 3). Although individual manuscript quality and review will be handled by dedicated writing teams, this committee will provide ongoing oversight by suggesting ideal journals for manuscript submission and ensuring writing teams are on task with their timelines.
FUTURE DIRECTIONS AND GOALS
Going forward, our research collaborative has targeted several areas for new research. These include innovation and technology in improving the quality of pediatric resuscitation, communication and teamwork during resuscitation, and characterizing and improving postresuscitation or postsimulation debriefing. Our overarching goals and objectives as a collaborative over the next 3 years will be to:
- Establish and maintain an efficient, functional, and funded infrastructure to support ongoing research on an international level.
- Identify and leverage new web-based collaboration tools to advance multicenter research.
- Identify additional best practices in researcher networks and adapt EXPRESS accordingly.
- Identify pediatric resuscitation and simulation-based research priorities and use these to guide future projects.
- Develop a structured application process for new membership, capable of efficiently and effectively connecting new members to new or ongoing research projects.
- Expand membership to include collaborators representing countries worldwide.
- Expand membership to include other professions and disciplines (eg, Anesthesiology, Surgery, Respiratory Therapy, and Qualitative Research Expertise), thus broadening the functional diversity of our collaborative and enabling us to stay informed of research outside the pediatric discipline.
- Continue to mentor and support young investigators interested in simulation-based research and education.
- Expand and build on the capabilities of the existing research portal to incorporate qualitative feedback and outcome measures generated by simulators.
- Disseminate knowledge and expertise by conducting courses and workshops at relevant simulation, resuscitation, and pediatric conferences.
To help achieve these goals, our collaborative plans to meet bi-annually through face-to-face meetings in conjunction with two major simulation conferences: International Meeting on Simulation in Healthcare and the International Pediatric Simulation Symposium and Workshops. At these meetings, project leaders will give updates on ongoing research and recent presentations. In addition, new research ideas will be discussed among focused working groups of interested members before grant proposals are prepared and submitted. Ultimately, we anticipate that these bi-annual meetings will grow in popularity and with this may require restructuring of the meeting agenda to accommodate the needs of our collaborative. To ensure long-term sustainability of EXPRESS, we have secured grant funding to build on our existing infrastructure. This funding will be used to enhance the research portal, expand our technological and administrative support, and to further develop other web-based applications; all of which will help to improve the efficiency of communication and workflow. Furthermore, our collaborative will be gathering for a face-to-face research retreat in the near future, aimed at discussing future plans for governance of the collaborative, outlining a strategy to secure both short- and long-term funding, and maintaining a heightened level of enthusiasm and dedication among all EXPRESS members.
The primary mission of the EXPRESS collaborative is to improve the delivery of medical care to critically ill children by answering important research questions pertaining to pediatric resuscitation, education, and simulation. In an era where web-based applications for social networks are burgeoning, the EXPRESS collaborative concept seeks to bring the synergy of professional networks to bear on important clinical problems. Comprising a team of simulation, pediatric resuscitation, education, research, and human factors experts, the EXPRESS collaborative has successfully conducted its first multicenter randomized-controlled trial, assessing the value of scripting and high-fidelity simulation in improving PALS-related educational outcomes. During this process, members of the collaborative have also developed and analyzed four separate assessment tools for use in simulation-based education and implemented an Internet-based, universally applicable and adaptable research portal capable of facilitating simulation-based research. Going forward, the collaborative aims to expand its membership internationally and collectively identify pediatric resuscitation and simulation-based research priorities, and use these to guide future projects.
EXPRESS Pediatric Simulation Research Investigators
Akira Nishisaki, MD; Mike Moyer, BS, MS; Marisa Brett-Fleegler, MD; Monica Kleinman, MD; Matthew Braga, MD; Susanne Kost, MD; Glenn Stryjewski, MD; Steve Min, MD; John Podraza, MD; Joseph Lopreiato, MD; Melinda Fiedor Hamilton, MD; Jonathan Duff, MD; Jeffrey Hopkins, RN; Kimberly Stone, MD, Jennifer Reid, MD, Douglas Leonard, MD; Kathleen Ventre, MD; Laura Corbin, MD; Kristine Boyle, MS; Marino Festa, MBBS; Frank Overly, MD; Stephanie Sudikoff, MD, Takanari Ikeyama, MD, Louis Halamek, MD; Stephen Schexnayder, MD; Jack Boulet, PhD; John Gosbee, MD; Laura Gosbee, MASc; and Matthew Richard, BSc.
The authors thank Anne Marie White, EXPRESS research coordinator, for her hard work and commitment to this project. They also thank the AV/IT team at CESEI for their support—Albert Ho and Ferooz Sekandarpoor for setting up the videotaping and processing hundreds of videos, Byron Tredwell for his innovative design of the research portal, Gary Cody for the beautiful posters, and Karim Qayumi and Marlene Woschee for their continued support of our research.
1. Okuda Y, Bryson EO, DeMaria S Jr, et al. The utility of simulation
in medical education: what is the evidence? Mt Sinai J Med
2. Weinburg ER, Auerbach MA, Shah NB. The use of simulation
training and assessment. Curr Opin Pediatr
3. Cheng A, Duff J, Grant E, et al. Simulation
in paediatrics: an educational revolution. Paediatr Child Health
4. Eppich WJ, Adler MD, McGaghie WC. The use of medical simulation
in the training of acute pediatric
emergencies. Curr Opin Pediatr
5. Gaba DM, Howard SK, Flanagan B, et al. Assessment of clinical performance during simulated crises using both technical and behavorial ratings. Anesthesiology
6. Curran VR, Aziz K, O'Young S, Bessell C. Evaluation of the effect of a computerized training simulator (ANAKIN) on the retention of neonatal resuscitation
skills. Teach Learn Med
7. Gilbart MK, Hutchisin CR, Cusimano MD, Regehr G. A computer-based trauma simulator for teaching trauma management skills. Am J Surg
8. Issenberg SB, McGaghie WC, Hart IR, et al. Simulation
technology for health care professional skills training and assessment. JAMA
9. Marshall RL, Smith JS, Gorman PJ, Krummel TM, Haluck RS, Cooney RN. Use of a human patient simulator in the development of resident trauma management skills. J Trauma
10. Holcomb JB, Dumire RD, Crommett JW, et al. Evaluation of trauma team performance using an advanced human patient simulator for resuscitation
training. J Trauma
11. Kory PD, Eisen LA, Adachi M, Ribaudo VA, Rosenthal ME, Mayo PH. Initial airway management skills of senior residents: simulation
training compared with traditional training. Chest
12. Rosenthal ME, Adachi M, Ribaudo V, Mueck JT, Schneider RF, Mayo PH. Achieving housestaff competence in emergency airway management using scenario based simulation
training: comparison of attending vs housestaff trainers. Chest
13. Hall RE, Plant JR, Bands CJ, Wall AR, Kang J, Hall CA. Human patient simulation
is effective for teaching paramedic student's endotracheal intubation. Acad Emerg Med
14. Eppich WJ, Brannen M, Hunt EA. Team training: implications for emergency and critical care pediatrics. Curr Opin Pediatr
15. Yee B, Naik VN, Joo HS, et al. Nontechnical skills in anesthesia crisis management with repeated exposure to simulation
-based education. Anesthesiology
16. Shapiro MJ, Morey JC, Small SD, et al. Simulation
based teamwork training for emergency department staff: does it improve clinical team performance when added to an existing didactic teamwork curriculum? Qual Saf Health Care
17. Ostergaard HT, Ostergaard D, Lippert A. Implementation of team training in medical education in Denmark. Qual Saf Health Care
18. Bond WF, Lammers RL, Spillane LL, et al. The use of simulation
in emergency medicine: a research
agenda. Acad Emerg Med
19. Fincher RM, White CB, Huang G, et al. Toward hypothesis-driven medical education research
: task force report from the Millenium Conference 2007 on educational research
. Acad Med
20. Boulet JR, Murray D, Kras J, Woodhouse J, McAllister J, Ziv A. Reliability and validity of a simulation
-based acute care skills assessment for medical students and residents. Anesthesiology
21. Devitt JH, Kurrek MM, Cohen MM, Cleave-Hogg D. The validity of performance assessments using simulation
22. Gordon JA, Tancredi DN, Binder WD, Wilkerson WM, Shaffer DW. Assessment of a clinical performance evaluation tool for use in a simulator-based testing environment: a pilot study. Acad Med
23. Malec JF, Torsher LC, Dunn WF, et al. The mayo high performance teamwork scale: reliability and validity for evaluating key crew resource management skills. Simul Healthc
24. Guise JM, Deering SH, Kanki BG, et al. Validation of a tool to measure and promote clinical teamwork. Simul Healthc
25. Vozenilek JA, Gordon JA. Future directions: a simulation
-based continuing medical education network in emergency medicine. Acad Emerg Med
26. Seropian MA, Driggers B, Taylor J, Gubrud-Howe P, Brady G. The Oregon simulation
experience: a statewide simulation
network and alliance. Simul Healthc
27. Gruppen LD. Improving medical education research
. Teach Learn Med
28. Regehr G. Trends in medical education research
. Acad Med
29. Cook DA, Beckman TJ, Bordage G. Quality of reporting of experimental studies in medical education: a systematic review. Med Educ
30. Reed DA, Kern DE, Levine RB, Wright SM. Costs and funding for published medical education research
31. Carline JD. Funding medical education research
: opportunities and issues. Acad Med
32. Reed DA, Cook DA, Beckman TJ, Levine RB, Kern DE, Wright SM. Association between funding and quality of published medical education research
33. Chen FM, Bauchner H, Burstin H. A call for outcomes research
in medical education. Acad Med
34. Whitcomb ME. Using clinical outcomes data to reform medical education. Acad Med
Emergency Care Applied Research
Network. The Pediatric
Emergency Care Applied Research
Network (PECARN): rationale, development, and first steps. Pediatr Emerg Care
36. Marshall JC, Cook DJ; Canadian Critical Care Trails Group. Investigator-led clinical research
consortia: the Canadian Critical Care Trails Group. Crit Care Med
37. Cobb JP, Cairns CB, Bulger E, et al. The United States critical illness and injury trails group: an introduction. J Trauma
38. Nunn T. The National Institute for Health Research
Medicines for Children Research
Network. Paediatr Drugs
39. Macleod ML, Dosman JA, Kulig JC, Medves JM. The development of the Canadian Rural Health Research
Society: creating capacity through connection. Rural Remote Health
40. Seropian M, Dillman D, Farris D. Statewide simulation
systems: the next step for Anesthesiology? Anesthesiol Clin
41. Smith T, Stone N, Bull R, et al.; Rural Interprofessional Education Network (RIPEN). Australian Rural Health Education Network's position on interprofessional education and practice in health care. Rural Remote Health
42. Guimera R, Uzzi B, Sprio J, et al. Team assembly mechanisms determine collaboration
network structure and team performance. Science
43. Joshi A, Roh H. The role of context in work team diversity research
: a meta-analytic review. Acad Manage J
44. Hong L, Page SE. Groups of diverse problem solvers can outperform groups of high-ability problem solvers. Proc Natl Acad Sci USA
45. Jehn KA, Northcraft GB, Neale MA. Why some differences make a difference: a field study of diversity, conflict and performance in workgroups. Adm Sci Q
Keywords:© 2011 Society for Simulation in Healthcare
Simulation; Pediatric; Resuscitation; Research; Collaboration