Journal Logo

Empirical Investigations

Use of Emergency Manuals During Actual Critical Events in China: A Multi-Institutional Study

Huang, Jeffrey MD; Wu, Jiayan MD; Dai, Christina BA; Zhang, Xianwei MD; Ju, Hui MD; Chen, Yiqi MD; Zhang, Chunyuan MD; Ye, Fan MD; Tan, Yi MD; Zong, Yongbo MD; Liu, Telong MD

Author Information
Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare: August 2018 - Volume 13 - Issue 4 - p 253-260
doi: 10.1097/SIH.0000000000000303
  • Free


Healthcare providers can respond to crises more efficiently by using operating room (OR) emergency manuals (EMs), books that provide procedures consistent with established medical guidelines in response to a list of critical events.1 Critical events, also referred to as crises, can be lethal in the OR, and the speed in which key actions are performed can determine the outcome of lives.2 Even when OR teams have the correct knowledge, however, this knowledge can stay “inert” when needed3,4 and high-stress situations can result in a higher incidence of omission errors.5,6 A simulation-based trial reported that during a variety of OR scenarios, teams missed 6% of critical actions when crisis checklists were accessible versus 23% of critical actions when they were not.7 For those reasons, EMs are being increasingly adopted and implemented in ORs.1

Studies performed on the clinical implementation or the use of EMs are lacking. A review article identified three studies8–10 that demonstrated evidence of implementation in the actual clinical environment.11 Two of these used surveys to study the use of cognitive aids, such as EM, in clinical settings. In each of these surveys, only half of the respondents reported that they either would use or had used cognitive aids in emergencies, citing barriers to usage as “events in the OR happen too quickly” and “insufficient people to help (eg, nobody available as reader)”.1 Another recent single-institute survey study demonstrated that since the clinical implementation of EMs, 45% of residents reported successful use of EMs during clinical critical events.1 The respondents in the study identified practiced use of EMs during immersive simulation trainings and self-review as positively influencing their subsequent effective EM use.1

Three different anesthesia EMs have been translated into Chinese by Jeffrey Huang, Hui Zhang, and Zhiqiang Liu (Stanford Operating Room Emergency Manuals, Harvard Ariadne Lab Operating Room Crisis Checklists, and Society for Pediatric Anesthesia Pedicrisis Critical Events Cards). On December 25, 2015, these three translated emergency manuals were published in the New Youth Anesthesia Forum, which is the largest anesthesia network in China. Within 6 months of publication, almost 125,000 copies have been downloaded.12

Official Chinese anesthesia organizations endorsed the implementation effort. The two top anesthesia societies encouraged Chinese anesthesiologists to incorporate the use of EMs in the management of critical events with appropriate multidisciplinary training to facilitate their usage.12 Throughout the country, many lectures about why and how to implement EMs were given and multidisciplinary training workshops were organized.

One year after the Chinese versions of EMs were published in the New Youth Anesthesia Forum, we distributed a multi-institutional survey to Chinese anesthesia providers with the goal of assessing how well EMs were being adopted and used clinically and identifying the factors associated with positive EM use.


Consent for this study was obtained from the local hospital authority, Zhongshan Traditional Chinese Medicine Hospital. Consent was obtained from individual hospital offices in charge of research. The translated EMs were made available for free download on December 25, 2015.

A WeChat group for EM was established to provide a means of communication for study participants and to allow the exchange of information on simulation training to enhance collaborative learning. WeChat is a free instant messaging service application for smartphones that was developed by Tencent in China and has more than 600 million users. It has been used by the media, government, hospitals, and enterprises in many fields of information services. Members of the EM group were encouraged to document their group studies or multidisciplinary simulation trainings and upload them onto the WeChat group for others to view.

The criteria to participate in this study were hospitals with anesthesia departments that held at least one group study of EMs and at least one simulation training within the past 6 months. For this study, self-review was defined as studying EM by oneself and group study was defined as reviewing EM with a group of people. Individual communication with each qualifying department was carried out to confirm the accuracy of that information before it was invited to join the study. Information on the total number of anesthesiologists in each participating department and the location of their EM simulation training (whether it took place in a simulation center or an OR) was collected. Training scenarios were designed by each hospital's simulation educators. The use of the EMs was fundamental to the simulator scenarios. To protect the confidentiality of physicians, we did not collect identifying information from respondents. No patient identifiers were collected. The survey questionnaire was adapted and modified from the survey used in the Stanford study1 by the author (J.H.) (see Appendix). It was then distributed to all respondents through WeChat using the WenJianxen software.

To maximize total responses, the survey did not contain any open-ended questions. Survey questions included the five-point Likert scale (strongly disagree to strongly agree), yes or no boxes, and multiple-choice answers. No monetary compensation was provided for any kind of participation in this survey.

Data on the frequency of reported EMs usage during critical events and frequency of self-review/group study and multidisciplinary simulation training were square root transformed. Square root transformed values were used to find the Pearson correlation coefficient to assess the relationships between EMs usage during clinical events and the following types of exposures: self-review or group study and multidisciplinary simulation training. Comparisons between the group that conducted simulation training in a simulation center and group that did simulation training in the OR was done using the χ2 test. Comparisons between the group working in a large practice and group working in a small/medium practice was done using the χ2 test. All tests were two-tailed with a type I error rate of 0.05. For quantitative variables, mean and SD were used.


The WeChat group contained approximately 2000 anesthesiologists, but many did not communicate whether they conducted group studies or simulation trainings and were therefore excluded from the study. We did not find any geographical correlation among departments that did not qualify for the study. Since no data were collected from nonqualifying departments, we could not examine for any patterns in city tier, hospital tier, and governance. Nine hospitals located in China met the inclusion criteria and were invited to participate in the study. These hospitals were located throughout the country, and their setting ranged from small rural towns with 200 beds to heavily populated metropolitan cities with 7000 beds. Hospitals in China are classified according to a three-tier system that ranges from grades 1 to 3 with 3 being the most capable of providing medical care, medical education, and conduct medical research. Seven hospitals in our study were grade 3, and two hospitals were grade 2. One of the hospitals adopted the Pedicrisis Critical Events Cards. The other eight hospitals adopted the Stanford Operating Room Emergency Manuals. The size of the anesthesiology departments in these nine hospitals varied widely and ranged from five to more than 120 anesthesiologists. The survey was sent to each hospital's champion, the person who led the implementation of EM in the department. The champion subsequently distributed the survey to each individual anesthesiologist in his/her department. Physician titles included resident physicians, attending physicians, and chief physicians. Chief physicians have seniority over attending physicians and resident physicians.

A total of 230 participants completed the survey. The demographic breakdown was 40% resident physicians, 43% attending physicians, and 17% chief physicians. The individual response rates from each hospital are shown in Table 1. The overall response rate was 56.4% (230/408).

Hospital Tiers, Response Rate, and Total Department Size

Clinical Uses, Self-review, and Simulation Training

Frequency of group study or self-review, participation in simulation training, and reported EM usage of each hospital are shown in Table 2. More than 70% (35%–100%) of all respondents reported using EM during at least one real critical event within the past 6 months in China. Of total respondents, 87.7% reported participating in self-review or group study of EMs at least once within the past 6 months. Of total respondents, 69% reported participating in multidisciplinary simulation training at least once in the past 6 months. Our analysis showed that EM usage during critical events showed a positive correlation with participation in multidisciplinary simulation training (R2 = 0.896, P = 0.0001, CI = 0.76–0.99) and self-review or group study (R2 = 0.5234, P = 0.0276, CI = 0.114–0.937).

Frequency of Self-review Participation, Simulation Training Participation, and EM Use During Critical Events

Frequency of Clinical Uses

The total number of respondents and EM usage of each provider are shown in Table 3. The average number of times that EM was used within 6 months by each anesthesiologist was twice. This was found by dividing the total number of reported EM usage in the past 6 months by the total number of the survey respondents. The data used to calculate this were taken from the survey question: “Select the number of times you've used the EM during or after a critical event for the past 6 months: 1, 2, 3, 4, ≥5.”

Average Frequency of EM Use Reports for Clinical Critical Events

High Fidelity vs. Low Fidelity

Independent comparisons of simulation training location and department size with EM usage are shown in Table 4. Of the nine hospitals participating in the study, four conducted EM simulation training in their simulation center and five did their EM simulation training in their OR. The location of the simulation training (OR vs. simulation center) did not affect the total number of EM usage during a critical event. In addition, there was no significant difference between the number of people who reported self-reviewing and the number of people who reported doing simulation training.

Comparison of Different Simulation Training Locations, Department Sizes, and Physician Titles on EM Usage

Anesthesia Department Size

The respondents were separated by department size (groups with >100 anesthesiologists and groups with <100 anesthesiologists). There was a significant difference in the use of self-review/group study, simulation training participation, and EM usage during critical events between the two groups with the smaller department having a higher rate of engagement in these activities (Table 4).

Clinical Experience (Physician Title)

Survey data showed no significant difference in the use of self-review/group study, simulation training participation, and EM usage during critical events among the three groups when separated by title (Table 4).

Responses to Likert scale survey questions are shown in Table 5.

Response to Survey Questions


These results of this study implied successful implementation of EM in multiple Chinese institutions and highlighted important factors that may have influenced such outcomes. Data from this survey of multi-institutional anesthesia providers showed that significantly higher levels of EM usage were associated with anesthesiologist participation in simulation training and anesthesiologist participation in EM group studies or self-review.

A four-element framework proposed to facilitate successful implementation of EM may help explain the promising results demonstrated in this study: create, familiarize, use, and integrate.13 Successful creation of EMs requires well-designed and easy-to-use manuals, familiarity with format, and integration into the culture of the institution. The manuals translated into Chinese and used in this study, Stanford Operating Room Emergency Manual and Society for Pediatric Anesthesia Pedicrisis Critical Events Cards, have been proven to be well-designed and easy to use in English-speaking settings.7,13,14 The Chinese translation of these EMs are accurate, precise, and fine-tuned. The organization, format, color coding, and text size were all kept consistent with the original English version. The EMs were made publicly available as free downloadable documents in the largest anesthesia network in China. They could also be printed out and placed in the OR at a very minimal cost.

Familiarization with EM before needing to use it increases the likelihood that it will be used and used more effectively in the future.11 A survey study conducted on a cognitive aid for cardiac arrest management showed that respondents who had learned about the aid from a formal orientation session were more likely to use it in an emergency than those who did not receive orientation to the aid.9 A key step to familiarize Chinese providers with EM was educating them with lectures and giving them opportunities to practice using EM through simulation. We conducted multiple lectures and published articles explaining reasons and ways to use EM to manage critical events. These educational resources were made readily accessible by being uploaded online. In addition, the two top anesthesia societies, Chinese Association of Anesthesiology and Chinese Association of Anesthesiologists, strongly endorsed the implementation of EMs. In a recent survey study, 65% of respondents believed that endorsement from the American Society of Anesthesiologists would help with checklists implementation.15 Therefore, the endorsement of EM usage by two highly influential Chinese anesthesia organizations12 may have played a role in the rapid adaptation to EM by the Chinese anesthesiologists in our study. This combination of education and official support helped us improve awareness, acceptance, and ultimately usage of EM in China: the webinar of the presentation during the September 2015 New Youth Anesthesia forum was viewed by more than 47,000 members.12 Within 6 months of publication, almost 125,000 copies on the translated EM have been downloaded,12 suggesting a broad acceptance of EM among Chinese providers. In addition, most respondents in our study agreed or strongly agreed to the following: “I definitely use EM to guide my actions during critical events” and “I will remind my colleagues to use EM during critical events.”

Aside from initial exposure, frequent review is also needed to maintain proficiency in EM.14 A recent study showed that within 4 months, providers demonstrated a 29% decrease in the mastery of pediatric resuscitation skills.16 Therefore, EM education should be integrated into anesthesiologists' vocational and continuing medical education so that they continuously understand when and how to use EMs appropriately. Review of EM every 4 months should be recommended.

Our data showed that self-review/group study of EM and multidisciplinary simulation training within the past 6 months showed a clearly positive correlation with EM usage during a critical event compared with the absence of review or simulation training. Self-review and group studying helps anesthesiologists identify how and when EMs should be used while simulations help build their confidence and comfort with using EM. Simulation-based medical education is recommended over traditional education for teaching technical skills in medicine because it is shown to be the superior learning method.17,18 One study investigating how simulation-based educational intervention affects emergent resuscitations showed a significant postsimulation improvement in written test and team performance.19 The strength of simulation-based education lies in the opportunity for sustained, goal-oriented, and deliberate practice for the acquisition of knowledge, skills, and attitudes without any risks of harm to patients.20,21 Simulation-based training provides an effective and immersive method of team-based training to integrate EM use with other crisis resource management skills.22 Operating room team members can build better teamwork skills and become familiarized with why, when, and how to use EMs, which will increase the chance of them using it effectively. In addition, most of our survey respondents agreed or strongly agreed with: “I believe that OR emergency checklist simulation training can improve clinical outcome” and analysis of the data showed a positive relationship between EM usage during critical events and receiving simulation training. These results are consistent with the conclusions of Goldhaber-Fiebert et al1 and support the existing literature that advocates simulation training as a powerful tool to improve clinical EM implementation and usage and ultimately enhance the quality of patient care being delivered.

Simulation is a technique, not one specific technology, and 11 dimensions of different applications should be considered when planning a simulation.23 When looking into the effects of training location on simulation training success, our data revealed that both simulation training in a high-fidelity simulation center and simulation training in the OR led trainees to meet similar learning objectives. Emergency manual usage during critical events did not differ significantly between those two groups. Operating room simulation training can focus on establishing effective ways to recognize signs and indications for EM as well as cultivate teamwork and dynamic decision-making skills. These findings allow for more flexibility when setting up future training sessions because having access to a high-fidelity simulation center may not prove necessary. In addition, even hospitals with limited resources can successfully train their anesthesiologists to use EM because simulation training in an OR does not require sophisticated nor expensive equipment.22 Hospitals can save on money and resources that would normally go toward simulation center training and use it to enhance other areas of patient care.

Before EMs were introduced in China, 97.1 % of hospitals in China had implemented World Health Organization Surgical Safety Checklists.24 This suggests that other types of cognitive aids were already implemented and integrated into the Chinese health care system. The pre-existing presence of cognitive aids in a hospital likely makes it easier to introduce additional cognitive aids such as EM.

The sociopolitical climate of China may have also played a role in the successes of EM implementation and usage shown in this study. The Chinese anesthesiology training system is markedly different and less standardized from that of the United States. In China, graduates of the same medical class may develop different professional levels depending on the hospital and region in which they work.25 According to the State Council, in 2015 China planned to establish nationally standardized residency training programs for medical graduates entering medical practice including anesthesiology.25 However, as of now, the standard of anesthesiology training is in the very early stages, and many practicing anesthesiologists have never received standardized training. The use rate and expenditure for health services are much higher in the urban areas of China.26 Larger hospitals generally offer better resources including human resources, education, and training. However, our data showed that self-review, simulation training participation, and reported EM use during critical events were actually higher in the small/medium practices than the larger practices. This can be attributed to the theory that even though anesthesiologists in smaller practices have fewer training and human resources, they were aware of their own limitations and demonstrated a strong desire to improve their health management skills. They recognized the importance of crisis management and patient safety and rapidly accepted EM as a valuable tool for both training and clinical practice. In addition, anesthesiologists who train in China receive an overall less uniform and less vetted for education than anesthesiologists who received their training in the United States. Therefore, it makes sense that the nine institutions we surveyed more readily and more rapidly adopted EM implementation efforts. One of the most important efforts done was the organization of simulation trainings, which encouraged rapid spread of EM awareness across the nine institutions included in this study.

This study found that the average number of times EM was used during critical events within the past 6 months is twice per anesthesiologist. This is higher than reported in previous literature.1 There are several limiting factors that may need to be considered. The anesthesiology training in China lacks standardization, so the perception of what constitutes a critical event among healthcare providers can vary. If survey respondents used EM during a critical event, the event should have theoretically fallen under the 23 critical events contained in the manual. However, because we did not provide a definition for “critical events” in the survey, the criteria that constitutes a critical event in this study may be more broadly defined than in previous literature.1 There are currently no data on the number of critical events in China. Another limiting factor is that all anesthesiologists working at the nine hospitals were invited to participate in the survey. As a result, some critical events may have included multiple anesthesiologists, which allows for the possibility of overlapping events and duplicate responses.

Because of the nature of this study, the accuracy and honesty of the survey responses cannot be verified. The survey instrument was not validated beyond the questionnaire. Selection-biased reporting may exist in this study because providers who use cognitive aids might be more likely to report events, or they may be more safety-conscious in the first place. Furthermore, the survey was only distributed to approximately 20% of the WeChat group because many group members did not have simulation training and group study or failed to communicate this information. Therefore, it is possible that the departments who qualified were more organized and motivated than the nonqualifying departments, and therefore, it may be more difficult to extrapolate the results of this study to a larger population. Finally, the absence of open-ended questions in the survey is another limitation. To ensure maximal completion of the survey, the questions were kept brief. However, open-ended questions can yield valuable qualitative data that detail experiences in greater depth than would be possible in a quantitative study.27 They could have provided insight into how people felt about the EM implementation process, allowed responders to explain or qualify their answers, and revealed barriers and factors that led to the usage or nonusage of EM such as the necessity for a reader. Including such questions in future survey studies is encouraged to gain insight into different and possibly unexpected factors that affect the EM implementation process. We plan to conduct a qualitative review component to follow up with this study in the future. Although we demonstrated that self-review and simulation training enhanced EM usage during critical events, we cannot provide evidence of improvements in clinical outcomes. The real value of cognitive aids such as EM is determined by whether it ultimately results in improved patient outcomes, which is an area for future research. Despite these limitations, this study is the first investigation on the use of EM in China after a recent implementation effort and identified two actions that positively affected EM usage: self/group study and simulation training. In addition, it provided evidence that simulation training can be successfully conducted outside the setting of a simulation center suggesting that efforts should be made to establish a national standard for EM simulation training. The results of this study can serve as a benchmark for comparison with future EM research.


This study demonstrated the nascent success of EM implementation in multiple Chinese institutions and identifies important factors that influenced this outcome, which can possibly be reproduced nationwide. A key finding in this multi-institutional study is the existence of a positive trend between self/group studying, participation in simulation training, and increased EM usage during a real critical event, highlighting the importance of these two practices. It supports the idea that simulation training is a powerful tool to enhance the implementation and usage of clinical EM and ensure the delivery of patient care in real emergency settings. In addition, simulation training in the OR, which requires lower costs and a simpler setup model than in a high-fidelity simulation center, was successful in helping trainees meet their learning objectives. The results taken from our study suggest that efforts to establish a national standard for EM simulation trainings across the nation can positively facilitate the adoption of EM. Further research will be needed to evaluate clinical outcomes and improvements in patient safety that resulted from the usage of EMs in these institutions.


The authors thank Jonathan Bilby, Albert Xinliang Liu, and Zheng Lin for their assistance with statistical analysis.


1. Goldhaber-Fiebert SN, Pollock J, Howard SK, et al. Emergency manual uses during actual critical events and changes in safety culture from the perspective of anesthesia residents: a pilot study. Anesth Analg 2016;123(3):641–649.
2. Ghaferi AA, Birkmeyer JD, Dimick JB. Variation in hospital mortality associated with inpatient surgery. N Engl J Med 2009;361:1368–1375.
3. Renkl A, Mandl H, Gruber H. Inert knowledge: analyses and remedies. Educ Psychol 1996;31:115–121.
4. Gaba DM. Perioperative cognitive aids in anesthesia: what, who, how, and why bother? Anesth Analg 2013;117:1033–1036.
5. Bourne LE Jr, Yaroush RA. Stress and cognition: a cognitive psychological perspective. NASA Technical Report 2003. Available at: Accessed June 16, 2016.
6. Staal MA. Stress, cognition, and human performance: a literature review and conceptual framework. NASA Tech Memo 2004;212824.
7. Arriaga AF, Bader AM, Wong JM, et al. Simulation-based trial of surgical-crisis checklists. N Engl J Med 2013;368:246–253.
8. Neily J, DeRosier JM, Mills PD, et al. Awareness and use of a cognitive aid for anesthesiology. Jt Comm J Qual Patient Saf 2007;33:502–511.
9. Mills PD, DeRosier JM, Neily J, et al. A cognitive aid for cardiac arrest: you can't use it if you don't know about it. Jt Comm J Qual Saf 2004;30:488–496.
10. Picard J, Ward SC, Zumpe R, et al. Guidelines and the adoption of ‘lipid rescue’ therapy for local anaesthetic toxicity. Anaesthesia 2009;64:122–125.
11. Marshall S. The use of cognitive aids during emergencies in anesthesia: a review of the literature. Anesth Analg 2013;117:1162–1171.
12. Huang J. Implementation of emergency manuals in China. APSF Newsletter 2016.
13. Goldhaber-Fiebert SN, Howard SK. Implementing emergency manuals: can cognitive aids help translate best practices for patient care during acute events? Anesth Analg 2013;117(5):1149–1161.
14. Clebone A, Burian BK, Watkins SC, et al.; Members of the Society for Pediatric Anesthesia Quality and Safety Committee The development and implementation of cognitive aids for critical events in pediatric anesthesia: The Society for Pediatric Anesthesia Critical Events Checklists. Anesth Analg 2017;124(3):900–907.
15. Krombach JW, Edwards WA, Marks JD, et al. Checklists and other cognitive aids for emergency and routine anesthesia care-a survey on the perception of anesthesia providers from a large academic US institution. Anesth Pain Med 2015;5(4):e26300.
16. Braun L, Sawyer T, Smith K, et al. Retention of pediatric resuscitation performance after a simulation-based mastery learning session: a multicenter randomized trial. Pediatr Crit Care Med 2015;16:131–138.
17. McGaghie WC, Issenberg SB, Cohen ER, et al. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta analytic comparative review of the evidence. Acad Med 2011;86:706–711.
18. Daniels K, Arafeh J, Clark A, et al. Prospective randomized trial of simulation versus didactic teaching for obstetrical emergencies. Simul Healthc 2010;5:40–45.
19. Hunt EA, Hohenhaus SM, Luo X, et al. Simulation of pediatric trauma stabilization in 35 North Carolina emergency departments: identification of targets for performance improvement. Pediatrics 2006;117(3):641–648.
20. Kneebone R. Simulation in surgical training: educational issues and practical implications. Med Educ 2003;37:267–277.
21. Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 2004;79:S70–581.
22. Goldhaber-Fiebert SN, Lei V, Jackson ML, et al. Simulation based team training: crisis resource management and the use of emergency manuals in the OR. MedEdPORTAL Publications; 2014. Available at: Accessed November 26, 2017.
23. Gaba DM. The future vision of simulation in healthcare. Simul Healthc 2007;2(2):126–135.
24. Zhu B, Wang Y, Huan G, et al. Anesthesia quality and patient safety in China: a survey. Perioper Saf Qual Assur 2017;2:71–76.
25. Zheng J, Zhang B, Yin Y, et al. Comparison of plastic surgery residency training in United States and China. Ann Plast Surg 2015;75:672–678.
26. Shi L. Health care in China: a rural–urban comparison after the socioeconomic reforms. Bull World Health Organ 1993;71(6):723–3.
27. Thomassen O, Brattebø G, Heltne JK, et al. Checklists in the operating room: help or hurdle? A qualitative study on health workers' experiences. BMC Health Serv Res 2010;10:342.
Survey Questions:

Emergency manuals; simulation; self-review; implementation

Copyright © 2018 Society for Simulation in Healthcare