Secondary Logo

Journal Logo

Validation of Simulated Difficult Bag-Mask Ventilation as a Training and Evaluation Method for First-Year Internal Medicine House Staff

Pastis, Nicholas J. MD, FCCP; Doelken, Peter MD; Vanderbilt, Allison A. EdD; Walker, John BHS; Schaefer, John J. III MD

Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare: February 2013 - Volume 8 - Issue 1 - p 20–24
doi: 10.1097/SIH.0b013e318263341f
Empirical Investigations

Introduction The past decade has witnessed the increased use of patient simulation in medical training as a method to teach complex bedside skills. Although effective bag-mask ventilation (BMV) is a critical part of airway management, the quality of training in this skill has been questioned.

Methods First-year internal medicine house staff (novices) were used to evaluate a computerized patient simulator as a tool to teach difficult BMV. A novice group and an expert group (certified registered nurse anesthetists and anesthesiologists) were tested to validate the simulator’s ability to distinguish between these 2 skill levels.

Results The difference between the novice and expert groups in the ability to perform difficult BMV was statistically significant (P < 0.0001). Brief training for novices led to a 100% pass rate and competence as measured by the simulator. Simulation training was effective in increasing the ability to ventilate a simulated difficult-to-ventilate patient (P < 0.0001).

Conclusions This study suggests that this computerized patient simulator was validated as a simulation model for teaching difficult BMV and differentiating skill levels in BMV. Using the simulator with brief training on difficult BMV allowed new internal medicine house staff to successfully ventilate a simulated difficult patient.

From the Division of Pulmonary, Critical Care, Allergy, and Sleep Medicine, Department of Medicine, and Department of Anesthesia and Perioperative Medicine, Medical University of South Carolina, Charleston, SC; and Center on Health Disparities and the Office of Assessment and Evaluation Studies, School of Medicine, Virginia Commonwealth University, Richmond, VA.

Reprints: Nicholas J. Pastis, MD, FCCP, Division of Pulmonary, Critical Care, Allergy, and Sleep Medicine, Department of Medicine, Medical University of South Carolina, 96 Jonathan Lucas St, 812 CSB, Charleston, SC 29425 (e-mail:

Dr Schaefer contributed mentorship, guidance concerning experimental design, and manuscript review for this study, but pursuant to the Medical University of South Carolina Conflict of Interest (COI) Policy, he did not participate in data collection, reduction, or analysis related to this study because of his potential COI, which includes simulator patent and copyright royalties. Dr Schaefer receives patent royalties from Laerdal Medical Corporation (SimMan/Baby/3G), and he is a nonmajority owner of SimTunes, which is a commercial outlet for Medical University of South Carolina licensed, copyrightable simulation training products. These amount to less than 25% of Dr Schaefer’s annual income.

The other authors declare no conflict of interest.

Competence in advanced cardiac life support (ACLS) is a requirement of internal medicine residency training. In the 2010 ACLS guidelines, the first priority in a cardiac arrest is effective circulation via chest compressions, and initial rounds of chest compressions should not be interrupted to place a supraglottic or subglottic airway. Therefore, effective bag-mask ventilation (BMV) is paramount. The ACLS guidelines state that all health care providers should be familiar with the use of the bag-mask device.1 Despite the movement in hospitals toward code teams for cardiopulmonary arrests, not all hospitals have such services, and among those that do, response times to respiratory arrests may vary.

Instruction in BMV in a traditional ACLS course is brief, and the training may involve as little as learning the ratio of breaths to cardiopulmonary resuscitation compressions. Resusci Anne is used but offers no opportunity to grade the skills of physicians in difficult BMV. The assumption that basic airway management skills and other resuscitation skills are mastered during ACLS training has been questioned.2–8

The most fundamental airway management skill, BMV, is required of all physicians for ACLS certification. In the training of physicians in high-risk low-frequency procedures such as emergency airway management, patient simulation has emerged as a means to supplement traditional procedural training with patients.9–12

Previously, it was shown that a computerized patient simulator (CPS) with scenario-based simulation training combined with one-on-one training was effective in training internal medicine residents in basic airway management.13 Currently, 2 important issues are lacking in simulation training for BMV: a validated “pass/fail” performance measure reflecting clinical practice standards has not been established, and training requirements beyond the most simple of BMV situations have been neither modeled nor incorporated into training of BMV skills. Traditionally, Resusci Anne manikins are used for airway management instruction in ACLS training and are relatively easy to ventilate (require <5 cm H2O to result in adequate inspiration). Evaluation of BMV usually incorporates a subjectively graded checklist and demonstration of easy BMV (rate and flow) for a short period.

In a study of 1502 patients receiving general anesthesia for surgical procedures, difficult BMV was reported in 5%,14 but this incidence has been reported as up to 15% in other studies.15 Difficult BMV may be even more common in hospitalized patients, especially those who are critically ill, who may have a higher incidence of upper or lower respiratory tract pathologic condition contributing to difficult BMV.16 It is vital for any physician who may be a first responder in cases of cardiac arrest to possess BMV skill. This may be lifesaving until a definitive airway is placed. Difficult BMV occurs because of various factors that are related to technique and/or the airway itself. The pathogenesis involves an interaction of these factors culminating in the final clinical scenario. Obesity, age older than 55 years, snoring, lack of teeth, presence of a beard, Mallampati class 3 or 4, and limited mandibular protrusion are all independent predictors.14,17,18

In addition, a difficult airway is more common in patients who experience difficult BMV,14,18 so clinicians should be familiar with corrective measures and management options for difficult BMV15 until personnel arrive with skill in endotracheal intubation. The American Society of Anesthesiologists identifies difficult ventilation as a factor that impacts the strategy used for a difficult airway.19

In this study, a model of increased airway resistance was used to represent difficult BMV. The rationale is as follows. The clinical causation of a patient that is “difficult to mask ventilate” may be multifactorial. Arguably, the biggest challenge, and therefore cause of failure to ventilate when performing BMV, is the critical ability to seal the mask to the face under conditions requiring increased positive pressure (the final common requirement).20 In this training model of difficult BMV, the focus is mask seal training (although there are other educational objectives); therefore, the choice of a model that creates difficulty to BMV and requires good masking skills is one of convenience and ability to standardize.

To further its educational value, the authors sought to validate a CPS ability to differentiate competence in simulated difficult bag-valve-mask ventilation among internal medicine house staff (novices) and an expert group of anesthesiologists and certified registered nurse anesthetists (CRNAs). After the validation process, the researchers conducted an educational intervention using the simulator to teach first-year internal medicine house staff on how to use BMV to ventilate a simulated difficult patient during ACLS training.

This study addresses 2 different research questions: (1) Can first-year house staff (novices) learn to ventilate a simulated patient under difficult BMV conditions during their ACLS training? (2) Can the CPS differentiate between different levels of skill in BMV?

Back to Top | Article Outline


The institutional review board at the Medical University of South Carolina (MUSC) approved this study. Oral and written informed consent was obtained from each subject. Data for this study were obtained from scoring reports that were generated by the software program and printed after each test with the simulator. Data reside at the simulation center on a password-protected computer and were de-identified by an honest broker employed by the simulation center to remove identifiable data elements of protected information (eg, name) and manage data transfer between the simulation and research systems. Results of simulation testing and other nonidentifiable data were stored in separate systems with a research study identification. Data were used exclusively for this research project.

The Basic Emergency Airway Management training course at the MUSC Simulation Center uses a CPS modified by inserting airflow restrictors into the 2 inspiratory pulmonary limbs in a standardized and reproducible manner along with a programmed scenario to support formative and summative feedback. This replicates the clinical scenario of a patient requiring appropriate skill in BMV focused on the mask seal and BMV. By using an oral airway, holding the mask properly, and achieving an adequate seal while maintaining chin lift, an operator should be able to pass a test with this modified device. The restrictors are simple rubber valves calibrated to require 28 (±1) cm H2O pressure to allow airflow through them. They are attached onto each inspiratory limb that also connects to the plastic lungs of the simulator.

Such restrictors, in combination with a grading scenario, although standardized, have not been validated with competent operators to support valid and reliable performance assessment. For the purpose of this study, the difficult BMV scenario is defined as one that a novice will fail without proper face seal adjustments and that most competent individuals and all experts will pass. Without validating such a system, the airway resistance may be too low (easy) or too high (difficult). Anesthesiologists and CRNAs are required to maintain competence in airway management including difficult BMV, so they were used to validate the simulator as a tool for teaching simulated difficult BMV.

Back to Top | Article Outline


This study includes 3 different groups. Group 1 is the novice group and comprised 34 first-year internal medicine house staff, group 2 comprised 16 CRNAs, and group 3 comprised 12 board-certified anesthesiologists—all employed by the MUSC. For the purposes of this validation study, the CRNAs and anesthesiologists were combined into 1 expert group.

To answer the question of whether the simulator is a valid tool for teaching and differentiating different skill levels in simulated difficult BMV, board-certified anesthesiologists and CRNAs practicing at an academic medical center were used to establish validity for this study. The CRNA and anesthesiologist groups were tested 1 time with no prior teaching or coaching. To simulate a patient who is difficult to ventilate with bag mask, the desired resistance was created with simple rubber restrictors placed in the airway of the simulator. Before starting each test, each participant was allowed to read a brief stem on the simulated patient who was apneic with a pulse at the start of the test. The same stem was used for each test in all groups, including pretesting and posttesting of internal medicine house staff (see Appendix).

All participants were allowed to ask for assistance from the supervising physician who could only squeeze the bag when asked, connect oxygen, or hand the participant the allowed equipment. The supervising physician was prohibited from providing suggestions or unsolicited assistance. Only oral and nasal airways, bag masks, and oxygen were available. Each subject was required to provide BMV for 2 minutes. These scoring standards were programmed into the simulator scenario such that, to run this test, any operator only needed to start the simulation. The test was performed automatically including grading for a period of 2 minutes.

Back to Top | Article Outline


Oxygen saturation (SpO2) was used as the outcome measure for grading participants in BMV. The SpO2 was programmed to drop very quickly (especially after dropping <93%) if airflow was not sensed. The only way airflow reaches the sensor in the chest of the CPS is through effective BMV. In this CPS, end-tidal carbon dioxide (CO2), respiratory rate, tidal volume, and SpO2 are interrelated such that, with inadequate BMV technique, inadequate ventilation will be reflected by a lower SpO2 during the 2-minute training and testing scenarios. The monitor did provide typical feedback including SpO2, capnographic result, respiratory rate, heart rate, and blood pressure. The program was developed, so SpO2 had a dependent relationship with adequate mask seal, tidal volume, and respiratory rate. Failure in BMV from various process issues led to inadequate ventilation and subsequent oxygen desaturation in the simulator. In addition, the didactic and hands-on training stressed looking for chest rise as another bedside tool to monitor for adequacy of ventilation during BMV.

Scoring reports had the following scale: high pass (>90%), pass (80%–90%), low pass (60%–79%), and unsatisfactory/fail (<60%). Oxygen desaturation less than 50% at any time stopped the scenario and was counted as a failure. The simulated patient’s SpO2 was visible during the entire scenario.

The house staff received training in difficult BMV, which was integrated into 2 days of their mandatory ACLS training held 1 week before their first week of clinical duty. One group of house staff was tested and trained on the first day, and a second group was trained on the second day. Before taking a pretest, these house staff received only a standard ACLS introduction to airway training with the Resusci Anne to verify that they knew the correct number of respirations per minute for ACLS. House staff were shown the standard “C hand position” for mask ventilation.

After the pretest, the house staff received a 15- to 20-minute didactic lecture by a critical care physician followed by 15 to 20 minutes of hands-on simulation instruction with primary attention to mask seal, use of artificial airways (oral and nasal), and progressing rapidly to the 2-person technique. The didactic lecture was structured to provide the following: an understanding of upper airway anatomy, identification of patient characteristics associated with difficult BMV, preoxygenation and length of time before significant oxygen desaturation, use of artificial airways (oral and nasal), and technique for proper BMV focusing on the application of maximum pressure centrally. Emphasis was placed on the mask seal that must be maximized while aligning the airway in a position for air to move unobstructed. This requires a coordinated effort between mandibular lift and centering pressure on the mask to avoid preferential leak from 1 side of the mask. The use of an oral or nasopharyngeal airway often serves as a critical maneuver to establish airflow. The 2-person technique is recommended when necessary to properly lift the mandible and simultaneously create a mask seal.21

The 2-person BMV is an approach to difficult BMV where the most experienced operator controls the mask using both hands to make a seal whereas the second person squeezes the bag with both hands. House staff were taught to recognize declining SpO2 and poor chest rise as indications to improve ventilation by changing to a 2-person technique when an assistant is available.

A practice simulation scenario was used, which provided feedback with adequate and inadequate BMV with the facilitator providing guidance when inadequate BMV was registered. After the instruction period during which all participants had to demonstrate effective BMV scenario through deliberate practice (required <20 minutes) on the difficult BMV simulator, the house staff were given a posttest.

Back to Top | Article Outline

Statistical Analysis

To answer the question of whether first-year internal medicine house staff can learn to ventilate a simulated difficult BMV patient, a 1-group pretest-posttest design was used to address the research question. This design was selected because the researchers were not using a control group. A control group was not selected for this pilot project because all of the students were going to be given the simulation as the educational intervention.

A nonparametric test, Mann-Whitney U test, was conducted to determine if the simulator could distinguish between a novice and an expert for BMV. Statistical significance was declared at P < 0.05. A nonparametric Wilcoxon test was performed to ascertain if the simulation training was effective in teaching internal medicine house staff to ventilate a simulated difficult patient.

Back to Top | Article Outline


Table 1 shows the results of the house staff pretest before training and of the posttest that took place after training. There is evidence that the mean scores on the posttest were positive and that the simulation training was effective in increasing knowledge and the ability to ventilate a simulated difficult patient (z = −5.477, P < 0.0001). All of the participants passed the posttest, although 6 participants scored in the low pass range. Overall, the simulated difficult-to-ventilate patient instructional intervention yielded positive results for house staff.



Without pretest training, 11 of 12 anesthesiologists and 13 of 16 CRNAs passed the test. Of the anesthesiologists, 8 scored high pass, 3 scored pass, and 1 failed. Of the CRNAs, 11 scored high pass, 2 scored pass, and 3 failed. The difference between the novice and expert groups was significant (U = 209, P < 0.0001).

Back to Top | Article Outline


The important findings of this study include the following: a simulator with calibrated airway restrictors is a valid tool to teach and test skill performance in 1 type of simulated difficult BMV. In a relatively short training session incorporated into standard ACLS training, first-year house staff can learn to ventilate a simulated difficult-to-ventilate patient. Additional testing is required to determine if successful BMV in this scenario translates to success with an actual patient who undergoes respiratory arrest.

In anesthesia training, simulation-based skill assessment has been validated as a tool for distinguishing the skills of experienced from less experienced house staff in recognition and management of simulated acute intraoperative events.22 In addition, simulation-based resuscitation courses have led to significant improvement in self-assessed theoretical knowledge and procedural skill for emergency medicine residents.23 To the authors’ knowledge, simulation-based skill assessment has not been validated as a tool for training novices in difficult BMV. This study used a novice and expert cohort to validate a difficult BMV scenario using resistance bands in the major airways of an assimilated patient. The results suggest that novices are unable to adequately ventilate and that, without specific training, expert skill is necessary to effectively ventilate this simulated patient. Although all CRNAs and anesthesiologists are expected to be competent in BMV, variability in the results of simulation testing is known to occur among individuals of similar skill level.24

This study demonstrated the feasibility and effectiveness of adding difficult BMV training to standard ACLS training. Furthermore, the researchers found that using the simulator with a 20-minute didactic lecture and 20 minutes of hands-on training on difficult BMV allowed for novice house staff to successfully ventilate a simulated difficult-to-ventilate patient. This educational intervention adds value to standard ACLS training for medical house staff, which traditionally does not provide hands-on training in difficult BMV.

Back to Top | Article Outline


Computerized patient simulators have provided reliable although not perfect measurements of participants’ abilities to manage simulated acute clinical events. However, it is acknowledged that, in the early phases of their use, their validity as yardsticks for clinical performance is controversial.22 As a tool to measure competence in 1 person BMV, we used a setting for difficulty that probably reflects a model for expertise rather than competence in that a significant number of CRNAs and 1 anesthesiologist did not pass on the simulator. Although the use of a 2-person BMV technique was available to participants, not all anesthesiologists and CRNAs took advantage of this technique when they realized how difficult BMV was on this simulated patient. Despite the difficulty in BMV in this simulated patient, 100% of first-year house staff passed after a short didactic and hands-on training session, which stressed the use of oral and nasal airways and the 2-person BMV technique to maximize mask seal.

Although the establishment of validity of this simulated difficult BMV scenario in this individual study is a first step, ultimately predictive validity is desired such that the ascertainment of “competence” in the simulated setting predicts performance in the true clinical setting. Further testing including clinical outcomes of house staff trained with this simulator was not performed as part of this study.

The methodology used for this study is also a limitation. Single-group pretest/posttest design does not allow for a control or comparison group related to the intervention. Instead, we compared the results of the participants from their pretest scores with those of their posttest scores. Because we did not use a control group, it is difficult to assess the significance of an observed change in this design. The change in learning we observed could be unrelated to the simulation intervention. However, this was a cost-effective approach to discern if the simulation intervention is worth further investigation. Therefore, conclusions drawn from this study should be used as a starting point for further research in simulation training for difficult BMV. It would be beneficial to conduct a similar study using a control group to determine if using simulation can assist with difficult BMV training during an ACLS program. In addition, a longitudinal evaluation with repeated testing of trainees should be performed to assess retention of this training.

The use of SpO2 as an outcome measurement for effective BMV in this study may be considered a weakness because exhaled tidal volume or end-tidal CO2 could have been the outcome measured for ventilation. However, in the programming of this CPS, end-tidal CO2, respiratory rate, and tidal volume are measured and are interrelated such that inadequate technique leading to reduced end-tidal CO2 would be reflected by a lower SpO2 during the 2-minute training and testing scenarios.

Back to Top | Article Outline

Future Directions

Although this study shows that a model for standardizing and validating the degree of difficulty for simulated BMV can be achieved, it suggests that further work could use settings that could be validated for a series of simulation scenarios that would support deliberate practice and evaluation for (a) 1-person BMV competence, (b) 1-person BMV expertise, and (c) 2-person BMV competence. Additional research is needed to determine the effectiveness of the educational intervention. More specifically, research studies could focus on retention of knowledge and/or translational impact with simulated difficult BMV patients for patient safety and clinical outcomes. Future directions looking at predictive validity will be necessary to close the loop between training in the simulated world and clinical practice in the real world as have been achieved in other fields of simulation such as commercial aviation and the military.

Back to Top | Article Outline


The strength of this study lies in the ability to standardize the degree of difficulty and take advantage of the ability to program a scenario that automatically monitors and applies standards to factors such as ventilation and time coupled to relevant physiology (ie, pulse oximetry readings) in a standardized manner. These characteristics help support generalization of simulation-based testing and form a foundation for further studies needed to validate such CPSs as surrogates for actual clinical events.

Back to Top | Article Outline


In this study, a standardized automated simulation was created to model difficult BMV. Results suggest that the current setting established validity for this study with a restrictor calibrated to 28 (±1) cm H2O. The results of this study suggest that an opportunity does exist for determining and validating a simulation model for assessing competence in difficult BMV. Data from this study suggest that using a simulator with a didactic lecture and brief hands-on training on difficult BMV can allow new house staff to successfully ventilate a simulated difficult patient. This model has the potential to be successfully incorporated into standard ACLS training for incoming residency house staff. Potentially, this educational intervention could be beneficial for house staff while they progress to real-life scenarios.

Back to Top | Article Outline


1. Neumar R, Otto C, Link M, et al.. Part 8: Adult Advanced Cardiovascular Life Support 2010 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation 2010; 122 (Suppl 3): S729–S767.
2. Abella B, Alvarado J, Myklebust H, et al.. Quality of cardiopulmonary resuscitation during in-hospital cardiac arrest. JAMA 2005; 293: 305–310.
3. Kaye W, Mancini M. Retention of cardiopulmonary resuscitation skills by physicians, registered nurses, and the general public. Crit Care Med 1986; 14: 620–622.
4. Kaye W, Mancini M. Teaching adult resuscitation in the United States: time for a rethink. Resuscitation 1998; 37: 177–187.
5. Lowenstein S, Hansbrough J, Libby L, et al.. Cardiopulmonary resuscitation by medical and surgical house-officers. Lancet 1981; 8248: 679–681.
6. Lum M, Galletly D. Resuscitation skills of first year postgraduate doctors. N Z Med J 1989; 102: 409–411.
7. Schwid H, Rooke G, Ross B, et al.. Use of a computerized advanced cardiac life support simulator improves retention of advanced cardiac life support guidelines better than a textbook review. Crit Care Med 1999; 27: 821–824.
8. Stross J. Maintaining competency in advanced cardiac life support skills. JAMA 1983; 249: 3339–3341.
9. Cavanaugh S. Computerized simulation technology for clinical teaching and testing. Acad Emerg Med 1997; 10: 939–943.
10. Friedrich MJ. Practice makes perfect risk-free medical training with patient simulators. JAMA 2002; 288: 2808–2812.
11. Gaba DM. Improving anesthesiologists’ performance by simulating reality. Anesthesiology 1992; 76: 491–494.
12. Issenberg SB, McGaghie WC, Hart IR, et al.. Simulation technology for health care professional skills training and assessment. JAMA 1999; 282: 861–866.
13. Mayo PH, Hackney JE, Mueck JT, et al.. Achieving house staff competence in emergency airway management: results of a teaching program using a computerized patient simulator. Crit Care Med 2004; 32: 2422–2427.
14. Langeron O, Masso E, Huraux C, et al.. Prediction of difficult mask ventilation. Anesthesiology 2000; 92: 1229–1236.
15. El-Orbany M, Woehlck H. Difficult mask ventilation. Anesth Analg 2009; 109: 1870–1880.
16. Walz JM, Zayarushi M, Heard S. Airway management in critical illness. Chest 2007; 131: 608–620.
17. Yildiz TS, Solak M, Toker K. The incidence and risk factors of difficult mask ventilation. J Anesth 2005; 19: 7–11.
18. Kheterpal S, Han R, Tremper KK, et al.. Incidence and predictors of difficult and impossible mask ventilation. Anesthesiology 2006; 105: 885–891.
19. Caplan R, Benumof J, Berry F, et al.. Practice guidelines for the management of the difficult airway. Anesthesiology 2003; 98: 1269–1277.
20. Benumof JL. Preoxygenation: best method for both efficacy and efficiency. Anesthesiology 1999; 91: 603–605.
21. Benumof JL. Definition and incidence of the difficult airway. In: Benumof JL, ed. Airway Management: Principles and Practice. Philadelphia, PA: Mosby Elsevier; 1996: 121–125.
22. Murray D, Boulet J, Avidan M, et al.. Performance of residents and anesthesiologists in a simulation-based skill assessment. Anesthesiology 2007; 107: 705–713.
23. Langhan T, Rigby I, Walker I. Simulation-base training in critical resuscitation procedures improves residents’ competence. CJEM 2009; 11: 535–539.
24. Henrichs B, Avidan M, Murray D, et al.. Performance of certified registered nurse anesthetists and anesthesiologist in a simulation-based skills assessment. Anesth Analg 2009; 108: 255–262.
Back to Top | Article Outline


A previously healthy 20-year-old man is 170 lb and 72 in tall. His vital signs are within normal limits. He is receiving moderate sedation for an elective outpatient procedure and becomes apneic. Youmust ventilate only withBMV until help arrives to place a definitive airway. There are no medications available to you, and you may ask the assistant for help.


Simulated difficult bag-mask ventilation; Computerized patient simulator; 2-Person bag-mask ventilation

© 2013 Society for Simulation in Healthcare