Birth is a complicated process in which 10% of all newborns require some assistance to start breathing. One percent, however, requires more extensive measures to survive.1 To provide a consistent approach during neonatal resuscitation, healthcare professionals in Europe are trained using the Newborn Life Support (NLS) guideline published by the European Resuscitation Council (ERC).2 Since the introduction of the NLS in 1999 and other comparable guidelines such as the Neonatal Resuscitation Program, research has repetitively shown that adherence to these guidelines is challenging and that errors during neonatal resuscitation are frequent. Previous research assessing real-life neonatal resuscitations reported deviation and error rates up to 55%.3–5 In addition, rapidly deteriorating skills and knowledge despite frequent resuscitation training have been well documented in literature.6–8 There is limited research on the effects of suboptimal adherence in patient outcome. However, several reports have shown that structured resuscitation training does improve patient outcome.9,10
Since the adoption of the NLS and analogous guidelines, numerous efforts have been made attempting to reduce errors during neonatal resuscitation. Checklists and cognitive aids have been proposed to improve adherence to guideline. Many are without any significant improvement.11,12
The cause of poor adherence to guideline remains largely unknown. A recent study under Dutch pediatricians revealed a large variation in knowledge, guideline execution, and resuscitation skills. This demonstrates the need for cues and quantitative feedback during neonatal resuscitation.13 Research into classification of errors during resuscitation has shown that errors of commission are abundant and repetitive and may be of clinical significance.14 It is thought that limited human ability in interacting with simultaneous stimuli and the high cognitive and technical demand during neonatal resuscitation might be explanatory for this phenomenon.15–17
Decision support tools (DSTs) are designed to fundamentally reduce cognitive load by decreasing cognitive demand and making decision making less prone to human error. A growing body of evidence suggests that the use of electronic decision support tools (eDSTs) during resuscitation is promising. To date, multiple studies have shown a significant reduction of errors and deviations and therefore improving adherence to guideline in different simulated resuscitation settings.18–22
The use of augmented reality in medical applications has shown to be an effective and versatile technique as interactive images can be displayed in the user's field of view without a significant disturbance of normal vision.23–26 In the field of neonatology, the use of augmented reality during simulated intubation has shown to be effective.25 Using the 2015 ERC NLS guidline, we designed an electronic decision support tool using augmented reality. We chose the Microsoft HoloLens as a platform.
The aim of this study is to evaluate adherence to guideline in subjects using an augmented reality electronic decision support tool during standardized simulated neonatal resuscitation compared with subjects working from memory alone.
Participants and Study Design
In this prospective randomized controlled pilot study, a total of 29 healthcare professionals participated. Subjects were included from 3 different hospitals: Radboud University Medical Center Amalia Children's Hospital, Nijmegen (level 3 neonatal unit), Rijnstate Hospital, Arnhem (level 2 neonatal unit), and Maasziekenhuis Pantein, Boxmeer (level 1 neonatal unit). The selection process of participants was based on availability of participants on the day the study team visited the selected hospitals. When the study team arrived, subjects could voluntarily apply to participate in the study. Subjects were only eligible to participate if their role in the hospital would require the skillset necessary to perform a full neonatal resuscitation. As a result, only pediatric residents, pediatricians, and neonatologists were able to participate. For allocation, participants were assigned to the control or intervention group by the study team using simple randomization procedures (computerized random numbers) moments before the start of the scenario. No randomization restrictions were applied. All participants performed 1 simulated neonatal resuscitation scenario with the aid of a nonobstructive nurse. Audiovisual recordings of all sessions were obtained and later reviewed by the study team.
The original Microsoft HoloLens version 1 (Redmond, WA) is an untethered mixed reality head-mounted display, enabling augmented visuals to be projected in the user's field of view. We developed an application for the Microsoft HoloLens using the commercially available Windows API and Unity engine called augmented reality in neonatology (ARNE). This application wirelessly connects the HoloLens to the instructor interface and downloads the desired vital parameters [heart rate (HR), saturation (Spo2), respiratory rate, and temperature] at a 1-second interval. It incorporates vital parameter data into the built-in NLS algorithm. The user interface presents vital patient data, current, and next appropriate steps of the algorithm using augmented visuals floating above the patient (Figs. 1, 2). It uses the built-in speakers and microphones to establish 2-way communication with the operator, giving the operator cues, evaluating whether an operation has been completed or requesting additional information for its analysis in absence of monitor data (examples are listed in Table 1). All vital parameters are designed to show the actual value including changing colors of the corresponding icons based on the appropriateness of the value. In addition to the figures, a demonstration of ARNE is available as supplemental digital content (see Video, Supplemental Digital Content 1, https://links.lww.com/SIH/A790, for a brief demonstration).
TABLE 1 -
Examples of Auditory Cues and Expected Operator Actions
||Operation to Be Performed and Interaction by User
|“Is the baby breathing?”
||Evaluate spontaneous breathing. Answer by saying “Yes” or “No.”
|“Inflate the lungs, 5 breaths with a two till three second inspiration and watch for chest rise”
||Start providing insufflation breaths. A timer will count down in which the 5 insufflation breaths are expected to be given (30 s). Recommended pressures are stated as well. Advance in protocol by saying “Next step.”
|“Low heartrate detected”
||ARNE detects the HR automatically from ECG or pulse oximetry sensors. For this, no input from the user is required.
|“Start compressions with a ratio of three compressions to one breath”
||Start with chest compression. Again, a timer will count down 30 s until the next moment of evaluation. Visual cues include to increase the fraction of inspired oxygen (Fio
2) and to prepare a UVC. Advance in protocol by saying “Next step.”
|“Place an umbilical catheter and continue basic life support”
||Place an umbilical venous catheter. The user can provide the answers “success” or “failed.” In the first case, ARNE will continue with the recommendation to provide adrenaline. If a UVC could not be placed, ARNE will recommend to (1) get help and continue basic life support or (2) get alternative intravenous access.
UVC, umbilical venous catheter.
A modified low-fidelity Neonatal Resuscitation Baby (Laerdal, Stavanger, Norway) was used in all simulated resuscitation scenarios. The manikin was modified by T.A.J.A., transforming it into a high-fidelity neonatal patient simulator. Head position, airway pressures, and chest compressions characteristics can be measured by positional, flow, and pressure sensors. Audio was added by small speakers to provide to ability to assess HR using a stethoscope. A simulated bedside patient monitor was used to display the vital parameters if the participant attached electrocardiogram and/or pulse oximetry sensors. The software controlling the manikin and the bedside monitor (HR, SpO2, and spontaneous breathing) was custom made and programmed by T.A.J.A. in C#. The control software ran from a computer using a wireless connection with the manikin and simulated bedside monitor, making it possible to operate the manikin and monitor from outside the room. A standardized script depending on the actions of the participant was run on this equipment during each simulation session.
Participants were requested to perform a real-time resuscitation at birth according to the NLS guideline. Beforehand, all subjects were familiarized with the equipment of the simulated delivery room. Subjects in the intervention group were instructed how to operate the HoloLens and ARNE but were not exposed to the NLS algorithm. All participants were assigned to the role of resuscitation team leader and in charge of airway management. The role of nonobstructive nurse was played by experienced resuscitation simulation operators. The nurse acted as in real clinical care but only on the participant's request. All scenarios were videotaped using a single wide angle high-definition camera (GoPro HD Hero 3 Black Edition) providing a full overview of the manikin, participant, and surroundings.
Each simulation included a briefing about the clinical history: a term newborn delivered by cesarean section because of fetal distress with a clinical suspicion of a placental abruption. At the initial assessment, the HR of the manikin was set at a rate of 40 beats per minute with no spontaneous breathing. Color and muscle tonus were provided by the operator of the mannequin and were “pale/blueish” and “weak,” respectively. The first set of chest compressions did not result in an increase of HR. Only after epinephrine and a fluid or blood bolus were provided via an umbilical venous catheter, HR and Spo2 would recover. After which, chest compressions could be ceased and ventilation had to be continued. At this time, the subject was instructed that the simulation had been completed.
Outcome Measures and Statistical Analysis
All video recordings were collected and subsequently independently evaluated for assessment metrics by K.D.T. and T.A.J.A. K.D.T. is a pediatric resident. T.A.J.A. is a neonatologist, NLS instructor, and course director with extensive experience in high-fidelity video-assisted real-time simulation training. A task was deemed appropriate when executed in accordance with the 2015 ERC NLS algorithm (see document, Supplemental Digital Content 2, https://links.lww.com/SIH/A791, for a complete case description and evaluation form). Each appropriate executed task was assigned a score of 1. The highest possible score was 40, based on the sum of tasks analyzed. Error classification was used to distinguish between errors of omission and commission in case of algorithm deviation.
Duration of time until the execution or completion of several critical steps in the algorithm was assessed to determine whether the use of the eDST would cause delay during simulated neonatal resuscitation.
Based on a nonnormal distribution of scores and errors, both groups were compared using the Wilcoxon rank sum test (Mann-Whitney). Cohen κ was used to assess interrater reliability for nominal data, comparing every distinct step in all subjects. For continuous data, the intraclass correlation coefficient was calculated based on a single measurement, absolute agreement, and 2-way mixed-effects model. SPSS 25.0 (IBM Chicago, IL) was used for statistical analysis. P values less than 0.05 were considered statistically significant.
From September to December 2019, a total of 29 healthcare professionals participated in the study. Eighteen of the 29 participants were randomly assigned to perform their NLS simulation with the help of our eDST. One subject in the intervention group was excluded from analysis because of device malfunction (see document, Supplemental Digital Content 3, https://links.lww.com/SIH/A792, which shows a flow diagram and Consolidated Standards of Reporting Trials checklist). As shown in Table 2, the 2 groups showed similar demographic characteristics (P > 0.05).
TABLE 2 -
Demographic Characteristics of the Study Group
||Decision Support Tool (n = 17)
||Control (n = 11)
|Age, mean (SD), y
|Sex (female), n (%)
|Function (resident), n (%)
|Years of experience, median (IQR [range])
||5 (2–17.5 [1–30])
||3 (2–16 [1–25])
|Previous NLS training, n (%)
|Years since last NLS training, median (IQR [range])
||3 (1–6.25 [0–13])
||1.5 (0–3.75 [0–7])
|Previous neonatal resuscitation, n (%)
|Years since last resuscitation, median (IQR [range])
||1 (0–4.75 [0–10])
||0 (0–3 [0–3])
To compare the difference in adherence to guideline between both groups, a total of 40 tasks per individual were assessed based on the NLS algorithm. The median [interquartile range (IQR)] test score in the eDST group was 34 (32.5–35.5) compared with 29 (27–33) in subjects working from memory alone (P = 0.004).
Subjects using the eDST showed significantly less errors of commission, for example, starting the Apgar clock late or providing ventilation breaths for less than 30 seconds. Median errors of commission were 2 (1–2.5) in the intervention group versus 4 (2–4) in the control group (P = 0.029).
Regarding delay, we measured if the initial assessment had been completed 30 seconds after birth and, as the NLS algorithm states, if the 5 insufflation breaths had been provided within 60 seconds after birth. In this study, analysis of both items showed no significant differences between intervention and control groups for the completion of initial assessment within 30 seconds (76 vs. 64%, P = 0.671) and completion of insufflation breaths within 60 seconds (59 vs. 55%, P = 1.000). Analysis of time to execution of distinct critical steps in the NLS algorithm showed no significant differences in both groups (shown in Fig. 3).
Agreement between the 2 individual raters was excellent. Agreement of nominal data was κ = 0.974. Intraclass correlation for continuous data was 0.999 (95% confidence interval = 0.999–1.000).
Healthcare professionals using our eDST showed a statistically significant improved adherence to the NLS guideline when compared with healthcare professionals working from memory alone. By using ARNE healthcare professionals were less likely to deviate from the algorithm and less prone to commit errors of commission. No significant differences were found between the eDST and control groups with regard to elapsed time in completion of the first assessment, provision of insufflation breaths, and other critical resuscitation steps. This demonstrates that the use of our eDST did not slow healthcare professionals down throughout their simulated resuscitation scenarios.
Neonatal resuscitation is a critical and time-sensitive intervention, which has a significant impact on mortality and life-long neurological morbidity. Deviations and errors during neonatal resuscitation may have a detrimental effect on patient outcome. In the control group, 3 subjects skipped critical steps in the NLS algorithm, compared with none in the intervention group. One subject did not provide ventilation breaths for 30 seconds before initiation of chest compressions, whereas 2 subjects did not provide chest compression before placing an umbilical venous catheter. Ventilation of the newborn lungs is crucial in neonatal resuscitation, because adequate ventilatory support may prevent further cardiorespiratory deterioration and may decrease chances of successful resuscitation.27
One previous study reported a significant reduction in guideline deviations when using NeoCue (MedicalCue, Inc, Mountain View, CA) during simulated neonatal resuscitation.21 NeoCue is an eDST, providing auditory and visual prompts extrapolated from the Neonatal Resuscitation Program algorithm based on manual input of heart and respiratory rate. Augmented reality in neonatology advances upon the NeoCue as it automatically acquires essential vital parameters from monitor data in simulated and real-life neonatal resuscitation, requiring no manual input of HR and breathing. The use of augmented reality rather than a tablet computer provides health care professionals with a continuous and undisturbed overview of vital parameters and instructions while keeping their hands free for essential interventions.
This study supports evidence from previous observations evaluating the use of eDSTs in healthcare. Electronic decision support tools concerning simulated life-threatening or resuscitation scenarios have shown a significant improvement in adherence to guideline and reduction of errors and deviations.18–22 Despite promising results, the adoption of eDSTs in clinical practice is still limited. Adoption of new technology or changes in long-standing routines often results in a temporarily increased user distraction or an additional required effort. In this study, subjects using the eDST were able to complete a greater number of correct actions without signs of delay in the algorithm, which may be explained by a reduced cognitive demand. However, successful neonatal resuscitation is not only limited to cognitive skills but requires technical and behavioral skills as well. Evaluation of technical and behavioral skills was outside the scope of this study.
Almost all participants agreed that an eDST is a welcome and helpful technology to be introduced in the clinical setting. However, the HoloLens version 1 represents an early instantiation of a commercially available augmented reality head-mounted display and subsequently has multiple drawbacks and technical limitations. Participants mentioned the limited field of view in which the holograms were projected to be inconvenient. Despite the fact that operation of ARNE was dependent on 6 distinct phrases, the speech recognition engine of the Microsoft HoloLens experienced considerable difficulties when being used in noisy environments. Other comments were bulkiness of the device and poor comfort when being worn for a prolonged amount of time or when combined with prescription glasses.
In addition to the technical limitations of the Microsoft HoloLens, this pilot study has several limitations, which must be acknowledged.
Our study is limited by a small sample size consisting of mainly pediatric residents. In our facility, all pediatric residents are required to undergo formal neonatal life support training. However, these participants are relatively young medical doctors and not fully trained consultants. It is conceivable that their limited experience and exposure to real neonatal resuscitation results in more deviations and errors. Nonetheless, even experienced pediatricians show the need for cues and feedback during resuscitation and may profit from the support of an eDST as well.13
All measurements were conducted during simulated resuscitation due to the ethical and organizational difficulties when conducting research in real-life critical conditions. However, high-fidelity simulation has been demonstrated to be beneficial as research methodology during neonatal resuscitation.28
None of the subjects in the intervention group obtained an optimal compliance even when all of the appropriate actions were displayed in our eDST. This could be due to limitations of the user interface or due to inadequate familiarity with the device.
Analysis only concerned the performance of the resuscitation team leader. A resuscitation team, consisting of at least 3 people, may yield different results. Therefore, it would be interesting to evaluate the effect of this eDST on team performance during simulated neonatal resuscitation.
The use of decision support tools based on augmented reality is a novel and promising approach to further improve neonatal resuscitation. This study shows an improved adherence to the NLS guideline during simulated neonatal resuscitation.
The authors thank Theo Peters, a neonatal intensive care unit nurse of the Radboudumc Amalia Children's Hospital Nijmegen, for his contribution as nonobstructive nurse during simulated resuscitation scenarios and for his technical support.
1. Kattwinkel J, Perlman JM, Aziz K, et al. Part 15: neonatal resuscitation: 2010 American Heart Association Guidelines for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation
. 2010;122(18 Suppl 3):S909–S919.
2. Wyllie J, Bruinenberg J, Roehr CC, Rüdiger M, Trevisanuto D, Urlesberger B. European resuscitation council guidelines for resuscitation 2015: section 7. Resuscitation and support of transition of babies at birth. Resuscitation
3. Carbine DN, Finer NN, Knodel E, Rich W. Video recording as a means of evaluating neonatal resuscitation performance. Pediatrics
4. Thomas EJ, Sexton JB, Lasky RE, Helmreich RL, Crandell DS, Tyson J. Teamwork and quality during neonatal care in the delivery room. J Perinatol
5. Williams AL, Lasky RE, Dannemiller JL, Andrei AM, Thomas EJ. Teamwork behaviours and errors during neonatal resuscitation. Qual Saf Health Care
6. Kaczorowski J, Levitt C, Hammond M, et al. Retention of neonatal resuscitation skills and knowledge: a randomized controlled trial. Fam Med
7. Binkhorst M, Coopmans M, Draaisma JMT, Bot P, Hogeveen M. Retention of knowledge and skills in pediatric basic life support amongst pediatricians. Eur J Pediatr
8. Yang CW, Yen ZS, McGowan JE, et al. A systematic review of retention of adult advanced life support knowledge and skills in healthcare providers. Resuscitation
9. Dempsey E, Pammi M, Ryan AC, Barrington KJ. Standardised formal resuscitation training programmes for reducing mortality and morbidity in newborn infants. Cochrane Database Syst Rev
10. Lindbäck C, KC A, Wrammert J, Vitrakoti R, Ewald U, Målqvist M. Poor adherence to neonatal resuscitation guidelines exposed; an observational study using camera surveillance at a tertiary hospital in Nepal. BMC Pediatr
11. Bould MD, Hayter MA, Campbell DM, Chandra DB, Joo HS, Naik VN. Cognitive aid for neonatal resuscitation: a prospective single-blinded randomized controlled trial. Br J Anaesth
12. Katheria A, Rich W, Finer N. Development of a strategic process using check-lists to facilitate team preparation and improve communication during neonatal resuscitation. Resuscitation
13. Binkhorst M, van de Wiel I, Draaisma JMT, van Heijst AFJ, Antonius T, Hogeveen M. Neonatal resuscitation guideline adherence: simulation
study and framework for improvement. Eur J Pediatr
14. Yamada NK, Yaeger KA, Halamek LP. Analysis and classification of errors made by teams during neonatal resuscitation. Resuscitation
15. Alvarez GA, Cavanagh P. The capacity of visual short-term memory is set both by visual information load and by number of objects. Psychol Sci
16. Pashler H. Dual-task interference in simple tasks: data and theory. Psychol Bull
17. Luck SF, Vogel EK. Multiple sources of interference in dual-task performance: the cases of the attentional blink and the psychological refractory period. In: Shapiro K, ed. The Limits of Attention: Temporal Constraints in Human Information Processing
. Oxford: Oxford University Press; 2001:124–140.
18. Field LC, McEvoy MD, Smalley JC, et al. Use of an electronic decision support tool improves management of simulated in-hospital cardiac arrest. Resuscitation
19. Low D, Clark N, Soar J, et al. A randomised control trial to determine if use of the iResus© application on a smart phone improves the performance of an advanced life support provider in a simulated medical emergency. Anaesthesia
20. Siebert JN, Ehrler F, Gervaix A, et al. Adherence to AHA guidelines when adapted for augmented reality glasses for assisted pediatric cardiopulmonary resuscitation: a randomized controlled trial. J Med Internet Res
21. Fuerch JH, Yamada NK, Coelho PR, Lee HC, Halamek LP. Impact of a novel decision support tool on adherence to Neonatal Resuscitation Program algorithm. Resuscitation
22. Lerner C, Gaca AM, Frush DP, et al. Enhancing pediatric safety: assessing and improving resident competency in life-threatening events with a computer-based interactive resuscitation tool. Pediatr Radiol
23. Pratt P, Ives M, Lawton G, et al. Through the HoloLens™ looking glass: augmented reality for extremity reconstruction surgery using 3D vascular models with perforating vessels. Eur Radiol Exp
24. Hanna MG, Ahmed I, Nine J, Prajapati S, Pantanowitz L. Augmented reality technology using Microsoft HoloLens in anatomic pathology. Arch Pathol Lab Med
25. Dias PL, Greenberg RG, Goldberg RN, Fisher K, Tanaka DT. Augmented reality-assisted video laryngoscopy and simulated neonatal intubations: A pilot study. Pediatrics
26. Wang S, Parsons M, Stone-McLean J, et al. Augmented reality as a telemedicine platform for remote procedural training. Sensors (Basel)
27. Kattwinkel J, Perlman JM, Aziz K, et al. Part 15: neonatal resuscitation: 2010 American Heart Association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation
28. Halamek LP. Simulation
as a methodology for assessing the performance of healthcare professionals working in the delivery room. Semin Fetal Neonatal Med