The Health Information for Economic and Clinical Health Act of 2009 encouraged the implementation of electronic health records (EHRs) in healthcare organizations, and by 2016, more than 75% of hospitals in the United States reported having an EHR system in place.1 The increased use of EHRs has resulted in unintended consequences that have contributed to medical errors that impair patient safety and outcomes.2 Unintended consequences are adverse events that are directly related to information technology and result from either an EHR that functions in a manner that the vendor did not intend or from inappropriate use of EHRs by healthcare professionals.2 There are no universal design standards for EHRs, and consequently, every EHR vendor requires unique training for end-users.3
Historically, EHR competency training has been achieved through didactic lectures with hands-on experience in a live classroom, and this method fails to teach learners how to be proficient with EHRs because the contextual factors that are present in real-world settings are not included.4 Contextual factors such as a hospital's workflow, sociotechnical aspects (ie, social interactions between clinicians and how individuals interact with technology), and organizational culture alter how healthcare providers actually use EHRs.5 The acquisition of EHR competencies can be enhanced by using clinical simulations that incorporate real-world sociotechnical (ie, contextual factors) with the use of an EHR.6 Failure to address the sociotechnical factors associated with EHRs may result in unintended consequences including impaired patient safety, poor documentation quality, and inefficient workflows.5 Additionally, the use of clinical simulations allows healthcare providers to train in a safe environment that incorporates contextual factors without affecting patient safety.7
The use of clinical simulations to teach healthcare professional students EHR competencies has been described in the literature.8 The authors of this article were unable to locate best practices for using clinical simulations to train healthcare providers to use EHRs. The purpose of this literature review was to summarize the current state-of-the-science on the use of clinical simulations to train healthcare professionals to use EHRs. The findings can be used to improve clinicians' acquisition of EHR competencies via clinical simulations and inform future research. A brief overview of clinical simulations appears in the following paragraphs to provide background knowledge. This literature review includes descriptions of the study findings and elements that are useful for designing clinical simulations for EHR training, which include (1) learning objectives used in EHR training; (2) measurements used for clinical simulation outcomes; (3) benefits; (4) limitations; and (5) design considerations.
REVIEW OF CLINICAL SIMULATION
The advantages of clinical simulations are well established. In particular, participants can acquire technical and nontechnical skills in a safe environment that will not compromise patient safety.9,10 Technical skills encompass the knowledge and physical ability to complete a particular task; and nontechnical skills (ie, human factors) include the cognitive and social processes needed to accomplish the technical skills.11 Using human patient simulators or standardized patients (human actors who have been trained to act out specific patient scenarios with consistent interactions with clinical simulation participants)12 in conjunction with clinical simulations assist participants in the acquisition of technical and nontechnical skills used to manage complex events.13–15 There is evidence that clinical skill training with well-designed clinical simulations can translate into improved clinical performance.14 Clinical simulation may also foster participant autonomy and decision-making skills.10
Clinical simulations have three distinct phases that include prebriefing, clinical simulation, and debriefing.15 During the prebrief phase, a description of what the clinical simulation will involve is provided to the participant and includes an overview of relevant information that is necessary to complete the scenario.15 After the clinical simulation is completed, debriefing involves providing scenario feedback for the purpose of facilitating participant learning.10,16,17 The format of debriefing (eg, instructor led, multimedia based, or self led) is not as important as the structure and content.14
Fidelity refers to the degree that the clinical simulation imitates real-life situations.18 High-fidelity clinical simulations attempt to replicate real-life situations including most of the contextual factors.19 Low-fidelity simulations involving EHRs may exclude all contextual factors and consist of just the keyboard, mouse, and computer display.18 For example, low-fidelity simulations are useful for the initial training of clinicians in the basic functionality of the EHR. The degree of fidelity is determined by the purpose of the clinical simulation.20 For the purpose of this article, the term clinical simulation will refer to high-fidelity clinical simulations.
In March 2019, the databases PubMed, CINAHL, and Scopus were searched for publications using the following pairs of keywords without quotations: (1) electronic health record and simulation and (2) electronic medical record and simulation. No time limit was set because of the limited amount of literature related to using clinical simulation with EHRs. Inclusion criteria were literature that mentions using clinical simulations to train healthcare providers to use EHRs. Exclusion criteria were non-English articles, not peer reviewed, and articles that focused on training students.
A total of 1832 articles were initially identified, with 523 articles remaining after duplicates were removed. The articles were initially evaluated by both authors of this literature review according to the title and abstract (see Figure 1). All of the articles that met the initial search criteria (n = 22) were electronically retrieved. Of these, 10 articles met all inclusion criteria. The additional articles that were excluded involved the use of clinical simulation to design and evaluate EHRs. The authors of this article appraised the quality of the retrieved articles using the Johns Hopkins Nursing Evidence-Based Appraisal Tool,22 with the evidence ranking as medium (n = 5) to low (n = 5) quality (see Figure 1 for grading). The thematic analysis of this literature was organized and conducted using NVivo qualitative data analysis software (version 12, 2018; QSR International Pty Ltd, Doncaster, Victoria, Australia) to organize the information for comparison and contrast.
SYNTHESIS OF STUDY FINDINGS
The articles in this review included a wide range of healthcare provider types, clinical settings, and study designs (see Table 1 for a summary of the literature). The healthcare provider groups included interdisciplinary teams (physicians, nurses, and pharmacists),6,24,31 physician residents,23,25–28 registered nurses,29 and anesthesia providers (nurse anesthetists and anesthesiologists).30 The authors of the reviewed articles self-identified their clinical settings as intensive care units (medical and surgical),23,28 family medicine clinics,26,27 an inpatient pediatric floor,25 an acute care facility,29 an outpatient clinic,31 and an anesthesia department.30 The most common study designs included preintervention/postintervention studies23,25,27 and case studies.24,30,31 Other approaches included randomized controlled trials,26,29 a 2 × 2 crossover design,28 and an expert consensus statement.6 See Table 2 for a summary of the identified best practices for using clinical simulations during EHR training, and a summary of the benefits and limitations are discussed in the following paragraphs.
Table 1 -
Summary of Literature
||Article Type and Appraisal Rating
||Summary of Important Findings
|Allen et al23
||Intensive care unit physician residents
|The purpose of this study was to determine if EHR simulation training can improve infection rates. Combining central line insertion simulation training and electronic documentation helped reduce central line–associated bloodstream infections post central line insertion. Documentation of compliance with sterile precautions also improved.
|Bordley et al24
||Interdisciplinary team (physician residents, nurses, and pharmacists)
|The purpose of this study was to improve interprofessional team dynamics with a focus on using the EHR to assist with recognizing patient safety issues. The findings were used to improve intensive care unit rounding practices.
|Mohan et al20
||Interdisciplinary team (members not defined)
|EHR-agnostic design principles for using clinical simulations with EHR training were presented and include focus on clinical context and not EHR functionality; use high-fidelity approach; use standardized and realistic cases; release of information from EHR in a sequential and realistic timeline to support “cause-and-effect” decision-making processes; and include interdisciplinary team.
|Orenstein et al25
||Preintervention/postintervention study with control group
|The purpose of this study was to assess if EHR simulation training is an effective approach to change participants use patterns in real clinical settings. There was improvement in information retrieval from EHRs that could improve recognition of patient safety issues.
|Reis et al26
||Randomized controlled trial
|The purpose of this study was to teach doctor-patient-computer communication skills. Simulation-based training was found to not be more effective than lecture-based training, but participants were more satisfied with simulation-based training.
|Shachak et al27
||Pre/post intervention study
|The purpose of this study was to evaluate whether EHR simulations could be used to teach doctor-patient-computer interaction skills. The authors concluded that EHR simulations can improve informatics competencies.
|Stephenson et al28
||2 × 2 crossover design
|The purpose of this study was to evaluate whether EHR simulations can improve recognition of patient safety issues. The authors concluded that clinical simulations with EHRs can improve physician residents' identification of patient safety issues.
|Steward et al29
||Randomized controlled trial
|The purpose of this study was to compare EHR simulation to the standard paper-based competency assessment tool on registered nurses self-reported confidence and competence scores. The authors report that there is no difference between simulation and paper-based EHR confidence and competence.
|Weintraub et al30
||Anesthesia providers (anesthesiologists and nurse anesthetists)
|The purpose of this case study was to train anesthesia providers to use a new proprietary electronic health record system prior to implementation. Participants reported high user satisfaction with using high-fidelity simulations to learn a new EHR system.
|Vuk et al31
||Interdisciplinary team (physicians and registered nurses)
|The purpose of this case study was to evaluate the use of clinical simulations for clinicians' acquisition of self-efficacy with EHRs. The authors report that this approach increased self-confidence and preparedness for using EHRs.
Table 2 -
Summary of Best Practices for Using Clinical Simulations for Electronic Health Record Training
|Establishing learning objectives
| 1. The learning objectives inform the clinical simulation design and content.
| 2. The primary learning objective is related to teaching participants how to use the EHR.
| 3. Most clinical simulations have a secondary learning objective related to improving real-world clinical practices or patient outcomes (eg, recognizing patient safety issues; adherence to evidence-based clinical practices; or EHR competence, confidence, or self-efficacy).
| 4. The secondary learning objective is the primary determinant of the content of the clinical simulation and the required content in the EHR.
|Selecting outcome measures
| 1. The most common outcome measures include checklists of expected observable behaviors of participants and self-reported measures using psychometric instruments.
| 2. It is beneficial to use patient outcome measures to demonstrate that the clinical simulation resulted in real-world improvements (eg, adherence to evidence-based guidelines or improvement in patient outcomes such as infection rates).
| 3. If psychometric instruments are used then validity and reliability needs to be evaluated to increase the dependability of the findings.
|Designing and implementing the clinical simulation
| 1. New EHR implementations or novice EHR users require didactic EHR training prior to clinical simulations or participants will focus primarily on using the EHR instead of the simulation.
| 2. Incorporate organization specific contextual factors for a realistic clinical simulation that will enable EHR training to translate into sustained behavior changes in clinical practice.
| 3. When using interdisciplinary teams, each discipline must have a role in the development of the clinical simulation to ensure that the final content is relevant for everyone involved.
| 4. Clinical scenario needs to be realistic, have appropriate level of fidelity, and contain standardized content to increase participant engagement and enhance the acquisition of skills.
| 5. The EHR training environment needs to be separate from the production EHR; contain enough data to mimic real-world scenarios; and contain all of the user-specific customizations, macros, clinical decision support tools, and order sets that the participant will normally use.
| 6. The temporal considerations include realistic sequence events in the clinical simulation, real-time release of information in the EHR that matches the clinical simulation, and enough prepopulated data in the EHR that allows participant identification of trend changes over time.
| 7. Individuals conducting the debriefing need to be adequately trained so they may effectively guide participants' self-reflection on their performance. Deliberate self-reflection is necessary to maximize experiential learning.
| 8. Debriefing may incorporate audio/video recordings, the documentation generated during the clinical simulation, or eye-tracking technology. These methods allow evaluation of interdisciplinary team dynamics or enhance an individual's self-evaluation.
Findings of the Studies in This Literature Review
The significant findings of the studies in this literature review involved the impact of clinical simulation-based EHR training on improved recognition of patient safety issues,6,24,28 EHR competency,27,29,31 confidence related to EHR use,27,29 and improving central line–associated bloodstream infection rates.23 Compared to individual healthcare professionals, interdisciplinary teams are more effective in identifying patient safety issues.6,24,28 Improved recognition of patient safety issues is related to the fact that each individual healthcare discipline focuses on specific information in the EHR, and this differing focus is complementary in interdisciplinary teams.24
Important findings were found for competency and confidence related to using clinical simulations for EHR training. The study by Steward et al29 found no difference between paper-based training and clinical simulation on competency or confidence of registered nurses using an EHR. Steward et al29 reported that a sample size of 240 participants was needed to achieve adequate power, but only 87 participants were recruited into the study. These findings may not be dependable because underpowered studies are less likely to detect real differences.32 Other studies reported large effect size differences in competency (Cohen's d = 0.6327 and 0.8431) and confidence (Cohen's d = 0.4427 and 1.2427) when clinical simulation was used for EHR training. There is some evidence that novice users of EHRs may benefit more from clinical simulations that are intended to improve competency and confidence when compared with more experienced users.27 The use of clinical simulations to teach EHR competencies has been shown to result in persistent behavior changes related to identifying patient safety issues.25,26
A study on EHR training with a clinical simulation involving central line insertions found a decrease in central line–associated bloodstream infections.23 The authors reported that the infection rate decreased by 85% in the medical intensive care unit that received the training intervention but the infection rates remained unchanged in the surgical intensive care unit that did not receive the training.23 Documentation of compliance with hand hygiene, barrier precautions, and use of chlorhexidine for central line insertion site preparation went from 65% to 100%.23 The cost savings to the hospital was estimated to be $1 669 000 because of the reduction in central line–associated bloodstream infections.23 This is the only study in this literature review that demonstrated a direct relationship of EHR training to either patient outcomes or financial impact.
Learning Objectives for Clinical Simulations Used in Electronic Health Record Training
Clinical simulation training to teach healthcare professionals EHR competencies is often combined with other learning objectives. Learning objectives are used to inform the design and content of the educational curriculum.33 The primary learning objective was to teach clinicians how to use the EHR, and all the studies included a secondary learning objective. Secondary learning objectives included improving real-world recognition of patient safety issues via an EHR,24,25,28 improving competence and confidence associated with using EHRs,27,29 improving self-efficacy with EHRs,31 improving central line–associated bloodstream infections by enhancing documentation quality and adherence to evidence-based guidelines,23 and improving clinician-patient-computer communication competencies.26 The secondary objective in all of the studies in this literature review was used to determine the content of the clinical simulation.
Measurements Used for Clinical Simulation Outcomes
The most used methods to measure outcomes associated with the clinical simulations included observer evaluations using checklists of expected behaviors24,26,28,29 and self-reported survey measures (attitudes,26,27 EHR competency,27 and confidence in using EHRs29,31). One study used the rate of central line–associated bloodstream infections that occurred within 48 hours combined with a retrospective chart review to ascertain that all elements were documented for sterile technique during central line insertion (ie, use of chlorhexidine prep, hand hygiene, and barrier precautions).23 A summary and critique of these measurement methods are presented below.
Checklists of Expected Behaviors
The most frequently used method of measuring outcomes was checklists. The checklists of expected behaviors were determined by the learning objectives and included a list of clinical scenario specific events that needed to be identified.24,26,28,29 For example, the use of checklists to identify patient safety issues was the basis for an evaluation of how interdisciplinary team communication contributed to patient outcomes, and the findings were used to improve communication skills.24,26,27 The patient safety issues included inappropriate medication administration (unnecessary medications, inappropriate dosages, and multiple medication orders for opiates), failure to provide deep vein thrombosis prophylaxis, failure to identify abnormal laboratory values (positive blood cultures, acute kidney injury, and hypercalcemia), and not recognizing trending changes in physiological parameters (tachycardia, hypoxemia, and respiratory distress).24 Another study used a checklist of observable behaviors necessary for effective clinician-patient-technology communication skills that included the clinician sharing EHR information with the patient, maintaining eye contact with the patient, displaying empathy for the patient, and using jargon-free dialogue.26
Reis et al26 used an unnamed psychometric instrument (measuring communication, computer skills, EHR information sharing, and overall performance) that reported high Cronbach's α (internal consistency) scores ranging from .86 to .91. A psychometric instrument is a survey that has questions that use Likert-based scales, and the scores for each individual question are summated for a total score that represents some theoretical concept (eg, confidence in using an EHR).34 In a previous study35 that described the development of the instrument used by Reis et al,26 the content validity was not established and the reported internal consistency reliability was low (Cronbach's α = 0.52). Cronbach's α quantifies how well the individual questions within a psychometric instrument measure the same theoretical concept.32,34 A Cronbach's α of 0 reflects no internal consistency and 1 indicates perfect internal consistency, with a score greater than .7 considered acceptable.36
Steward et al29 adapted the Confidence Scale (C-Scale) to measure an individual's confidence during the generation of electronic documentation. No assessment of the reliability or validity of the C-Scale was reported.29 The original content validity and reliability were reported as acceptable (Cronbach's α = .85) in the original development of the C-Scale.37 The original C-Scale was designed to measure nurse confidence in physical assessment skills and not intended to be used as a measure of confidence in electronic documentation.37 It is essential to use psychometric instruments that have established validity and reliability so that the results are dependable, and when questions in an instrument are altered, it must be reevaluated.34 The findings of Reis et al26 and Steward et al29 may not be dependable or generalizable since the psychometric properties of the instruments were not adequately assessed for validity and reliability.
Benefits of Using Clinical Simulations for Electronic Health Record Training
The benefits of using clinical simulation for EHR training include evaluating interdisciplinary team communication,6,24 incorporating the sociotechnical factors that alter the way EHRs are used,6,24,28,31 improving the recognition of patient safety issues in real-world settings,6,24 attainment of EHR competencies that can translate into clinical practice,6,25 improving clinician-patient-technology interaction competencies to maximize interactions with the patient,6,26 and providing training in an environment that does not affect patient safety.6,30,31 The use of clinical simulations has been shown to result in sustained behavioral changes associated with the use of an EHR.25 Clinical simulations provide an active learning environment that enhances knowledge acquisition by increasing learner engagement.25 Additionally, clinical simulation training for EHRs may improve end-user acceptance of new system implementations while reducing unintended consequences.30,31
The role of communication among the interdisciplinary team members and with the patient is important.6 The use of interdisciplinary teams in clinical simulations for EHR training improves information sharing and fosters shared decision-making.6 High-quality interdisciplinary team communication improves the recognition of patient safety issues and prevention of medical errors.24,25 Including a standardized patient as part of the clinical simulation increases the interactivity, engagement, and satisfaction of simulation participants.31 Each healthcare professional discipline has unique patient safety issues that they may fail to recognize, and training interdisciplinary teams to use EHRs has been shown to improve the recognition of these patient safety issues.24 For example, registered nurses were most likely to recognize the lack of nutrition and the presence of multiple opiates prescribed to a patient, and these patient safety issues can be relayed to the attending physician.24
Healthcare providers can improve clinician-patient-technology competencies during clinical simulations by practicing documentation generation and information retrieval while communicating with the patient.27 If the healthcare providers' attention is focused on using the EHR, then the social interactions with the patient are impaired.27 The use of clinical simulations to teach EHR skills can help increase the social interactions with the patient by promoting eye contact, maintaining rapport, and facilitating psychological/emotional communication.27
Limitations of Using Clinical Simulations for Electronic Health Record Training
The limitations of using clinical simulations for EHR training include (1) difficulty in acquiring all possible interdisciplinary team members because of availability or financial cost24; (2) current EHR training environments have limited datasets that do not replicate real-world environments because they cannot include real patients6; (3) the real-time cascade of events is not similar to clinical practice (eg, laboratory values and physiological vital signs are often not released in real time)6; (4) the use of clinical simulations requires more human, financial, and physical resources when compared with standard didactic approaches6,25; and (5) the impact of clinical simulation on patient outcomes has not been definitively proven.25 The EHR training environment used with clinical simulations rarely captures the complex nature of real-world patient care events because of the limited amount of data within the EHR and the inability to provide a chronological release of relevant information.6 The most substantial limitation to using clinical simulations is the need for experienced simulation personnel and the associated cost of implementation.8 Additionally, clinical simulation training may not be beneficial for improving outcomes of rare events because the low sample size makes it difficult to assess the impact of any intervention.23
Design Elements and Considerations When Developing Clinical Simulations
Current EHR training focuses exclusively on the structure and content of the graphical user-interface without incorporation of the contextual factors that alter how healthcare professionals interact with information technology.6 Integrating the contextual factors into the clinical simulation helps to create a realistic scenario that includes the mental effort that the healthcare professional will expend when using the EHR in clinical settings.6 Clinical simulations need to focus on EHR functionality within the context of real-world patient care events that have increased mental effort.6 Electronic health record training that has artificially reduced mental effort may not catch potential decision-making errors that could result in medical errors.6,26
In general, designing clinical simulations often includes a literature review to find relevant content followed by assessment and approval by content experts.38,39 Many clinical simulations are developed according to course learning objectives and the unique knowledge, skills, and attitudes associated with each specific discipline participating in the clinical simulation.40 All of the literature in this review designed clinical simulations based on unique EHR training goals, which supported improved use of EHRs in clinical settings. Other design elements and considerations include clinical scenario content, EHR content, temporal considerations, and debriefing for participant evaluations.
Clinical Scenario Content
The clinical scenario content needs to be realistic, have an appropriate level of fidelity, and contain standardized content.6,28 The specific scenario used in the clinical simulation will be determined by the educational objectives and the specific disciplines represented in the interdisciplinary team.41 A representative from each discipline should be involved during the design of the clinical simulation to ensure that the final product is relevant to their unique needs.41 The clinical simulation needs to have a fidelity high enough to mimic a real-world environment closely.28 The realism provided by a high-fidelity environment will help engage the participants and enhance the acquisition of technical and nontechnical skills.6 Using realistic scenarios will improve the ability of clinicians to translate learned skills into real-world applications.6 Additionally, the clinical simulation needs a standardized structure and format because inconsistent implementation of the scenario will have a negative impact on the effectiveness of the training.6
Electronic Health Record Content
Electronic health record training has specific requirements for the content in the EHR database. Training for EHRs needs to occur in an environment that is separate from the production EHR used in clinical practice not only because of privacy concerns and the potential to for negative impacts on patient safety, but also because of inadvertent alteration of documentation in patients' medical records.6 The EHR training environment must replicate the production EHR that includes all user-specific customizations, macros, clinical decision support tools, and order sets.6,28,31 The number of test patients and the content included for each need to be similar to real-world settings (eg, radiology reports, laboratory values, medications, and previous documentation from interdisciplinary team members).24 An EHR training environment that includes a limited number of patients or minimal documentation content for each patient will artificially reduce the mental effort required to use the EHR and will reduce the usefulness of the training because of the failure to include important contextual factors.24 Also, the content of the training EHR needs to support the clinical workflows present in the clinical simulation to achieve the training objectives effectively.6
There are temporal considerations for the design of the clinical simulation and the content of the EHR. The clinical simulation needs a realistic sequence of events that occur in real-time because participants will be better able to translate the acquired technical and nontechnical skills into clinical practice.6,31 Patient information in the EHR needs to be released over time and coordinated with events in the clinical simulation.6 For example, a clinical simulation about a patient in acute respiratory failure needs to have laboratory values for arterial blood gas results populated in the EHR shortly after the blood sample is obtained. Additionally, the realism of the clinical simulation is improved if previous patient data are present that allows participants to evaluate trend changes over time.6,31
The articles in this review evaluated the learners in the clinical simulations for EHR training using debriefing techniques combined with other technologies (eg, video/audio recordings and eye-tracking). Debriefing facilitates experiential learning and occurs after the clinical simulation is concluded.42 The purpose of debriefing is to promote critical reflection on the clinical simulation experiences.42 An assumption of debriefing practices is that learning does not occur primarily through experiences but on the deliberate reflection on that experience.42
Debriefings should be interactive, contain bidirectional communication between the instructor and participant, and contain reflective discussions.42 The instructor conducting the debriefing needs to present statements and questions in a positive manner that focuses on the actions performed during the clinical simulation without implying that the participants made an error (eg, refer to errors as “opportunities for improvement”).42 Implying that participants performed poorly will impair learning because negative emotional responses have been proven to prevent effective learning.43 There are multiple approaches to debriefing that are summarized in the literature.42,43 A commonly used debriefing framework that is easy to use is Debriefing With Good Judgment.42,43
The Debriefing With Good Judgment framework includes three phases named reaction, analysis, and summary. The reaction phase is focused on participants' emotional feelings and presenting the facts about the clinical simulation.43 The first step of the reaction phase is to ask a question similar to “How did you feel?” and then followed up by the instructor summarizing what the clinical simulation was about. The purpose of the reaction phase is to allow the participants to reduce negative emotional responses that may impair effective learning.43 The analysis phase includes a reflective discussion to explore why the participants did specific actions during the clinical simulation with a focus on exploring the mental frame (ie, reasoning process) that led to mistakes.43 Identifying why a mistake was performed allows the participants to adjust their mental frame and prevent similar mistakes in the future, whereas only identifying specific errors results in an unchanged mental frame that may allow the error to occur again.43 The summary phase involves reinforcing the lessons learned and important issues discovered in the analysis phase.42,43
The evaluation of clinical simulations during debriefings can also incorporate video or audio recordings,24 assessment of the final documentation,23,24 and eye-tracking hardware.24 Video-based debriefing has been shown to be equivalent in effectiveness to the other debriefing approaches and allows participants to evaluate their own performance visually.14 Video-based debriefing also allows participants to analyze interdisciplinary team dynamics that include physical interactions and communication.24 Analyzing the documentation that was generated during the clinical simulation can help participants identify areas for self-improvement related to documentation quality.24
Eye-tracking technology can measure mental workload using pupil dilation changes that reliably reflect alterations in mental effort,44 capture video of the graphical-user-interface or patient interactions,24 and identify specific locations in the graphical-user-interface on which the participant was visually focused.44 An assumption of eye-tracking is that the location on which individual fixates (ie, is visually focused) represents where information is being retrieved and mentally processed.44 An eye-tracker can generate a video that identifies where an individual physically gathered information to inform clinical decision making.44 For example, eye-tracking can reveal that a nursing student did not evaluate a patient's drug allergies prior to administration of a medication (as evidenced by no video recording of a visual fixation on allergies in the EHR or verbalization by the student for oral verification of drug allergies from the patient). The current cost and expertise needed to use eye-tracking technology can limit the usefulness of this approach (per eye-tracker cost currently ranges from a minimum of $5000 for monitor mounted and $20 000 for eye goggles).44
As previously discussed, most articles in this literature review combined clinical simulations for EHR training with secondary goals such as recognition of patient safety events, improving documentation quality, and minimizing infection rates. The authors' descriptions of how the clinical simulations were designed and the specific methods of measuring outcomes were thorough, with checklists of expected behaviors as the primary approach. The previous discussions described how to establish learning objectives, select outcome measures, and design and implement clinical simulations. However, important issues that were not addressed by the articles in this literature review included pretraining for the EHR system prior to clinical simulation and considerations specific to the resources needed for the clinical simulation laboratory.
Pretraining in Electronic Health Records Prior to Clinical Simulation
If clinical simulation participants have no foundational knowledge of how to use the EHR, then most of the participants' mental effort will be on learning the EHR at the expense of the clinical simulation.8 Conventional classroom training for EHRs is insufficient to teach new users to be proficient and a significant amount of additional on-the-job training will be required to obtain proficiency.4 The traditional didactic approach to EHR training results in clinicians becoming distracted from direct patient care because of the mental effort required to learn the EHR.4 Combining conventional EHR classroom training that occurs before the simulation-based training is a more efficacious approach if the participants are novice users of the EHR, but more experienced participants may forgo this additional training.4
Simulation laboratories need specific human, equipment, environmental, and financial resources.45 Some of the human resources include personnel trained in simulation techniques, information technology support personnel, and adequate staff for the daily management of the simulation laboratory.45 Personnel who support and train the instructors who will conduct the clinical simulations are crucial for successful achievement of the learner objectives.8 Developing clinical simulations and the process used for successful debriefing require instructor training to achieve proficiency and these roles are often filled by specialists with advanced educational preparation.45
The financial impact of the required resources may make a simulation laboratory cost-prohibitive for smaller hospitals. Some of the required equipment and environmental needs may include patient simulators, a control room, simulated patient care areas, conference room, wall-mounted suction equipment, oxygen outlets, air compressors, conference room(s), wall-mounted video cameras, and desktop computers.45 The initial acquisition cost of a simulation laboratory can range from $100 000 to several million dollars.46 Combining resources with other organizations (eg, other hospitals or universities) in the same geographical region may facilitate the creation of a new simulation laboratory.8
Multiple implications for research were identified in this literature review. Future research needs to include psychometric instruments that have demonstrated validity and reliability in the respective study population to improve dependability and generalizability of the findings. It would be beneficial to compare the effectiveness of clinical simulations to train novice versus experienced EHR users because this will have a direct impact on designing the EHR training. Additionally, the findings in this review need to be replicated with larger sample sizes.
There were several important limitations to this literature review. It is possible that the use of different search keywords may have identified other relevant articles. The majority of articles in this literature review focused on EHR training for clinicians who have already been using the EHR in clinical practice, and more research is needed on the use of clinical simulation for the initial EHR training after new system implementations. The articles in this literature review are medium to low quality. Additionally, the studies in this literature review used psychometric instruments without fully describing their validity and reliability, so their findings may not be dependable.
Healthcare professionals can acquire EHR competencies through well-designed clinical simulations. The benefits of using simulation-based EHR training that incorporates the organization's sociotechnical factors include improvement of interdisciplinary team communication, clinical performance, clinician-patient-technology communication skills, and recognition of patient safety issues. Some of the significant limitations include the difficulty in coordinating all of the interdisciplinary team members required to design and implement the clinical simulation and the availability of the financial resources needed to operate the simulation laboratory. What this article adds to the literature is a summary of evidence-based guidelines to design and evaluate the use of clinical simulation to train healthcare professionals to use EHRs. Design considerations for EHR training using clinical simulations involve establishing learner objectives, identifying outcome measures, establishing content requirements of both the clinical simulation and EHR, and providing adequate debriefing.
1. Sittig DF, Wright A, Ash J, Singh H. New unintended adverse consequences of electronic health records. Yearbook of Medical Informatics
. 2016;1: 7–12.
2. Zheng K, Abraham J, Novak LL, Reynolds TL, Gettinger A. A survey of the literature on unintended consequences associated with health information technology: 2014-2015. Yearbook of Medical Informatics
. 2016(1): 13–29.
3. Singh H, Spitzmueller C, Petersen NJ, et al. Primary care practitioners' views on test result management in EHR-enabled health systems: a national survey. Journal of the American Medical Informatics Association
. 2013;20(4): 727–735.
4. Stroup K, Sanders B, Bernstein B, Scherzer L, Pachter LM. A new EHR training
curriculum and assessment for pediatric residents. Applied Clinical Informatics
. 2017;8(4): 994–1002.
5. Wilbanks BA. An integrative literature review of contextual factors in perioperative information management systems. Computers, Informatics, Nursing
. 2013;31(12): 622–628.
6. Mohan V, Woodcock D, McGrath K, et al. Using simulations to improve electronic health record use, clinician training
and patient safety: Recommendations from a consensus conference. AMIA Annual Symposium Proceedings
. 2017;2016: 904–913.
7. Gold JA, Stephenson LE, Gorsuch A, Parthasarathy K, Mohan V. Feasibility of utilizing a commercial eye tracker to assess electronic health record use during patient simulation. Health Informatics Journal
. 2016;22(3): 744–757.
8. Wilbanks BA, Watts PI, Epps CA. Electronic health records in simulation education: literature review and synthesis. Simulation in Healthcare
. 2018;13(4): 261–267.
9. Gaba DM. The future vision of simulation in healthcare. Simulation in Healthcare
. 2007;2(2): 126–135.
10. Grant JS, Moss J, Epps C, Watts P. Using video-facilitated feedback to improve student performance following high-fidelity simulation. Clinical Simulation in Nursing
. 2010;6(5): e177–e184.
11. Wunder LL. Effect of a nontechnical skills intervention on first-year student registered nurse Anesthetists' skills during crisis simulation. AANA Journal
. 2016;84(1): 46–51.
12. Yeung F, Yuan C, Jackson DS, Chun MBJ. Gone, but not forgotten? Survey of resident attitudes toward a cultural standardized patient examination for a general surgery residency program. Health Equity
. 2017;1(1): 150–155.
13. Grant JS, Dawkins D, Molhook L, Keltner NL, Vance DE. Comparing the effectiveness of video-assisted oral debriefing and oral debriefing alone on behaviors by undergraduate nursing students during high-fidelity simulation. Nurse Education in Practice
. 2014;14(5): 479–484.
14. Leblanc VR. Review article: simulation in anesthesia: state of the science and looking forward. Canadian Journal of Anaesthesia
. 2012;59(2): 193–202.
15. Brett-Fleegler M, Rudolph J, Eppich W, et al. Debriefing assessment for simulation in healthcare: development and psychometric properties. Simulation in Healthcare
. 2012;7(5): 288–294.
16. Johnson KB, Syroid ND, Drews FA, et al. Part task and variable priority training
in first-year anesthesia resident education: a combined didactic and simulation-based approach to improve management of adverse airway and respiratory events. Anesthesiology
. 2008;108(5): 831–840.
17. Park CS, Rochlen LR, Yaghmour E, et al. Acquisition of critical intraoperative event management skills in novice anesthesiology residents by using high-fidelity simulation-based training
. 2010;112(1): 202–211.
18. Jensen S, Nohr C, Rasmussen SL. Fidelity in clinical simulation: how low can you go?Studies in Health Technology and Informatics
. 2013;194: 147–153.
19. Gold JA, Tutsch AS, Gorsuch A, Mohan V. Integrating the electronic health record into high-fidelity interprofessional intensive care unit simulations. Journal of Interprofessional Care
. 2015;29(6): 562–563.
20. Mohan V, Scholl G, Gold JA. Intelligent simulation model to facilitate EHR training
. AMIA Annual Symposium Proceedings
. 2015;2015: 925–932.
21. Moher D, Liberati A, Tetzlaff J, Altman DG; The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Medicine
. 2009;6(7): e1000097. doi:10.1371/journal.pmed.1000097.
22. Newhouse RP, Dearholt SL, Poe SS, Pugh LC, White KM. Johns Hopkins Nursing Evidence-Based Practice Model and Guidelines
. Indianapolis, IN: Sigma Theta Tau International Honor Society of Nursing; 2007.
23. Allen GB, Miller V, Nicholas C, et al. A multitiered strategy of simulation training
, kit consolidation, and electronic documentation is associated with a reduction in central line-associated bloodstream infections. American Journal of Infection Control
. 2014;42(6): 643–648.
24. Bordley J, Sakata KK, Bierman J, et al. Use of a novel, electronic health record-centered, interprofessional ICU rounding simulation to understand latent safety issues. Critical Care Medicine
. 2018;46(10): 1570–1576.
25. Orenstein EW, Rasooly IR, Mai MV, et al. Influence of simulation on electronic health record use patterns among pediatric residents. Journal of the American Medical Informatics Association
. 2018;25(11): 1501–1506.
26. Reis S, Sagi D, Eisenberg O, et al. The impact of residents' training
in electronic medical record (EMR) use on their competence: report of a pragmatic trial. Patient Education and Counseling
. 2013;93(3): 515–521.
27. Shachak A, Domb S, Borycki E, et al. A pilot study of computer-based simulation training
for enhancing family medicine residents' competence in computerized settings. Studies in Health Technology and Informatics
. 2015;216: 506–510.
28. Stephenson LS, Gorsuch A, Hersh WR, Mohan V, Gold JA. Participation in EHR based simulation improves recognition of patient safety issues. BMC Medical Education
. 2014;14: 224.
29. Steward DJ, Mullinix C, Wu Q. Written versus simulation-based evaluation methods to assess competency and confidence in the use of electronic medical records. Journal of Continuing Education in Nursing
. 2018;49(6): 262–268.
30. Weintraub AY, Deutsch ES, Hales RL, Buchanan NA, Rock WL, Rehman MA. Using high-technology simulators to prepare anesthesia providers before implementation of a new electronic health record module: a technical report. Anesthesia and Analgesia
. 2017;124(6): 1815–1819.
31. Vuk J, Anders ME, Mercado CC, Kennedy RL, Casella J, Steelman SC. Impact of simulation training
on self-efficacy of outpatient health care providers to use electronic health records. International Journal of Medical Informatics
. 2015;84(6): 423–429.
32. Polit DF, Beck CT. Nursing Research: Generating and Assessing Evidence for Nursing Practice
. 9th ed. Philadelphia, PA: Wolters Kluwer/Lippincott Williams & Wilkins; 2012.
33. Iwasiw CL, Goldenberg D. Curriculum Development in Nursing Education
. Burlington, MA:Jones & Bartlett Publishers; 2014.
34. Waltz CF, Strickland O, Lenz ER. Measurement in Nursing and Health Research
. 4th ed. New York: Springer Pub; 2010.
35. Reis S, Cohen-Tamir H, Egger-Dreyfus L, Eisenberg O, Shachak A, Ziv A. The patient-doctor-computer communication Israeli study—pilot report. The International Journal of Person Centered Medicine
. 2015;1(4): 507–512.
36. Huck SW. Reading Statistics and Research
. 5th ed. Boston, MA: Pearson/Allyn & Bacon; 2008.
37. Grundy SE. The Confidence Scale: development and psychometric characteristics. Nurse Educator
. 1993;18(1): 6–9.
38. Chiu M, Tarshis J, Antoniou A, et al. Simulation-based assessment of anesthesiology residents' competence: development and implementation of the Canadian National Anesthesiology Simulation Curriculum (CanNASC). Canadian Journal of Anaesthesia
. 2016;63(12): 1357–1363.
39. Daniels RG, McCorkle R. Design of an evidence-based “second victim” curriculum for nurse anesthetists. AANA Journal
. 2016;84(2): 107–113.
40. Banerjee A, Slagle JM, Mercaldo ND, et al. A simulation-based curriculum to introduce key teamwork principles to entering medical students. BMC Med Educ
. 2016;16(1): 295.
41. Jensen S, Kushniruk AW, Nohr C. Clinical simulation: a method for development and evaluation of clinical information systems. Journal of Biomedical Informatics
. 2015;54: 65–76.
42. Sawyer T, Eppich W, Brett-Fleegler M, Grant V, Cheng A. More than one way to debrief: a critical review of healthcare simulation debriefing methods. Simulation in Healthcare
. 2016;11(3): 209–217.
43. Bowe SN, Johnson K, Puscas L. Facilitation and debriefing in simulation education. Otolaryngologic Clinics of North America
. 2017;50(5): 989–1001.
44. Wilbanks BA, McMullan SP. A review of measuring the cognitive workload of electronic health records. Computers, Informatics, Nursing
. 2018;36(12): 579–588.
45. King M. Developing a high-fidelity simulation program in a nursing educational setting. Health Care Management (Frederick)
. 2018;37(3): 235–249.
46. Hanberg A, Brown SC, Hoadley T, Smith S, Courtney B. Finding funding: the Nurses' guide to simulation success. Clinical Simulation in Nursing
. 2007;3(1): e5–e9.