Milano, Christina E. MD; Hardman, Joseph A. MD; Plesiu, Adeline MD; Rdesinski, Rebecca E. MSW, MPH; Biagioli, Frances E. MD
Dr. Milano is assistant professor, Department of Family Medicine, Oregon Health & Science University, Portland, Oregon.
Dr. Hardman is assistant professor, Department of Medicine, Oregon Health & Science University, Portland, Oregon.
Dr. Plesiu is an intern, Kettering Medical Center, Kettering, Ohio.
Ms. Rdesinski is a research associate, Department of Family Medicine, Oregon Health & Science University, Portland, Oregon.
Dr. Biagioli is associate professor, Department of Family Medicine, Oregon Health & Science University, Portland, Oregon.
Funding/Support: Dr. Milano, Ms. Rdesinski, and Dr. Biagioli were partially funded by an NIH grant supporting this study (National Cancer Institute grant 1R25CA158571-01A1; Integrating Patient Centered EHR and HIT Curriculum into BSS Medical Education; primary investigator, Frances Biagioli, MD).
Other disclosures: None reported.
Ethical approval: This study was approved by OHSU’s institutional review board.
Previous presentations: Portions of this work were presented at the Association of American Medical Colleges Western Regional Conference, Irvine, California, May 2013; at the 38th Annual Society of Teachers of Family Medicine Medical Student Education Conference, Long Beach, California, February 2012; at the Epic User’s Group Annual Meeting, Verona, Wisconsin, September 2011; and at the 37th Annual Society of Teachers of Family Medicine Medical Student Education Conference, Houston, Texas, January 2011.
Correspondence should be addressed to Dr. Milano, Oregon Health & Science University, 3181 SW Sam Jackson Park Rd., Mail Code: FM, Portland, OR 97239; telephone: (503) 418-3973; e-mail: firstname.lastname@example.org.
Studies have demonstrated that electronic health record (EHR) use improves the quality, efficiency, and cost of patient care.1,2 The Medicare and Medicaid EHR Incentive Programs, therefore, provide financial incentives to providers that “demonstrate meaningful use of certified EHR technology.”3 Although academic health centers are at the forefront of EHR implementation,4 most medical trainees receive little directed, uniform education in optimal use of EHRs.5 Further, because of concerns about liability, reimbursement/billing, and patient safety,6,7 medical students’ access to EHRs is often restricted,5 making them unable to document encounters, enter history, or create orders. This lack of education and access prevents students from learning to use EHRs in a “meaningful” way. Rather than being given opportunities to experience in simulated settings the mistakes and frustrations inherent in using EHRs, students often embark on a haphazard and variable learning curve in patient care settings, where their EHR education is directed by faculty with differing skill levels. Given this model, few students are likely to graduate feeling fully proficient or comfortable with their ability to use EHRs.
To address this training gap, Oregon Health & Science University (OHSU) implemented the Simulated EHR (Sim-EHR) curriculum in 2011. The aims of this innovative curriculum are to provide learners with a safe, supervised setting in which they place orders for a virtual patient in a simulated chart while also learning the fundamentals of chronic disease management and health care maintenance in the context of an EHR. In this article, we describe the Sim-EHR curriculum, including its development and implementation, and share participant feedback and lessons learned.
The OHSU Sim-EHR Curriculum
OHSU is located in a city and a state—Portland, Oregon—that both have high EHR adoption rates.8,9 OHSU implemented its ambulatory and inpatient EHR platforms in 2005 and 2008, respectively. All first-year OHSU medical students receive a brief EHR training session and are then granted access to write notes, enter history, and pend orders. However, faculty comfort with use of the EHR varies within OHSU, and training sites outside OHSU have different policies on student EHR access. In academic year 2011–2012, third-year OHSU medical students participating in focused interviews during their family medicine clerkships reported using three to six different EHRs during their rotations. Some students reported regular use of EHRs during their clerkships, yet others reported they had never written an order in an EHR.
Curriculum development and objectives
We developed the Sim-EHR curriculum in 2010 to transform the haphazard, exposure-based EHR education described above into a uniform EHR experience with defined objectives and clear assessment measures.
We identified EHR-related general and specific objectives for the curriculum using existing guidelines10 as well as input from OHSU faculty members responsible for EHR implementation (see Table 1). We designed the objectives to impart general concepts common to all EHRs rather than related to a specific EHR platform. In the Sim-EHR curriculum, trainees learn to maintain an accurate EHR (correct/update social, surgical, and family histories, allergy alerts, and problem and medication lists), apply prevention and chronic disease guidelines in the EHR context, and order appropriate referrals/consultations, laboratory tests, imaging studies, and medications. With these objectives in mind, we created a virtual patient and a simulated chart in which trainees can practice these skills.
Simulated chart and virtual patient
One Sim-EHR chart was created for use in an EHR training environment (EPIC Train) that is identical to the OHSU EHR system (EPIC Production). The simulated chart includes a virtual patient’s outpatient, emergency, operative, and consultation notes; medication list; laboratory, imaging, and pathology results; electrocardiogram (ECG) tracings; and pulmonary function test results. The chronic disease states we chose for the virtual patient include those that have a large body of evidence-based guidelines, including diabetes, hyperlipidemia, hypertension, and chronic obstructive pulmonary disease. We provided details of the medical and social histories to encourage learners to alter their recommendations on the basis of a patient’s individual risks. For example, to teach trainees to apply osteoporosis screening guidelines, we created a 60-year-old female virtual patient who has multiple osteoporosis risk factors in her family and medical histories. To teach clinical reasoning and data interpretation skills in the EHR context, we varied the presentation of medical issues: some are represented with results only (e.g., elevated fasting glucose levels), some are “buried” in the scanned documents (e.g., an ECG), and some appear as previously diagnosed in the notes. We included mistakes (e.g., incorrect entries) and omissions (e.g., no allergies listed) to provide training in identifying and correcting errors and to emphasize safety and professionalism in maintaining accurate patient records.
We tailored the simulated chart compo nents to delineate varying trainee skill levels. For example, an early learner may be able to correct the family history section using a prior clinic note; a higher-level trainee should be able to read an ECG and add a diagnosis of left ventricular hypertrophy to the problem list; and a more advanced learner should be able to identify the diagnosis of diabetes on the basis of elevated hemoglobin A1c and glucose levels and also place multiple orders for appropriate evidence-based care.
Implementation in family medicine clerkship
The Sim-EHR curriculum was implemented for OHSU medical students in January 2011 as part of the required third-year family medicine clerkship. At the beginning of the academic year, an IT specialist duplicates the simulated chart to create one copy per learner; each copy bears a unique patient name. During the first week of the five-week clerkship, students receive their virtual patient’s name and attend a half-hour orientation session where a faculty facilitator provides an overview of the activity. The facilitator instructs learners to imagine they are interns taking care of a patient who has been recently discharged from the emergency department and has received fragmented care over the last five years because of a lack of health insurance. Learners must review and correct the chart in preparation for meeting this patient, create an evidence-based plan of care for prevention/surveillance of the patient’s chronic diseases, and place indicated orders. At this session, the facilitator also distributes the learner guide, which outlines the objectives and grading guidelines and includes screenshots illustrating the step-by-step mechanics of manipulating a patient chart and placing orders.
Students are given two weeks to work in the chart. Small groups of six to eight students meet with a faculty facilitator during the second week of the clerkship for a two-hour session to discuss the first week’s work and the content of the “correct” chart (patient problem list, allergies, and history sections). They come to agreement on this content before beginning the next week of work in the chart, which involves editing the medication list, updating the health maintenance tool, and placing orders for future studies, referrals, and treatments (based on evidence-based guidelines). While the students are instructed to work individually on their charts during the first week, they are actively encouraged to work in groups for completion of the second week’s assignment. During the third week of the clerkship, the small groups assemble again for one hour to share the remainder of their work. Other topics touched upon in these sessions include the art of triaging care for complex patients, population-based EHR applications, and the professionalism, patient safety, and systems-based care reasons for excellent EHR chart hygiene habits.
To aid in these discussions, hard-copy summaries of each student’s chart work are available. In addition, the facilitator projects the open EHR onto a screen so that any student’s work can be displayed and reviewed by the group. To ensure consistent facilitation across groups, facilitators are asked to spend at least one hour prior to the first small-group session reviewing the simulated chart and the accompanying faculty guide, which outlines the session agenda and evidence-based recommendations for the virtual patient’s medical problems.
Students’ work in the virtual chart is assessed for this exercise, which counts for 5% of the family medicine clerkship grade. Faculty facilitators independently grade students’ work in the charts using a standardized rubric that addresses the three general objectives (see Table 1): chart maintenance skills (accuracy of history, allergies, and medication sections), knowledge of chronic disease management and disease prevention, and EHR skills (ability to place orders). For example, one point is given for each correct diagnosis included in the problem list. Students receive their simulated chart grades at clerkship completion. Those with questions about their grades are invited to review their work in the chart with a faculty member.
The EHR includes an audit trail with time and date stamps indicating when a student first starts work on the chart and documentation of the total amount of time spent on the exercise. Students who access the record within 24 hours of the start of the assignment receive a small increment of “extra credit” for their timely initiation of the work; they are made aware of this during the orientation session.
The main resource the Sim-EHR curriculum requires is faculty time to create and update the virtual patient/simulated chart, the grading rubric, and the learner and faculty guides. The curriculum also requires a core group of faculty small-group facilitators, one of whom will lead the orientation and be readily available to respond to learners’ questions during the week they are working on their charts. An IT specialist skilled in the EHR training environment is also needed to build and duplicate the simulated chart and address any IT issues that may arise. Importantly, the institution’s EHR system must include a segregated training environment that is not wiped clean of its data every day.
National Institutes of Health grant funding (NIH 1R25CA158571-01A1) supported faculty time for project development and startup (0.1 full-time equivalent [FTE]) and IT personnel time (0.2 FTE). This same grant funds ongoing IT support (0.1 FTE) and faculty evaluation and dissemination efforts (0.1 FTE). Getting support from OHSU and its hospital system was also critical. Early endorsement by the medical school dean and our academic medical center’s chief information officer helped ensure that students have remote access to the EHR training environment.
Chart completion to date
The curriculum remains part of the required family medicine clerkship. As of January 2014, three years after implementation, 406 students had completed simulated charts, both on campus and at remote clerkship sites in Oregon. The curriculum was also implemented over four hours of the OHSU internal medicine clinic orientations in July 2012 and July 2013; as of January 2014, 21 interns had completed simulated charts.
Small-group faculty facilitators met regularly from January to December 2011 to compare content discussed during and overall experiences of the small groups. The facilitators described students as actively engaged in the group discussions and identified the following common discussion points across groups: differing opinions regarding which diagnoses are chronic diseases as well as how and what to prioritize given the complexity of the patient’s issues; frustration over inadequacies of certain EHR functions; and uncertainty around how to respond to electronic queries when completing orders. They also reported that students were surprised as they discovered from one another the elements of the patient’s chart they had missed in their own reviews. On the basis of this feedback, we made some revisions to the faculty and learner guides.
In August 2012, we e-mailed survey invitations to the 129 students and 12 IM interns who had by then completed the exercise to explore their perceptions of the Sim-EHR curriculum’s educational effectiveness. All of the students had learned their final grade in the clerkship. The survey, which was completed anonymously and hosted at SurveyMonkey, contained two questions regarding effectiveness of the activity and the facilitation.
All 12 interns and 67 (52%) of the students responded. The low response rate among students may be attributable to the long period of time (up to 18 months) between completion of the activity and receipt of the survey. Among respondents, about half (51%) of the students and almost all (92%) of the interns rated the activity as “effective” or “very effective”; the remaining 49% of students were evenly split between ratings of “neutral” and “ineffective.”
On the basis of student comments on the survey, it appears that many students felt more comfortable with finding information, inputting orders, and updating a health maintenance tool after completing the Sim-EHR curriculum. Some students recognized the value of the activity (“Vastly improved my chart navigating and documentation skills”; “I think EHR navigation is a huge part of the ‘job’ of being a doctor, and it should be explicitly taught like this”; “Helpful in preparing us for residency”). Others, however, expressed frustration with its timing (“This should be part of transition to clerkship, not the [family medicine] rotation”; “Probably most instructive for those who are in the first few blocks of the year”) and its opportunity costs (“Too time consuming for pay off”; “Please just let us have time to study”).
Identifying strengths and gaps in the status quo
In September 2012, we conducted a preliminary analysis of student score data. When comparing data for students who completed the Sim-EHR exercise early versus later in their third year, we identified modest improvement in the more experienced students’ ability to place orders and update the chart, including the allergy section and medication list. However, we found no difference between the earlier and later students’ ability to use a health maintenance tool to create routine disease screening, prevention, and management alerts. We also found no difference in performance between all of the students and incoming interns, but we had data for only a small number of interns.
Such an analysis of score data may serve as a proxy for prioritizing which EHR skills need to be explicitly taught prior to graduation. We found universally low performance among both students and interns on specific objectives related to the knowledge of disease prevention, which suggests that there may be gaps in the predoctoral curriculum relating to the use of clinical decision support tools and the identification of evidence-based prevention guidelines. During our small-group sessions, students practice manipulating commonly used online tools and guidelines such as the World Health Organization’s Fracture Risk Assessment Tool (FRAX; http://www.shef.ac.uk/FRAX) and the University of Edinburgh’s cardiovascular risk calculator (http://cvrisk.mvm.ed.ac.uk/calculator/calc.asp). Potential enhancements to the Sim-EHR curriculum include placing an increased emphasis on training students to use these and other similar decision support tools in tandem with their EHR work.
Initially, students had three weeks to complete the Sim-EHR chart work, which was due in week four of the five-week clerkship. Audit trail data revealed that a majority of students completed the work one to seven days before the due date (i.e., they did not use the full period). Students participating in a June 2011 focus group reported that the timing of the Sim-EHR chart work competed with their final exam study time. Faculty facilitators reported that variable preparation by students, some of whom did minimal chart work, decreased the effectiveness of the small-group discussions. As a result of the audit trail analysis and comments, we changed the duration and timing of the activity. As noted above, Sim-EHR assignments are now distributed in week one of the clerkship, and the small-group discussions are held in weeks two and three. There is likely a point of diminishing returns with regard to time spent charting and resultant improved performance and educational benefit.
The Sim-EHR exercise was initially graded pass/fail. Six months after implementation, because of variable student completion and lack of small-group preparation as described above, the significance of this activity was increased. As indicated above, it now represents 5% of the family medicine clerkship grade. This change resulted in overall improvement in performance, effort, and group session participation.
Looking Forward and Conclusions
Our experience with the Sim-EHR curriculum thus far is limited to third-year OHSU medical students and a small group of OHSU internal medicine interns. As of January 2014, two other academic health centers in the United States are planning to implement the Sim-EHR curriculum. At OHSU, we plan to expand the curriculum to family medicine interns and physician assistant students. We are considering methods for identifying control groups so that we may better evaluate the effectiveness of the intervention. In addition, we recently experimented with larger student groups (17–20 students) during the chart review sessions. We found that this resulted in richer discussions and opportunities for “team teaching”; it also reduced the required number of faculty facilitators. We expect to continue experimenting with group size and venue moving forward.
The Sim-EHR curriculum can be adapted to teach many aspects of medical care and for use with learners at many different levels. For example, a simulated chart could be created to teach about liver or renal disease, or a longitudinal aspect of care could be added. Elements of the exercise could be assigned by year of training. For example, first-year medical students could identify chart omissions and errors, second-year students could create problem lists and diagnoses based on test results, and third-year students could return to the chart to apply evidence to “treat” the patient.
In conclusion, we believe the Sim-EHR curriculum has a wide range of potential applications in clinical environments. While EHR use can decrease error and provide systems for delivering quality care, EHR adoption brings new challenges that can be addressed with sufficient hands-on training. The Sim-EHR curriculum is a way to reinforce, in a safe learning environment, important behaviors required for maintaining a well-organized chart that reflects current standards for chronic disease and routine prevention.
Acknowledgments: The authors wish to thank Gretchen R. Scholl, Benjamin Schneider, MD, Shawn Blanchard, MD, William Toffler, MD, and David Thomas.