The extensive implementation of electronic health record (EHR) systems has altered how medical information is recorded and how physicians interact with patients.1 Consequently, medical students now must learn how best to use an EHR for effective documentation of clinical encounters. In particular, proper entry of patient notes and orders into EHRs represents an important clinical skill that students ought to develop and practice in medical school. Yet, little is known about student experiences with EHRs during their clinical education from students themselves. Moreover, shifts in student experiences with health records over time, particularly during the time period when EHR use dramatically increased, have yet to be examined.
Underscoring the need for students to learn to use EHRs during their clinical education, the Alliance for Clinical Education (ACE) issued a policy statement arguing that medical educators should determine the competencies associated with effective EHR use.2 Subsequently, the Association of American Medical Colleges defined documentation of clinical encounters in either written or electronic format as a core entrustable professional activity (EPA) for entering residency.3,4 More specifically, EPA 5 summarizes the competency for documenting a patient encounter in the record, and EPA 4 focuses on the ability to enter and discuss orders and prescriptions.4 Such official recognition of the importance of student use of EHRs, particularly in terms of opportunities for students to enter notes and orders, codifies the need for student experience with EHRs during medical school.
Historically, various barriers have limited student use of EHRs during their medical education. Among them are (1) hospital policies prohibiting students from entering orders, often due to patient care and liability concerns; (2) software-user license restrictions that limit trainee use of EHRs in some hospital settings; and (3) guidelines from the Centers for Medicare and Medicaid Services (CMS) centered on billing and reimbursement issues. Examining student exposure to EHR systems in light of these challenges provides a way to highlight potential areas where educational opportunities that facilitate student use of EHRs may be limited.
Previous studies of student EHR use tend to rely on reports of medical educators rather than students and/or focus on individual clerkships or institutions. One study surveyed U.S. clerkship directors and found that 64% of institutions allowed student use of EHRs, but one-third of those institutions did not allow student documentation within the record.5 Another study surveyed U.S. and Canadian medical school deans and revealed a similar pattern—roughly one-half of the hospitals associated with the medical schools in the study indicated that students could enter notes into EHRs.6 A recent article highlights survey findings showing high overall levels of student access to EHRs, but wide variation in the type of access across settings; for example, 33% of nonhospital ambulatory care centers provided read-only access during the 2013–2014 academic year, whereas 16% of university-owned hospitals did so during the same time period.7 A survey of clerkship directors showed that 58% of internal medicine programs used EHR systems, with students entering information into the record in about one-half of those programs.8
Two recent specialty-specific studies shed additional and consistent light on the topic.9,10 A survey of family medicine preceptors in the outpatient setting found that although most preceptors allowed students to access EHRs, only 62% permitted students to enter notes.9 A second recent study surveyed emergency medicine clerkship directors and found that 63% of clerkships allowed students to document clinical encounters in EHRs.10 In contrast to these findings and unique in its focus on students’ perspectives, a survey of third-year students at a single institution showed that the vast majority of students—97%—indicated that they either frequently or always use EHRs to document clinical encounters.11
Recent increases in EHR use raise questions about the content and scope of EHR training in medical school, with potential implications for safe and effective patient care. Some medical schools have begun to develop formal curricula for students to learn how to effectively use the EHR while preserving strong doctor–patient relationships.7,12–14 To inform how medical students are taught to develop and practice the underlying knowledge and skills associated with competent EHR use, a better understanding of the degree to which students currently use EHRs, for what specific activities they use them, and how these patterns have changed over time is required. Despite identification of documentation as an essential clinical skill that should be learned in medical school,2–4 actual student interface with and entry of information into EHRs has not been extensively explored from the student’s vantage point using large national samples. The purpose of this study is to fill this gap by examining student accounts of EHR use during a time period in which implementation of EHR systems dramatically increased. Students’ reports of their paper health record (PHR) use are also examined to aid in interpreting findings related to student use of EHRs.
Data for this study come from an online survey administered immediately after examinees completed the Step 2 Clinical Knowledge (CK) portion of the United States Medical Licensing Examination (USMLE). This postexamination survey includes sections related to the students’ examination experiences and other thematic sections related to contemporary issues in medical education and training. Multiple forms of the survey were used, with one including questions about students’ use of EHRs and PHRs during inpatient experiences in six core clinical clerkships. These clerkships are typically completed during the third year of medical school and include family medicine, internal medicine, obstetrics–gynecology, pediatrics, psychiatry, and surgery. During the five consecutive years that the survey was administered, survey forms were randomly assigned to students taking the Step 2 CK examination such that 25% of examinees received the health record survey from August 2011 to July 2014, and 50% of examinees received it from August 2014 to July 2016.
For both EHRs and PHRs, the survey included questions related to whether or not students participated in the following health record activities during their clerkships: (1) used a health record for review or entry of patient information and (2) entered notes or orders that became part of the official health record after appropriate authorization. Medical students who indicated that they entered notes or orders were asked about the types of notes or orders they entered, including (1) admission history and physical examination notes, (2) progress notes, (3) admission orders, and (4) postadmission orders.
The initial dataset included 83,301 medical students from 142 U.S.-based Liaison Committee on Medical Education (LCME)-accredited medical schools/campuses who planned to graduate between 2012 and 2016 and completed the USMLE Step 2 CK examination. A subset of 17,202 students who received the health record survey and provided valid responses to the health record questions were selected from this dataset for analysis purposes.
Descriptive statistics were computed for responses to all health record questions, and analysis of variance (ANOVA) techniques were used to evaluate changes in health record use by graduation year. Several steps were taken to create the final dataset from which the analyses were done. First, six binary variables were created to represent the six health record activities asked about in the survey (used a record, entered information into a record, entered admission history and physical examination notes, entered progress notes, entered admission orders, and entered postadmission orders). This was done for each clerkship separately by health record format, resulting in a total of 72 dichotomous variables per student (2 record formats * 6 activities * 6 clerkships). In all cases, the reference category indicated “did not” engage in the activity.
Next, these dichotomous variables were averaged across clerkships for each student, yielding 12 new variables representing the proportion of clerkships in which a student engaged in specific health record activities during their clinical training; 6 of these variables focused on EHR use and 6 on PHR use. Proportions were then averaged across students and examined by expected graduation year. For the survey question asking about entry of information into a record, proportions also were averaged across only students who indicated that they used a record. All proportions were then transformed into percentages. Differences in average percentages for graduation years 2016 and 2012 were calculated, and one-way ANOVAs were used to test whether changes were statistically significant. All analyses were performed using IBM SPSS Statistics Version 23.0 (IBM Corp., Armonk, New York). This study was reviewed by the American Institutes for Research Institutional Review Board and qualified for exempt status because it involved very minimal or no risk to study subjects.
The final sample used for analysis included 17,202 students from 142 LCME-accredited medical schools reflecting a broad range of clinical curricula and hospital settings. These students took the Step 2 CK examination for the first time between August 2, 2011 and July 13, 2016 under standard testing conditions and expected to graduate medical school between 2012 and 2016. Of the original 83,301 students, 25% of 55,454 (13,864) and 50% of 27,847 (13,924) were randomly assigned to the health record survey. Thus, the response rate for the health record survey was 62% (17,202/27,788).
Table 1 provides a comparison of mean percentages of inpatient clerkships in which a student used and entered information into EHRs and PHRs by graduation year. As shown, students used EHRs during the majority of their inpatient clinical clerkships in all graduation years. Over the study period, the mean percentage of inpatient clerkships in which a student used an EHR increased significantly from 78% to 93%. With respect to PHRs, Table 1 indicates that, on average, a much lower percentage of students engaged with PHRs during their inpatient clerkships and that the mean percentage of clerkships in which a student interfaced with a PHR declined significantly from 34% to 11%.
Table 1 further indicates that the mean percentage of inpatient clerkships in which a student entered information into an EHR increased from 59% to 72% over the study time period. Examined differently, of the students who indicated that they used an EHR, the percentage of clerkships in which they entered notes or orders remained constant at 76%. For PHRs, the patterns are different. For all students, the mean percentage of clerkships in which a student entered notes or orders decreased from 26% to 7%, and for students who indicated that they used a PHR, the percentage decreased from 79% to 69%.
Panel A of Figure 1 represents the mean percentages of inpatient clinical clerkships in which a student used and entered information into health records by graduation year. These averages are based on all student responses. The mean percentage of clerkships in which a student used an EHR is greater than the mean percentage of clerkships in which a student entered information into an EHR for each graduation year, and the rate across graduation years at which both student use of and entry of information into EHRs changed appears similar. For each graduation year, the mean percentages of clerkships in which a student used and entered information into PHRs is much lower than the corresponding percentages for EHRs. Unlike EHRs, use of and entry of information into PHRs have both declined.
Panel B of Figure 1 also shows the mean percentages of inpatient clerkships in which a student used and entered information into health records by graduation year, but the entry percentages displayed in Panel B are based on only students who indicated that they used a record. Of note is the increasing gap between the mean percentage of clerkships in which a student used an EHR and the mean percentage of clerkships in which a student entered information into an EHR. While EHR use increased over the study period, the mean percentage of clerkships in which a student entered information into EHRs remained stable. For students who used a PHR, the mean percentage of clerkships in which a student entered information into a record declined slightly over time.
When the specific types of notes or orders that students entered are examined, the picture becomes more complicated. Table 2 provides mean percentages of inpatient clerkships in which a student entered different types of notes and orders into EHRs and PHRs by graduation year. The mean percentage of clerkships in which a student entered notes into an EHR increased from 45% to 54% for admission history and physical examination notes, and from 50% to 63% for progress notes. Students reported entering admission orders and postadmission orders into EHRs in less than a quarter of their inpatient clerkships. From 2012 to 2016, mean percentages decreased by 5% for admission orders and 4% for postadmission orders.
For PHRs, students did not enter admission history and physical examination notes or progress notes in the majority of their inpatient clerkships, with decreases over time. The mean percentage of clerkships in which a student entered notes into PHRs decreased significantly from 2012 to 2016; for admission history and physical examination notes it decreased by 14%, and for progress notes it decreased by 17%. This decline was lower for admission and postadmission orders, with decreases of 6% and 7%, respectively.
Figures 2 and 3 display the mean percentages of inpatient clerkships in which a student entered notes or wrote orders by type of note/order and graduation year. Figure 2 shows this information for EHRs and illustrates how student entry of notes during their clinical education increased over time, whereas student writing of orders declined. Figure 3 shows the same information for PHRs and illustrates that the mean percentages of clerkships in which a student engaged in all four of these activities has fallen over time. It further shows how students did not enter notes and, to an even lesser extent, orders in the vast majority of their clerkships.
Focusing on the inpatient component of clinical training in core third-year clerkships, this study finds that medical students used EHRs in the majority of their clerkships and that this use increased from 2012 to 2016. This is anticipated given the increased use of EHR systems in clinical settings and expectations that medical students learn to appropriately use EHRs to ensure safe and effective patient care. There also has been an increase in student entry of information into EHRs, but for students who indicated that they used an EHR, the mean percentage of clerkships in which they entered information into EHRs has remained constant. This may reflect the notion that policies or practices regulating student permissions for EHR use and entry of information during inpatient clerkships have not changed despite increased EHR use in the clinical environment. Not surprisingly, the mean percentage of clerkships in which a student used a PHR was low and decreased over time, likely reflecting replacement of PHR systems with EHR systems in inpatient settings.
One important finding relates to the mean percentage of clerkships in which a student reported read-only access to EHRs. Specifically, in 2016, read-only access was reported for 21% of student inpatient clerkship experiences (16% when examined by only those students who indicated they used a record). The mean percentage of clerkships in which students entered notes remained below 65%, and, on average, students wrote orders in fewer than 20% of their clerkships. Notably, the mean percentage of clerkships in which a student entered orders actually decreased significantly over the study period.
These findings suggest that students may not be receiving sufficient experience with order-writing or note-writing skills in particular during their inpatient clerkships. Consequently, they could be missing a significant educational opportunity to learn and practice an important skill needed in graduate medical education and subsequent practice. Indeed, one study found that deans view medical student notes as a valuable part of the patient record, serving to enhance student education and a patient care team’s ability to provide effective medical care.15 As mentioned, possible issues contributing to low levels of student entry of information into EHRs include health system policies, EHR system capabilities, and billing issues. Regardless of the reason(s), the potential for students to have received inadequate educational experiences during their third-year clinical education remains. Although some students may have the opportunity to develop EHR skills during their fourth-year rotations, it seems important that students receive experience engaging with EHRs during their core clerkships as fourth-year experiences are generally quite variable.
As EHRs become an integral part of the U.S. health care system, medical educators will need to effectively incorporate EHR training into medical school curricula to ensure patient safety and high-quality care. ACE proposed comprehensive practice guidelines2 for student documentation in EHR systems, and EPAs specifically address these competencies.3 The results of this study reveal patterns in student EHR use at the national level as indicated by students themselves. These findings highlight possible areas in need of increased attention and may help to inform discussions about the appropriate scope of student EHR use, the expectations of successful student utilization of EHR systems, and the challenges of training students to use EHRs. Notably, a recent modification to the requirements for student clinical documentation put forward by the CMS may offer greater opportunity for students to document in EHRs.16
One limitation of this study is that all students did not provide survey responses for all questions. Survey responses, therefore, included small amounts of missing data. There is no clear theoretical reason to think that the health record experiences of this student group vary in meaningful ways from those who completed all survey questions, as the missing data are likely due to time constraints associated with the survey format. Although a formal analysis of students with missing data was beyond the scope of this descriptive study, future research could examine this group more closely. Variation in EHR use by medical school was not addressed, yet it is reasonable to assume that student use of EHRs may differ in meaningful ways both within and across institutions. Indeed, preliminary analyses of the data used in this study revealed school-to-school variation in student EHR use. Because this may reflect variation in policies and functionality across schools and hospitals, which could pose possible impediments to student EHR use,7 future research should address this area. Additionally, this study did not address health record use in outpatient environments during core clerkships. Although other studies have focused on outpatient settings for specific disciplines at selected institutions,9,17 additional future research is still needed using national samples. Similarly, this study did not address educational experiences after core inpatient clerkships, and future study is needed to better understand these types of experiences as well. Focusing on the inpatient setting in core third-year clinical clerkships, the present study provides information regarding student accounts of their EHR use during core inpatient clinical clerkships for a national sample of U.S. medical students, with potential implications for enhancing medical school curricula. Although documentation of health records is recognized as an important medical student competency, and EHR systems now represent a very common health record format, findings suggest that students may receive inadequate experience with EHRs, specifically with respect to order and note writing. This in turn could translate into a lack of preparedness for future training and practice.
1. Lown BA, Rodriguez D. Commentary: Lost in translation? How electronic health records structure communication, relationships, and meaning. Acad Med. 2012;87:392394.
2. Hammoud MM, Dalymple JL, Christner JG, et al. Medical student documentation in electronic health records: A collaborative statement from the Alliance for Clinical Education. Teach Learn Med. 2012;24:257266.
3. Association of American Medical Colleges Medical Education Cluster. Core Entrustable Professional Activities for Entering Residency (CEPAER). 2013.Washington, DC: Association of American Medical Colleges;
4. Association of American Medical Colleges. Core Entrustable Professional Activities for Entering Residency Curriculum Developers’ Guide. 2014.Washington, DC: Association of American Medical Colleges;
5. Hammoud MM, Margo K, Christner JG, Fisher J, Fischer SH, Pangaro LN. Opportunities and challenges in integrating electronic health records into undergraduate medical education: A national survey of clerkship directors. Teach Learn Med. 2012;24:219224.
6. Friedman E, Sainte M, Fallar R. Taking note of the perceived value and impact of medical student chart documentation on education and patient care. Acad Med. 2010;85:14401444.
7. Welcher CM, Hersh W, Takesue B, Stagg Elliott V, Hawkins RE. Barriers to medical students’ electronic health record access can impede their preparedness for practice. Acad Med. 2018;93:4853.
8. Mintz M, Narvarte HJ, O’Brien KE, Papp KK, Thomas M, Durning SJ. Use of electronic medical records by physicians and students in academic internal medicine settings. Acad Med. 2009;84:16981704.
9. White J, Anthony D, WinklerPrins V, Roskos S. Electronic medical records, medical students, and ambulatory family physicians: A multi-institution study. Acad Med. 2017;92:14851490.
10. Wittels K, Wallenstein J, Patwari R, Patel S. Medical student documentation in the electronic medical record: Patterns of use and barriers. West J Emerg Med. 2017;18:133136.
11. Heiman HL, Rasminsky S, Bierman JA, et al. Medical students’ observations, practices, and attitudes regarding electronic health record documentation. Teach Learn Med. 2014;26:4955.
12. Duke P, Frankel RM, Reis S. How to integrate the electronic health record and patient-centered communication into the medical visit: A skills-based approach. Teach Learn Med. 2013;25:358365.
13. Wald HS, George P, Reis SP, Taylor JS. Electronic health record training in undergraduate medical education: Bridging theory to practice with curricula for empowering patient- and relationship-centered care in the computerized setting. Acad Med. 2014;89:380386.
14. Silverman H, Ho YX, Kaib S, et al. A novel approach to supporting relationship-centered care through electronic health record ergonomic training in preclerkship medical education. Acad Med. 2014;89:12301234.
15. Friedman E, Sainte M, Fallar R. Taking note of the perceived value and impact of medical student chart documentation on education and patient care. Acad Med. 2010;85:14401444.
16. Department of Health and Human Services. Centers for Medicare and Medicaid Services. Medicare claims processing. https://www.cms.gov/Regulations-and-Guidance/Guidance/Transmittals/2018Downloads/R3971CP.pdf
. Published February 2, 2018. Accessed April 27, 2018.
17. Rouf E, Chumley HS, Dobbie AE. Electronic health records in outpatient clinics: Perspectives of third year medical students. BMC Med Educ. 2008;8:13.