With the increasing focus on the dissemination of health information technology (HIT) and the electronic health record (EHR), questions regarding the impact of these technologies on clinical practice become of paramount importance. Studies conducted thus far outline both the benefits of HIT and EHRs, such as reduction in medication errors and improved patient outcomes,1,2 and their unintended consequences, such as reduced efficiency, lower quality of care, increased possibility of medical errors,3,4 and disruptions in clinicians’ workflows, particularly if the design of the HIT or EHR does not match clinical work practices.5
Despite the growing importance of HIT and EHRs, the actual patterns of clinicians’ use of these technologies remain poorly understood. Many studies do not discriminate between different types of tasks clinicians perform using electronic systems or only focus on particular types of HIT, such as computerized provider order entry3,6,7 and electronic documentation.8,9 However, the rich functionality of modern EHRs suggests that a wide variety of tasks related to patient care may now be performed using computing systems. Moreover, new generations of clinicians increasingly rely on computing and information technologies in different aspects of their lives. The question then becomes how these increasing expectations and experiences with technologies change the way clinicians practice medicine, engage with patients, and carry out their professional duties.
Previous studies of resident workflows suggested that physicians in training spend little time on direct patient care and the majority of their time on educational and administrative tasks.10,11 These observations remained consistent for several decades,12,13 leading to a standing concern about residents’ time allocation. The introduction of HIT and EHRs inspired an ongoing debate in the medical community, not only about their benefits and limitations14,15 but also about their impact on how residents spend their time, particularly with regard to the proportion of time residents spend on direct patient care.10 The study reported here contributes to this discussion by investigating the use of computing and information technologies by residents in inpatient care settings. The goal of the study was to understand not only how much time residents spend using computers compared with other activities but also what residents use computers for, thus providing a snapshot of the role of HIT in modern patient care. To accomplish this, we conducted a time and motion study of medical residents at a large urban teaching hospital. To add to the previous research in this area, the study expanded on a validated taxonomy of clinical activities by introducing a set of fine-grained categories describing computer use that allowed us to examine these activities with an unprecedented level of detail.
We conducted this time and motion study in June and July 2010 at NewYork-Presbyterian/Columbia University Medical Center (NYP/CUMC), a large urban teaching hospital in New York, New York. NYP/CUMC had over 2,300 beds and discharged over 110,000 patients in 2009, with an average length of stay of 6.4 days.16
We conducted this study with residents (first-, second-, and third-year) on the general medicine service. Patient care on the general medicine service was practiced in teams; the physicians on each team included an attending physician, a fellow, a second- or third-year resident, a first-year resident (intern), and a medical student.
In 2004, NYP/CUMC deployed a commercial EHR system (Allscripts Sunrise, Allscripts, Chicago, Illinois). Before that, NYP/CUMC used WebCIS, which was developed in-house. The Allscripts Sunrise EHR system included a number of modules, separated into tabs (e.g., results, flowsheets, orders). Most licensed independent practitioners entered their notes directly into the EHR via a keyboard and mouse as opposed to using dictation.
In addition to the EHR, all residents had two-way pagers, which were used as the main means of communication on the general medicine service. No work-related mobile or handheld devices were actively used during the study.
At the time of the study, 16 residents rotated on the general medicine service (8 second- or third-year residents and 8 interns). Four (25.0%) of these residents participated in a pilot study, conducted immediately prior to the study described here. Another 7 (43.8%) participated in the final data collection study—3 (42.9%) of these participants were interns, who began their first rotations 3 to 4 weeks prior to the study, and 4 (57.1%) were second- or third-year residents. The 7 participants were randomly selected from the pool of rotating residents with the assistance of chief residents. All residents who were approached agreed to participate in the study, and there were no withdrawals. Each participant was observed for the entire duration of one of his or her shifts. The participants were introduced to the purpose of the study (time and motion study of their activities); there was no compensation for participating.
This study used a time and motion design to allow us to capture data on clinical activities with a high level of detail.10 During the course of the study, an experienced observer (L.M.) shadowed each participant for the 7 to 14 hours of their day shift (typically starting between 7:00 am and 8:00 am and ending between 3:00 pm and 9:00 pm). We did not include night shifts in the study. While shadowing participants, the observer captured all their activities using a custom-developed iPad application. Notably, the application allowed us to capture multiple activities simultaneously to account for multitasking (e.g., when participants were viewing a patient’s record while talking on the phone). The observer had extensive expertise with qualitative research methods, including observations of clinicians’ use of electronic documentation systems17 and clinical work practices.18
During the data collection sessions (i.e., the residents’ shifts), the observer kept detailed field notes describing the captured activities and the context of these activities. We conducted informal interviews with study participants as member checks to confirm our findings and interpretations of the findings. The interviews were conducted during the days following the observations as one-on-one conversations and lasted between 15 and 45 minutes.
The study was approved by the institu tional review board of Columbia University Medical Center; all participants consented to participate in the study prior to observation.
Taxonomy of clinical activities
For this study, we used a modified tax onomy of clinical activities developed by Overhage et al19 and refined by Pizziferri et al,20 which we expanded to allow for a fine-grained examination of computer-based activities. Specifically, we added six new subitems to the looking-up-data category—viewing the patient list, viewing flowsheets, viewing results, viewing imaging data, viewing general patient data, and viewing a visualization of patient data. In addition, we introduced several new subitems describing different activities related to managing patient care (e.g., managing handoff to-do lists and clearing to-do flags). Because dictation was not used, we did not include the category of reviewing dictation in our taxonomy. The final breakdown of the computer-based activities category is given in Table 2.
The analytic approach used in this study was inspired by the work of Zheng et al5 that described multiple analytical tools to visualize and uncover hidden regularities embedded in the sequential execution of patient care tasks in a clinical workflow. The frequencies and durations of activities were calculated using Excel (version 14.5.8 for Mac, Microsoft, Redmond, Washington). The activity visualization tool used in Figure 1 was custom developed using the D3 visualization package.21 The aggregated activity view (heat map visualization) used in Figure 2 was generated using an Excel stacked bar chart feature. Finally, we reviewed all field notes taken during observations and grouped them in accordance with the taxonomy based on the category of activities they referred to. We also conducted a thematic analysis of notes and transcripts from interviews. These notes, transcripts, and themes were used to provide context and explanations for data captured with the time and motion study.
General patterns of activities
The total times spent by residents on different clinical activities per shift are presented in Table 1. On average, out of 720.2 minutes of shift time, the residents spent 364.5 minutes (50.6%) using computers (Computer read/write); in comparison, they spent 67.8 minutes (9.4%) interacting with patients (Patient). After computer-based activities, the second most time-consuming activity residents did during their shift time was talking with others in person (Talking, 292.3 minutes [40.6%]); this included conversations about both patient care and general social topics. In addition to reading and writing notes on the computer, residents spent a quarter of their shift time (186.0 minutes [25.8%]) handling paper (Paper read/write), primarily printouts of the electronic sign-out note; residents in the study often printed this note in the morning and used it throughout the day to capture updates and handwritten to-do lists. The residents spent just over an hour of their shift time in rounds (Rounds, 79.7 minutes [11.1%]). Because the general medicine service covered several floors of the hospital building, the residents spent a considerable amount of their shift time walking or waiting (Moving/waiting, 80.0 minutes [11.1%]). The residents spent 54.0 minutes (7.5%) of their shift time talking on the phone (Phone); most of this time was spent in consultations and on managing patients’ discharges. The residents spent 22.8 minutes (3.2%) of their shift time engaged in nonwork or personal activities (Personal). The remaining activity category, Looking for (used for activities related to searching for documents or people), was insignificant and took 2.4 minutes (0.3%) of residents’ shift time. Notably, because the study allowed for the capturing of different activities that happened at the same time, the total time of captured activities (1,149.4 minutes) exceeded the total observation time per shift (720.2 minutes on average) by 429.2 minutes (59.6% of shift time), suggesting that 59.6% of residents’ time was spent multitasking. Figure 1 shows aggregated times spent on different activity categories for each participant.
Table 2 presents a more detailed breakdown of the residents’ computer-based activities per shift. On average, over half of this time, 189.9 minutes (52.1%) was spent on documentation (writing, viewing, and reading notes). The residents spent considerably more time writing their own notes (128.7 minutes [35.3%]) than reading notes composed by others (27.3 minutes [7.5%]). Notably, the residents spent another 33.9 minutes (9.3%) viewing the list of available notes. This most commonly occurred in situations where residents were expecting a note (from an attending physician or a consultant) and were checking to see whether the note had been made available.
On average, the residents spent over an hour (70.8 minutes [19.4%]) looking up patient data. This time included viewing the patient list, a necessary step when switching between patients (25.5 minutes [7.0%]); viewing flowsheets, which covered patient vital signs, intakes and outputs, respiratory parameters, and nursing assessments (25.1 minutes [6.9%]); viewing results, which included laboratory test results and radiology, cardiology, and pathology reports in either the new or legacy EHR system (15.9 minutes [4.4%]); viewing imaging data (3.3 minutes [0.9%]); viewing general patient data (0.7 minutes [0.2%]); and viewing a visualization of patient data, a custom-developed feature, which allows patient data to be viewed on a timeline (0.3 minutes [0.1%]). The residents also spent 48.1 minutes (13.2%) managing orders, including viewing (23.9 minutes [6.6%]) and writing (24.2 minutes [6.6%]) orders, and just under 10 minutes using computer-mediated communication, such as paging (9.2 minutes [2.5%]) and e-mail (0.4 minutes [0.1%]). Other activities, related to logistics, managing to-do lists, reference look-up, and managing medications, each took less than 10 minutes on average. Finally, the residents spent 24.0 minutes (6.6%) on computer-based activities not included in the taxonomy; these were classified in the Other category.
Figure 2 shows a heat map visualization of the activities captured for each participant during the observed clinical shifts. As is evident from the visualization, patient interactions (black) happened mainly in three different ways: as part of the morning prerounding activities, after rounds (usually to carry out the established care plans), and sometimes before the end of the shift. The figure also shows that there was a visible difference in time spent with patients between interns and residents; the informal interviews confirmed that the participants perceived interns as being primarily responsible for direct patient contact, unless it involved procedures that required a higher level of skill or expertise. Both computers and paper were most commonly used in the morning while preparing for rounds, after rounds to update care plans, and later in the day to document changes in patients’ conditions and to check off completed to-do list items. In the morning hours, the use of computers was tightly coupled with the use of paper. During this time, the residents copied important bits of data from the EHR onto their paper notes to have this information available for rounds. In the afternoon, however, the use of computers was tightly interspersed with communication, as residents often engaged in discussions with others in person and on the phone while using computers.
In this study, we examined how residents (first-, second-, and third-year) on the general medicine service in a large urban teaching hospital spent their shift time, with a particular focus on their use of computers. Consistent with previous reports, our study showed that residents spend considerably more time interacting with computers (over 50% of their shift time) than in direct contact with patients (less than 10% of their shift time). The difference we found, however, was even more pronounced than what had been reported in earlier studies (e.g., 40% computer use and 12% patient care10). The study may have identified two factors that might contribute to the high level of computer use: inefficiencies in the design of the EHR system and an increasing reliance on computing systems for access to patient data.
Inefficiencies in the design of the EHR system
First, the study indicated that a signif icant portion of the computer-based activities was dedicated to documentation (128.7 minutes, 35.3%). This finding is higher than similar findings in previous reports (e.g., 21% of time spent on documentation22,23), including a previous report from our institution.24 This observation raises a question about whether electronic documentation is inevitably time consuming and burdensome or whether there are limitations in the design of the current electronic documentation systems that inflate documentation time. This study may have highlighted several aspects of electronic documentation that contributed to inefficient use of time spent documenting. For example, residents spent 33.9 minutes (9.3%) of their shift time viewing the list of available notes rather than reading them. Our field notes suggested that these situations often occurred when residents were waiting for an attending physician to “drop” their progress note, which would make the discussed care plan official, before proceeding with the planned activities. Because the EHR did not alert the residents to newly posted notes, they had to periodically look for updates. In addition, the high degree of fragmentation in the organization of the patient record, reflected in the six distinct items within the looking-up-patient-data category (which corresponded to six different areas of the EHR containing patient data), may have required residents to spend a considerable amount of time consolidating data from these different areas. Finally, the reliance on desktop computers and their positioning away from patients made it impossible for clinicians to integrate computer-based activities with more direct patient care activities.
Increasing reliance on computing systems for access to patient data
Second, the study suggested that half of residents’ computer-based activities time was spent on activities not related to documentation but, rather, to reviewing patient data (looking up patient data, 70.8 minutes [19.4%]); managing and coordinating patient care (managing orders, 48.1 minutes [13.2%], and communicating, 9.6 minutes [2.6%]); and other activities (other, 24.0 minutes, [6.6%]). These findings suggest that delivery of patient care necessitates frequent updates to orders and to-do lists to allow members of patient care teams to carry out their respective responsibilities in an efficient and effective manner. Moreover, during the informal interviews, residents who previously completed specialty rotations, such as nephrology, reported decreased reliance on patient contact and increased reliance on information stored within the EHRs (e.g., the available laboratory test results). With the growing amount and richness of patient data available only through computing systems, these trends are likely to amplify. Questions remain, however, as to the role of patients in helping clinicians to interpret these data and ways in which computing systems can help facilitate engagement between patients and clinicians, rather than diminish it.
These findings further reinforce the importance of continuous focus on improvements to the design and usability of EHRs.25,26 They also suggest the need to reenvision the EHR as a dynamic tool for facilitating and coordinating complex multidisciplinary patient care and for enhancing communication within patient care teams and between clinicians and patients, rather than as a static record of patient care.
This study has a number of limitations. First of all, it was conducted with a limited number of participants on one general medicine service within one large urban teaching hospital. As such, it has limited generalizability beyond these settings. However, the scale of the study is comparable with other recent time and motion studies of resident workflows. Moreover, the distribution of activities captured in the study is likely to be different in specialty units, such as critical care units, or for different types of residents, such as surgical residents. In addition, the study was conducted in June and July, at the time when new interns begin their residency and do not yet have established work patterns. Further research can show whether these patterns change over time as interns gain more experience. Complementing time and motion studies of clinical workflows with analysis of EHR usage logs (similar to Hripcsak et al24) could allow for an expansion of the number of participants and examinations of differences in workflows between physicians in different subspecialties and of different parameters (time of day, severity of patients’ conditions, etc.) on clinical work.
This time and motion study investigated how residents on the general internal medicine service of a large urban teaching hospital spent their shift time, with a particular focus on their use of computers. The study may have uncovered a number of inefficiencies in the design of the EHR system that led to inefficient use of time for documenting and reviewing patient data, suggesting that improvements in the design and usability of EHRs may help to streamline computer-based activities. Arguably, the practice of medicine may have reached the “point of no return” in regard to its reliance on computing systems. The question then becomes not whether and how much clinicians should use computers, but what they should use them for and to what degree the use of computing systems can support clinical care activities.
1. Amarasingham R, Plantinga L, Diener-West M, Gaskin DJ, Powe NR. Clinical information technologies and inpatient outcomes: A multiple hospital study. Arch Intern Med. 2009;169:108114.
2. Wong DH, Gallegos Y, Weinger MB, Clack S, Slagle J, Anderson CT. Changes in intensive care unit nurse task activity after installation of a third-generation intensive care unit information system. Crit Care Med. 2003;31:24882494.
3. Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. 2006;13:547556.
4. Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2005;116:15061512.
5. Zheng K, Haftel HM, Hirschl RB, O’Reilly M, Hanauer DA. Quantifying the impact of health IT implementations on clinical workflow: A new methodological perspective. J Am Med Inform Assoc. 2010;17:454461.
6. Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: The nature of patient care information system-related errors. J Am Med Inform Assoc. 2004;11:104112.
7. Ash JS, Sittig DF, Dykstra R, Campbell E, Guappone K. The unintended consequences of computerized provider order entry: Findings from a mixed methods exploration. Int J Med Inform. 2009;78(suppl 1):S69S76.
8. Ammenwerth E, Spötl HP. The time needed for clinical documentation versus direct patient care. A work-sampling analysis of physicians’ activities. Methods Inf Med. 2009;48:8491.
9. Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nurses: A systematic review. J Am Med Inform Assoc. 2005;12:505516.
10. Block L, Habicht R, Wu AW, et al. In the wake of the 2003 and 2011 duty hours regulations, how do internal medicine interns spend their time? J Gen Intern Med. 2013;28:10421047.
11. Moore SS, Nettleman MD, Beyer S, et al. How residents spend their nights on call. Acad Med. 2000;75:10211024.
12. Parenti C, Lurie N. Are things different in the light of day? A time study of internal medicine house staff days. Am J Med. 1993;94:654658.
13. Finkler SA, Knickman JR, Hendrickson G, Lipkin M Jr, Thompson WG. A comparison of work-sampling and time-and-motion techniques for studies in health services research. Health Serv Res. 1993;28:577597.
14. Cimino JJ. Improving the electronic health record—Are clinicians getting what they wished for? JAMA. 2013;309:991992.
15. Wang CJ, Huang AT. Integrating technology into health care: What will it take? JAMA. 2012;307:569570.
16. NewYork-Presbyterian. 2010 annual report. 2010. http://nyp.org/pdf/annual_report_2010.pdf
. Accessed November 12, 2015.
17. Mamykina L, Vawdrey DK, Stetson PD, Zheng K, Hripcsak G. Clinical documentation: Composition or synthesis? J Am Med Inform Assoc. 2012;19:10251031.
18. Mamykina L, Wolf CG. Kellog W; ACM Digital Library; ACM Special Interest Group on Supporting Group Work; ACM Special Interest Group on Computer-Human Interaction. Evolution of contact point: A case study of a help desk and its users.Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work. 2000:New York, NY: ACM; 4148.
19. Overhage JM, Perkins S, Tierney WM, McDonald CJ. Controlled trial of direct physician order entry: Effects on physicians’ time utilization in ambulatory primary care internal medicine practices. J Am Med Inform Assoc. 2001;8:361371.
20. Pizziferri L, Kittler AF, Volk LA, et al. Primary care physician time utilization before and after implementation of an electronic health record: A time-motion study. J Biomed Inform. 2005;38:176188.
21. Bostock M, Ogievetsky V, Heer J. D³: Data-driven documents. IEEE Trans Vis Comput Graph. 2011;17:23012309.
22. Hollingsworth JC, Chisholm CD, Giles BK, Cordell WH, Nelson DR. How do physicians and nurses spend their time in the emergency department? Ann Emerg Med. 1998;31:8791.
23. Gabow PA, Karkhanis A, Knight A, Dixon P, Eisert S, Albert RK. Observations of residents’ work activities for 24 consecutive hours: Implications for workflow redesign. Acad Med. 2006;81:766775.
24. Hripcsak G, Vawdrey DK, Fred MR, Bostwick SB. Use of electronic clinical documentation: Time spent and team interactions. J Am Med Inform Assoc. 2011;18:112117.
25. Thyvalikakath TP, Dziabiak MP, Johnson R, et al. Advancing cognitive engineering methods to support user interface design for electronic health records. Int J Med Inform. 2014;83:292302.
26. Walji MF, Kalenderian E, Tran D, et al. Detection and characterization of usability problems in structured data entry interfaces in dentistry. Int J Med Inform. 2013;82:128138.