Tele-ICU services are increasingly used in the United States, with estimates ranging from 11% to 20% of ICU beds being monitored by tele-ICUs (1–3). The drivers for tele-ICUs in the United States are workforce shortages, technological improvements, and demographic changes related to age (4–6). Tele-ICUs work through interfacing clinical data and shared access to electronic medical records (EMRs) fed to a central location, which has capabilities to interact with the bedside location(s) through advanced audiovisual technology (7,8). Its adoption has resulted in increased access for patients to specialty services, bridging the widening supply-demand critical-care provider gap and realizing synergies for services offered through hub-and-spoke models of care delivery. More recently, the continuous model of tele-ICU has further evolved into managing ICU bed access, capacity, and throughput across the spectrum as well as providing care standardization for large healthcare systems with positive financial results (9,10).
Tele-ICU implementation has been associated with reductions in mortality, length of stay (LOS), and costs in some studies but not others (11–14), indicating that additional factors, about which we have gained a better understanding in recent years, affect its value potential:
- 1) level of authority of the tele-ICU to co-manage (15–17)
- 2) robust protocols for ensuring best practice adherence (10,18,19)
- 3) timely use of performance data (18)
- 4) tele-ICU physician review of all new admissions within 1 hr of arrival and quick alert response times (18)
- 5) leadership (18,19)
- 6) perceived value (19)
- 7) addition of a logistics center geared at optimizing ICU bed access, care standardization, capacity, and throughput (10)
- 8) a standardized mortality ratio greater than 1 pre-implementation baseline performance (13).
Tele-ICUs generally perform a multitude of tasks of different priorities and with different team members involved at different steps. Balancing tasks in varying workload situations is crucial to ensure reliable yet flexible tele-ICU performance. For example, as ICU admissions cluster during peak hours (20,21), workflow efficiency must be optimized to ensure the evidence-based goal of evaluating new patients within 1 hour of admission to the ICU (18). Overall, how to optimize multidisciplinary tele-ICU intrinsic operational workflows, protocols, and priorities that in aggregate lead to tele-ICU benefit requires further study (22). We therefore set out to
- 1) Develop an operational model that accurately reflects the existing workflows and priorities and their effect on core tele-ICU tasks at baseline
- 2) Use discrete-event simulation and multi-class priority queuing modeling on this operational model to identify workflow changes that optimize operational efficiency
- 3) Implement the predicted favorable workflow changes and validate the simulation model through prospective correlation of actual-to-predicted change in performance measures linked to patient outcomes.
Operational changes can target a variety of factors. For this study, we chose change in process structure or changes in hierarchical priorities in processes.
In this study, we create a discrete-event simulation model reflective of day-to-day operations in the tele-ICU in that it accurately predicts both baseline targeted performance measures as well as the overall effect of operational optimization on these targeted performance measures.
MATERIALS AND METHODS
We studied 7,387 patient admissions to a continuous model tele-ICU of a large medical system in Valhalla, NY, during a 18-month period (January 2016 to June 2017). The first quarter of 2016 (2016Q1) constituted the baseline pre-intervention phase and included 1,155 patients. Simulations were developed and conducted in the second quarter of 2016 (2016Q2). Workflows were changed on July 1, 2016. The post-intervention phase included the third quarter of 2016 (2016Q3) to the second quarter of 2017 (2017Q2) with a total of 6,232 patient admissions. Our tele-ICU covers nine ICUs across the Hudson Valley of New York State, serving a quaternary care center, two community hospitals and one critical access hospital. It admits approximately 5,500 patients each year and continuously monitors an average of 100 patients daily. The tele-ICU team is comprised of one tele-intensivist, one data coordinator, and three registered nurses working 12-hour shifts with shift change occurring at 7 am and 7 pm, for the whole team, on a daily basis.
We collected each patient’s anonymized admission and discharge dates and times, and times and lengths of the first video assessment performed by tele-intensivist and by tele-nurse for each new patient, the lengths of any follow-up video(s) performed by tele-intensivist and by tele-nurse for each new patient, each new patient’s time from admission to first video assessment by tele-intensivist and tele-nurse (time to first video assessment [TFVA]). The daily, weekly, and yearly admission and discharge patterns as well as LOS patterns are available in the eSupplement (Supplemental Digital Content 1, http://links.lww.com/CCX/A118).
We focused on the following evidence-based performance metric categories:
- 1) Wait time until the tasks are performed
- 2) Utilization measures related to the clinician productivity in the tele-ICU
- 3) Quality of care measures such as time spent interacting with patients through video.
The Westchester Medical Center/New York Medical College Committee for the Protection of Human Subjects waived the need for Institutional Review Board approval as the study does not meet institutional criteria for Human Subjects Research.
Simulation Model and Validation
Discrete-event simulation has been widely applied to improve healthcare delivery systems (23,24). Simulation modeling is used to mimic and replicate the operations of a real system, which can capture key characteristics and functions of the system. We developed a discrete-event simulation model to investigate the operations of the tele-ICU center using Rockwell Arena 15.0 software (Rockwell Automation, Coraopolis, PA). This model allows us to gain insights of operational changes on real tele-ICU systems. Once derived, the model must be validated to ensure that it is a suitable replication of real-life scenarios. This is performed by comparing the model outputs to empirically collected tele-ICU data to test for validation.
Figure 1 provides a flowchart of baseline tele-ICU center processes. These processes are divided into four basic categories:
- 1) Crisis intervention
- 2) Proactive monitoring
- 3) Best practice adherence
- 4) New patient evaluation
“Crisis intervention,” as a task, is given the highest priority in our model. In real-life scenarios, in the tele-ICU, both the tele-ICU nurse and intensivist will stop all other tasks to help an ICU patient in crisis.
“Proactive monitoring” was given the second highest priority. This is typically an intermittent task usually performed in-between other tasks. In our simulation model, it did not preempt other tasks.
“Best practice adherence” is a task derived from using tele-ICU software (eCare Manager 4.1.1; Royal Philips, Amsterdam, The Netherlands) to audit predefined best practices in the ICU. These included venous thromboembolism prophylaxis in patients meeting requirement, stress-ulcer prophylaxis in patients meeting requirement, low tidal volume ventilation (< 7.5 cc/kg of ideal body weight) and targeting glycemic control in any patient with two blood glucoses greater than 180 mg/dL in a 24-hour period. In terms of task hierarchy, these were considered nonurgent but must be completed within certain predefined time periods.
“New admission evaluation,” while being one of the most complex tasks being performed was given the contextually lowest baseline priority of the aforementioned tasks.
As shown in Figure 1A, the typical tele-ICU process involves a tele-ICU nurse performing the first video assessment of a newly admitted ICU patient and then drafting an ICU admission note for the tele-intensivist to review prior to their first video assessment. This is done in an effort to maximize the scalability of the intensivist efficiency. The tele-intensivist will then perform a first video session with the patient, edit and complete the note before sending it to the EMR. Higher acuity patients will often be seen by the tele-intensivist and/or tele-nurse again via follow-up video assessments.
The daily arrival process of admissions, that is, tasks related to new patient evaluation and risk prediction, in the simulation model is assumed to follow a nonstationary Poisson process based on the empirical data collected and is shown in Figure e1 (Supplemental Digital Content 2, http://links.lww.com/CCX/A119).
We collected baseline data during 2016Q1. This time period followed the baseline workflows shown in Figure 1.
To validate our simulation model, we set a run length of 90 days (approximately one quarter of a calendar year) with a warm-up period of 30 days for each replication in the simulation model. Discrete-event simulation models start with completely empty systems (no patients currently being treated by tele-ICU or waiting to be processed). A warm-up period as used here allows the simulation model to reach a steady-state equivalent to the current status of the tele-ICU center. This prevents under-estimating the system’s long-term performance measures, such as average waiting times. A 30-day warm-up period insures that our system reaches steady state to more accurately reflect the reality of the tele-ICU center.
The simulation model performed 100 replications, where each replication represents a 90-day period of operations for the tele-ICU. A replication represents one complete run of our simulation model. The number of independent and identically distributed replications used in the model determines the width of CIs of the estimated variables (25). We use 100 replications because this provides a half-width on the CI for the baseline case that is less than 6% of the mean value (Table 2).
Changing Priority of Tasks
As mentioned previously, the original priority ordering of tasks performed by the tele-ICU team are “(crisis intervention) > (proactive monitoring) > (best practice adherence) > (new patient evaluation).” Because crisis intervention and proactive monitoring are time sensitive, we did not change their priorities. However, best practice adherence is less time sensitive and can be completed at any time within the typical shift and therefore could change priorities with new admission evaluations, which are also less time sensitive. Thus, we made a priority change of new admission evaluation over best practice adherence, while leaving the priority of crisis intervention and proactive monitoring unchanged. In other words, the new priority ordering of tasks after the change was “(crisis intervention) > (proactive monitoring) > (new patient evaluation) > (best practice adherence).” Our expectation was that this change would result in shorter wait times for new patients to be evaluated by the tele-ICU team but could result in delays to the adherence of best practices; since we did not change the priorities of crisis intervention and proactive monitoring, there should not be changes in wait times for these two tasks.
Changing Sequencing of the Tele-ICU Admission Process
As described previously, a typical tele-ICU admission process involves a tele-nurse performing the first video session with each new patient and then after reviewing the tele-nursing note the tele-intensivist performs a video session with that same patient and completes the tele-nursing note sending it to the EMR. In this scenario, if a new admission has not been seen by a tele-nurse, the tele-intensivist will not start a video session with this patient. From a queueing theory and operational point of view, the current admission process represents a two-stage tandem queue (26,27). In this type of queue, the server at the second stage (i.e., a tele-intensivist) might be “starved” if the first-stage server (i.e., tele-ICU nurses) work slower than the second stage during certain periods of time. Such systems lead to operational inefficiency due to this starving effect (28). We proposed to increase the flexibility of the process by allowing the tele-intensivist or tele-nurse to begin the first-video session with a patient, based on who is available first. In other words, a tele-intensivist can perform the video interaction with a new admission if they are free, even if this patient has not had a video session yet with a tele-nurse. From an operations framework, such a change is indicative of creating “flexible servers” since each server (clinician) can perform their task based on availability instead of hierarchy. This scenario has been shown to add operational efficiency in a variety of clinical and nonclinical settings (29,30).
Based on recent literature, we adopted the performance goal that all new admissions should be evaluated by video session within 1 hour of being admitted (18).
In order to examine the effects from both priority changes as well as sequencing changes on tele-ICU performance, we designed the following simulations with four possible variable changes (Table 1): 1) the average number of admissions per day; 2) the average length of the first video assessment by the tele-intensivist; 3) the percentage of follow-up videos by the tele-intensivist; and 4) the operational changes.
After reviewing the data collected from January 1, 2016, to March 31, 2016, the observed number of performed first videos assessments ranged from 23 to 27 per day in the study period. Based on these data, we chose these two extremes of the range values for the two levels of this particular variable for our model. These data set also indicated that the average duration of the first video assessment by the tele-intensivist was approximately 120 seconds. Given that comprehensive first video assessments are desirable, we chose 120 and 180 seconds as inputs for our model. Approximately 35% of new patients were seen again in follow-up video assessments by the tele-intensivist in the first quarter of 2016. A team goal was to increase engagement through a higher percentage of follow-up video assessments. Thus, the levels of this variable were input as 30% and with a higher goal of 50%. In order to fully examine the impacts of the proposed operational changes (both “priority” changes and “sequencing” changes), we set four levels for this factor: no change, priority change only, process change only, and both priority and process change.
We performed a full-factorial experiment (27), that is, 32 design points in total. Each design point, that is, each combination of levels of the four factors, is evaluated using 200 replications of our discrete-event simulation model.
Based on simulations performed in 2016Q2, the actual tele-ICU workflow process structure and hierarchical process priority as outlined in the previous section was changed on July 1, 2016. Post-intervention performance data on 6,232 patient admissions as outlined above was collected for 2016Q3 to 2017Q2.
Table 2 displays the simulated and observed actual baseline average TFVA with 95% CIs. The observed baseline average TFVA are well within the CIs of the simulated average TFVA. We find that our simulation model is an excellent approximation of the baseline process, and thus is validated by the actual observed data.
Simulating the various changes in inputs as listed in Table 1, the output metrics delivered a variety of results. Figure 2, A and B, illustrate the simulated effect that these model inputs have on average time to new patient evaluation by the tele-intensivist and tele-ICU nurse, respectively. As evidenced, the length of first-video assessments and the percentage of follow-up video assessments performed by tele-intensivists have relatively small impacts on TFVA as compared with the impact that the changing number of admissions and operational changes yielded. Specifically, if the length of a first-video assessment increases from 120 seconds to 180 seconds, the average TFVA increases by only 13% and 7.2%, respectively. Similarly, if the percentage of follow-up video assessments performed by the tele-intensivist increases from 30% to 50%, this increases the TFVA by the tele-intensivist by 3.7% and leaves the average TFVA by a tele-nurse approximately unchanged.
In contrast, the average number of admissions per day expectedly has a much greater effect on the average TFVA by either the tele-intensivist or tele-nurse. On increasing from 23 to 27 new admissions per day the TFVA increases by 64% for the tele-intensivist and by 89% for the tele-nurse.
When making operational changes related to priority, the average TFVA is reduced by 71% (tele-nurse) and 56% (tele-intensivist). On making changes to process, the average TFVA is reduced by 26% (tele-nurse) and 22% (tele-intensivist).
There is concern that increasing engagement through more frequent and/or longer video assessments by the tele-intensivist could potentially come at the expense of completing other time-sensitive tasks and lead to longer TFVA. Therefore, we ran simulations on our model of the effect that longer video times would have on completion of other tasks. With both the priority and the process change implemented in the tele-ICU center, Figure 3A displays average TFVA by tele-intensivist and tele-nurse when average video length increases. Similarly, we also ran simulations on our model of the effect that increasing the percentages of follow-up video assessments by tele-intensivists and tele-nurses would have on completion of other tasks and have found similar results to the simulations performed in Figure 3A (Fig. e7, Supplemental Digital Content 8, http://links.lww.com/CCX/A125; legend, Supplemental Digital Content 1, http://links.lww.com/CCX/A118).
In this simulation model and analysis, we assumed that the arrival of crisis interventions is independent of the wait times to complete other tasks. In other words, the arrival of crisis interventions is considered exogenous to our system being modeled. However, there likely exists interdependence between crisis interventions and the other tele-ICU tasks. For example, previous studies have shown that the arrival rate of crisis interventions (cardiopulmonary arrests) is correlated with proactive monitoring (31). Any increase in crisis interventions could potentially require more utilization of the tele-ICU team and detract from the performance of the other time-based tasks. Additional crisis intervention tasks could delay proactive monitoring further, effectively creating a negative feedback loop causing even more crisis intervention tasks.
We adapted an operations model called multi-class priority queuing system with state-dependent arrivals to evaluate the effects of varying delays to proactive monitoring activities by tele-intensivists and nurses on the respective wait times for other tasks (for further methodological details see eSupplement, Supplemental Digital Content 1, http://links.lww.com/CCX/A118) (32,33). Figure 3B illustrates that other proactive monitoring tasks are affected less than first video assessments, but that best practice wait times are disproportionately negatively affected especially for tele-intensivists when the proactive monitoring delay threshold is reduced.
After full implementation of both priority and process changes as simulated by our model, we observed a reduction in the average TFVA to 50 minutes for tele-intensivists and 49 minutes for nurses, respectively (Table 2). These observed average TFVA are well within the CIs of the simulated average post priority and process change TFVA (Table 2).
With the increasing adoption of tele-ICUs in the healthcare industry, further research and novel operational tools are needed to optimize operational processes specifically tailored for tele-ICUs, particularly as it relates to maximizing the efficiency of a scarce workforce. Because tasks performed by clinicians in tele-ICUs vary in terms of urgency, performance can likely be improved by prioritizing tasks correctly. Process changes that promote flexibility of tele-clinicians improve operational performance by reducing starvation of dependent operators, consistent with the findings from the literature in other industries (29,30).
Priority change reduces average TFVA more than process change. The improvement from making both the priority change and the process change simultaneously reduces the average TFVA with a tele-nurse by 75% and tele-intensivist by 62%, which is smaller than the additive improvement of making the priority change and process change separately. In other words, the improvements from the priority and process changes are not additive but correlated to some degree.
Our findings suggest that implementing both the priority change and the process change can bring the most improvement in reducing the average TFVA. Furthermore, these improvements allow for other changes that can increase engagement with the bedside clinical team and potentially offer higher quality of care, such as longer initial video assessment times and more frequent follow-up video assessments. We also show that this improvement only results in modest increases in delays to perform other tasks such as best practices and proactive monitoring.
After full implementation of the two modeled operational changes, the average TFVA for both tele-intensivists and tele-nurses decreased to less than 60 minutes, thus meeting evidence-based operational goals. These results are also extremely consistent with the predictions of our discrete-event simulation model. The observed average TFVA for tele-intensivists of 50 minutes was well within the CIs of the simulated average post priority and process change TFVA of 55 minutes. This further validates our simulation model and suggests that our results may be robust to implementing these changes in tele-ICUs more generally.
Modeling the likely effects of contemplated or planned workflow priority and process changes prior to their implementation can save time and safeguard against unforeseen effects uncovered by the model. Although we agree that the workflow changes modeled in this study are rather uncomplicated, workflow process and task priority modeling will become increasingly valuable as the number of involved variables and operational complexities increase.
We thank Dr. Michael Ries for stimulating discussions on the subject of tele-eICU workflow planning.
1. Lilly CM, Zubrow MT, Kempner KM, et al.; Society of Critical Care Medicine Tele-ICU CommitteeCritical care telemedicine: Evolution and state of the art.Crit Care Med2014422429–2436
2. New England Healthcare Institute Massachusetts Technology CollaborativeCritical Care, Critical Choices: The Case for Tele-ICUs in Intensive Care.2010Available at: https://www.nehi.net/writable/publication_files/file/teleicu_critical_care_critical_choices.pdf
. Accessed November 27, 2019
3. Lilly CM, Fisher KA, Ries M, et al. A national ICU telemedicine survey: Validation and results.Chest201214240–47
4. Udeh C, Udeh B, Rahman N, et al. Telemedicine/virtual ICU: Where are we and where are we going?Methodist Debakey Cardiovasc J201814126–133
5. Angus DC, Kelley MA, Schmitz RJ, et al.; Committee on Manpower for Pulmonary and Critical Care Societies (COMPACCS)Caring for the critically ill patient. Current and projected workforce requirements for care of the critically ill and patients with pulmonary disease: Can we meet the requirements of an aging population?JAMA20002842762–2770
6. Halpern NA, Tan KS, DeWitt M, et al. Intensivists in U.S. Acute care hospitals.Crit Care Med201947517–525
7. Ramnath VR, Ho L, Maggio LA, et al. Centralized monitoring and virtual consultant models of tele-ICU care: A systematic review.Telemed J E Health201420936–961
8. Fusaro MV, Becker C, Pandya S, et al. International teleconsultation on conjoined twins leading to a successful separation: A case report.J Telemed Telecare201824482–484
9. Lilly CM, Motzkus CA. ICU telemedicine: Financial analyses of a complex intervention.Crit Care Med2017451558–1561
10. Lilly CM, Motzkus C, Rincon T, et al.; UMass Memorial Critical Care Operations GroupICU telemedicine program financial outcomes.Chest2017151286–297
11. Young LB, Chan PS, Lu X, et al. Impact of telemedicine intensive care unit coverage on patient outcomes: A systematic review and meta-analysis.Arch Intern Med2011171498–506
12. Wilcox ME, Adhikari NK. The effect of telemedicine in critically ill patients: Systematic review and meta-analysis.Crit Care201216R127
13. Fusaro MV, Becker C, Scurlock C. Evaluating tele-ICU implementation based on observed and predicted ICU mortality: A systematic review and meta-analysis.Crit Care Med201947501–507
14. Chen J, Sun D, Yang W, et al. Clinical and economic outcomes of telemedicine programs in the intensive care unit: A systematic review and meta-analysis.J Intensive Care Med201833383–393
15. Hawkins HA, Lilly CM, Kaster DA, et al. ICU telemedicine comanagement methods and length of stay.Chest2016150314–319
16. Thomas EJ, Lucke JF, Wueste L, et al. Association of telemedicine for remote monitoring of intensive care patients with mortality, complications, and length of stay.JAMA20093022671–2678
17. Thomas JT, Moeckli J, Mengeling MA, et al. Bedside critical care staff use of intensive care unit telemedicine: Comparisons by intensive care unit complexity.Telemed J E Health201723718–725
18. Lilly CM, McLaughlin JM, Zhao H, et al.; UMass Memorial Critical Care Operations GroupA multicenter study of ICU telemedicine reengineering of adult critical care.Chest2014145500–507
19. Kahn JM, Rak KJ, Kuza CC, et al. Determinants of intensive care unit telemedicine effectiveness. An ethnographic study.Am J Respir Crit Care Med2019199970–979
20. Arulkumaran N, Harrison DA, Brett SJ. Association between day and time of admission to critical care and acute hospital outcome for unplanned admissions to adult general critical care units: Cohort study exploring the ‘weekend effect.’Br J Anaesth2017118112–122
21. Kuijsten HA, Brinkman S, Meynaar IA, et al. Hospital mortality is associated with ICU admission time.Intensive Care Med2010361765–1771
22. Granja C, Janssen W, Johansen MA. Factors determining the success and failure of eHealth interventions: Systematic review of the literature.J Med Internet Res201820e10235
23. Yang M, Fry MJ, Raikhelkar J, et al. A model to create an efficient and equitable admission policy for patients arriving to the cardiothoracic ICU.Crit Care Med201341414–422
24. Katsaliaki K, Mustafee N. Applications of simulation within the healthcare context.J Oper Res Soc2011621431–1451
25. David W, Kelton RPS, Deborah A, et al. Simulation With Arena1998New York, NYMcGraw-Hill
26. Gross DS, John F, Thompson JM, et al. Fundamentals of Queueing Theory2008Hoboken, NJJohn Wiley & Sons
27. Montgomery DC. Design and Analysis of Experiments2017Hoboken, NJJohn Wiley & Sons
28. Jacobs FR, Chase RB. Operations and Supply Chain Management2018New York, NYMcGraw-Hill
29. Graves SC, Tomlin BT. Process flexibility in supply chains.Manag Sci200349839–981
30. Van Mieghem J. Investment strategies for flexible resources.Manag Sci1998441021–126
31. Brown H, Terrence J, Vasquez P, et al. Continuous monitoring in an inpatient medical-surgical unit: A controlled clinical trial.Am J Med2014127226–232
32. Bitran G, Caldentey R. Two-class priority queueing system with state dependent arrivals.Queueing Syst200240355
33. Abouee-Mehrizi H, Baron O. State-dependent M/G/1 queueing systems.Queueing Syst201682121