Healthcare environments are ecology of heterogeneous technologies with ever-increasing complexities among hardware configurations, software applications, and device manufacturers and vendors.1 Currently, medical devices not only capture patient vital signs, but aggregate, store, and transmit to other networks, devices, and applications within the healthcare information technology network. Ensuring device security and health information privacy is of paramount concern to many healthcare facilities. The 3rd annual benchmark study on Patient Privacy and Data Security by the Ponemon Institute identified data breaches as a regular occurrence in the healthcare industry with an average economic cost of approximately $2.4 million.2 A significant amount was due to insider negligence, lost or stolen devices,2 and placing an ever-increasing importance on network and information security best practices. The Healthcare Industry Cybersecurity Task Force suggests steps for organizations to address medical device security including a recommendation to use single- or 2-factor authentication to better establish trust between clinicians and the devices they interact with.3 Securing these devices to ensure compliance is proving difficult within healthcare organizations because some of the measures taken, the simplest of which is the username and secure password combination, can inhibit productivity and be a source of cognitive overload and frustration.4,5 Single Sign-On (SSO) has emerged as a means of addressing authentication challenges, providing the user the ability to log into a network once and be able to navigate multiple applications without having the need to reauthenticate themselves in the network. This is an ideal scenario reducing user frustration, increasing compliance, user satisfaction, and efficiency.1
Our evaluation was conducted in a pediatric treatment and research facility focused on children's catastrophic diseases, particularly cancer, located in the southeast region of the United States. The outpatient clinic, that is, the Assessment Triage Unit (ATU), served as the pilot site. The ATU comprised 20 assessment rooms equipped with either a standalone Connex® Spot Monitor (Hillrom, Skaneateles Falls, NY) or a wall-mounted version with wireless transmission to the Cerner electronic health record (EHR) (Cerner CareAware VitalsLink® [Cerner, North Kansas City, MO]). The hospital clinical informatics department, in collaboration with researchers from Welch Allyn, was interested in evaluating the effect of the pilot go-live to identify potential usability and system behavior issues encountered with the deployment, prior to expanding the service hospital-wide.
There was interest in obtaining staff perception on the deployment of an SSO solution for medical devices, that is, “badge scan-and-go” using an optical bar code reader versus the 2-factor log-in method (username and password) that was in use.
The ATU is a high-volume clinic with an average of 200 patient visits per day. Clinicians see an average of 10 to 15 patients a day, assessing vital signs with each patient encounter. This is the 1st stop for patients where, in addition to having vital signs assessed, they often have an IV started and laboratories drawn before proceeding to a specialty clinic or a treatment area.
Prior to this initiative, the clinical informatics department received multiple complaints from staff about the frequent need to log into the vital sign devices to obtain subject vital signs, especially in high-volume clinics, as well as frustrations with devices timing out while the clinician worked with the patient to complete other tasks. Combined with the need to optimize clinician time and enhance their daily clinical experience, clinical informatics identified a need to find a user-friendly solution that also provided data security. This study received approval from the IntegReview institutional review board (IRB). The project was determined to be a quality improvement initiative involving nonhuman subject research and hence exempt from review per the hospital IRB.
An observational preimplementation and postimplementation study of the SSO solution was performed by collecting quantitative time data, comparing clinician time savings, system usability, staff preference, and satisfaction by survey methodology on different workflows, that is, the 2-factor authentication method (username and password) and the SSO method (badge scan-and-go).
Subjects and Sample Size
A convenience sample of unit clinicians who were available on the preimplementation data collection day was approached and consented prior to their participation. A deidentification method was developed to observe the same set of clinicians during both the preimplementation and postimplementation phases based on their availability during the study period. The same set of clinicians was required to also complete the usability and workflow comparison survey postimplementation. The usability survey included open-ended questions asking about the training they received on the SSO system, as well as what they liked and disliked about it. The workflow comparison survey assessed their satisfaction with using both methods and questions about their impressions of both workflows.
Patients and parents of the children visiting the ATU for their routine vital sign and laboratory draws were approached, and verbal permission sought to allow the researcher to be present in the room to observe and time the clinician during the log-in workflow being assessed. The preimplementation study data were collected over a period of 2 days in the 1st week of March 2019 prior to the “go-live” of the SSO in the last week of March. Study observations and data collection were performed during the clinic hours of 7 am to 5 pm. A total of 16 clinicians agreed to participate and consented to participating in the study.
At 6:30 am of the go-live implementation, before the start of the normal clinic workday, ATU staff were trained on the new system by a Welch Allyn clinical trainer. Postimplementation data were collected similarly over 2 days in the last week of April 2019 during clinic hours. Staff had been using the system for a period of 4 weeks prior to the postimplementation data collection. Staff were provided the SSO usability survey and the workflow comparison surveys to complete at this time.
Data collection for both the preimplementation and the postimplementation timing was collected by a trained clinical researcher. The 2-factor authentication workflow in the clinic comprised logging into the EHR by manually entering the username and password on the clinical workstation and then proceeding to log into the vital sign device at the beginning of each individual assessment. The clinician's next step consisted of assessing the patient's blood pressure (BP), pulse, temperature, and occasionally pulse oximetry readings. Nursing assessment of pain, respirations, and modifiers such as left or right arm used for BP, sitting or supine position during assessment, or temperature measurement route was manually entered into the vital sign device using the touchscreen keyboard. These actions were often done simultaneously as the device was obtaining readings. Height and weight were obtained by a digital seca scale (Seca Corporation, Chino, CA) with Wi-Fi capability that transmitted the readings into the vital sign monitor. Once readings were obtained, the clinician would press the save button on the device, which would instantly transmit data to the EHR and the clinician would then continue to provide care for the patient. For the next patient, the clinician's process of logging into the workstation and vital sign device would then be repeated. During the study, the timing of data collection began when the clinician started to enter their username and password for the 2-factor method until they obtained vital signs and saved the data.
The SSO workflow included the clinician logging in with their username and password manually into any workstation in the clinic to access the EHR, after which they would remain authenticated by the system for a period of 8 hours. Following this, the clinician would then proceed to log into the vital sign device at the beginning of an assessment by scanning the barcode on their badge by an optical scanner and then proceed to take measurements. When the assessment was complete, the clinician would press the save button. For the SSO method, measurement of time began as soon as the clinician reached for the barcode scanner until they obtained vital signs and saved the data.
A time study was chosen for the consistency in obtaining precise times taken for the vital sign collection in the outpatient clinic,6 as well as the ability to assess the difficulties encountered with log-in attempts with both workflows.
For this study, time taken to obtain height and weight measurements was omitted. This was done because of an inconsistency in the method as well as scales used for these measurements. Other than the scale located inside each room, clinicians also used a supine scale for height and weight for infants and a built-in floor scale for those using a wheelchair. In 3 instances where BP was not obtained by the clinical staff during workflow observations, the timed study data were not included in the analysis.
Descriptive statistical summaries were computed for workflow time (in seconds) for the vital measurements obtained, separately for each method. Independent-samples t tests were carried out to analyze differences of workflow times between the 2 methods. A minimum set of 52 observations starting with clinician log-in into the vital sign device, measuring BP, pulse, heart rate, respiratory rate, and oximetry until the clinician pressed save on the device, which transmitted the date to the electronic medical record, was collected for both methods.
Usability survey data were analyzed using a 5-point Likert scale and consolidated into 3 categories (“strongly agree/agree,” “neutral,” and “disagree/strongly disagree”) and reported as percentages. Free text questions were analyzed using descriptive methodology. Additional data analysis for this article was generated using SAS Software (version 9.4, ©2002-2012; SAS Institute Inc, Cary, North Carolina).
The mean time for device-only log-in and vital sign collection from the 2-factor method to SSO decreased from 144.2 seconds to 110 seconds, with a time savings of 34.07 seconds per log-in (P = .0003) (Figure 1, Table 1). The difference in time for device log-in reduced from an average of 22.9 seconds to 9.8 seconds (P = <.0001) (Figure 1, Table 1) (see Supplemental Digital Content 1, http://links.lww.com/JONA/A740, illustrating the histograms of time required for collection of vital signs including log-in times only with both methods).
With the assumption that each clinician logs into the device an average 10 to 15 times a day, the annual number of events of logging in and collecting vital signs amounts to 604 800. This calculates out to an annual average time savings of 761.6 hours for a single clinic. The financial impact estimates conservatively translate the time savings for this clinic into annual savings of $16 755 (Table 2). The average cost of individual SSO for each device is $250, with a total investment for 16 devices being $4000. By implementing this solution, there is an estimated return on investment of $12 755.
A total of 14 clinicians responded and completed the usability survey. Thirteen also completed the workflow comparison survey. Most of the staff were credentialed licensed practical nurses (13) with 1 RN among the respondents (Supplemental Digital Content 2, http://links.lww.com/JONA/A741).
SSO Usability Survey Responses
Eighty-six percent of the respondents agreed that SSO would save them time and the scanning capability to read their badge was easy to use (Figure 2). Seven percent disagreed with these statements, and 7% were neutral in their response. Ninety-three percent agreed that it was easy to log into the device, understood the benefit of using SSO, and preferred to use it as compared with the manual workflow, and 7% disagreed with these statements. Seventy-one percent of the participants felt that using SSO provides for better data security, 21% were neutral in their response, and 7% disagreed. The survey included open-ended questions to allow free text responses. Themes that emerged from responses to these questions include the following verbatim responses:
Describe Aspects You Liked Most About the SSO Workflow
Easy to use. Most passwords require to have upper case/lower case and symbols for security, which makes typing to sign in time consuming.
It was quick, and I don't have the problem of accidentally mistyping my password.
Faster even with taking off badge.
Quick, accurate, secure.
Please Describe Your Impressions of Training Received to Use the SSO Workflow
Easily understood and to convey to coworkers that weren't present that day.
It was really easy and did not take long.
Very helpful, easy, educational.
Easy to understand how to use.
Please Describe Which Aspects You Liked Least About the SSO Workflow and Things That Could Be Improved
That the info on the patient doesn't stay in the history if it sends. Sometimes it sends but disappears between it and chart, so it would be nice to have it in both for at least in the last 5 to 10 patients.
Sometimes device kicks me out.
For the workflow comparison questionnaire, we saw a shift to the right, with participants going from “moderately dissatisfied” for the 2-factor method, to “very satisfied” for the SSO solution (Supplemental Digital Content 3, http://links.lww.com/JONA/A742). Responses that emerged from questions on their impressions of logging in with a 2-factor method and SSO showed a preference for the SSO workflow. Clinicians responded to their log-in impression with 2-factor log-in as “outdated” and “time-consuming”. For SSO, the most common responses included “faster,” “saves time,” “easier and convenient.” Clinicians identified the top 3 benefits of logging in with the SSO as less frustration with password log-outs, faster time to log-in while ensuring security, and a quicker start with their daily clinical work providing them enough time savings to get ready and set up earlier for their day.
The scope of our study combined the evaluation of both efficiency and satisfaction with the system, as opposed to other studies that look at each individually.5,8 This combination provides a more complete perspective of the impact of technology implementation. Our study showed that SSO had a favorable impact on efficiency and was the preferred method by staff. Our return on investment and cost savings are based on actual device costs, timed study observations, and estimates of labor cost of staff in the clinic based on the US Department of Labor estimates.7
Despite the limitations of a small sample size in a single clinic, as well as the potential Hawthorne effect resulting in clinicians inadvertently trying to speed up device log-ins, we received positive validation of the implementation prior to the hospital-wide launch. Our cost savings analysis did not account for information technology personnel and deployment cost, due to difficulty in accurately assessing personnel time and facility cost. Despite this, our study shows that SSO provides cost savings over time for the organization. This result is similar to what has been seen in other studies.8
Implications for Nurse Leaders
Our study offers a replicable survey methodology and cost calculator that nurse leaders can implement to achieve similar results in their institutions. In addition, our study highlights the importance of clinician input into the adoption and assessment of technologies that can potentially impact and improve their workflow.
Providing high-quality patient care should be a primary mission of any healthcare organization. To create an environment in which caregivers can be effective and efficient, organizations must adopt processes and technology that improve caregivers' workflow. Increasing productivity can maximize the use of available resources and, at the same time, increase staff morale. While enhancing staff morale is not often seen as a key to improving an organization's fiscal state, there is a proven correlation between nurse satisfaction and patient satisfaction, which impacts an organization's revenues.9,10 By implementing technology that provides a proven return on investment and increases staff engagement, nursing leadership can successfully meet what often seem like competing goals. The resulting improvement in patient care brings the nurse leader the satisfaction of attaining a win-win-win outcome, a rare occurrence in the challenging environment of healthcare.
The authors acknowledge Pamela Goodman, senior clinical analyst, Clinical Informatics; Teresa Browning, MBA, BS, RN-AD, Process Re-engineering, Clinical Informatics; Tammy Teams, RN, CPN, manager, Ambulatory Care Unit; Catherine O'Malley, RN, resource nurse, Ambulatory Care Unit; and Dana Matheny, PMP, Nursing Administration.
1. Heckle RR, Lutters WG. Tensions of network security and collaborative work practice: understanding a single sign-on deployment in a regional hospital. Int J Med Inform
2. Poneman Institute LLC. Third Annual Benchmark Study on Patient Privacy and Data Security
. Sponsored by ID Experts: December 2012. https://www.idexpertscorp.com/knowledge-center/single/third-annual-benchmark-study-on-patient-privacy-data-security
. Accessed June 11, 2019.
3. US Department of Health and Human Services. Healthcare industry Cybersecurity task force June 2017, report on improving cybersecurity in the healthcare industry. https://www.phe.gov/Preparedness/planning/CyberTF/Documents/report2017.pdf
. Accessed June 11, 2019.
4. Yan JBA, et al. Password memorability and security: empirical results. IEEE Security Privacy
5. Hope P, Zhang X. Examining user satisfaction with single sign-on and computer application roaming within emergency departments. Health Inform J
6. Lopetegui M, Yen PY, Lai A, Jeffries J, Embi P, Payne P. Time motion studies in healthcare: what are we talking about? J Biomed Inform
7. Bureau of Labor Statistics, US Department of Labor. Occupational Outlook Handbook. Licensed Practical and Licensed Vocational Nurses
. Accessed June 6, 2019.
8. Gellert GA, Crouch JF, Gibson LA, Conklin GS, Webster SL, Gillean JA. Clinical impact and value of workstation single sign-on. Int J Med Inform
9. McHugh MD, Kutney-Lee A, Cimiotti JP, Sloane DM, Aiken LH. Nurses' widespread job dissatisfaction, burnout, and frustration with health benefits signal problems for patient care. Health Aff
. Accessed November 21, 2019.
10. Mehta SJ. Patient satisfaction reporting and its implications for patient care. AMA J Ethics