Development, Validation, and Results of a Survey of Personal Electronic Device Use Among 299 Anesthesia Providers From a Single Institution

Abstract

BACKGROUND: 

The pattern of perioperative use of personal electronic devices (PEDs) among anesthesia providers in the United States is unknown.

METHODS: 

We developed a 31-question anonymous survey of perioperative PED use that was sent to 813 anesthesiologists, anesthesiology residents, and certified registered nurse anesthetists at 3 sites within one health system. The electronic survey assessed patterns of PED use inside the operating room (OR), outside the OR, and observed in others. Questions were designed to explore the various purposes for PED use, the potential impact of specific hospital policies or awareness of medicolegal risk on PED use, and whether PED was a source of perioperative distraction.

RESULTS: 

The overall survey response rate was 36.8% (n = 299). With regard to often/frequent PED activity inside the OR, 24% reported texting, 5% reported talking on the phone, and 11% reported browsing on the Internet. With regard to often/frequent PED activity outside the OR, 88% reported texting, 26% reported talking on the phone, and 63% reported browsing the Internet. With regard to often/frequent PED activity observed in others, 52% reported others texting, 14% reported others talking on the phone, and 34% reported others browsing the Internet. Two percent of respondents self-reported a distraction compared to 15% who had observed a distraction in others. Eighty percent of respondents recognized PED as a potential distraction for patient safety.

CONCLUSIONS: 

Our data reinforce that PED use is prevalent among anesthesia providers.

See Article, p 266

KEY POINTS

• Question: What is the pattern of use and perceived risks of perioperative personal electronic device (PED) use among anesthesia providers?
• Findings: PED use is common in the perioperative setting and more often observed in others than self-reported, and the perceived risk of distraction is high.
• Meaning: Given the prevalence of perioperative PED use, further research is needed to quantify and determine the potential impact of perioperative PED use on patient safety.

The introduction of the Palm Pilot personal digital (TCL Technology) assistant in 1997 and the subsequent release of the Epocrates software platform (athenahealth) in 1998 ushered a new era into modern medicine. The use of digital handheld devices was heralded as a major advance in the delivery of care and patient safety.¹ Over the ensuing 23 years, handheld technologies have evolved into personal electronic devices (PEDs) and become ubiquitous. Despite the unprecedented convenience of instant access to patient records, pharmacopeia, and medical literature, the net impact that PEDs play in the delivery of medical care is opaque. On the one hand, the use of PEDs in the health care environment may result in improved communication between the providers,² improved hospital throughput,³ and improved clinical decision making.⁴ On the other hand, the ability to participate in non-health care activities introduces the opportunity for distraction, medical errors, and patient harm.^5–7

Despite the omnipresence of PEDs in the health care setting and growing recognition of the possibility of harm, data regarding use of PEDs in the perioperative setting are lacking. The use of PED in nurses and perfusionists has been described.^8,9 However, no contemporary validated PED survey study exists, which investigates anesthesia providers in the United States, a workforce that prides itself on vigilance and patient safety. To that end, we developed, validated, and conducted a survey of anesthesia providers (residents, nurse anesthetists, and anesthesiologists) at 3 hospitals within our institution that assesses the current state of inside the operating room (OR) PED use, outside the OR PED use, and observed PED use in others.

METHODS

Survey Development and Execution

This study was approved by the institutional review board, and the requirement of written informed consent was waived. A literature search was performed using the search phrase [“personal electronic device” OR “smartphone”] AND [“perioperative” OR “anesthesia” OR “anesthesiology” OR “operating room”]. Search results were reviewed, and relevant topics and questions were identified (S.B.P., J.R.R., S.P., C.R.R.) and collated. After the research team reached consensus, the 31-question survey was developed (Supplemental Digital Content, Appendix 1, https://links.lww.com/AA/D649). Questions were designed in 2 general forms: ordinal frequency rating scale and yes/no. Nine questions measured PED use either by the respondents themselves or observed in others and represented the key outcomes of interest. Other information collected included topics such as awareness of hospital policy on perioperative PED use, purpose of PED use (eg, social media, education, and shopping), experience with distraction related to PED use, awareness of lawsuit related to PED use, and demographics.

An e-mail with a link to the web survey was sent to all anesthesia providers at our institution (n = 812). Experts participating in the content elicitation were included in the distribution list and were allowed to participate in the survey. Reminder e-mails were sent to nonresponders 2 and 4 weeks after the initial e-mail. The web survey was closed to data collection 2 weeks after the second reminder e-mail. To promote accurate reporting, the survey was anonymous, meaning the investigators were unable to discern who had and who had not responded. Therefore, we were unable to compare available characteristics such as age and role for respondents versus nonrespondents. However, we were able to compare aggregate-level descriptive characteristics of participants to those of the target population using a χ² goodness-of-fit test.

Survey Validation

Content validity is the extent to which the questions in the questionnaire cover the breadth of the topic that is the subject of the questionnaire. Face validity of a questionnaire is an assessment of whether the questions included appear to address the concepts of interest.¹⁰ Both of these types of validity are subjective and were established through expert review of the draft survey by stakeholders within the department of anesthesiology, but otherwise not involved in this research.

To assess the validity and reliability of the newly developed survey, we took a simple random sample of 50 respondents with complete data. To facilitate analysis and interpretation, we investigated combining related questions to create a scale score. We assessed internal consistency reliability using Cronbach alpha to determine whether questions about the same setting (eg, inside the OR PED use) could be scaled together; alpha >0.7 supports combining question into a single score.¹¹ We hypothesized that questions about the same setting (eg, 3 questions about PED use inside the OR) would correlate higher with each other than with questions about a different setting (eg, 3 questions about PED use outside the OR). To address this hypothesis, we assessed scaling success, which is the frequency with which questions correlate more highly with their own scales than with other scales, after correcting for overlap.

We removed the 50 subjects used for the validation step from the dataset and evaluated our research questions using the data for the remaining respondents. We described the association between the 3 composite scale scores (inside the OR PED use, outside the OR PED use, and observed PED use by others) and participant characteristics and attitudes about PED use assessed in the survey. To maintain anonymity of respondents and to minimize spurious comparisons due to small sample size, categories with ≤5 respondents were not reported or analyzed.

Statistical Analysis

Nonresponse bias among anesthesia providers was tested between the reference population and the observed survey participants using the χ² goodness-of-fit test. Scaling success was assessed with the item-to-total correlation, adjusted for overlap. That is, to adjust for overlap within a scale, an item was removed from the scale and correlated with the mean score of the 2 remaining items in the scale. Univariate analysis was used to compare the 3 perioperative PED use scale scores to the participant characteristics, attitudes, and behaviors. Independent t tests were used to assess significance among dichotomized class variables, and ANOVA was used to analyze significance of age across the 4 groups. The effect size between the 2 groups was measured using Cohen d that was calculated by taking the difference between the group means and dividing by the pooled standard deviation. The standard interpretation for effect size using Cohen d is: small (≤ 0.2), moderate (0.2 to 0.8), and large (≥ 0.8).¹² All data were analyzed with the SAS version 9.4 software (SAS, Inc).

RESULTS

Survey Validation

Content validity was assessed by sending the 31-question survey to 4 attending anesthesiologists within one department to ask if any questions were superfluous and/or any additional questions needed. Face validity was assessed by asking these same 4 attending anesthesiologists if the wording of the questions was clear and if the questions in the different settings appeared to actually assess use in the different settings: For example, does a question about PED use inside the OR appear framed to assess PED use inside the OR versus general PED use. The 4 anesthesiologists were also asked to time how long it took them to fill out the survey to make sure that we could estimate to would-be respondents the time-commitment necessary to fill out the questionnaire. One of our research team (S.B.P.) met with each of the 4 attending anesthesiologists and brought their comments back to the research team. No changes were requested, and the average time required to fill out the survey was approximately 4 to 5 minutes.

The survey was sent to 813 anesthesia providers (82 anesthesia residents/fellows, 471 nurse anesthetists, and 260 anesthesiologists). Two hundred and ninety-nine (36.8%) completed at the survey (Table 1).

Table 1. - Survey Participant Characteristics and Nonresponse Bias Analysis

Age, y	Residents/fellows			In practice
	Survey respondents	Reference populationa	P value	Survey respondents	Reference populationb	P value
	n = 24	N = 82		n = 246	N = 260
<30	7 (29%)	20 (24%)	.55	--	--	--
≥30	17 (71%)	62 (76%)		--	--
<40	--	--	--	95 (39%)	244 (33%)	.19
40–49	--	--		77 (31%)	233 (32%)
50–59	--	--		48 (19%)	150 (21%)
≥60	--	--		26 (11%)	104 (14%)

Respondents do not sum to 299 due to missing data on age and/or role.

^aResidents/fellows.

^bPhysicians/advanced practice providers.

Table 2. - Item-to-Scale Correlations,^a Validation Sample (n = 50)

Item	Scale
	Outside OR	Inside OR	Others OR
Q2—text outside OR	0.65	0.27	0.20
Q3—call outside OR	0.62	0.40	0.21
Q4—browse Internet outside OR	0.66	0.38	0.21
Q5—text inside OR	0.42	0.64	0.53
Q6—call inside OR	0.35	0.57	0.45
Q7—browse Internet inside OR	0.30	0.67	0.38
Q8—observe others texting inside OR	0.17	0.44	0.74
Q9—observe others calling inside OR	0.30	0.48	0.59
Q10—observe others browsing Internet inside OR	0.18	0.47	0.72

Abbreviation: OR, operating room.

^aItem-scale Pearson correlation corrected for overlap (relevant item removed from scale). To support scaling success, correlations in bold should be larger than those in adjacent columns for the same question.

In a random sample of 50 respondents with complete data, scaling success was demonstrated with questions within a scale correlating more highly with that scale than another scale (Table 2); we observed 100% scaling success for all 3 scales. Furthermore, Cronbach alpha supported combining questions within each of the 3 settings to create a single mean score for inside the OR PED use (alpha = 0.78), outside the OR PED use (alpha = 0.79), and observed PED use by others (alpha = 0.82). The mean scores for the 3 composite scales were correlated as follows: inside OR and outside OR, r = 0.50; inside OR and observed others, r = 0.37; and outside OR and observed others, r = 0.22. Large correlations (ie, ≥0.8) would indicate redundancy across the scales. The small-to-moderate correlations observed here confirmed that the 3 scales were measuring different constructs.

Postvalidation Results

After removing the random sample of 50 respondents used for survey validation, there were 249 remaining respondents. Supplemental Digital Content, Appendix 2, https://links.lww.com/AA/D649, provides a summary of survey response data. With regard to often/frequent PED activity inside the OR, 24% reported texting, 5% reported talking on the phone, and 11% reported browsing on the Internet. With regard to often/frequent PED activity outside the OR, 88% reported texting, 26% reported talking on the phone, and 63% reported browsing the Internet. With regard to often/frequent PED activity observed in others, 52% reported others texting, 14% reported others talking on the phone, and 34% reported others browsing the Internet. With regard to policies and lawsuits, 74% of respondents were aware of their hospital’s PED policy, 40% responded that they considered a PED use policy reasonable and rational, and 71% had heard of a lawsuit related to PED use. With regard to PED-related distractions, 2% of respondents self-reported a distraction compared to 15% who had observed a distraction in others. With regard to surrogates for addiction, 31% of respondents reported trying to cut down on their PED use, and 21% felt the need to start using PED after putting it down. Finally, 80% of respondents recognized PED as a potential distraction for patient safety.

The associations between participant characteristics and other attitudes and the 3 composite scale scores are summarized in Table 3. Data are not presented for use of PED for online shopping and banking due to ≤5 individuals reporting this behavior “frequently” or “often.” Similarly, data are not reported for the question “Have you ever had a significant distraction within the operating room because of your personal electronic device” due to 5 or fewer individuals responding “yes.”

Table 3. - Univariate Analyses (3 Different Perioperative PED Use Scale Scores Compared to the Participant Characteristics, Attitudes, and Behaviors) Research Sample, N = 249

	Inside OR scale		Outside OR scale		Observed others OR scale
Nonstratified mean scale score (SD)	1.8 (0.9)		3.6 (0.9)		2.9 (1.1)
	Mean (SD)	P value	Mean (SD)	P value	Mean (SD)	P value
Q13 Does your hospital have a policy regarding the use of personal electronic devices in the operating rooms that you are familiar with?
Yes	1.75 (0.90)	.082a	3.63 (0.89)	.724	2.85 (1.11)	.274
No/don’t know	1.98 (0.93)		3.67 (0.89)		3.03 (1.01)
Q16 Have you ever heard about any law suits related with health care and use of personal electronic device data?
Yes	1.82 (0.92)	.716	3.67 (0.87)	.331	2.99 (1.04)	.035 a
No	1.77 (0.89)		3.55 (0.91)		2.65 (1.17)
Q17–22 Please rate the proportion of your personal electronic device usage during work
Job-related tasks
Never/rarely/sometimes	1.42 (0.59)	<.001 b	3.07 (0.95)	<.001 c	2.55 (0.93)	.001 b
Often/frequently	2.02 (0.98)		3.95 (0.68)		3.09 (1.12)
Social media
Never/rarely/sometimes	1.76 (0.88)	.095a	3.53 (0.91)	<.001 b	2.81 (1.08)	.012 a
Often/frequently	2.04 (1.02)		4.12 (0.52)		3.31 (1.02)
Entertainment
Never/rarely/sometimes	1.76 (0.89)	.031 a	3.55 (0.91)	<.001 b	2.83 (1.08)	.004 b
Often/frequently	2.14 (1.00)		4.14 (0.60)		3.42 (0.93)
Educational use
Never/rarely/sometimes	1.58 (0.71)	<.001 b	3.33 (0.97)	<.001 b	2.71 (1.01)	.008 a
Often/frequently	2.05 (1.03)		3.95 (0.67)		3.09 (1.12)
Q24 Have you ever recognized in others or heard about a significant distraction within the OR because of a personal electronic device?
Yes	1.84 (0.94)	.825	3.69 (0.80)	.705	3.28 (1.06)	.028 a
No	1.80 (0.91)		3.62 (0.91)		2.84 (1.08)
Q25 Do you think you should reduce the time spent with your own personal electronic device?
Yes	1.80 (0.97)	.878	3.69 (0.77)	.560	3.03 (1.02)	.244
No	1.82 (0.89)		3.62 (0.94)		2.85 (1.11)
Q26 Sometimes, after I stop using my personal electronic device, I have the need to start using it again
Yes	1.96 (1.09)	.281	3.86 (0.77)	.054a	3.19 (1.12)	.037 a
No	1.77 (0.86)		3.57 (0.91)		2.82 (1.06)
Q27 Do you think or recognize that the use of the personal electronic devices can affect your attention and thus patient safety?
Yes	1.74 (0.86)	.017 a	3.55 (0.88)	.007 a	2.93 (1.04)	.396
No	2.09 (1.06)		3.95 (0.85)		2.77 (1.26)
Age		.090		<.001		.148
(1) <40 y	1.94 (0.93)	1 vs 4b	3.83 (0.75)	1 vs 2a	2.88 (1.08)	1 vs 3a
(2) 40–49 y	1.79 (0.95)	2 vs 4a	3.63 (0.85)	1 vs 3a	2.95 (1.02)	1 vs 4a
(3) 50–59 y	1.83 (0.89)	3 vs 4b	3.61 (0.84)	1 vs 4c	3.14 (1.17)	2 vs 4a
(4) ≥60 y	1.39 (0.69)		2.79 (1.18)	2 vs 4c	2.48 (1.13)	3 vs 4b
Role
Resident/fellow	2.17 (0.99)	.097a	4.00 (0.51)	.006 a	3.41 (0.76)	.038 b
In practice	1.79 (0.91)		3.59 (0.92)		2.85 (1.10)

T test of mean score difference between 2 independent groups except for age, which was assessed with ANOVA.

P values in bold are statistically significant (P < .05).

Abbreviations: ANOVA, analysis of variance; OR, operating room; PED, personal electronic device; SD, standard deviation.

^aScore difference is associated with a small effect size >0.2.

^bScore difference is associated with a moderate effect size >0.5.

^cScore difference is associate with a large effect size >0.8.

Respondents were more likely to use their PED for job-related tasks often/frequently compared to never/rarely/sometimes; and PED use for job-related tasks was significantly associated with PED use/observations in all 3 settings. Although few respondents reported often/frequent PED entertainment use, those who reported often/frequent use were more likely to use/observe PED across all 3 settings. With educational use, respondents were more evenly split with regard to never/rarely/sometimes versus often/frequent PED use. However, for those who reported often/frequent use, they were more likely to use/observer PED across all 3 settings. Utilizing one’s PED for social media purposes was more commonly reported outside of the OR among respondents. Significantly more respondents who often/frequently used their PED for social media were more likely to report PED use outside the OR.

Individuals who had heard of lawsuits related to PED use in health care were more likely to report observing others’ use of PED inside the OR, but this awareness was not related to their personal PED use inside or outside of the OR. Individuals who had noticed a distraction caused by PED use were more likely to report observing PED use inside the OR by others. Individuals who reported a need to start using their PED after putting it down were more likely to report observing PED use in others. Individuals who reported recognizing the use of PED can affect their attention were less likely to report using their PED inside or outside the OR.

As respondent age increased, the mean score of outside the OR use decreased (ie, the older the respondent, the less outside the OR PED use). In general, the oldest respondent age group had the lowest mean scores for inside the OR PED use (small and moderate effects sizes), outside the OR PED use (all large effect sizes), and observes PED use in others (small and moderate effect sizes). In general, residents/fellows reported more PED use inside or outside the OR, and observed more PED use in others than those in the “In Practice group” (small-moderate effect size). This association was statistically significant for outside the OR and observes in others classifications. In general, the more the individuals did job-related tasks/educational use/entertainment/social media on their PED, the more likely they reported PED use inside the OR, outside the OR, or observing PED use in others.

DISCUSSION

We found that PED use is prevalent in the perioperative setting, and most providers recognize the potential for PED use to negatively impact patient safety. To our knowledge, this is the first study to validate a survey of perioperative PED use among anesthesia providers in the United States. Content and face validity were incorporated in the survey development, and strong internal validity and reliability were observed. As questionnaire validation is an ongoing process, future users should confirm the psychometric properties when used in a new setting and sample.

By delineating between self-observed PED use and PED use observed in others, we sought to minimize self-reporting bias. We found observations of PED use in others approximately 2 to 3 times that of self-reported inside the OR PED use. One possible explanation for this may be an awareness that intraoperative social media use—which unlike text or phone messaging cannot serve a patient-care purpose—is clearly inappropriate, leading to a greater degree of self-reporting bias. By differentiating between inside versus outside of OR use, our survey results demonstrate awareness of and an effort to lower PED use. If we consider self-reported outside OR use as the baseline PED use behavior, intraoperative PED use reductions were observed across all domains, both by self-report (eg, for texting, 24% intraoperative versus 88% outside) as well as observed in others (52% intraoperative versus 88% outside).

To discern deterrents against PED use, we asked questions about the impact of the possibility of lawsuits, hospital sanction, and addiction on PED use. We found no association between knowledge of malpractice lawsuits or awareness of hospital PED policy and lowered PED use (both inside and outside of OR). Possible explanations for these observations include the ubiquity and possible necessity for PED use, as well as the potential addictive nature of PED use and the implied inability to reduce use.¹³ Indeed, in our survey, surrogate questions adapted from the CAGE score¹⁴ for alcohol misuse to evaluate PED addiction were not associated with a decrease in PED use inside or outside the OR.

Our survey results among anesthesia providers are consistent with previous investigations of PED use among other medical professionals. Among nursing students, Cho and Lee⁸ demonstrated that 28% and 25% of the respondents were distracted by externally and internally initiated PED use, respectively. In addition, 43% of the respondents reported that they had witnessed other students’ distraction by PED. Self-reported use was lower than PED use observed in others.⁸ Similarly, Katz-Sidlow et al⁷ performed a cross-sectional survey of residents and faculty PED during inpatient attending rounds. Nearly 50% of respondents used PED during inpatient rounds. Although the most common reported application was for patient care, still 19% of respondents reported missing important clinical information due to PED distraction.⁷ A majority of respondents agreed that PED can be a significant source of distraction. In a survey of behaviors during cardiac surgery, PED use was observed in all members of the operative team, with the anesthesia providers responsible for nearly half of all interactions. Specific interactions included the sharing of vacation photos, members of the team “taking a selfie,” and perusal of home listings.⁵

Within the field of anesthesiology, Pinar et al¹⁵ performed a survey among anesthesia providers in Turkey with 955 respondents. Ninety-four percent stated they had used PED during anesthesia patient care. Phone calls (65%), messaging (46%), social media (35%), and Internet browsing (34%) were the most common applications. Overwhelming majorities responded that they never or seldom used PED during critical stages (97%) or were never distracted by PED (87%); however, nearly half (41%) reported observing PED distraction in colleagues.¹⁵ Slagle et al¹⁶ observed anesthetics in Tennessee, evaluating for noncare activities such as PED use. Distractions accounted for 2% of case time with a median distraction time of 2.3 seconds. Forty-nine percent of distractions were personal in nature; the most common distractions were personal Internet browsing, including educational content, and the use of PED for phone calls. Distractions occurred almost exclusively during the maintenance phase of anesthesia.¹⁶ Distractions in the OR have led to lawsuits with settlements made in >80% of cases with a median payment of approximately $725,000.¹⁷ The lack of association in our data among knowledge of lawsuits, hospital PED policy, and lowered PED use suggests potent denial mechanisms at work.

Not all data on PED use has negative implications with regard to patient safety. Soto et al² surveyed >4000 participants at the American Society of Anesthesiologists annual meeting and found 65% used pagers as their primary means of perioperative communication. Forty-five percent of these respondents reported significant delays in communications with 35% having observed medical error/injury as a result. Only 18% used cell phones as their primary means of perioperative communication, with such respondents reporting significantly less delays and medical error/injury than the pager group.² In a follow-up study, the investigators surveyed attendings, residents, and certified registered nurses (CRNAs) and found that 51% were unaware of whether their institution had a PED use policy and 21% reported at least 2 positive answers to a modified survey of addiction for PED. Nearly all respondents felt that the positives outweighed negatives of PED use.¹⁸ A higher proportion (71%) of our respondents were aware of litigation related to PED. Even in the face of this awareness, our data also revealed similar responses to questions evaluating PED addiction as 31% of our respondents have tried to cut down PED use and 21% feel the need to restart using their PED after stopping.

Our study is not without limitations. This effort was conducted across a single health care system, leaving the opportunity for sampling bias. However, our health care system spans 3 geographically different states with unique perioperative practice parameters. Additionally, it was noted by our respondents that several anesthesia providers have arranged to have communications from their hospital’s paging system diverted to their PED. Our questions did not account for PED use for this purpose. There are a number of PED-based clinical decision support and cognitive aid applications that may assist in patient care.^19–21 However, PEDs at our institution have not been incorporated into routine perioperative care (eg, note writing, laboratory follow-up, communications with OR teams, and monitoring). We reiterate that if this survey were to be used for another institution or organization, future users should confirm the psychometric properties. Although we asked about respondents’ knowledge of perioperative PED-related policies, other institutions have specific guidelines that may affect usage patterns. We did not ask questions specifically about PED use during critical phases of anesthetic care (eg, induction or emergence). Finally, we note that our references on this topic range as far back as 2006, and we acknowledge that the dynamic changes in the advancement and ubiquity of PED make it hard to compare even relatively recent years of literature to our current state.

In conclusion, we present and validate survey data of anesthesia providers from 3 hospitals in 3 different states within a single institution. Our data reinforce that PED use is prevalent among anesthesia providers who strive for vigilance in one of the highest acuity settings. Awareness of either lawsuit regarding PED use or hospital policy regulating PED use did not appear to deter anesthesia providers from utilizing their PED use in the perioperative setting. Fortunately, nearly 80% of our respondents acknowledge the impact PED use can have on provider attention and thus patient safety. Future efforts are warranted to further examine the nuances between safe and unsafe PED use during clinical care.

DISCLOSURES

Name: Steven B. Porter, MD, FASA.

Contribution: This author helped design the study, collect the data, and write the manuscript.

Name: J. Ross Renew, MD, FASA, FASE.

Contribution: This author helped design the study, collect the data, and write the manuscript.

Name: Stephania Paredes, MD.

Contribution: This author helped design the study, collect the data, and write the manuscript.

Name: Christopher R. Roscher, MD.

Contribution: This author helped design the study and write the manuscript.

Name: Matthew F. Plevak, BS.

Contribution: This author helped collect the data and write the manuscript.

Name: Kathleen J. Yost, PhD.

Contribution: This author helped design the study, collect the data, and write the manuscript.

This manuscript was handled by: Richard C. Prielipp, MD.