BELING, JENNIFER BSN, RN; LIBERTINI, LINDA S.; SUN, ZHIYUAN MS; MASINA, V. MARIA MSN, RN; ALBERT, NANCY M. PhD, RN
Computers with Internet access have become commonplace in US businesses and homes; 70% of Americans go online in any given day.1 Many businesses and universities use the Internet to conduct business, such as distance learning, administration of exams, data collection,2 and dissemination of research findings.3 Using the Internet to collect survey and research data reduced costs, provided a more efficient distribution and collection of data,4,5 decreased potential for errors, allowed participants to feel more comfortable in responding to sensitive questions,6,7 and offered faster distribution to study subjects as well as the potential for quicker returns when compared with traditional mail.6
Only a few studies assessed characteristics and attitudes of completing electronic surveys in general. Information management students with positive attitudes toward computer surveys were more likely to perceive that the technology was advantageous for them.2 College students given a paper-and-pencil assessment of self-concept and a computer version of the same assessment preferred the computer version even though it took more time.8 No studies addressed respondents' preferences toward Internet research survey completion by a team of healthcare investigators in a hospital setting. Furthermore, no studies were found that addressed disease registry participant preference for Internet versus paper surveys sent through traditional mail in response to ongoing requests for data.
In hospitals with multiple and/or large patient registries that require ongoing data collection, an Internet based data collection system could promote data validity and reliability that are critical to new knowledge. Paper surveys create a two-step process of data entry: patient's response and research personnel's input into the database. Although an electronic data collection system would not alter the quality of responses received, it would eliminate user-interface errors since data are transferred directly into the clinical registry and checks for consistency are performed, preventing implausible missing values.
In our Colorectal Surgery Research department, people who previously had a surgical experience receive questionnaires by mail to determine disease progression and quality of life. Questionnaires are mailed at a frequency of multiple times per year at irregular prespecified intervals to one time per year. These registries maintain follow-up data for the life of the patient or in the long term, for example, 20 years. Since the volume of surveys continue to grow as new surgical encounters occur, the annual costs for completing registry work also continue to grow. In 2007, the average cost of mailing surveys was $1200 per database and did not include returned mail postage and personnel time to search addresses or assess current patient status. Ultimately, research personnel's time could be used to conduct ongoing quality monitoring procedures that ensure optimal data quality. The purposes of this study were to understand patient preferences for using Internet versus paper health surveys and learn if patient characteristics and survey length predict preference for completing health surveys by the Internet versus paper.
Setting and Sample
This study used a cross-sectional, descriptive, correlational design. After receiving institutional review board approval from a large tertiary-care hospital in Northeast Ohio, approximately 10 800 patients with a history of colorectal surgery who previously participated in registry research projects were our potential subjects. For each of five registries: Crohn disease (n = 1400), colorectal cancer (n = 4100), laparoscopic surgery (n = 2100), pelvic pouch (n = 3200), and pelvic floor dysfunction (n = −600), 250 patients were randomly selected to participate. Inclusion criteria were being 18 years or older, having current address listed in the database, and having received and completed at least one postdischarge follow-up survey in the last 2 years. There were no other exclusions to participation.
Data Collection Methods
The 1250 surveys were printed on colored paper, 250 per color to differentiate responses by registry type. Then, after using a computer to randomize cases from each of the five registries, surveys were mailed to potential participants with a cover letter, a research information sheet that provided details about the research and confidentiality of responses, and a self-addressed, prestamped return envelope. Only one request was sent to prospective participants.
An 18-item investigator-developed, anonymous survey was used to collect data. There were three sections: Internet use (6 items), Internet versus paper survey completion (2 items), and patient characteristics (10 items). We created a total Internet use/access Internet score composed of six questions related to Internet themes studied: (1) I have reliable Internet access most days of the week, (2) my Internet access is private, (3) I have Internet navigation skills, (4) the Internet is secure for private health information exchange, (5) I use the Internet to complete online health surveys, and (6) I would like to access my health information online. Content validity of the six items making up the total Internet use/access Internet score was assessed using the Lynn methodology.9 Sixteen healthcare professionals, including physicians, administrators, clinical nurses, and research personnel provided feedback on item relevance. The mean individual item content validity before revisions was .85 and improved to .89 after excluding one item with a content relevance score below the cutoff. In this study, the six-item total Internet use/access Internet items were assessed for reliability. Cronbach's α was .76.
Categorical variables were summarized using frequencies and percents. Continuous variables were summarized using mean and SD. A χ2 test was used to analyze the univariate association between survey responses and categorical patient characteristics. If 25% or more of the cells had expected counts less than 5, then Fisher exact test was used. If Fisher exact test was needed for a table larger than 2 × 2, Monte Carlo estimation with 10 000 samples was applied. The associations between survey responses and continuous patient characteristics were assessed using Spearman rank correlation. After a total Internet use/access Internet score was created, univariate associations between the total score and using a secure Web-based computer system, owning a computer, marital status, living with, education, age, or sex were analyzed using a two-sample t test, analysis of variance, or simple regression if appropriate. When analysis of variance was used, if an overall main effect was detected, multiple comparisons using Bonferroni adjustment were performed to detect which groups differed. For six paired comparisons, a Bonferroni-adjusted significance level was .0083. For three paired comparisons, a Bonferroni-adjusted significance level was .0167. A multiple linear regression was used to assess the association between the total Internet use/access Internet score and owning a computer, using a secure Web-based computer system, and preference for completing health surveys by the Internet after controlling for age, education, and marital status. Unless otherwise defined, all analyses were performed at a significance level of .05. All analyses were performed using SAS 9.1 software (SAS, Cary, NC).
Of 526 respondents (a 42.1% response rate), mean age was 56.9 (SD, 12.4) years, and participants were more often female (55%), white (96%), married (79%), and educated beyond high school (80%). Most participants owned their computer (89%), and 33% used the Internet to complete online health surveys. In this sample, 43% were familiar with our secure Web-based electronic health records system, and 20.5% used the electronic health record system to access medical information.
In this sample, 50% preferred to receive paper surveys at home, and 26% did not have a preference between Internet or paper survey completion (Figure 1). Participants preferring to complete research surveys by the Internet were younger (P < .001), more often educated beyond high school (P = .01), computer owners (P < .001), and users of an e-health medical record system (P < .001). Participants with a history of completing health surveys by the Internet were more likely married or divorced (rather than single or widowed) (P = .001; Figure 2), computer owners (P < .001), and had a history of accessing health records electronically (P < .001). Sex, ethnicity, self-rated health, and registry survey length were not associated with preference for or history of completing health surveys by the Internet.
Internet Use/Access Score and Participant Characteristics
Based on the total use/access Internet score, participants who were more likely to complete an Internet-based health research survey had access to a secure Web-based computer system (P < .001), were computer owners (P < .001), and preferred completing health surveys by the Internet (P < .001; significance level = .005 after Bonferroni correction). Characteristics of participants who were more likely to complete an Internet-based health research survey were younger age (estimate, −0.11; 95% confidence interval [CI], −0.15 to −0.07; P < .001), educated beyond high school (P < .00; significance level = .02 after Bonferroni correction), and married or divorced (P = .002, significance level = .008 after Bonferroni correction). After controlling for age, education, and marital status, the total Internet use/access Internet score was significantly associated with owning a computer (estimate, 7.2; 95% CI, 5.5-8.9; P < .001), using a secure Web-based computer system (estimate, 4.1; 95% CI, 3.0-5.3; P < .001), and preferring to complete health surveys by the Internet (estimate, 7.3; 95% CI, 5.8-8.8; P < .001 with significance level of .005 after Bonferroni correction).
The goals of this study were to determine the preference registry participants had for completing surveys electronically and factors associated with interest or preference for completing research surveys by the Internet. Surprisingly, 50% of respondents preferred to complete research surveys by paper from home, and 26% were neutral to the survey completion methodology. This study suggests that Internet survey research is a viable alternative to paper-based surveys for registry research data collection, but paper-based surveys should remain an option if it is important to receive generalizable data based on participant characteristics. In a healthcare environment, other investigators found that preoperative surgery patients and those with low-back pain treated in a spine clinic were willing and interested in completing computer-based surveys while in a medical center.10,11 Our study provides new knowledge that surgically treated patients enrolled in research registries are also willing and interested in completing computer-based research surveys via the Internet.
Except for ethnicity, patient characteristics in this study were broad in scope and were comparable to respondents' characteristics in other studies of computer-based surveys.10,11 In multiple studies, younger adults and those with greater education and/or household income were higher Internet users1,12 and were more likely to seek healthcare information online.12 After controlling for age, urban, low-income public housing adults were more likely to own a computer and use it regularly when they had higher education, employment, and higher income.13 Of healthcare-based studies, receptionists who were younger were more likely to be users of an electronic medical record and were more likely to have nonnegative reactions to computerized systems; however, marital status was not studied, and education was not associated with overall user reaction to electronic medical records.14
Our results add to the literature in that we found an association between married or divorced versus single marital status as a factor in completing computer-based health research surveys. When sociodemographic factors were assessed in online versus offline health information seekers, marital status was not associated with the source of health information.12 However, in a study of rural patients with cardiovascular disease who had similar characteristics to patients in our study, those who accessed health information through the Internet were more often married.15 Marital status might be a more important characteristic of people who have experienced healthcare issues and been treated in a hospital environment than of the general population. Alternately, marital status may be a more important factor for electronic research survey requests than for general Internet use or searching for healthcare literature. Since no other research focused on electronic or paper research survey completion, future research is needed to determine the importance of marital status in facilitating completion of electronic health research surveys.
In our study, completing health research surveys electronically was associated with owning a computer, having secure computer access, and having a general desire to complete health surveys electronically, even after controlling for significant personal characteristics. While we did not assess the exact same characteristics as other investigators, our findings are similar in that people more likely to complete surveys online were more likely to use a computer for other reasons.12 No other investigators asked participants about secure computer access; thus, more research is needed to learn if this is an important variable to completion of research surveys or completion of any survey that asks for personal information.
There were several limitations to this study. The study used a convenience sample of primarily white adults with gastrointestinal diseases who were already research subjects, located in a large Midwestern hospital. Generalizability of results to other regions of the United States, in nonwhite groups, in patients who are not already involved in a clinical research registry, or in patients with different medical problems is unknown. However, our patient characteristics and preferences were comparable with results of current published studies on electronic versus paper survey and general interest in seeking online health information. In our study, the preference for completing health research surveys by the Internet or paper was based on survey items that were assessed for content validity prior to study implementation and met study needs. In the literature, published data on information technology use were obtained from investigator-designed surveys, many of which were not shown to be content-valid data collection instruments prior to being used in research. Moreover, in some projects, multiple regression analyses were not performed to learn the mediators of outcome variables. Finally, other unstudied electronic health survey and patient demographic variables could be important variables.
In February 2009, the US government implemented the American Recovery and Reinvestment Act. An opportunity was created for healthcare improvement in the United States through the use of health information technology that promotes meaningful use of electronic health records.16 Clinical researchers will want to utilize electronic methods of data collection, in conjunction with the Internet. Thus, Internet-based health research survey completion should be offered as an option to paper-based research surveys. Newly identified advantages of using Internet-based health research surveys should be investigated as more healthcare centers convert fully to electronic health records and implement Internet-based data collection. An increase in electronic survey completion may shift role responsibilities for research personnel from collecting data to ensuring high-quality data, which may minimize data disparities and improve the value of data.
2. Huang H, Liaw S. Exploring users' attitudes and intentions toward the Web as a survey tool. Comput Hum Behav
3. Schmeida M. Telehealth Innovation in the American States
[dissertation]. Ann Arbor, MI: ProQuest; 2005.
4. Stewart S. Casting the net: using the Internet for survey research. Br J Midwifery
5. Kypri K, Gallagher SJ, Cashell-Smith ML. An Internet-based survey method for college student drinking research. Drug Alcohol Depend
6. Wharton CM, Hampl JS, Hall R, Winham DM. PCs or paper-and-pencil: online surveys for data collection. J Am Diet Assoc
7. Kaplowitz MD, Hadlock TD, Levine R. A comparison of Web and mail survey response rates. Public Opin Q
8. Vispoel WL. Computerized versus paper-and-pencil assessment of self-concept: score comparability and respondent preferences. Meas Eval Couns Dev
9. Lynn MR. Determination and quantification of content validity. Nurs Res
10. VanDenKerkhof EG, Goldstein DH, Blaine WC, Rimmer MJ. A comparison of paper with electronic patient-completed questionnaires in a preoperative clinic. Anesth Analg
11. Koestler ME, Libby E, Schofferman J, Redmond T. Web-based touch-screen computer assessment of chronic low back pain. A pilot study. Comput Inform Nurs
12. Cotton SR, Gupta SS. Characteristics of online and offline health information seekers and factors that discriminate between them. J Soc Sci Med
13. McNeill LH, Puleo E, Bennett GG, Emmons KM. Exploring social contextual correlates of computer ownership and frequency of use among urban, low-income, public housing adult residents. J Med Internet Res
14. Al-Azmi SF, Al-Enezi N, Chowdhury RI. Users' attitudes to an electronic medical record system and its correlates: a multivariate analysis. HIM J
15. Lear SA, Araki Y, Maric B, Kaan A, Horvat D. Prevalence and characteristics of home Internet access in patients with cardiovascular disease from diverse geographical locations. Can J Cardiol
For more than 8 additional continuing education articles related to electronic information in nursing, go to NursingCenter.com\CE.
© 2011 Lippincott Williams & Wilkins, Inc.