Skip Navigation LinksHome > August 2011 - Volume 29 - Issue 8 > Challenges of Using the Internet for Behavioral Research
CIN: Computers, Informatics, Nursing:
doi: 10.1097/NCN.0b013e3181fc4153
Feature Article

Challenges of Using the Internet for Behavioral Research


Free Access
Article Outline
Collapse Box

Author Information

Author Affiliations: College of Nursing (Dr Loescher) and College of Public Health (Ms Hibler, Mr Hla, and Dr Harris), The University of Arizona, and Skin Cancer Institute at the Arizona Cancer Center (Mss Hiscox, Harris, and Loescher), Tucson.

This work was funded by a Laurence B. Emmons Award from The University of Arizona College of Nursing and from the Skin Cancer Institute at the Arizona Cancer Center.

The authors have disclosed that they have no significant relationships with, or financial interest in, any commercial companies pertaining to this article.

Corresponding author: Lois J. Loescher, PhD, RN, College of Nursing, The University of Arizona, PO Box 210103, Tucson, AZ 85721 (preferred contact method) (

Collapse Box


Using the Internet in behavioral research remains a challenge. We developed a video intervention and conducted a pilot study that was designed to deliver and test the intervention via the Internet. One aim of this study was to evaluate the feasibility of using the Internet to both deliver the intervention and collect data from participants. This article summarizes procedures for delivering the intervention and survey via the Internet, obstacles encountered during delivery of the intervention and data collection, and lessons learned that can be applied to future research involving the Internet.

Using the Internet in behavioral research remains a challenge. We conducted a pilot study that was designed to deliver and test a video intervention to enhance skin cancer early detection in patients with melanoma. A secondary aim was to evaluate the feasibility of using the Internet to deliver the intervention and collect data from participants. The results reported here pertain to the second aim; procedures for delivering the intervention and survey via the Internet, obstacles encountered during intervention delivery and data collection, and lessons learned that can be applied to future research involving the Internet will be summarized.

Back to Top | Article Outline


Use of the Internet by the general population continues to increase worldwide. Approximately 73% of Americans have Internet access, about 60% go online daily, and more than 60% use e-mail each day.1 About eight of 10 users seek health information from the Internet,2 and many of those have stated that the Internet helped them to better understand their condition.3 The ability to access data more quickly via broadband Internet has greatly increased since 2005.4 However, as Internet access and use expand in the general population, many aspects of using this technology for behavioral research lag behind.

Benefits and barriers to using the Internet in research have been well documented.5-7 A major benefit is the ability to enroll individuals who would otherwise be unable to or prefer not to participate in person. In some cases, data collection instruments can be customized in terms of appearance, sequence, and delivery modes.8,9 Surveys usually can be posted easily on the Internet, and there are decreased costs for data collection and data entry.7,9-11 Major barriers to using the Internet as a research method include the often high level of skill and resources needed for Web site construction, data security, and participant knowledge or capability for navigating the Internet.7,12 Other barriers include limited participant access, which may be attributed to bandwidth limitations or lack of a computer, and issues with operating system compatibility.12 Barriers in common with mailed surveys include lack of sufficient guidance for answering questions, which increases chances for invalid responses13 and missing data.8

Although there are numerous reports describing the use of Web sites as interventions, there are few studies that involved prospectively using the Internet to both deliver a video intervention and collect survey data. Most publications describing Internet-delivered videos pertain to professional education or skills training, videoconferencing or telemedicine, or use of video games. Video sent to participants by mail or viewed at the study setting has been compared with information delivered on a study-specific Web site.14-16 We found only two studies that delivered a video intervention via the Internet. One randomized trial of smoking cessation involved obtaining consent, enrolling participants, delivering a video intervention, collecting survey data, and sending e-mail reminders via the Internet.17 Another randomized trial of physical activity and diet in middle-schoolers delivered video clips as an intervention via the Internet.14

Why is video an important intervention to consider? In the short term, video has been shown to be equivalent to written information on Web sites as a decision tool for cancer screening16 or was significantly more effective than a Web site for influencing knowledge and decision making about cancer screening.15 Video interventions have significantly increased risk reduction behavior.14,17 Evaluation of longer-term effects of video has not been undertaken, and some issues that arise when using this method merit further consideration. These are addressed in this article.

Back to Top | Article Outline


An interdisciplinary team developed and produced an evidence-based video on skin cancer early detection. The video was made available in QuickTime (Apple, Inc, Cupertino, CA) and Windows Media (Microsoft, Redmond, WA) formats. The video required a broadband connection of 40 Kbs to 1 Mbs (LAN) to be viewed without problems from popular Internet browsers. The video was hosted on a streaming server at the university's computer center on a secure server. E-mail boosters provided supplemental information on skin cancer early detection.

To assess the feasibility of using the Internet to deliver the intervention and collect data, we recruited adults who were at least 18 years of age, with a history of melanoma and self-reported ability to use the Internet, e-mail, and e-mail attachments. Using procedures approved by the institutional review board, patients consented to participate in the study during a clinic visit. At that time, we collected participants' contact information, including their e-mail addresses, and provided detailed instructions on how to use the technology for the study. Within 24 hours, participants received e-mailed instructions to access the link for survey 1 (baseline). The Internet version of this survey was created using Microsoft Solutions, which included Internet Information Services on a Windows 2003 Web server, with programming in ASP.Net 2.0 (, Courtland Manor, NY). This suite of products was the technology standard for the study host Web site. Participants had to complete all questions, or they would not be able to submit the survey. After completing survey 1, participants received the link to the video and instructions for accessing it. To track video usage, participants clicked on a link to indicate that they had watched the video, which then updated a counter. At months 1 and 2, participants received the e-mailed boosters. At month 3, participants completed survey 2 (posttest). We managed data using a database in Microsoft SQL server 2005 (Microsoft, Redmond, WA).

Conducting a study via the Internet requires precise administrative procedures. A secure area on the study Web site allowed the study administrator to add participants as they were enrolled. The administrator sent participants an encrypted e-mail message with unique log-in information, which included a randomly generated password to access the protected part of the Web site. The administrator also received a copy of the e-mail, which served as validation that participants were enrolled. A participant's e-mail address served as the log-in for the account. The e-mail message also suggested changing the password after logging in for the first time. With each Web site log-in, the system queried the database to determine the participant's status in the study process. Depending on the information in the database, the participant would see a link for the surveys or the video. The survey data transmissions were protected by 128-bit encryption based on a certificate purchased from Thawte Inc (Mountain View, CA). Thawte is a well-known provider of secure socket layer (SSL) data encryption solutions used for data privacy as it travels over the Internet. This certificate allowed for secure data transmission that met the requirements of the institutional review board.

Back to Top | Article Outline


Video Intervention

Viewing the video via the Internet presented the most challenges for participants (Table 1). The primary issues encountered involved programs for viewing the video and Internet connection speed. Most participants were able to open and view the video without trouble using either QuickTime or Windows Media Player; however, some participants reported problems with these programs that we were able to resolve, and for others, we were unable to identify the specific cause via e-mail and phone. A few participants stated that their Internet connection was not fast enough to support viewing the video; these participants had to watch the video at work, at the home of a friend or relative, or at a public library. Although we knew when participants opened the video, there was no way for us to determine if they actually watched the video from start to finish.

Table 1
Table 1
Image Tools

Whether these issues have occurred in previous Internet-based research is unclear. Internet-delivered video typically is not mentioned as an intervention in reviews of Internet-based research.5,18 There were no reports of technical difficulties in the one other Internet-based study that included video as an intervention.17 In another study, participants viewed the video on the Internet in a middle-school computer laboratory, with in-person technology assistance available.14 Because Internet-delivered video has not been extensively used as an intervention, a clear explanation of the methods, challenges with delivery, and troubleshooting efforts that occur during studies would benefit researchers who are interested in using this technology.

Back to Top | Article Outline

Despite pilot testing and revising the surveys prior to using them on the Internet, we encountered unforeseen survey challenges, including formatting of the Internet version, barriers faced by participants during completion of the surveys, and accessing the data during analysis. Because each survey item was tied to a specific data element in the database, adding, deleting, or otherwise modifying any item (including changes such as font size, which had a different appearance on the Internet survey, spacing between questions, or response options) once programmed would have required database and programming changes. Upon the initiation of data collection, we had to freeze changes to the survey to have a consistent data set for evaluation. The difficulty making changes stemmed from several factors including added costs for technical support and the timeline of the study. Any suggestions for improving the survey by the study administrators, however minor, could not be implemented without delaying the study.

Although other investigators have emphasized the importance of preliminary testing and revision prior to releasing a survey on the Internet,9,19 few have discussed other survey issues that may arise during data collection. For example, current literature emphasizes the importance of using surveys that are adaptable for use over the Internet and the ease of adapting pencil-and-paper surveys for the Internet, but does not describe procedures for that adaptation.7,19 Perhaps using an Internet survey design resource, such as Survey Monkey (Menlo Park, CA) or SurveyGizmo (Boulder, CO), would aid in creation of Internet- and user-friendly surveys. However, success of using these surveys in behavioral research has not been well documented, and the underlying technology may be subject to additional scrutiny from the researcher's institutional review board.

A technical challenge faced by some participants that applied to both the surveys and the video was cross-browser compatibility. Because the study Web site and secure areas were developed using Microsoft tools, Internet Explorer was able to adjust to some of the anomalies in the coding and display the Web pages as designed. This was not the case when participants used Mozilla Firefox (Mountain View, CA) or Safari (Apple Inc, Cupertino, CA). With those browsers, we found some pages that did not display properly, although we were able to isolate and fix the problems with the coding within those pages. Other investigators described similar issues.19 Finally, the data were stored in the Microsoft SQL server; however, the Internet technology staff would have to send the data files manually to the administrator for analysis. Eventually, we wrote code in SAS 9.0 (Cary, NC), which allowed for remote access of the database, making this process much more efficient.

Back to Top | Article Outline
E-mail Boosters

There were fewer problems with the e-mail boosters. Only one participant reported not being able to open a PDF attachment embedded in the booster; however, forwarding the message to a different e-mail address allowed him to open the file. We did not know how many e-mails were opened and read by participants, an issue that was experienced by other investigators.20 One way to better track e-mailed information would be to include a request both for a delivery and a read receipt for each message. This, however, would not inform researchers of whether participants opened attachments. Some participants responded to the e-mailed boosters with questions pertaining to their medical treatment at the clinic where they were recruited. The study administrator, who was not a medical professional, could not answer these questions and forwarded them to the clinic staff.

Back to Top | Article Outline

Issues with administration of the project primarily involved the participants' passwords and problems with study e-mails not reaching participants. Data-entry errors occurred during participant enrollment, including misspelled names and invalid e-mail addresses. When these were recognized, they were corrected by the administrator through the screens where updates could be made to participant demographic information. There were also a few instances where participants were unable to log on to the secure area of the Web site, owing largely to forgotten passwords or transposing zeros for the letter "O" or the number 1 for the lowercase "L" within the password. We reset their passwords in these situations. Some of the e-mail messages were automatically marked as spam or junk mail by recipient e-mail systems. A few participants had to retrieve e-mails from spam or junk mail folders. A few systems requested the administrator to identify the reason for contacting the participant and obtain permission for the e-mail to be accepted.

In future studies, we will incorporate lessons learned from this experience into the design phase. We will explore existing survey Web sites, which may have systems built that address some of the issues we experienced. This option may be less costly and resource intensive than building our own Web site. It may be necessary to incorporate in-person Internet training for participants or offer terminals they could use to complete the surveys with technical assistance available. We will explore methods to better track viewing of video interventions. Finally, we plan to take advantage of the capability of many statistical programs to directly access databases. These changes will help make the research more cost-efficient and easily managed by staff.

Back to Top | Article Outline


We learned from this pilot study that delivery of a video intervention and data collection using the Internet is not flawless, but it is feasible. Internet-delivered video interventions are rarely reported in the literature. The information gained from this study can be used to help researchers use the Internet for conducting behavioral research. We will continue to explore the efficacy of using advanced technology for providing health information to a broader portion of the population.

Back to Top | Article Outline


The authors acknowledge the contribution of the University of Arizona Information Technology Services, the Cutaneous Oncology Group at the Arizona Cancer Center, and the College of Nursing Writers Group.

Back to Top | Article Outline


1. Horrigan J. A typology of information on communication technology users. Accessed September 3, 2009.

2. Fox S. Reports: health information online. Accessed September 3, 2009.

3. Sabel MS, Strecher VJ, Schwartz JL, et al. Patterns of Internet use and impact on patients with melanoma. J Am Acad Dermatol. 2005;52(5):779-785.

4. Jones S, Murphy F, Edwards M, James J. Using online questionnaires to conduct nursing research. Nurs Times. 2008;104(47):66-69.

5. Ahern NR. Using the Internet to conduct research. Nurse Res. 2005;13(2):55-70.

6. Duffy ME. Methodological issues in Web-based research. J Nurs Scholarsh. 2002;34(1):83-88.

7. Rhodes SD, Bowie DA, Hergenrather KC. Collecting behavioural data using the World Wide Web: considerations for researchers. J Epidemiol Community Health. 2003;57(1):68-73.

8. Cantrell MA, Lupinacci P. Methodological issues in online data collection. J Adv Nurs. 2007;60(5):544-549.

9. Huntington A, Gilmour J, Schluter P, Tuckett A, Bogossian F, Turner C. The Internet as a research site: establishment of a Web-based longitudinal study of the nursing and midwifery workforce in three countries. J Adv Nurs. 2009;65(6):1309-1317.

10. Duffy M. Web-based research: an innovative method for nursing research. Can Oncol Nurs J. Spring 2000;10(2):45-49.

11. Henry J, Palmer BW, Palinkas L, Glorioso DK, Caligiuri MP, Jeste DV. Reformed consent: adapting to new media and research participant preferences. IRB. 2009;31(2):1-8.

12. Smith B, Smith TC, Gray GC, Ryan MA. When epidemiology meets the Internet: Web-based surveys in the Millennium Cohort Study. Am J Epidemiol. 2007;166(11):1345-1354.

13. Best S, Krueger B. Internet Data Collection. Thousand Oaks, CA: Sage Publications; 2004.

14. Frenn M, Malin S, Bansal N, et al. Addressing health disparities in middle school students' nutrition and exercise. J Community Health Nurs. 2003;20(1):1-14.

15. Frosch DL, Kaplan RM, Felitti VJ. A randomized controlled trial comparing Internet and video to facilitate patient education for men considering the prostate specific antigen test. J Gen Intern Med. 2003;18(10):781-787.

16. Ilic D, Egberts K, McKenzie JE, Risbridger G, Green S. Informing men about prostate cancer screening: a randomized controlled trial of patient education materials. J Gen Intern Med. 2008;23(4):466-471.

17. Swartz LH, Noell JW, Schroeder SW, Ary DV. A randomised control study of a fully automated Internet based smoking cessation programme. Tob Control. 2006;15(1):7-12.

18. Eysenbach G, Wyatt J. Using the Internet for surveys and health research. J Med Internet Res. 2002;4(2):E13.

19. Strickland OL, Moloney MF, Dietrich AS, Myerburg S, Cotsonis GA, Johnson RV. Measurement issues related to data collection on the World Wide Web. ANS Adv Nurs Sci. 2003;26(4):246-256.

20. Napolitano MA, Fotheringham M, Tate D, et al. Evaluation of an Internet-based physical activity intervention: a preliminary investigation. Ann Behav Med. 2003;25(2):92-99.


Data collection; Internet; Multimedia; Nursing methodology research

© 2011 Lippincott Williams & Wilkins, Inc.



Article Tools



Article Level Metrics

Search for Similar Articles
You may search for similar articles that contain these same keywords or you may modify the keyword list to augment your search.