Internet Option in a Mail Survey: More Harm Than Good?

Ziegenfuss, Jeanette Y.; Beebe, Timothy J.; Rey, Enrique; Schleck, Cathy; Locke, G. Richard III; Talley, Nicholas J.

doi: 10.1097/EDE.0b013e3181e09657
Letters

Division of Health Care Policy and Research; Department of Health Sciences Research; Survey Research Center; ziegenfuss.jeanette@mayo.edu (Ziegenfuss, Beebe)

Division of Digestive Diseases; Hospital Clinico San Carlos; Complutense University; Madrid, Spain (Rey)

Division of Biomedical Statistics and Informatics; Mayo Clinic; Rochester, MN (Schleck)

Division of Gastroenterology and Hepatology; Mayo Clinic; Rochester, MN (Locke)

Division of Gastroenterology and Hepatology; Mayo Clinic; Jacksonville, FL (Talley)

Article Outline
Back to Top | Article Outline

To the Editors:

In population surveys, an Internet option may improve the response rate, if prospective respondents prefer one method of data collection over another.1 An Internet response option, is relatively inexpensive and may appeal especially to younger people2 (who often have lower survey participation).3 However, there is no viable Internet sampling frame for population surveys, necessitating initial contact through another mode, such as a mailed survey. Due to the lack of universal Internet penetration in the United States, the option of sending a Web address alone is not feasible, as some respondents will not be able to access the survey.

An emerging approach to harness the benefits of an Internet option is to mail a survey and offer an option to complete the survey online. The evidence is inconclusive, as some have observed no impact on response rate,2,4 and others have suggested that this approach actually lowers participation in general population surveys.1,5

We conducted a 16-page survey in March 2008 (described elsewhere6). The questionnaire was mailed to 770 randomly-selected Olmsted County, MN residents, aged 25–65 years, through the Rochester Epidemiology Project.7 This sampling frame contains data on age and sex, allowing us to compare response status on these factors. Our sample was randomized to receive a mail survey with or without the option of completing the survey on the Internet. The survey packets for the 2 groups were equivalent with the exception of a Web address and personal identification number (PIN) in the cover letter for the Internet option. All individuals received a topically related book as an incentive. A reminder letter was mailed to nonresponders 3–6 weeks after the initial mailing; the Web address and PIN were again included with the Internet option.

The response rate was 26%. The inclusion of the Internet option reduced the overall response rate (23% compared with 30%) (Table). Unlike what some have proposed, we saw no improvement in response rates for young adults aged 25–34 years. For all older-age categories, the response rates differed modestly in the expected direction.

Of 388 individuals randomized to the Internet option, only 7 completed the survey online; the remaining 81 completed the survey on paper and mailed back the form. The small number completing the survey online prohibits us from making any conclusions about the online respondents.

The overall decrease in response rate when offering an Internet option in mailed surveys is consistent with other recent studies.1,5 Finding no difference in response rates for adults aged 25–34 years is consistent with a Norwegian trial that saw no difference for persons aged 25–40.4 For older participants, including an online option could cause some potential respondents to put aside the task to complete later online, and never reengage.5 We concur with the recommendation by Dillman et al1 that availability of multiple response modes is likely to bring about more harm than good.

Jeanette Y. Ziegenfuss

Timothy J. Beebe

Division of Health Care Policy and Research

Department of Health Sciences Research

Survey Research Center

ziegenfuss.jeanette@mayo.edu

Enrique Rey

Division of Digestive Diseases

Hospital Clinico San Carlos

Complutense University

Madrid, Spain

Cathy Schleck

Division of Biomedical Statistics and Informatics

Mayo Clinic

Rochester, MN

G. Richard Locke III

Division of Gastroenterology and Hepatology

Mayo Clinic

Rochester, MN

Nicholas J. Talley

Division of Gastroenterology and Hepatology

Mayo Clinic

Jacksonville, FL

Back to Top | Article Outline

REFERENCES

1.Dillman DA, Smyth JD, Christian LM. Internet, Mail, and Mixed-Mode Surveys: the Tailored Design Method. 3rd ed. New York: Wiley; 2009:chap 8.
2.Friedman EM, Clausen NA, Hartzell M. The net effect: A comparison of internet and mail survey respondents. In: Statistics as a Unified Discipline. 2004:3519–3524. Available at: http://www.mathematica-mpr.com/publications/pdfs/internetmailsurvey.pdf.
3.Elliott MN, Edwards C, Angeles J, Hambarsoomians K, Hays RD. Patterns of unit and item nonresponse in the CAHPS® hospital survey. Health Serv Res. 2005;40(6 pt 2):2096–2119.
4.Brogger J, Nystad W, Cappelen I, Bakke P. No increase in response rate by adding a web response option to a postal population survey: A randomized trial. J Med Internet Res. 2007;9:e40.
5.Griffin DH, Fischer DP, Morgan MT. Testing an internet response option for the American Community Survey at US Department of Commerce. Paper presented at: The 2001 AAPOR Conference; 2001; Montreal, Quebec, Canada.
6.Talley NJ, Phillips SF, Melton J III, Wiltgen C, Zinsmeister AR. A patient questionnaire to identify bowel disease. Ann Intern Med. 1989;111:671–674.
7.Melton LJ III. The threat to medical-records research. N Engl J Med. 1997;337:1466–1470.
© 2010 Lippincott Williams & Wilkins, Inc.