Web-based and Mailed Questionnaires

A Comparison of Response Rates and Compliance

Abstract

Background: 

We assessed response rates and compliance for a printed questionnaire and a Web questionnaire in a Swedish population-based study and explored the influence of adding personalized feedback to the Web questionnaire.

Methods: 

We assigned 875 subjects to 1 of 3 groups: printed questionnaire, plain Web questionnaire, or Web questionnaire with personalized feedback. The questionnaire had 2 parts, first a general section and then a dietary section.

Results: 

The response rate for the general section was 64% for the printed questionnaire, compared with 50% for the Web questionnaire with feedback. For the dietary questionnaire, the rates were reversed, resulting in a total response rate for the dietary questionnaire that did not differ between printed and web questionnaire with feedback.

Conclusions: 

Interactivity in the Web questionnaire increased compliance in completion of the second section of the questionnaire. Web questionnaires can be useful for research purposes in settings in which Internet access is high.

ArticlePlus

Click on the links below to access all the ArticlePlus for this article.

Please note that ArticlePlus files may launch a viewer application outside of your web browser.

• https://links.lww.com/EDE/A159

The Internet is an unexploited resource for collecting self-reported information in epidemiologic studies. Web-based questionnaires are easy to administer and offer several advantages, including immediate checks for incomplete or implausible answers, reminder messages to the respondent, automatic summarization of answers, personalized feedback, inclusion of illustrations or sounds to clarify complex questions, and hiding nonrelevant follow-up questions. Web questionnaires require no expense for printing, postage, manual check of incomplete answers, and transfer of data to an electronic format. The major cost for Web questionnaires is development of the system for handling the questionnaires; thus, once the system is established, the extra cost to add a few thousand or even a few hundred thousand participants to the study is relatively small.

However, access to the Internet can be biased with regard to age, sex, and education, among other factors. This differential access has restricted the use of Web-based methods to studies in specific groups with access to Internet, such as university students,¹ employees at certain companies,² or known Internet users.^3,4 Also, substantial proportion of the general population is unfamiliar with Web questionnaires and might hesitate to answer a Web questionnaire due to lack of experience or worries about security issues. However, these obstacles are likely to diminish over time. We explored response rates and compliance in a population-based study in Sweden, in which Internet access is estimated to be 80% in the working population.

METHODS

Study Population and Design

The study base comprised all persons 20 to 59 years of age living in a middle-sized county in Sweden in 2002. The county has the same distribution of inhabitants living in city (80%) and rural areas (20%) as the average in Sweden. We randomly selected 875 eligible persons from the Swedish Population Registry. Participation required filling out a questionnaire and undergoing several 24-hour recall interviews about physical activity. There were 3 versions of the questionnaire: (1) traditional printed questionnaire, (2) regular Web questionnaire, and (3) interactive Web questionnaire with personalized feedback.

The eligible sample was assigned randomly at the outset to one of the 3 versions of the questionnaires. All were sent an invitation letter informing them about the study. After 2 weeks, one third of the group was sent a printed questionnaire and two thirds were sent a letter with information on how to access the Web questionnaire, including details on use of the Web browser, the URL to our Web questionnaire, and an individual username. In addition, half of the Web group (one third of the total) was given the option of personalized feedback about their energy expenditure and body mass index. All nonrespondents were reminded after 3 weeks by a letter and contacted by phone after an additional 3 weeks, if necessary.

The questionnaire was divided into 2 parts. The first part was a general survey of lifestyle factors, such as physical activity, weight, height, smoking, and education, as well as the respondent's Internet habits. On the last page of this section, respondents were asked if they would be willing to answer additional questions about diet. Web respondents answering “yes” were linked to the second part of the questionnaire, whereas respondents to the printed questionnaire were sent the dietary questionnaire by returning mail. Personalized feedback was given to the interactive group on their intake of fiber, calcium, vitamin C, iron, and the composition of lunch and dinner meals. All basic questionnaires were followed by a validation study on physical activity.

The ethical committee at the Karolinska Institutet approved the study. Answering the questionnaire was considered to be informed consent.

Technical Aspects of the Web Questionnaire

We collaborated with a commercial Swedish Web survey company (Netsurvey²), which developed software to meet our requirements with respect to layout, feedback, and interactivity. An individual username for login provided the same identification in the Web questionnaires as in the printed questionnaires. This username prevented multiple answers from the same respondent or answers from individuals other than the invited respondents and allowed us to direct reminders to the nonrespondents only. The respondent used the username to establish an encrypted connection, using Secure Socket Layer, with the Web questionnaire system at Netsurvey. The username was linked to a unique number that identified the person and the study. The questionnaire answers were stored at Netsurvey along with the unique identification number.

To minimize typing errors by the respondent, the system immediately checked for implausible answers (eg. letters instead of numbers, unrealistic weight or height, or missing answers). When the system discovered an error, the respondent was given the chance to change the answer before moving on to the next page.

RESULTS

Among the subjects given the printed questionnaire, the response rate for the general section was 64% (Table 1). Response rates for the groups using Web-based questionnaires were 51% for the questionnaire without feedback and 50% with feedback. Additional details regarding response rates can be found in a supplementary table, available with the electronic version of this article.

TABLE 1: Response Rates for Each Group by Type of Questionnaires; Printed Questionnaire, Basic Web-based Questionnaire, and Web Questionnaire With Personalized Feedback

Compliance (willingness to answer the second part of the questionnaire) was higher for the Web questionnaires than for the printed questionnaire. Fifty-three percent of those who completed the general section of the paper questionnaire went on to complete the dietary part, compared with 58% of those responding to the Web questionnaire and 64% of those who were given the interactive Web questionnaire. Thus, the total response rate for the dietary questionnaire was similar for the 3 groups (34%, 29 and 32% respectively).

There were no differences in response to the 3 questionnaire options by age, body mass index, and current smoking, and only small differences by sex, education, and food habits (Table 2). The self-reported time spent answering the questionnaires did not differ between the groups. In general, those responding to the Web questionnaires were also more frequent users of the Internet and more able to arrange for privacy when using the Internet, assuring that no one was watching the screen.

TABLE 2: Description of the Study Population and Internet Habits by Type of Questionnaire

DISCUSSION

To the best of our knowledge, this is the first population-based study comparing the use of Web questionnaires with a similar printed questionnaire. Half of the people invited to use the Web questionnaire completed the general section of questionnaire. The willingness to answer a second part of the questionnaire was higher with the Web questionnaire than with the printed questionnaire, which suggests that those that responding to the Web questionnaire found the process more appealing than those who responded to the mailed questionnaire.

Web questionnaires can be used for research purposes in population-based settings in which Internet access is high, although we found that the initial response rate was lower than for the traditional printed questionnaire. In comparison, the willingness to answer a second questionnaire was higher when using a Web questionnaire instead of a printed questionnaire. Personalized feedback in the Web questionnaire further increased the compliance rate for a second questionnaire. Total response rates for the second part of the questionnaire were similar for the printed and the Web questionnaires.

ACKNOWLEDGMENTS

We thank Netsurvey for the use of their Web-survey software and hardware.