The ability to recall historical physical activity accurately is important in chronic disease research. Epidemiologists and physical activity researchers often use self-reports of physical activity to identify the associations between physical activity and health outcomes. These surveys rely heavily on recall processes to obtain information about physical activity done at some point in the past, with recall frames as short as 1 wk(5) or as long as a lifetime (30). Efforts to validate self-reports of physical activity typically demonstrate moderate associations between recalled information and both direct (physical activity records, motion detectors) and indirect measures (fitness and fatness) of physical activity. However, at best, the surveys fail to explain more than 45% of the variance in these measures of physical activity(26). The ability of these surveys to only partially account for the overall variance raises questions about data quality, which, in turn, is one of the primary obstacles to drawing conclusions about the relation between physical activity and various health outcomes.
What accounts for the unexplained variance in these measures of physical activity? We contend that it is attributable to error in the cognitive operation employed in recalling and reporting physical activity. For example, some error may be due to respondents omitting the types of activities that they have done in the past. Other error may be attributable to their inability to accurately recall the details of a past physical activity, such as the duration of a particular episode. Factors that influence the degree of bias include the interval between the activity in question and the recall episode, the salience of the activities recalled, the social desirability of the responses, the personal characteristics of the respondent (e.g., age, sex, race), the behavior of the interviewer, and the interviewing techniques used to obtain the information (15).
Bias in survey research is important since it may over-shadow otherwise detectable relations between physical activity and other important factors or cause such a relation to appear when, in reality, there is none. For example, evidence suggests that physical activity during adolescence is inversely related to the incidence of cancerous tumors of the breast, colon, and reproductive system (24,37). Further, lifetime physical activity is reported to be inversely related to osteoporosis(30). However, if subjects in these studies consistently overreported a particular, highly salient activity, the relation between exercise and these outcome measures may have emerged despite the fact that, if the activity were consistently reported more accurately, the relation would not be detected. This type of bias, termed differential recall bias, can also be seen in case-control studies. In these studies, subjects with a specific disease report exposure differently than those without the disease. These differential errors of recall either inflate a risk estimate between an exposure and outcome or bias it toward a null effect.
While few reported studies have applied what is known about memory and decision making to enhance the accuracy of recall of physical activity data, research that applies cognitive principles to data collection procedures has recently received more attention in the epidemiological literature(54). To explore the issue of reporting bias in more detail, we now turn to a discussion of ways in which bias in reporting physical activity can be diminished. We will do this by first reviewing a subset of the research that has investigated the validity of physical activity self-reports and then focusing on cognitive research that has investigated ways of improving the validity and reliability of responses to surveys. As part of this discussion, we explore ways in which this knowledge can be employed to improve the accuracy of physical activity reports.
Studies of the Recall of Physical Activity
In 1988, Baranowski (2) published a review of the memory processes hypothesized to influence self-reports of physical activity. He identified only one study that specifically focused on the accuracy of recalled physical activity. In this study, Sallis et al.(50) used physical activity diaries to compare the ability of respondents to recall a broad spectrum of moderate, hard, and very hard activities on the Seven Day Recall physical activity survey. While recall of hard and very hard activities was quite accurate, Sallis et al. found that recall of moderate activities was quite poor. This pattern was consistent in men and women and across leisure, occupation, home, and conditioning activities.
Sallis et al.'s findings are typical of studies with similar foci that have been conducted since that time (22,31). For instance, Slattery and Jacobs (51) found that recall accuracy varied depending on how vigorous the activities were. These investigators used a subset of the Coronary Artery Disease in Adolescents Study (CARDIA) subjects to compare the accuracy of recalled physical activity with physical activity reports from 4 yr earlier. Physical activity was measured using the CARDIA Physical Activity History (PAH) survey. Correlations between recalled physical activity and the original reports were highest for vigorous physical activities (r = 0.57) and slightly lower (r = 0.45) for moderate activities.
Cunningham (17) obtained similar results in studying the characteristics associated with the ability to recall leisure-time physical activities done 1 wk earlier. In this study, subjects were more likely to underestimate the amount of walking and other light intensity activities they had engaged in. With more intense activities, this bias was less likely to arise. Blair et al. (5) also obtained results suggesting that recall of vigorous activities was more accurate than less strenuous activities. Using the Minnesota Leisure Time Physical Activity Questionnaire (MNLTPA) to identify the accuracy of physical activity recall up to 10 yr ago, these investigators found that correlations between the recall of the activity and earlier reports were highest for vigorous activities. They also noted that these correlations decreased with the passage of time.
To stimulate the recall of vigorous activities, Richardson et al.(48) employed a novel approach, using a calendar of the past year as a recall aid. While no specific retrieval cues were used, respondents were encouraged to use the calendar to stimulate recall of their physical activities over the past year. With this technique, these investigators showed high levels of agreement between 1-yr recall of vigorous physical activities on the MNLTPA and physical activity records kept during that year.
Findings from other studies suggest that differences in the validity and reliability of self-reports of physical activity emerged across different groups of respondents. Falkner et al. (19) studied the reliability of recalling physical activity from the previous 32 yr in the Buffalo Blood Pressure Study. Moderate correlations between realled and recorded physical activity scores were obtained (approximately 0.50). However, Falkner et al. found that recall accuracy varied depending on age, education, and contextual factors. For example, older males and subjects with less education tended to overstimate their recorded physical activity. On the other hand, younger males and subjects with more education tended to underestimate their physical activity. Additionally, recall varied depending on whether the subjects were working. For instance, younger males tended to provide more reliable responses for nonworking days than for working days.
Kledges et al. (29) found that physical activity recall varied depending on the sex and the obesity of the subjects. When asked to report their physical activities from the previous hour, males tended to overestimate their physical activities, whereas females tended to underestimate their activities. Obesity also influenced recall accuracy, as obese subjects consistently underestimated their physical activity levels in the previous hour. In another study that highlighted individual differences, Cumming and Klineberg (16) demonstrated that the level of agreement between past reports of physical activities and recalled physical activity was poor in elderly people. They found that accurate recall tended to be lower in males, older subjects, and those with cognitive impairments. Finally, Cunningham (17) found no differences in recall abilities across subjects of different genders, body masses, or physical fitness levels.
Taken together, this body of research suggests that two factors influence the validity and reliability of physical activity self-reports: 1) the characteristics of the activity, and 2) the characteristics of the respondent.
Survey Methods Research and Cognitive Psychology
We now turn to a presentation of the collaborative efforts of cognitive psychologists and survey researchers to increase the accuracy of survey responses (20,27,28,57). This collaboration has resulted in a shift of research foci away from factors related to the characteristics of the respondents and behaviors of the interviewers, which was the traditional focus of survey methods research, to a more task-oriented approach. As such, research aimed at reducing errors on surveys has begun to address the mental processes underlying the question-answering processes. This, of course, has implications for obtaining accurate histories of physical activity. This section will outline the cognitive processes involved in recall and provide suggestions about how researchers can apply the model to physical activity survey research.
The Cognitive Model
Several cognitive models have been developed that provide a framework for identifying the influence of various cognitive operations hypothesized to underlie the question-answering process(12,17,33,38). In general, these models postulate that four basic stages of cognitive processing influence the way in which respondents answer questions. These stages are the following: 1) comprehension, 2) retrieval, 3) decision-making, and 4) response generation. During the comprehension stage, respondents perceive, interpret, and then store the question in a short-term memory buffer. Following this, during the retrieval stage, respondents use the information in the question to determine retrieval objectives, to generate retrieval cues, and then to use these retrieval cues to search their memory. In the decision-making stage, respondents integrate and evaluate the retrieved information. If the retrieved information satisfies the retrieval objectives, then the respondent will generate a response. However, if respondents deem this information inadequate, then they will try different strategies to recall the information using estimating or heuristic methods. If necessary, respondents will attempt to derive numerical estimates or to make inferences about the recalled behavior. In the response generation stage, respondents convert the output from the decision-making stage into a form that meets the response requirements of the question.
Using the model to identify possible errors. We will use the College Alumnus Study survey (45) to apply this model to the following physical activity question: “How many city blocks or their equivalent do you walk each day?”
The survey defines the length of a city block as 12 blocks per mile.
While recall errors may arise in all four stages in a number of different ways, we focus on several of the more prominent types of errors that could influence the accuracy of the response to this question. First, in the comprehension stage, ambiguities exist in the question that diminish the control researchers have over how respondents define critical terms. For example, the term “walk” is vague. Specifically, the question does not specify what distinguishes walking from jogging and running. Nor does the question provide an adequate definition of a “city block.” This information is presented to the respondent, but it is not translatable into a simple metric. Additionally, the failure of the question to state clearly a reference period (the period of time a respondent should consider in responding to a question) leads to a similar problem. Because the distance walked for many people varies with location and over time, failing to specify the reference period could drastically influence the distances that the respondents report.
Second, in the decision-making stage, respondents must carry out many mental operations that could contribute to error. Because respondents rarely encode walking, they must infer the distance walked for any given walking episode using other information from the episode. For example, a respondent might only be able to remember the starting and end points of a particular walking episode and might therefore infer the distance walked by calculating the distance between these two points. Estimating distance in this manner may be errorprone, because in doing so, the respondent fails to take into account such important factors as the route traveled. Additionally, because respondents might mentally compute averages from walking episodes over several days, response errors could also arise from mistakes in these mental computations.
Third, in the response generation stage, respondents may have to convert the distance derived in the decision-making stage to the form required in the answer. For example, respondents who calculated the average distance walked in miles would then have to translate miles into “city blocks,” which is the metric requested by the question. For respondents who have done a lot of walking, this translation would be difficult and potentially subject to error.
Using this model as a framework, it is possible to identify potential sources of error in a questionnaire, and in doing so, begin the process of making improvements in the instrument. A frequently used method for identifying where in the question-answering process these errors arise is to use cognitive think aloud interviews(32,33,61). This type of interview is a widely used method for examining tasks that require extensive memory or decision-making operations (18). These interviews use verbal protocols to enhance the understanding of unseen cognitive activity occurring in the interval between the onset of a task and the execution of a response. Whereas other methods rely solely on inference to understand these processes, cognitive think-aloud interviews directly access at least some of the underlying cognitive processes. From the standpoint of interpretation, this method in not completely free of inference, however, strong arguments have been made about its effectiveness in understanding what is happening cognitively as an individual makes a response (18).
There have been isolated cases in which researchers who are interested in health outcomes have used this technique to refine their information-gathering instruments. For example, Subar et al. (54) conducted a study of dietary history in which they used cognitive think-aloud interviews with food frequency questionnaires. These interviews demonstrated that respondents found it difficult to answer questions about the frequency and portion size of different foods they had eaten and also experienced problems computing average frequencies for aggregated food items or for seasonal foods. As a result of this work, Subar et al. were able to pinpoint ambiguities in the survey, and by rewording and reordering problematic questions, they were able to increase the comprehensibility of the questions and the overall validity of their data.
Recalling the Type, Frequency, and Temporal Sequencing of Recalled Events
When asking about physical activities, the accurate retrieval and reporting of three types of information is crucial to obtaining valid data: 1) the types of activities that were done, 2) the frequencies of the activities, and 3) and when the activities were done (temporal sequencing). We now turn to a more detailed discussion of the processes influencing the accurate recall of each.
Types of Physical Activities Recalled
Reporting the types of activities respondents did at a point in the past is a critical component in most types of survey research. Several factors influence what respondents remember about past activities including: 1) how respondents encode and store the information in memory, and 2) how they retrieve the information from memory. In the following discussion of memory factors, we focus primarily on retrieval, discussing the way in which memory-related errors potentially arise during this process.
The role of memory. Two theoretical conceptions of autobiographical memory are critical to understanding the cognitive operations that influence physical activity self-reports. The primary theoretical foundation of autobiographical memory is the distinction between general, abstract memories and memories for specific events or experiences. One of the first memory researchers to draw attention to the distinction between general and specific memories was Tulving (59). He hypothesized that two forms of memory exist: episodic memory and semantic memory. According to Tulving, episodic memories have spatial and temporal associations, containing details about specific experiences from a particular time and place in the past. Semantic memories, on the other hand, contain information about general classes of events that tend not to have temporal or spatial associations.
Emerging views of autobiographical memory postulate that respondents nest episodic and semantic memories within a hierarchial structure(1,7,13,14,35,43,49). For example, Conway and Rubin (14) proposed a hierarchical model with three levels. According to this conception, the top level contains general knowledge corresponding to lifetime periods, the intermediate level contains general knowledge of events, and the bottom level contains memories for specific experiences. Respondents order memory traces at the bottom level according to their temporal relations. Within this framework, respondents recall the most recent events first and the most distant events last. Support for this hierarchical model of autobiographical memory has come from studies assessing memory for paired associations(3), memory for past diary entries(35), retrieval speed under different retrieval instructions (1), and retrieval speed when varying the informativeness of retrieval cues (46,47).
One of the critical assumptions that this latter model makes is that memory traces corresponding to specific events (the bottom level of Conway and Rubin's model (14)) interact with general memory traces(the intermediate level) during retrieval, resulting in what cognitive psychologists call “intrusions” (7). Specifically, respondents supplement fragments of memories for specific events with general information about that type of event. While this makes recollections of an experience coherent and clear, it contributes to inaccuracies. Research illuminating the differences between general and specific memory traces shows that, over time, memories for specific events lose their distinctiveness (34). When this happens, the memories often blend with other similar memories, making it more difficult to retrieve episodic information and increasing the likelihood that what is remembered is inaccurate.
A particularly salient example of this process was reported by Smith and his associates (52,53), who attempted to measure the frequency of intrusions in self-reports of dietary habits. As part of this study, respondents kept a diary of all food eaten for either a 2- or a 4-wk period. They were asked to recall what they had eaten immediately after this period or after a 2-, 4-, or 6-wk interval. Overall, 34% of all food consumption reports were intrusions. Additionally, as the retention interval increased from immediate recall to 6 wk, the percentage of intrusions increased dramatically from 28% immediately after the period during which the diaries were kept to 38% 6 wk after the diary recording period ended. These findings suggest: 1) memory traces corresponding to typical eating patterns were influencing the recall of dietary information from specific eating episodes and 2) memory traces for general dietary information had a larger influence on memory for specific episodes as the retention interval increased.
The intrusion of semantic information into the recall of specific experiences is especially problematic for survey researchers interested in obtaining information about highly routinized or frequently repeated events. A number of methods have been identified that potentially minimize this problem. For example, Means et al. (40) attempted to reduce the influence of general knowledge about doctor visits on memory for specific visits to the doctor. To do this they used a decompositional technique to aid in the recall of specific doctor visits. These investigators first had respondents recall event-specific details from their most recent visit to the doctor using a “guided memory” technique. Using this technique, subjects were encouraged to visualize the location of the event and to recall event-specific details. They repeated the procedure for the first visit and then for each subsequent visit. This procedure, in conjunction with a method designed to promote more accurate dating of events, had a profound effect on respondent reports, increasing the percentage of doctor's visits accurately reported from 32% to 67%.
Implications for physical activity instruments. The level of detail about physical activity habits obtained in survey research is often limited by survey space and time constraints. However, we contend that, even with these constraints, the above cognitive strategies can be used to improve the accuracy of recall on most physical activity surveys.
The above discussion of autobiographical memory suggests that the ability to recall specific details about physical activity events may be clouded by general memories of past participation habits. To illustrate how this might occur with self-reports of physical activity, we focus on a question from the Seven-Day Recall questionnaire (6), which is a popular survey that asks questions about participation in activities done in the past week:
“Did you participate in any moderate activities this past weekend?”
Examples of moderate activities are presented to respondents prior to the presentation of the question. Several cognitive difficulties may arise in answering this question including the aforementioned problem with intrusions. Consider the case of a respondent whose Saturday morning routine includes walking briskly every weekend. If the respondent failed to walk during the previous weekend, but forgot much of what he or she did on those days, then the respondent would reconstruct the weekend by substituting information from memory traces corresponding to general patterns of exercise for this forgotten information. Therefore, because the respondent usually walks on Saturdays, he or she may inaccurately report walking during the past weekend.
One way to avoid these types of intrusions between semantic and episodic memory traces for highly routinized events would be to use the basic technique of Means et al. (40). For example, given the example illustrated above, respondents might be asked to remember the most recent time that they walked briskly. They would identify unique, event-specific details about that particular walking episode. After recalling this episode, they would then be asked to recall the next most recent walking episode, and continue repeating this procedure until they had recalled all of the potential occasions within a particular reference period.
Another method of enhancing the distinction between episodic and semantic memory traces is to employ a procedure that focuses on setting up appropriate retrieval cues for the episodic events (41). First, an interview would ask the respondent questions that established retrieval cues:“Have you every had periods in your life where you have taken up vigorous activities? Why did you want to be more vigorously active? How did you go about becoming more vigorously active? How successful were your attempts?” The interviewer would then follow these cues with more specific questions about vigorous physical activity exposures: “How old were you when you initiated your first attempt at becoming more vigorously active?” “What activities did you do?” “How long did you do these activities?” The latter question could be framed in minutes, hours, days, weeks, months, or years. This approach would then be executed for each period of vigorous activity initiated at some time in the past.
Each of these techniques employs slightly different procedures to enhance the recall of specific experiences. In both cases, the likelihood of intrusions biasing physical activity self-reports is minimized by forcing respondents to cognitively distinguish between discreet episodes.
Frequency of Recalled Events
Identifying how often respondents engaged in an activity at sometime in the past is important when computing summary scores (e.g., kcal·wk-1). Reporting the frequency of a behavior or an event relies extensively on both retrieval and decision-making processes. In the cognitive and survey research literatures, it has been hypothesized that respondents use several different estimation processes in estimating frequency, including episode enumeration (56), rate-based estimation, heuristic-based estimation (60), and estimation based on some innate sense (25). We will focus on episode enumeration and rate-based estimation, as these strategies have been the primary focus of survey methods investigations.
Sudman and Bradburn (56) proposed a memory model in which they claimed that individuals answer questions about frequency by using an episode enumeration process. With episode enumeration, respondents retrieve all discrete episodes of a particular behavior during a specific period and then count these episodes. However, research conducted since that time suggests that individuals vary the way in which they generate frequency estimates based on situational and contextual factors(42). For example, Blair and Burton(4) suggested that the tendency to use episode enumeration and the accuracy of this method of estimating frequency, varies with the length of the reference period and with the frequency of the behavior in question. The investigators observed that as event frequency increased, the likelihood of using episode enumeration decreased. Burton and Blair(11) extended these findings by exploring the conditions under which respondents are biased to use episode enumeration. They found that respondents were more likely to use episode enumeration if they were asked about infrequent, vivid events or if the question had a short reference period. They also observed that respondents used episode enumeration when they were given more time to respond to the question. However, this applied only to events that were easily retrievable.
With a rate-based strategy, on the other hand, respondents determine how frequently they engage in a behavior based on the average frequency of their participation habits. Respondents then multiply the frequency by the length of the reference period. This strategy works well to recall behaviors that are performed frequently and at regular intervals. For example, in answering the question “How many times during the past year did you play singles tennis?” the accuracy of rate-based strategies or episode enumeration strategies will depend on the regularity and frequency of usual tennis playing. Those who routinely play tennis every Sunday afternoon can most accurately estimate frequency by multiplying one game of tennis per week by 52 wk. On the other hand, respondents who play tennis infrequently or at irregular intervals, will be more accurate when they attempt retrieve and count the specific times that they have played tennis each month.
Implications for physical activity instruments. There is considerable variability in the frequency with which people perform physical activities. Some people perform physical activities very frequently and on a regular basis. Others do not. Thus, it may be more prudent to base the strategy for recalling the frequency and duration of physical activities on the length of the reference period. For short reference periods, such as 7-30 d, questions may be more effective if they use episode enumeration techniques. For those with longer reference periods, rate-based strategies may be more effective in eliciting information about the recall of physical activities.
For example, the MNLTPA (58) asks respondents to recall the months in the past year in which they engaged in various household and recreational activities. For those months in which they engaged in an activity, they are asked to identify the number of times they conducted the activity. It is reasonable to assume that with a reference period of 1 yr, respondents would be unlikely to rely on episode enumeration, especially if they engaged in the activity with any regularity. Therefore, with this questionnaire, strategies which encourage respondents to use rate-based strategies would, in all likelihood, be most beneficial.
Temporal Sequencing of Recalled Events
Because most physical activity surveys ask about activities conducted during a particular period of time, remembering when an activity was done is an important component of accurate self-reports. Reference periods in these surveys may vary from days to years; therefore, identifying the information retrieval and decision-making strategies that influence the process of dating events is critical to producing accurate self-reports.
The ability to date a particular event is dependent on first being able to retrieve the event. Once it is retrieved, respondents can obtain temporal information about the event in two ways (23). Some memory traces have intact temporal tags that respondents have access to when they retrieve the event. Cognitive psychologists term these “landmark events” (36), and they usually represent significant public (e.g., presidential elections, holidays) or personal events(e.g., birthdays, anniversaries).
For other memory traces, such direct temporal associations are inaccessible. In these cases, when temporal information is not directly available, the date of the event can be inferred based on other information. One way in which this can be done is through associated events or episodes. Bradburn et al. (8) claimed that each individual event is part of a group of events that are connected by temporal and causal links. Accessing information about an event at this level can enable information from other events to be accessed at the same time as the event in question. If one or more of these related events have a date associated with them, then a date can be inferred on the basis of the temporal relation between these events and the event in question. Therefore, if a respondent tended to only play tennis during his or her summer vacation (which presumably has a more memorable date than an individual tennis game), then remembering the date(s) on which he or she played tennis might be facilitated by remembering that these activities took place during summer vacation.
A different approach to methods used in recalling temporal information is posed by Brown et al. (10). They suggest that temporal information about an event is inferred through the accessibility or the clarity of one's memory for the event. Thus, events that have occurred more recently are easier to retrieve and therefore will be more clearly remembered. According to Brown et al., an event's date is first calculated based on the accessibility of the event. It is then adjusted based on other information, such as the range of possible dates and the importance of the event.
Forward telescoping. Most items in physical activity surveys are asked in relation to a reference period (e.g., the past 4 wk). However, locking a respondent into a fixed reference period increases the chances of recall errors through a process called forward telescoping. Forward telescoping can artificially inflate estimates of how often a behavior has occurred by displacing events forward in time (55). Means et al. (40) have demonstrated the effects of forward telescoping on the recall of doctor's visits. When respondents were asked to recall the number of times they had visited the doctor in the past year, respondents were able to correctly report 38% of all recorded doctor's visits in the past year. However, approximately 28% of the recalled visits were subject to forward telescoping. In this case, respondents reported a date that was more than 15 d later than the actual doctor's office visit.
Brown et al. (9,10) hypothesized that forward telescoping occurs when individuals incorrectly attribute the clarity of the memory of a past event as the most recent occurrence of the event. This observation was shown in a study were respondents were asked to date high-knowledge public events (e.g., assassinations) and low-knowledge public events. The high-knowledge public events were more susceptible to forward telescoping than the low-knowledge events. According to this interpretation, recall of vigorous activities is more likely to be inflated on physical activity surveys. Because these events have strong, physiological cues (e.g., increased heart rate and breathing), they are more likely to be clearly remembered and therefore more susceptible to forward telescoping compared with less intense activities. We note, however, that this susceptibility to telescoping could be offset by other factors, such as the fact that vigorous activities tend to be highly salient and are therefore less likely to be forgotten. Thus, despite this, their heightened memorability might override the influence of telescoping which, in turn, would explain why past research suggests that self-report data for these activities tends to be more valid than self-reports of other types of activities.
Methods of minimizing forward telescoping. Several researchers have proposed methods to reduce the risk of forward telescoping in survey research. One method is to use bounded interview procedures(44). Here respondents are interviewed several times in succession, using identical reference periods for each interview. With each subsequent interview, interviewers repeatedly remind the respondents of the events they had reported in earlier surveys, and in doing so, presumably reduce the likelihood that these events will be included in responses to the current questions.
A second method focuses on enhancing the ability of respondents to remember a specific event. This is done by asking the respondents to recall why and how they did specific behaviors before respondents are asked about the temporal patterns of the behaviors (41). Means and her associates(38,41) employed this technique with a great deal of success in a survey focusing on smoking cessation activities. Prior to asking detailed questions about the dates of their past smoking cessation attempts, Means et al. asked respondents why they wanted to stop smoking, how they attempted to stop smoking, and the outcome of their cessation attempts. The responses to these questions presumably established relevant retrieval cues for subsequent questions addressing the dates of their smoking cessation efforts. Compared with a sample of respondents who were not asked these questions about their smoking history, respondents who were asked“contextual” questions more accurate dated their smoking cessation efforts.
A third method focuses on increasing the accuracy of dating an event. This can be done by using a calendar that contains various public and personal landmark events (36,39,40). It is assumed that the respondent can use the dates of these landmark events to infer the date of the event of interest. When using both public (e.g., eruption of Mount St. Helens) and private landmarks (birthdays), Loftus and Marburger(36) were able to significantly reduce the influence of forward telescoping on frequency estimates about crime victimizations. In dating medical events, Means et al. (40) also used public and personal landmarks to enhance recall, and they observed a 15% increase in the number of doctor's visits that respondents were able to accurately date within 15 d of the actual visit.
Implications for physical activity instruments. Methods used to reduce the amount of forward telescoping in physical activity research can be applied to the following question from the MNLTPA(58):
“Have you performed dancing (ballroom, square and/or disco) in the past 12 months?”
If a respondent didn't dance within the past 12 months, but did dance 13 months ago, reports of dancing could be inflated because of forward telescoping. Given the long reference period and the likelihood that no direct temporal association exists within the memory trace, it is possible that the respondent could misestimate the actual date of the event and include it in the above response.
Forward telescoping could be reduced with this scenario in one of three ways: 1) repeatedly reminding respondents that the time frame is for the last 12 months only, 2) having respondents recall contextual elements of any dancing experiences, or by 3) referring to a calendar. Perhaps the most effective and least burdensome way of counteracting forward telescoping is use the latter method, incorporating a calendar with landmark events into the protocol (39). Steps an interviewer could use to minimize errors attributable to forward telescoping are the following:
- Note the interview date and the beginning of the reference period.
- Highlight prominent public landmark events that occurred during this time.
- Encourage the respondent to recall public and personal events during this time. It is important that respondents also recall landmark events near the beginning of the reference period to serve as a boundary for the reference period.
- Encourage the respondent to recall physical activity events associated with public and personal landmark events. The associations would be through temporal associations occurring with the events (e.g., “I went out dancing a couple of days before my birthday, which occurred before the period they are asking about”).
By structuring the interview in this manner, the respondent has the appropriate cues at his or her disposal to more accurately date events around the beginning of the reference period, and in doing so, minimize the tendency to overreport the types and frequency of activities in which one has engaged.
Limitations to Using Cognitive Methods in Survey Research
We recognize several issues that complicate the application of cognitive methods to physical activity surveys and potentially compromise their effectiveness. First, research situations arise that are not conducive to in-person interviewing, thereby necessitating the use of alternative modes of data collection, such as self-administered questionnaires. This, of course, poses a problem for certain methods that are most effectively employed when the respondent has face-to-face contact with an interviewer. For instance, an interview method using a calendar with landmark events(40) is much more difficult to administer and presumably much less effective with a self-administered questionnaire. Because there is no interviewer controlling the progression of the interview, the process of focusing on the calendar and eliciting personal landmark events from the respondent may be seriously compromised.
Second, the effectiveness of many of the techniques described above is highly dependent on how often a respondent engages in a particular type of activity, which, in turn, may be highly variable across respondents(42). For example, the effectiveness of different strategies in eliciting frequency information varies depending on the frequency and regularity of the activity and the length of the reference period. Because the frequency and regularity of a given activity can be highly variable across individuals, it becomes difficult to employ a strategy that will maximize the effectiveness of these estimates for all individuals.
A third important issue is the resource investment necessary to identify and address shortcomings in physical activity instruments. Past researchers in epidemiology and other areas that rely extensively on self-report data have employed cognitive think-aloud interviews to obtain this information(33,54,61). This method is generally effective in accessing cognitive processes underlying responses to questions, thereby illuminating potential sources of respondent error. However, this method adds significantly to the amount of preliminary pilot work that a particular instrument requires during its development, which has proved to be too great a cost for many researchers.
Addressing these issues remains a challenge for cognitive psychologists, survey researchers, and epidemiologists who focus on physical activity. While attempts to improve the quality of self-reported physical activity data requires the expenditure of resources that would otherwise be used to investigate more substantive issues, we contend that the potential validity problems arising from the failure to adequately measure physical activity outcomes by far outweighs these concerns.
In this paper, we have attempted to address the issue of data quality in physical activity surveys by outlining a method of developing and modifying physical activity instruments that taps our understanding of cognition. Specifically, we presented a cognitive model of the question-answering process that has been used successfully in the past to identify problematic questions in surveys addressing such topics as alcohol and drug use(21). Using this basic approach, it is possible to develop new physical activity instruments or to modify existing instruments so that inaccuracies arising from problems related to cognitive processing are minimized. We also highlighted different cognitive-based methods that have been employed in survey research, specifically focusing on the reports that a respondent is usually required to generate with physical activity surveys: 1) remembering an event or activity, 2) estimating how often or how long they engaged in an activity, and 3) remembering when they engaged in an activity. While many researchers who rely on self-reported physical activities have incorporated these methods into their interview protocols, we believe their apparent success in improving data quality in survey research warrants their more widespread use in research investigating physical activity.
1. Anderson, S. J. and M. A. Conway. Investigating, the structure of autobiographical memories. J. Exp. Psychol. Learn. Mem. Cognit.
2. Baranowski, T. Validity and reliability of self report measures of physical activity: an information-processing perspective.Res. Q. Exerc. Sport
3. Barsalou, L. W. The content and organization of autobiographical memories. In: Remembering Reconsidered: Ecological and Traditional Approaches to the Study of Memory
, U. Neisser and E. Winograd(Eds.). Cambridge, England: Cambridge University Press, 1988, pp. 193-243.
4. Blair, E. and S. Burton. Cognitive processes used by survey respondents to answer behavioral frequency questions. J. Consum. Res.
5. Blair, S. N., M. Dowda, R. R. Rate, et al. Reliability of long-term recall of participation in physical activity by middle-aged men and women. Am. J. Epidemiol.
6. Blair, S. N., W. L. Haskell, P. Ho, et al. Assessment of habitual physical activity by a seven-day recall in a community survey and controlled experiments. Am. J. Epidemiol.
7. Bower, G. H., J. B. Black, and T. J. Turner. Scripts in memory and text. Cognit. Psychol.
8. Bradburn, N. M., L. J. Rips, and S. K. Shevell. Answering autobiographical questions: the impact of memory and inference on surveys.Science
9. Brown, N. R., L. J. Rips, and S. K. Shevell. The subjective dates of natural events in very-long-term memory. Cognit. Psychol.
10. Brown, N. R., S. K. Shevell, and L. J. Rips. Public memories and their personal content. In: Autobiographical Memory
, D. C. Rubin (Ed.). Cambridge, England: Cambridge University Press, 1986, pp. 137-158.
11. Burton, S. and E. Blair. Task conditions, response formulation processes, and response accuracy for behavioral frequency questions in surveys. Public Opinion Q.
12. Cannell, C. F., P. V. Miller, and L. Oksenberg. Research on interviewing techniques. In: Sociological Measurement
, S. Leinhardt (Ed.). San Francisco: Jossey-Bass, 1981, pp. 389-437.
13. Conway, M. A. and D. A. Bekerian. Organization in autobiographical memory. Mem. Cognit.
14. Conway, M. A. and D. C. Rubin. The structure of autobiographical memory. In: Theories of Memory
, A. E. Collins, S. E. Gathercole, M. A. Conway, and P. E. M. Morris (Eds.). Hillsdale, NJ: Erlbaum, 1993, pp. 103-137.
15. Coughlin, S. S. Recall bias in epidemiologic studies.J. Clin. Epidemiol.
16. Cumming, R. G. and R. J. Klineberg. A study of the reproducibility of long-term recall in the elderly. Epidemiology
17. Eisenhower, D., N. A. Mathiowetz, and D. Morganstein. Recall error: sources and bias reduction techniques. In: Measurement Errors in Surveys
, P. P. Biemer, R. M. Groves, L. E. Lyberg, N. A. Mathiowetz, and S. Sudman (Eds.). New York: Wiley & Sons, 1991, pp. 127-144.
18. Ericsson, K. A. and H. A. Simon. Verbal reports as data. Psychol. Rev.
19. Falkner, K. L., M. Trevisan, M. Zielezny, J. Freudenheim, W. Winkelstein, and R. P. Fisher. Relative validity of recall of physical activity in the distant past (Abstract). Am. J. Epidemiol.
20. Fienberg, S. E. and J. M. Tanur. Combining cognitive and statistical approaches to survey design. Science
21. Forsyth, B., J. Lessler, and M. Hubbard. Cognitive evaluation of the questionnaire. In: Survey Measurement of Drug Use: Methodological studies
, C. F. Turner, J. T. Lessler, and J. C. Gfroerer(Eds.). Washington, DC: U.S. Government Printing Office, 1992, pp. 13-52.
22. Friedenreich, C. M. Improving long-term recall in epidemiologic studies. Epidemiology
23. Friedman, W. J. Memory for the time of past events.Psychol. Bull.
24. Frisch, R., G. Wyshak, J. Witschi, N. L. Albright, T. E. Albright, and I. Schiff. Lower lifetime occuring of breast cancer and cancers of the reproductive system among former college athletes. Int. J. Fertil.
25. Hasher, L. and R. T. Zacks. Automatic processing of fundamental information. Am. Psychol.
26. Jacobs, D. R., Jr., B. E. Ainsworth, T. J. Hartman, and A. S. Leon. A simultaneous evaluation of 10 commonly used physical activity questionnaires. Med. Sci. Sports Exerc.
27. Jobe, J. B. and D. J. Mingay. Cognition and survey measurement: history and overview. Appl. Cognit. Psychol.
28. Jobe, J. B., R. Tourangeau, and A. F. Smith. Contributions of survey research to the understanding of memory. Appl. Cognit. Psychol.
29. Kledges, B. C., L. H. Eck, M. W. Mellou, W. Fulliton, G. W. James, and C. L. Hanson. The accuracy of self-reports of physical activity. Med. Sci. Sports Exerc.
30. Kriska, A. M., R. B. Sandler, J. A. Cauley, R. E. Laporte, D. L. Hom, and C. Pambianco. The assessment of historical physical activity and its relation to adult bone parameters. Am. J. Epidemiol.
31. Lee, M. M., A. S. Whittemore, and D. L. Jung. Reliability of recalled physical activity, cigarette smoking, and alcohol consumption. Ann. Epidemiol.
32. Lessler, J. T. and M. Holt. Using response protocols to identify problems in the U.S. Census Long Form. Proceedings of the Section on Survey Research Methods
, American Statistical Association, 1987, pp.262-266.
33. Lessler, J. T., R. Tourangeau, and W. Salter. Questionnaire design in the cognitive research laboratory. In: Vital and Health Statistics
, Series 6, No. 1. Washington, DC: U.S. Government Printing Office, 1989, pp. 1-55.
34. Linton, M. Transformations of memory in everyday life. In: Memory Observed: Remembering in Natural Contexts
, U. Neisser(Ed.). San Francisco: Freeman, 1982.
35. Linton, M. Ways of searching and the contents of memory. In: Autobiographical Memory
, D. C. Rubin (Ed.). Cambridge, England: Cambridge University Press, 1986, pp. 50-67.
36. Loftus, E. F. and W. Marburger. Since, the eruption of Mt. St. Helens, has anyone beaten you up? Improving the accuracy of retrospective reports with landmark events. Mem. Cognit.
37. Longnecker, M., P. Gerhardsson, M. De Verdier, H. Frumkin, and C. Carpenter. A case-control study of physical activity in relation to risk of cancer of the right colon and rectum in men. Int. J. Epidemiol.
38. Means, B., K. Habina, G. E. Swan, and L. Jack. Cognitive research on response error in survey questions on smoking. In:Bital and Health Statistics
, Series 6, No. 2. Washington, DC: U.S. Government Printing Office, 1992, pp. 1-95.
39. Means, B. and E. F. Loftus. When personal history repeats itself: decomposing memories for recurring events. Appl. Cognit. Psychol.
40. Means, B., A. Nigam, M. Zarrow, E. F. Loftus, and M. S. Donaldson. Autobiographical memory for health-related events. In: Vital and Health Statistics
, Series 6, No. 2. Washington, DC: U.S. Government Printing Office, 1989, pp. 1-37.
41. Means, B., G. E. Swan, J. B. Jobe, and J. L. Esposito. An alternative approach to obtaining personal history data. In:Measurement Errors in Surveys
, P. P. Biemer, R. M. Groves, L. E. Lyberg, N. A. Mathiowetz, and S. Sudman (Eds.). New York: Wiley & Sons, 1992, pp. 167-183.
42. Menon, G. Judgments of behavioral frequencies: memory search and retrieval strategies. In: Autobiographical Memory and the Validity of Retrospective Reports
, N. Schwarz and S. Sudman (Eds.). New York: Springer-Verlag, 1994, pp. 161-172.
43. Neisser, U. Nested structure in autobiographical memory. In: Autobiographical Memory
, D. C. Rubin (Ed.). Cambridge, England: Cambridge University Press, 1986, pp. 71-81.
44. Neter, J., and J. Waksberg. A study of response errors in expenditures data from household interviews. Journal of the American Statistical Association
. 59:17-55, 1964.
45. Paffenbarger, R. S., A. L. Wing, and R. T. Hyde. Physical activity as an index of heart attack risk in college alumni.Am. J. Epidemiol.
46. Reiser, B. J., J. B. Black, and R. P. Abelson. Knowledge structures in the organization and retrieval of autobiographical memories. Cognit. Psychol.
47. Reiser, B. J., J. B. Black, and P. Kalamarides. Strategic memory search processes. In: Autobiographical Memory
, D. C. Rubin (Ed.). Cambridge, England: Cambridge University Press, 1986, pp. 100-121.
48. Richardson, M. T., A. S. Leon, D. R. Jacobs, B. A. Ainsworth, and R. C. Serfuss. Accuracy of the Minnesota LTPA Questionnaire.J. Clin. Epidemiol.
49. Schooler, J. W. and D. J. Hermann. There is more to episodic memory than just episodes. In: Theoretical Perspectives on Autobiographical Memory
, M. A. Conway, D. C. Rubin, H. Spinnler, and W. A. Wagenaar (Eds.). Dordrecht, The Netherlands: Kluwer Academic, 1992, pp. 241-261.
50. Sallis, J. F., W. L. Haskell, P. D. Wood, et al. Physical activity assessment methodology in the five-city project. Am. J. Epidemiol.
51. Slattery, M. L. and D. R. Jacobs Jr. Assessment of ability to recall physical activity of several years ago. Ann. Epidemiol.
52. Smith, A. F., J. B. Jobe, and D. J. Mingay. Retrieval from memory of dietary information. Appl. Cognit. Psychol.
53. Smith, A. F., J. B. Jobe, and D. J. Mingay. Question-induced cognitive biases in reports of dietary intake by college men and women. J. Health Psychol.
54. Subar, A. F., F. E. Thompson, A. F. Smith, et al. Improving food frequency questionnaires: A qualitative approach using cognitive interviewing. J. Am. Diet. Assoc.
55. Sudman, S. and N. M. Bradburn. Effects of time and memory factors on response in surveys. J. Am. Stat. Assoc.
56. Sudman, S. and N. M. Bradburn. Response Effects in Surveys: A Review and Synthesis
. Chicago: Aldine, 1974, pp. 1-264.
57. Tanur, J. M. and S. E. Fienberg. Cognitive aspects of surveys: yesterday, today, and tomorrow. J. Official Stat.
58. Taylor, C. B., T. Coffey, K. Berra, R. Iaffaldano, D. Casey, and W. L. Haskell. Seven day activity and self-report compard to a direct measure of physical activity. Am. J. Epidemiol.
59. Tulving, E. Elements of Episodic Memory
. New York: Oxford University Press, 1983, pp. 1-364.
60. Tversky, A. and D. Kahneman. Availability. A heuristic for judging frequency and probability. Cognit. Psychol.
61. Willis, G. B., P. Royston, and D. Bercini. The use of verbal report methods in the development and testing of survey questionnaires.Appl. Cognit. Psychol.