BANDA, JORGE A.1,2; HUTTO, BRENT1,2; FEENEY, ANNA1,2; PFEIFFER, KARIN A.3; McIVER, KERRY2; LAMONTE, MICHAEL J.4; BLAIR, STEVEN N.2,5; VENA, JOHN5; HOOKER, STEVEN P.1,2
It is well documented that engaging in regular physical activity (PA) helps with the control of body weight and is associated with a reduced risk of many adverse health conditions (5,18). As a result of these associations and the high prevalence of physical inactivity (24) and obesity (15) in the United States, many research studies are conducted in an effort to promote PA and to better understand its determinants. Accurately measuring PA is thus important, particularly for those working in public health and research settings who would like to conduct PA surveillance, in understanding the association between PA and health-related outcomes and to determine the effectiveness of intervention efforts.
Objective measures of PA (i.e., pedometers and accelerometers) are not commonly used in public health and epidemiological research because of a lack of familiarity with them, perceptions that they are not feasible in studies involving a large number of participants, and because of their high cost. As a result, most public health professionals and investigators have primarily relied on self-report PA questionnaires. However, the research literature has consistently demonstrated that there are limitations with self-report PA questionnaires, which present several concerns. PA questionnaires are prone to recall biases, which can lead to the misclassification of individual and population PA habits (20). This misclassification of PA may lead to biased effect estimates between PA exposure and health outcomes in epidemiological studies and intervention trials.
The misclassification of PA habits may be particularly concerning when using self-report questionnaires in population subgroups, such as racial/ethnic minorities or midlife and older adults, for whom subgroup-specific questionnaires typically have not been developed (29). A concern is that bias or a lack of validity could be present in questionnaires among certain population subgroups (i.e., lower education, older adults). The Behavioral Risk Factor Surveillance System (BRFSS) PA questions, the Aerobic Center Longitudinal Study (ACLS) PA short survey (PASS), and the ACLS PA long survey (PALS) are three questionnaires where such concerns might apply. The BRFSS is a telephone health survey system that tracks health conditions and risk factor behaviors in all 50 states, the District of Columbia, Puerto Rico, the US Virgin Islands, and Guam (6). The current BRFSS PA questions have been shown to be reliable (3,8). However, the validity of the current BRFSS PA questions is less established because published validations studies have included small sample sizes, samples that offer limited generalizability to US adults, and a variety of PA criterion measures (21-23).
The ACLS is an epidemiological cohort study that examines the relationship of PA, cardiorespiratory fitness (CRF), and other health factors with a variety of health-related outcomes (2). The PASS and PALS were validated in a large sample of ACLS participants (13). The sample in this earlier study was composed of well-educated Caucasian males, which limits the generalizability of findings in the previous validation study. Also, since completion of the earlier study, changes have been made to the questionnaires that have yet to be validated. Thus, the purposes of this study were to compare moderate- and vigorous-intensity PA (MVPA minutes per day) determined from the BRFSS PA questions, the PASS, and the PALS to accelerometer-determined MVPA minutes per day in a diverse group of midlife and older adults; and to determine whether these comparisons varied by participant demographics and accelerometer MVPA bout length.
Study participants were recruited through flyers that were posted in public locations, presentations at local senior centers, and personal referrals. Potential participants contacted research staff to receive additional information about the study and to determine eligibility. Persons who were ≥45 yr, ambulatory, and without cognitive impairment were eligible to participate. We did not aim to recruit a specific number of participants, choosing instead to recruit as many participants as possible. The study protocol was approved by the Institutional Review Board of the University of South Carolina, and the procedures used in the study were in accordance with institutional guidelines.
Once verbal informed consent was obtained, participants completed a telephone questionnaire to answer the BRFSS PA questions and to obtain demographic characteristics and self-reported height, weight, and health status. After this phone call, participants were mailed a packet of materials containing a written consent form, an information sheet, the PASS, the PALS, a daily log, an Actical® accelerometer, and written and visual instructions on how to properly wear the accelerometer. Participants were contacted by telephone 3 d after the package was mailed to confirm receipt, to remind them to start wearing the accelerometer, and to answer any questions.
Each participant was instructed to first complete the PASS and PALS and to then wear the Actical™ accelerometer for seven consecutive days beginning the next day. After the participant wore the accelerometer for 7 d, she/he returned it along with her/his signed consent form, the PASS, the PALS, and the PA log to research staff in a preaddressed, stamped envelope. Research participants were contacted by telephone if the envelope containing the equipment and documents was not received within 2 wk from the date it was initially mailed. Once the materials were received from the participants, they were reviewed for completeness.
The Actical® uses a single internal "omnidirectional" accelerometer that senses motion in all directions but is most sensitive within a single plane. It detects low-frequency (0.35-3.5 Hz) g-forces (0.05g-2.0g) common to human movement and generates an analog voltage signal that is filtered and amplified before being digitized by an A-to-D converter at 32 Hz (17), and it has been established as a valid measure of energy expenditure (11,19). The digitized values were summed during 1-min epochs in this study. Study participants were instructed to wear the accelerometer, which was attached to a neoprene waistband that allowed for easy but secure positioning of the device over the right side of the hip. A visual depiction of how to wear and position the device was provided in the materials mailed to the participant. Data were downloaded using a serial port computer interface.
Although accelerometry provides an objective measure of PA, it does have limitations. Accelerometry is not the gold standard method for measuring PA, and it is limited in its ability to capture certain types of PA (i.e., stationary activities), which may lead to the underestimation of PA. In addition, there is currently no consensus standardized approach to accelerometer field methods (i.e., wear time criteria) or to processing accelerometer data, including the selection of appropriate accelerometer count cut points to identify time spent in MVPA. Because other objective PA assessment methods, such as indirect calorimetry or doubly labeled water, generally are not feasible in public health or research settings, accelerometry often is the preferred criterion measure against which PA questionnaires are compared.
The current BRFSS PA questions are interviewer administered and ask participants whether they participate in moderate-intensity physical activities for at least 10 min at a time in a usual week (time frame is unspecified). Examples of moderate-intensity activities are given to queue participant recall. If participants respond affirmatively, they are then asked how many days per week and how many hours and minutes per day they spent performing these activities. A similar approach is used to query participation in vigorous-intensity physical activities. The PASS is a self-administered questionnaire that provides definitions of moderate- and vigorous-intensity PA and asks for the total hours during the past 7 d spent in moderate-intensity (lasting at least 8-10 min per session, accumulating to at least 30 min·d−1) and vigorous-intensity (lasting at least 20 min per session) PA (13). The PALS is self-administered and queries participant participation during the past 3 months in several traditional aerobic activities and sports. If participants answer yes to an activity, they are then asked how often (number of sessions per week) and how much time (average duration of each session) they spent performing the activity (13). Participants are also allowed to write in activities that are not listed on the survey, along with the frequency and duration they performed that activity. The questionnaires used in the present study are available upon request from the corresponding author.
The criterion for including accelerometer data in the analysis was ≥4 d of valid wear (25,26), with a valid day being ≥12 h of valid wear time. SAS macros were used to determine nonwear time (≥20-min periods of zero counts). Valid wear time was determined by subtracting nonwear time from 24 h. Participants without ≥4 valid days of data (n = 9) were not included in the analysis, and invalid days were also not included in the analysis. Accelerometer data were organized as minutes of MVPA per day (MVPA min·d−1). MVPA minutes per day were determined by using both 1- and 10-min MVPA bout methods. In the 1-min MVPA bout method, minutes that exceeded the MVPA cut point were counted as MVPA minutes if they occurred in bouts lasting at least 1 min. In the 10-min MVPA bout method, minutes that exceeded the MVPA cut point were counted as MVPA minutes if they occurred in bouts of ≥10 consecutive minutes. To determine MVPA minutes per day, we divided the total number of MVPA minutes from valid days by the number of valid days. An accelerometer count cut point of 1065 was used to identify minutes spent in MVPA (12).
All statistical analyses were conducted using SPSS (Chicago, IL, version 16.0). Mean, median, SD, and interquartile range (IQR) were calculated for questionnaire- and accelerometer-determined MVPA minutes per day. Because of the skewed distribution of the PA data determined from questionnaire and accelerometry, natural log transformations were conducted on them. Unadjusted and adjusted (adjusting for age, body mass index (BMI), race, and gender) Pearson correlation coefficients with 95% confidence intervals (95% CI) were calculated to determine the correlation between log-transformed questionnaire- and accelerometer-determined MVPA minutes per day. Bland-Altman plots were constructed to compare the differences in untransformed MVPA minutes per day obtained from questionnaire and accelerometry. Cohen κ coefficients (unweighted) with 95% CI were calculated to determine the agreement between questionnaires and accelerometry and between individual questionnaires for categorizing participants as having ≥150 MVPA min·wk−1. Data were also analyzed to investigate differences in PA assessment methods by accelerometer bout length.
Valid accelerometer data were available in 71 (88.8%) of the 80 participants enrolled in the study. Participants with valid data had a mean ± SD of 6.1 ± 1.1 d of valid data, whereas those with invalid data had a mean of 1.5 ± 1.2 d of valid data. Those with valid data had a mean ± SD of 13.8 ± 1.2 h of valid wear time per day. Descriptive statistics for those with valid accelerometer data is reported in Table 1. The participants included in the analysis were primarily Caucasian (74.6%), female (69%), older (age = 57.4 ± 9.9 yr), and overweight (BMI = 27.9 ± 4.9 kg·m−2) but were otherwise apparently healthy adults.
Table 2 lists MVPA minutes per day determined from questionnaires and accelerometry, and Table 3 lists Pearson correlation coefficients between questionnaire- and accelerometer-determined MVPA minutes per day. After adjusting for demographic variables, significant correlations were observed between PASS and 1-min bout accelerometer-determined MVPA minutes per day (r = 0.23, P < 0.05) and between PALS and 1- and 10-min bout accelerometer-determined MVPA minutes per day (r = 0.31 and r = 0.37, respectively, P < 0.01 each; Table 3). The Bland-Altman plots (Fig. 1) suggest a systematic overreporting of BRFSS-determined MVPA minutes per day when compared with accelerometry. The same general pattern existed in the Bland-Altman plots for the PASS and PALS (plots not shown). Cohen κ coefficients documented poor agreement between questionnaires and 1- and 10-min bout accelerometer-determined MVPA minutes per week (Table 4) and between individual questionnaires (Table 5) for classifying participants as having ≥150 MVPA min·wk−1.
We found low to moderate correlations and agreement between questionnaire- and accelerometer-determined MVPA minutes per day in midlife and older adults. Compared with accelerometry, the questionnaires we examined typically resulted in more time spent in MVPA, with differences varying by PA questionnaire, accelerometer MVPA bout length, and participant demographics.
Although the adjusted correlations indicated that PASS- and PALS-determined MVPA minutes per day were significantly correlated with 1-min MVPA bout accelerometer-determined MVPA minutes per day, the observed correlations were low to moderate in strength (Table 3), and median MVPA minute per day varied considerably between measures (6.7-19 min·d−1 or 46.9-133 min·wk−1). These large differences in MVPA present a serious concern because they can lead to erroneous findings in PA surveillance, intervention efforts, and epidemiological studies. For example, a recent study examining the association between daily energy expenditure and mortality among older adults found stronger protective effects with objective rather than self-report methods, suggesting that self-report PA may lead to the underestimation of potential health benefits of PA (14).
When a 10-min MVPA bout criterion was applied to accelerometer data, only the PALS significantly correlated with accelerometer-determined MVPA minutes per day (Table 3). This raises a concern because MVPA minutes per day from the BRFSS PA questions, which were designed to measure MVPA that occurs in bouts of ≥10 min, was not significantly correlated with 10-min MVPA bout criteria accelerometer data. In addition, there was a difference of 38.3 min·d−1, or 268.1 min·wk−1, in median MVPA minutes per day determined from the BRFSS PA questions and accelerometry, indicating that participants reported substantially higher levels of PA than indicated by objective data from accelerometry. It has recently been reported that significant gaps exist between respondents' interpretations of some self-report PA questions and researchers' assumptions about what the questions are intended to ask (1). In one study using cognitive interviews, it was apparent that respondents did not comprehend the concept of intensity, often counted the same activity more than once, and understood activities that were grouped together in a single category (e.g., moderate intensity) to be definitive lists rather than examples (1). These issues are inherent in the BRFSS PA questions and could partially explain the large differences in MVPA minutes per day determined from the BRFSS PA questions and accelerometry.
Although not designed to measure MVPA occurring in bouts of ≥10 min, the PASS and PALS performed better than the BRFSS PA questions. However, there was a large absolute difference in median MVPA minutes per day determined from the PASS and accelerometry (46.8 min·d−1 or 327.6 min·wk−1) and the PALS and accelerometry (21.1 min·d−1 or 147.7 min·wk−1), again raising concerns that participants are reporting higher levels of PA than directly measured by accelerometry. It should be noted that a strict 10-min bout criterion was used in this study (i.e., we did not allow for a 1- to 2-min break). Allowing for such a break may have resulted in different results, such as more favorable outcomes for the 10-min bout criteria analysis. However, it should be noted that there is no standard procedure in place because strict use and relaxed use (i.e., allowing for a 1- to 2-min break) methods are both used in studies involving accelerometry.
Previous research found that 1-min MVPA bout accelerometer-determined MVPA minutes per day were significantly correlated with BRFSS-determined MVPA minutes per day in a large sample of Australian adults aged 18-75 yr (r = 0.24, P < 0.01) (22). In addition, Strath et al. (21) reported that the 10-min MVPA bout accelerometer-determined MVPA minutes per day were not significantly correlated with the BRFSS-determined MVPA minutes per day in a small sample of adults aged 30.0 ± 10.5 yr (r = 0.10). Limited published information is available on the agreement between the PASS and PALS and objective measures of PA habits, with previous work having relied on CRF as an indirect criterion measure of usual PA habits (13). Kohl et al. (13) reported that an aerobic PA index created from the PALS was not significantly correlated with CRF among a large sample of Caucasian males (r = 0.05). However, a weak but significant correlation was observed between CRF and an aerobic PA index created from the PASS (r = 0.14, P = 0.03). It should be noted that although related, CRF and PA are essentially two different measures, with CRF being an outcome of not only PA habits but also genetic endowment and health status. As a result, care should be taken when comparing the results from the present study to this previous study.
It is important to note that because of the PA data being skewed, we chose to focus on median MVPA minutes per day rather than on mean MVPA minutes per day in our interpretations. However, if mean MVPA minutes per day had been used, which is often the case in practice, the differences observed between PA measures would have been larger. For example, median MVPA minutes per day from the questionnaires differed from the 1-min bout accelerometer-determined MVPA minutes per day by 6.7-19 min·d−1 or 46.9-133 min·wk−1. In comparison, mean MVPA minutes per day from the questionnaires differed from the 1-min bout accelerometer-determined MVPA minutes per day by 4.3-40.2 min·d−1 or 30.1-281.4 min·wk−1.
Examination of the Bland-Altman plots suggests that at higher levels of MVPA, there were large differences between questionnaires and accelerometry, with participants tending to report more MVPA than directly measured by the accelerometer. In addition, the agreement observed between questionnaires and accelerometry for classifying persons as having ≥150 MVPA min·wk−1 was poor (Table 4). These results are similar to previous research, indicating that there is poor agreement between self-report PA questionnaires and accelerometry for categorizing PA (22,27). In a study involving Australian adults aged 18-75 yr, Timperio et al. (22) found poor agreement (κ = 0.14, 95% CI = −0.01 to 0.29) between the BRFSS PA questions and accelerometry in classifying persons as having ≥150 MVPA min·wk−1. In addition, in a study involving primarily obese African American adults aged 24-67 yr, Wolin et al. (27) reported poor agreement between the International Physical Activity Questionnaire (IPAQ)-short and the 1-min MVPA bout (κ = 0.21, 95% CI = −0.04 to 0.47) and the 10-min MVPA bout (κ = 0.04, 95% CI = 0.01-0.06) accelerometry in classifying persons as meeting PA guidelines. However, in a study involving healthy-weight adults aged 30.6 ± 9.9 yr (n = 25), Strath et al. (21) provided evidence of agreement between the BRFSS PA questions and the 10-min MVPA bout accelerometry in classifying persons as meeting PA guidelines (κ = 0.61, P < 0.05).
Finally, the agreement observed between questionnaires for classifying persons as having ≥150 MVPA min·wk−1 was poor (Table 5). Similar results can be seen in a recent report by Carlson et al. (4), where the National Health Interview Survey, the National Health and Nutrition Examination Survey, and the BRFSS produced considerably different prevalence estimates. The results from this present study should be interpreted with caution as the wording, time frames, and specific PA items used among the questionnaires are different. Unfortunately, a direct comparison of these results is not possible because this topic has not been previously investigated in these questionnaires or by most comparison and validation studies to date. This issue can be extended to the correlation analysis. The results indicate that the BRFSS PA questions and the PASS, which both used a 1-wk time frame, compared similarly to the 1- and 10-min bout accelerometer data and produced similar median MVPA minutes per day (Tables 2 and 3). In comparison, the PALS performed better than BRFSS PA questions and the PASS when compared with accelerometry (Tables 2 and 3). Unfortunately, it is not clear whether the PALS compared better with accelerometry because of its extended time frame (previous 3 months) or because it is a more comprehensive measure.
Strengths in our study include working with midlife and older adults, a group in which the accuracy of PA questionnaires has not been fully explored; an ethnically and gender-diverse sample; the use of the 1- and 10-min MVPA bout criteria on accelerometer data; and the use of multiple PA questionnaires. Limitations in our study include our small sample size and a limited number of men and African Americans, which prevented us from fully exploring associations by various demographic categories. Although our sample size may seem small, it is similar in size to those previously comparing questionnaire and objectively measured PA (9,16,21,28). In addition, although the results of this study may not generalize to younger populations, they do fill a gap in the research literature pertaining to midlife and older adults. The results may have also varied if another accelerometer count cut point was used to determine the time spent in MVPA. However, the cut point used in this study was previously determined to be valid for a diverse group of midlife and older men and women, and this is the only cut point for the Actical accelerometer created specifically for this population (12).
Because of the limitations associated with accelerometry, it is possible that the lack of agreement observed in this study is due to the limitations in both self-report methods and accelerometry. However, this is highly unlikely because large differences in MVPA between self-report methods and accelerometry have been reported extensively in the research literature and have been found in studies using different types of accelerometers, different accelerometer processing guidelines (i.e., wear time criteria and MVPA bout criteria), and with different accelerometer MVPA count cut points (10,22,27,28). As a result, the self-report questionnaire is likely the largest source of bias, not the accelerometer. This is supported by work indicating that significant gaps exist between respondents' interpretations of self-report PA questions and researchers' assumptions about what the questions are intended to ask (1). Finally, the results may have differed if we had compared accelerometer-measured PA with other self-report PA questionnaires that have been specifically designed for use with persons aged ≥50 yr.
Researchers and practitioners need to be aware of the limitations associated with PA assessed by the BRFSS PA questions, the PASS, and the PALS. Every attempt should be made to obtain objectively measured PA data whenever possible to reduce measurement errors and limitations associated with self-reported PA habits (7). In addition, further study is needed to better understand and standardize the accelerometer bout definitions used to assess MVPA and other aspects of PA habits.
This study was supported by a grant from Mini Mitter Respironics.
The authors have no conflict of interest to declare.
The results of the present study do not constitute endorsement by the American College of Sports Medicine.
The authors thank Chris Davis and Ellen Henderson for their assistance in collecting the data.
1. Altschuler A, Picchi T, Nelson M, Rogers JD, Hart J, Sternfeld B. Physical activity questionnaire comprehension: lessons from cognitive interviews. Med Sci Sports Exerc
2. Blair SN, Kohl HW 3rd, Paffenbarger RS Jr, Clark DG, Cooper KH, Gibbons LW. Physical fitness and all-cause mortality: a prospective study of healthy men and women. JAMA
3. Brown WJ, Trost SG, Bauman A, Mummery K, Owen N. Test-retest reliability of four physical activity measures used in population surveys. J Sci Med Sport
4. Carlson SA, Densmore D, Fulton JE, Yore MM, Kohl HW 3rd. Differences in physical activity prevalence and trends from 3 U.S. surveillance systems: NHIS, NHANES, and BRFSS. J Phys Act Health
. 2009;6(suppl 1):S18-27.
6. Centers for Disease Control and Prevention (CDC) Website [Internet]. Behavioral Risk Factor Surveillance System; [cited 2009 May 13]. Available from: http://www.cdc.gov/BRFSS/
7. Clark BK, Sugiyama T, Healy GN, Salmon J, Dunstan DW, Owen N. Validity and reliability of measures of television viewing time and other non-occupational sedentary behaviour of adults: a review. Obes Rev
8. Evenson KR, Eyler AA, Wilcox S, Thompson JL, Burke JE. Test-retest reliability of a questionnaire on physical activity and its correlates among women from diverse racial and ethnic groups. Am J Prev Med
. 2003;25(3 suppl 1):15-22.
9. Harada ND, Chiu V, King AC, Stewart AL. An evaluation of three self-report physical activity instruments for older adults. Med Sci Sports Exerc
10. Harris TJ, Owen CG, Victor CR, Adams R, Ekelund U, Cook DG. A comparison of questionnaire, accelerometer, and pedometer: measures in older people. Med Sci Sports Exerc
11. Heil DP. Predicting activity energy expenditure using the Actical activity monitor. Res Q Exerc Sport
12. Hooker SP, Feeney A, Hutto B, et al. Validation of the Actical®
activity monitor in middle-aged and older adults. J Phys Act Health
. (in press).
13. Kohl HW, Blair SN, Paffenbarger RS Jr, Macera CA, Kronenfeld JJ. A mail survey of physical activity habits as related to measured physical fitness. Am J Epidemiol
14. Manini TM, Everhart JE, Patel KV, et al. Daily activity energy expenditure and mortality among older adults. JAMA
15. Ogden CL, Carroll MD, Curtin LR, McDowell MA, Tabak CJ, Flegal KM. Prevalence of overweight and obesity in the United States, 1999-2004. JAMA
16. Pettee Gabriel K, McClain JJ, Lee CD, et al. Evaluation of physical activity measures used in middle-aged women. Med Sci Sports Exerc
18. Physical Activity Guidelines Advisory Committee. Physical Activity Guidelines Advisory Committee Report, 2008
. Washington (DC): US Department of Health and Human Services; 2008. 683 p.
19. Puyau MR, Adolph AL, Vohra FA, Zakeri I, Butte NF. Prediction of activity energy expenditure using accelerometers in children. Med Sci Sports Exerc
20. Sallis J, Owen N. Physical Activity and Behavioral Medicine
. Thousand Oaks (CA): Sage Publications; 1999. 240 p.
21. Strath SJ, Bassett DR Jr, Ham SA, Swartz AM. Assessment of physical activity by telephone interview versus objective monitoring. Med Sci Sports Exerc
22. Timperio A, Salmon J, Rosenberg M, Bull FC. Do logbooks influence recall of physical activity in validation studies? Med Sci Sports Exerc
24. Troiano RP, Berrigan D, Dodd KW, Masse LC, Tilert T, McDowell M. Physical activity in the United States measured by accelerometer. Med Sci Sports Exerc
25. Trost SG, McIver KL, Pate RR. Conducting accelerometer-based activity assessments in field-based research. Med Sci Sports Exerc
. 2005;37(suppl 11):S531-43.
26. Ward DS, Evenson KR, Vaughn A, Rodgers AB, Troiano RP. Accelerometer use in physical activity: best practices and research recommendations. Med Sci Sports Exerc
. 2005;37(suppl 11):S582-8.
27. Wolin KY, Heil DP, Askew S, Matthews CE, Bennett GG. Validation of the International Physical Activity Questionnaire-Short among blacks. J Phys Act Health
28. Yore MM, Ham SA, Ainsworth BE, Kruger J, Reis JP, Kohl HW 3rd, Macera CA. Reliability and validity of the instrument used in BRFSS to assess physical activity. Med Sci Sports Exerc
29. Young D, Miller K, Wilder Y, Yanek Y, Becker D. Physical activity patterns of urban African Americans. J Community Health