Physical activity provides important health benefits for children and adolescents, and public health authorities in the United States and around the world have called for young people to engage in moderate-to-vigorous physical activity (MVPA) for at least 60 min·d−1 (1,2). However, most children and youth in developed countries do not meet that guideline (3). Accordingly, many public health organizations consider promoting physical activity in children and youth as a high priority (4).
In public health, surveillance refers to the continuous, systematic collection, analysis, and interpretation of data (5). Robust surveillance systems support public health initiatives by providing information needed to effectively target intervention efforts and by producing data that enable monitoring progress toward goals (5). In the United States, some surveillance systems include measures of physical activity in children and adolescents, including the National Health and Nutrition Examination Survey (NHANES) and the Youth Risk Behavior Survey (YRBS). However, each of these measures has significant limitations.
Recent cycles of NHANES have included both accelerometry as an objective measure of physical activity and an interviewer-administered physical activity survey (6). Although the NHANES protocol provides a great deal of information, it is limited by high staff and respondent burden. The school-based YRBS includes a self-report of physical activity in high school students (grades 9–12, ages 13–18 yr), but its items have limited validity compared with accelerometry and provide limited information regarding participation in specific forms of physical activity (7).
An ideal surveillance system for monitoring physical activity in youth would provide both a valid estimate of the respondents’ physical activity level and information regarding the specific types of physical activity in which they have engaged. The former would provide the basis for determining the prevalence of compliance with a physical activity guideline. The latter would be useful in designing public health initiatives to increase physical activity. Given the limitations of youth physical activity measures in existing surveillance systems, the purpose of this study was to apply state-of-the-art psychometric methods to develop a youth physical activity self-report instrument that could be used in public health surveillance systems.
METHODS AND RESULTS
The Evaluation of Activity Surveys in Youth (EASY) study took place between 2012 and 2015 and used a mixed methods qualitative/quantitative sequential research design to develop a physical activity self-report instrument for youth. The study was conducted as five separate substudies using three data collection protocols in three independent samples of middle school students. Each substudy used a cross-sectional study design. All study protocols were approved by the University of South Carolina Institutional Review Board, and parent consent and child assent were obtained before data collection. The study was conducted in a sequential manner such that the findings from each substudy informed the methods of the subsequent substudies. Accordingly, in the following section, we present both methods and results for each substudy. All analyses were performed in SAS (version 9.4). The flow of the substudies and samples is shown in the supplemental figure (see Figure, Supplemental Digital Content 1, Summary of study protocol, http://links.lww.com/MSS/B201).
The purpose of the first substudy was to identify specific forms of physical activity that middle school students (grades 6–8, ages 11–13 yr) most frequently perform. Focus groups were conducted with 162 students in grades 6 to 8. Students were recruited through physical education classes at two middle schools in a single school district in Columbia, SC. Enrollments at both schools (n = 884 and 1017) were demographically diverse (% Black, 50.3 and 38.2; % White, 35.8 and 50.1; % Hispanic, 7.0 and 5.4). The percentage of children eligible for free or reduced price lunch was 50.2 and 39.0. Twenty-four focus group meetings were conducted, with four separate groups for each grade and gender. On average, seven students participated in each focus group.
Before conducting these sessions, the investigators developed a list of 22 forms of physical activity, based on existing youth physical activity surveys. This list was used in selecting specific discussion prompts, and the same list was used with both boys and girls. Discussions were led by an experienced focus group leader, and a second research staff member served as the recorder. Participants were asked to mention the physical activities they performed at any time during the year. The leader prompted the groups by enquiring about school and non–school sport participation; classes or lessons; activities done at home, at a friend’s house, at church, or at a gym or community center; before- and after-school activities; and games or other free-play activities.
After completion of the focus groups, the frequency with which specific forms of physical activity had been reported was tabulated and cross-referenced with the investigator-generated list of 22 activities. Many of the physical activities mentioned were specific activities such as soccer or basketball. Others were broader and categorical, such as doing household chores. Because our goal was to select a limited number of activities for inclusion in the draft instrument, we opted to include specific forms of physical activity only if we felt that the activity was not likely to be reported by respondents in a broader categorical item. For example, we did not include soccer as an item because we felt that students would include it in their responses to categorical items such as “play on an organized school sports team” or “play on an organized, non–school sports team.” We did include activities such as “do weight training” and “walk your dog” because they were frequently mentioned in the focus groups and they did not clearly fit into a categorical item. The initial list of 22 items was modified and expanded to produce a final list of 30 forms of physical activity. These activities are listed in Table 1.
The purpose of the second substudy was to apply Item Response Theory (IRT) to identify an appropriate survey item response format for use in an instrument designed to assess physical activity levels of youth. Participants were 537 students in grades 6 to 8 recruited from six middle schools in one school district, two middle schools in a second school district, and an evening program at a local church. Enrollments in the seven schools were demographically diverse. Across the seven schools, mean enrollment was 636 (range, 531–901), and the race/ethnicity profiles were as follows (mean, range): % Black (33.2, 9.1–53.4), % White (58.5, 33.2–87.8), and % Hispanic (5.4, 1.2–13.2). The mean percentage of students eligible for free or reduced price lunch was 55.9 (range, 29.7–76.8).
Participants were randomly assigned to complete one of three versions of a survey instrument that included the list of 30 physical activities. The versions differed in the level of detail reported for each form of physical activity. With version 1, participants (n = 185; 52.4% girls) reported whether they had performed each of the activities in the previous 7 d, using only a yes/no response format. Version 2 asked participants (n = 182; 55.5% girls) to report yes/no for each activity and, for the “yes” items, to report the number of days (1–7) on which they performed the activity. Version 3 (n = 170; 57.1% girls) asked respondents to report yes/no for each activity, the frequency of participation, and a rating of the intensity at which the activity was performed. Participants were oriented to the intensity rating scale using figures and written descriptions. For each reported physical activity, one of the following intensities was selected: light, moderate, hard, or very hard (8). The mean time ± SD to complete each version of the instrument was 4.0 ± 1.6, 5.1 ± 2.0, and 6.5 ± 2.5 min, for versions 1, 2, and 3, respectively.
For each of the three versions of the instrument, data were analyzed using two different Rasch model approaches. A “traditional” Rasch method was used with version 1 (dichotomous data) (9), and the Rasch Rating Scale Method (10) was used with versions 2 and 3 (ordinal data). Person- and item-separation scale characteristics and reliability measures were examined, and point-biserial correlations with the total scale were evaluated to ensure that all items exhibited a positive relationship across the latent construct. Infit mean square and outfit mean square were used to assess the fit between items and the Rasch model. In addition, item-construct maps (11) were used to determine each item’s endorsability relative to the construct of physical activity.
The complete findings of the IRT analyses have been published elsewhere (12). Briefly, all versions of the instrument were found to be acceptable. Person separation ranged from 1.40 to 2.25, and person reliability ranged from 0.66 to 0.84 for each of the different versions. Item separation ranged from 4.77 to 7.10, and item reliability ranged from 0.96 to 0.98. The IRT analyses indicated that version 2 (yes/no plus frequency) was most appropriate for use in middle school students based on the similarity of item fit and item spread along the latent dimension. Version 1 (yes/no only) was generally acceptable but would not allow for frequency of participation to be assessed. Version 3 (yes/no plus frequency and intensity) was not selected because adding intensity did not improve measure scores, item fit, or item spread (12).
The third substudy was designed to identify the specific survey items that would comprise the physical activity self-report instrument. Participants were 342 middle school students recruited from physical education classes in two schools. Participating schools were the same as those for substudy 1, and the demographic profile of those schools is provided above. Data were collected in the school setting during two sessions. During the first data collection session, height and weight were measured, and participants were fitted with an accelerometer (ActiGraph GT1M and GT3X models; Pensacola, FL). During the second data collection session, accelerometers were collected and participants completed the 30-item self-report instrument. The instrument was administered twice to assess test–retest reliability. The two administrations were separated by 15–20 min. Characteristics of the sample are summarized in Table 2.
Participants were instructed to wear the accelerometer on an elastic belt over the right hip for seven consecutive days, except while engaging in water activities (swimming, bathing, showering, etc.). The accelerometers were set to record data in 30-s epochs, and periods of 60 min of consecutive zeroes were defined as non–wear time and excluded from the analyses. To be included in the analyses, participants must have worn the accelerometer for at least 8 h on at least 3 d. Age-specific cut points for the age-group represented (11- to 14-yr-olds) were used to classify MVPA (11-yr-olds, ≥1030 counts per minute; 12-yr-olds, ≥1110 counts per minute; 13-yr-olds, ≥1197 counts per minute; 14-yr-olds, ≥1290 counts per minute).
Accelerometer wear time met the specified standard (8 h, 3 d or more) for 264 students, and characteristics of this analysis sample are summarized in Table 2. Accelerometer-derived MVPA (min·d−1; mean ± SD) was calculated. Biserial correlations were calculated between yes/no responses for each item and accelerometer-derived MVPA minutes per day, and Spearman correlations were calculated to determine the association between the number of days reported for each item and accelerometer mean MVPA minutes per day (see Table 1). To determine the test–retest reliability of each item, kappa statistics were calculated for the percent of students indicating yes for each reported activity for administrations 1 and 2, and interclass correlation coefficients were calculated for the reported number of days of each activity between administrations one and two.
As presented in Table 1, statistically significant associations between item responses and MVPA as assessed by accelerometry were observed for 15 of the 30 items. For 12 of those items, significant associations were observed for both the yes/no and the frequency of participation (0–7) responses. Strongest associations were observed for the following six items: play actively during recess or other free time at school; play on an organized, non–school sports team; play nonorganized sports; do weight training; ride your bike or other wheeled toys for fun or exercise; and play actively in your neighborhood. All 30 items were found to be reliable based on the findings of the test–retest protocol. As presented in Table 1, kappa statistics varied from 0.73 to 0.94, and intraclass correlation coefficients varied from 0.79 to 0.96.
Three steps were taken in selecting items for inclusion in the final instrument. First, the six items that correlated most highly with MVPA were selected for inclusion in the instrument. Those items are listed above. Second, five items were included in the instrument because it was deemed important, from a surveillance perspective, that children’s participation in those activities be monitored. Those included were as follows: have PE/gym classes, play on an organized school sports team, walk or bike to or from school, participate in physical activity in an after-school program, and participate in physically active classes or lessons. Finally, the IRT person-item map (Fig. 1) was reviewed to ensure that the instrument included items that were appropriately distributed across the breadth of the latent domain, including items at low and high intensity. To meet this criterion, the following three items were included: walk for fun or exercise, run/jog for fun or exercise, and play actively at home. This produced a 14-item instrument which is shown in Figure 2.
To assess reliability and concurrent validity of the 14-item instrument from substudy 3, cumulative scores for those 14 items were calculated for the first and second administrations of the preliminary 30-item instrument. This substudy used test administration data from the sample of youth included in substudy 3. Mean ± SD scores for the first and second responses to the 14 items were 28.0 ± 15.2 and 25.6 ± 13.7, respectively. Response to the set 14 items was highly reliable. An intraclass correlation coefficient of 0.93 was observed, and the Spearman correlation coefficient was 0.91 (P < 0.001). The Spearman correlation coefficient for the association between the cumulative score for the 14 items (first administration) and minutes per day of MVPA derived from accelerometry was 0.33 (95% CI = 0.22–0.43, P < 0.001).
Finally, substudy 5 was conducted to determine the sensitivity and specificity with which the new self-report instrument detected objectively measured compliance with the current federal physical activity guideline for children and youth, 60 or more minutes of MVPA per day (1,4). Using the same sample of participants and data from substudies 3 and 4, the percentage of participants who accumulated ≥60 min of MVPA per day was calculated. In this sample, 23.1% of boys and 5.4% of girls accumulated 60 or more minutes of MVPA per day (1). Receiver operator characteristic curves were used to determine the optimal instrument score for detecting ≥60 min of accelerometer-derived MVPA per day. Sensitivity, specificity, and area under the curve were calculated, and the Youden Index (13) was used to identify the instrument score that best discriminated between meeting and not meeting the guideline. The optimal score to identify children who accumulated ≥60 min of MVPA per day was 22 (sensitivity = 0.90, specificity = 0.44, area under the curve = 0.68).
The product of this study is a physical activity self-report instrument that is suitable for public health surveillance of physical activity behavior in youth. The instrument consists of 14 items, each of which indicates recent frequency of participation in a form of physical activity that is common in U.S. youth. The individual items were selected using Rasch analysis, a procedure for application of IRT, and students’ responses to the items were found to be highly reliable. The composite of responses to the 14 items was also found to be highly reliable. When the composite score was compared with objectively measured physical activity, the score manifested acceptable concurrent validity. Further, a composite score was identified that showed acceptable sensitivity and specificity in differentiating between youth who met or did not meet a current physical activity guideline, as determined by objective measurement.
Surveillance of physical activity behavior in children and youth has been implemented in the United States on a limited and uneven basis. The most consistently applied method is the Youth Risk Behavior Surveillance System (YRBSS), which is administered in alternate years to state-based samples of high school students (14). Since its inception in 1991, this survey protocol has included varying numbers of physical activity items. In recent years, only a single item has been included, and that item queries frequency of participation (expressed as days of the previous seven) on which the respondent engaged in 60 or more minutes of physical activity that “…increased your heart rate and made you breathe hard some of the time.” Although earlier iterations of YRBSS physical activity items have been shown to be acceptably reliable and valid when compared with objectively measured physical activity (7,15), we have not found published documentation of the psychometric properties of the specific item that has most recently been included in the survey. Other limitations of this YRBSS item include its systematic use only with high school students and its failure to provide information about participation in any specific forms of physical activity. By contrast, the instrument developed in the present study has high reliability and acceptable concurrent validity and provides information about participation in 14 types of physical activity that are common in U.S. youth.
The importance of physical activity surveillance in youth has been recognized internationally; however, surveillance methodologies and instrumentation have varied considerably across countries. In an effort to standardize surveillance practices to allow for international comparisons, the World Health Organization and the Centers for Disease Control and Prevention launched the Global School-based Student Health Survey (16). Overall physical activity participation of youth (ages 13–17) is assessed through one item, identical with that of the YRBSS. In addition, many countries conduct physical activity surveillance through their own comprehensive health behavior surveys with physical activity subsections (17–19). For example, the physical activity subsection of the Canadian Health Measures Survey (17) collects detailed information on the frequency, duration, and settings in which physical activity and active transportation take place. The Health Survey for England (19) queries youth on the type, frequency, and duration of formal and informal physical activities in which they participated over the past week. This method offers a detailed account of specific activities in which youth participate; however, the result is an extensive survey that often overestimates the time youth spend in physical activity (20), and there is little to no psychometric data to support the utility of the physical activity items.
In the present study, middle school youth identified specific types of physical activities, and 14 of them, after application of rigorous item selection procedures, were included in the final instrument. In addition to emerging from the Rasch analysis, these items were deemed to be meaningful sources of physical activity in U.S. youth. Thus, the EASY instrument offers a method to assess both youth’s compliance with physical activity guidelines and prevalence of participation in forms of physical activities that are widely accessible to youth. This information can inform public health interventions. For instance, approximately 30% of youth in the present study reported walking or biking to or from school during the previous week, and the mean frequency was about 1.4 d·wk−1. This information points to the need to increase this specific form of physical activity and provides guidance in focusing participation in public health initiatives aimed at increasing active transport to school. Further, this information provides a basis for evaluating the effectiveness of such initiatives.
This study has both important strengths and significant limitations. It is a strength that state-of-the-art methods were used to identify the items that comprise an instrument that is designed for surveillance of physical activity behavior in youth. Further, validity of the instrument was determined by comparing its self-reported information with objectively measured physical activity. Also, it is a strength that sensitivity and specificity of the instrument were assessed and receiver operator characteristic curves were examined in identifying an instrument score that differentiated between youth who met and did not meet a physical activity guideline as objectively determined. The product is an instrument that detects prevalence both of compliance with a physical activity standard and of participation in 14 forms of physical activity that are common among U.S. youth. However, it is a limitation that the instrument was developed only with youth of middle school age. Participating youth were residents of one metropolitan area in the Southeastern United States, and therefore it is not certain that the instrument’s psychometric properties would be the same if a more diverse sample of youth was assessed. Further, youth in other regions of the United States or other countries may prioritize and report forms of physical activity differently compared with this current sample of U.S. youth. However, the instrument includes many forms of physical activity common to youth, regardless of region or nationality, including walking for fun or exercise, playing actively in the neighborhood, and playing on organized non–school sports teams.
In summary, this study produced an instrument that is suitable for inclusion in surveillance systems that seek to assess, by self-report, health behaviors of middle school–age youth. The instrument is concise, reliable, and valid when compared with objectively assessed physical activity. Importantly, the instrument provides information on both overall physical activity level as well as participation in selected, specific forms of physical activity. Future studies should use the methods used here in developing physical activity self-report instruments for children and youth that are younger and older than those included in the present study. Further, it would be desirable to replicate this study with a participant group that is known to be nationally representative of children and youth in the United States.
This study was funded by a grant (R21CA158577) from the National Cancer Institute, National Institutes of Health. The authors express their appreciation to Dale Murrie, MEd, who assisted with collecting the data and Gaye Groover Christmus, MPH, who edited the manuscript. The authors also thank the administrators, teachers, and students in the participating schools. The authors declare that there are no conflicts of interest. The results of the study do not constitute endorsement by the American College of Sports Medicine. The authors declare that the results of the study are presented clearly, honestly, and without fabrication, falsification, or inappropriate data manipulation.
1. U.S. Department of Health and Human Services. 2008 Physical Activity Guidelines for Americans
. U.S. Department of Health and Human Services; 2008.
3. Pate RR, Freedson PS, Sallis JF, et al. Compliance with physical activity guidelines: Prevalence in a population of children and youth. Ann Epidemiol
4. Physical Activity Guidelines for Americans Midcourse Report Subcommittee. President’s Council on Fitness Sports and Nutrition: Physical Activity Guidelines for Americans Midcourse Report: Strategies to Increase Physical Activity Among Youth
. Washington (DC): 2012.
6. Trost SG, McIver KL, Pate RR. Conducting accelerometer-based activity assessments in field-based research. Med Sci Sports Exerc
. 2005;37(11 Suppl):S531–43.
7. Troped PJ, Wiecha JL, Fragala MS, et al. Reliability and validity of YRBS physical activity items among middle school students. Med Sci Sports Exerc
8. Pate RR, Ross R, Dowda M, Trost SG, Sirard J. Validation of a three-day physical activity recall instrument in female youth. Pediatr Exerc Sci
9. Schumaker RE: Rasch measurement: the dichotomous model. In: Introduction to Rasch Measurement
. Edited by Smith EV, Smith RM. Maple Grove (MN): JAM Press; 2004.
10. Smith EV Jr, Conrad KM, Chang K, Piazza J. An introduction to Rasch measurement for scale development and person assessment. J Nurs Meas
11. Bond TG, Fox CM. Applying the Rasch Model: Fundamental Measurement in the Human Sciences
. Mahwah (NJ): Lawrence Erlbaum; 2007.
12. DiStefano C, Pate R, McIver K, Dowda M, Beets M, Murrie D. Creating a physical activity self-report
form for youth using Rasch methodology. J Appl Meas
13. Youden WJ. Index for rating diagnostic tests. Cancer
14. Centers for Disease Control and Prevention (CDC), Brener ND, Kann L, et al. Methodology of the Youth Risk Behavior Surveillance System—2013. MMWR Recomm Rep
15. Prochaska JJ, Sallis JF, Long B. A physical activity screening measure for use with adolescents
in primary care. Arch Pediatr Adolesc Med
17. Tremblay M, Wolfson M, Connor Gorber S. Canadian Health Measures Survey: rationale, background and overview. Health Rep
18. Australian Bureau of Statistics. Australian Health Survey: Physical Activity, 2011–2012
. Canberra: Australian Bureau of Statistics; 2013.
19. National Obesity Observatory. Physical Activity Surveillance in England: What is Measured and Where are the Gaps?
London: NHS; 2009.
20. Basterfield L, Adamson AJ, Parkinson KN, et al. Surveillance of physical activity in the UK is flawed: validation of the Health Survey for England Physical Activity Questionnaire. Arch Dis Child
PHYSICAL ACTIVITY MEASUREMENT; SELF-REPORT; ADOLESCENTS; POPULATION SURVEILLANCE
Supplemental Digital Content
© 2018 American College of Sports Medicine