Secondary Logo

Cross-validation of Waist-Worn GENEA Accelerometer Cut-Points

WELCH, WHITNEY A.1; BASSETT, DAVID R.1; FREEDSON, PATTY S.2; JOHN, DINESH3; STEEVES, JEREMY A.1; CONGER, SCOTT A.1; CEASER, TYRONE G.1; HOWE, CHERYL A.2; SASAKI, JEFFER E.2

Medicine & Science in Sports & Exercise: September 2014 - Volume 46 - Issue 9 - p 1825–1830
doi: 10.1249/MSS.0000000000000283
APPLIED SCIENCES
Free

Purpose The purpose of this study was to determine the classification accuracy of the waist gravity estimator of normal everyday activity (GENEA) cut-points developed by Esliger et al. for predicting intensity categories across a range of lifestyle activities.

Methods Each participant performed one of two routines, consisting of seven lifestyle activities (home/office, ambulatory, and sport). The GENEA was worn on the right waist, and oxygen uptake was continuously measured using the Oxycon mobile. A one-way chi-squared test was used to determine the classification accuracy of the GENEA cut-points. Cross-tabulation tables provided information on under- and overestimations, and sensitivity and specificity analyses of the waist cut-points were also performed.

Results Spearman rank order correlation for the GENEA gravity-subtracted signal vector magnitude and Oxycon mobile MET values was 0.73. For all activities combined, the GENEA accurately predicted intensity classification 55.3% of the time, and it increased to 58.3% when stationary cycling was removed from the analysis. The sensitivity of the cut-points for the four intensity categories ranged from 0.244 to 0.958, and the specificity ranged from 0.576 to 0.943.

Conclusion In this cross-validation study, the proposed GENEA cut-points had a low overall accuracy rate for classifying intensity (55.3%) when engaging in 14 different lifestyle activities.

1Department of Kinesiology, Recreation, and Sport Studies, University of Tennessee, Knoxville, TN; 2Department of Kinesiology, University of Massachusetts, Amherst, MA; and 3Department of Health Sciences, Northeastern University, Boston, MA

Address for correspondence: Whitney A. Welch, M.S., Department of Kinesiology, University of Wisconsin–Milwaukee, Enderis Hall Room 416, P.O. Box 413, Milwaukee, WI 53201; E-mail: wawelch@uwm.edu.

Submitted for publication August 2013.

Accepted for publication January 2014.

To measure population compliance with the 2008 national physical activity guidelines and progress toward meeting physical activity goals (e.g., as identified in Healthy People 2020), researchers have developed several monitoring tools that can be used to determine activity levels within large populations (4). Most large cohort studies use physical activity questionnaires to estimate activity levels because of their low participant burden and ease of administration. However, a study using data from the US National Health and Nutrition Examination Survey has found that moderate to vigorous physical activity measured by accelerometer-based activity monitors yields a stronger association with physiological biomarkers (2). This provides strong evidence that objectively measured physical activity has superior validity, compared with subjective questionnaires. Accelerometer-based physical activity monitors enable researchers to estimate the intensity of activities toward the goal of developing a more complete picture of physical activity habits.

Some factors influence the accuracy of activity monitors in classifying physical activity intensities, including the number of monitoring axes, the placement site of the monitor on the body, and the type of activities an individual performs (3). Activity monitor use has increased in physical activity research, and it is therefore important that these devices are accurately calibrated and that researchers analyze the data correctly.

The gravity estimator of normal everyday activity (GENEA) is a triaxial accelerometer-based activity monitor that can be worn on the waist, wrist, or ankle. Recent research has shown the GENEA, whether worn on the waist or wrist, has excellent accuracy for classifying intensity of physical activity (8). On the basis of data from 16 structured activity bouts, Esliger et al. (8) developed intensity classification cut-points (sedentary, light, moderate, and vigorous)for the GENEA worn at the waist. Upon further analysis of the accuracy of their cut-points, Esliger et al. (8) reported a criterion validity of r = 0.87 and a classification accuracy of 95%. However, Esliger et al. (8) determined classification accuracy on the same sample used to develop the cut-points, which may have artificially inflated the accuracy rate (10). Thus, the purpose of this study was to determine whether the cut-points developed by Esliger et al. for the waist-worn GENEA accelerometer are accurate for predicting intensity categories across a range of lifestyle activities (home/office, ambulatory, and sport) in an independent sample.

Back to Top | Article Outline

METHODS

Adult participants (N = 139, age range = 20 to 60 yr) from the University of Tennessee, Knoxville (UTENN), the University of Massachusetts, Amherst (UMASS), and the surrounding communities were recruited to participate in this study. Upon final analysis, nine participants were excluded for being left-side dominant and 24 participants were excluded because of incomplete data; therefore, the final sample totaled 106 participants (n = 64 UTENN, n = 42 UMASS). Inclusion criteria for participation included the following: age 20–60 yr, apparently healthy, and free from chronic disease or any musculoskeletal injuries that may affect activity. Each participant signed an informed consent form approved by the institutional review boards of the UTENN and UMASS.

Back to Top | Article Outline

Protocol

The study sample consisted of 48% males and 51.9% females (n = 106; mean age, 39.1 ± 11.2 yr old; mean body mass index, 25.5 ± 4.6 kg·m−2). Before testing, participants filled out a Physical Activity Readiness Questionnaire, Healthy History Questionnaire, and Physical Activity Status questionnaire. The Physical Activity Readiness Questionnaire was used to assess participants’ current health status to engage in physical activity. The Physical Activity Status questionnaire was used to determine the types of activities the participants engaged in during their leisure time. This information was used when assigning activity routines. For example, someone who had never played basketball would not be asked to play basketball during their routine as to avoid inaccurate metabolic responses as a result of the new movements and activity. Participants’ height (stadiometer) and weight (Tanita BC-418, Tanita Corporation of America, Inc., Arlington Heights, IL (UTENN); physicians’ scale, Detecto, Webb City, MO (UMASS)) were measured, and participants’ body mass index was calculated (kg·m−2). The study protocol has been previously described (17), although the following section will briefly review the methods.

Participants were instructed not to consume nicotine, caffeine, other stimulants or food for 4 h before reporting to the laboratory and to avoid any planned exercise for 24 h before arrival. Each individual was randomized to perform one of two routines, which consisted of seven activities each. Activities in routine one included the following: filing papers, vacuuming, self-paced walking, treadmill walking at 6.4 km·h−1, cycling at 49 W, basketball, and treadmill running at 9.6 km·h−1. Activities in routine two included the following: computer work, treadmill walking at 4.8 km·h−1, cycling at 98 W, moving a box, treadmill walking at 4.8 and 6.4 km·h−1 with a 5% grade, and tennis. Each activity lasted 7 min with at least a 4-min break in between activities.

Oxygen uptake (V˙O2) was measured using the Oxycon mobile portable metabolic unit (CareFusion, San Diego, CA). V˙O2 data were collected breath by breath and averaged over minutes 4 through 6 of the collection period. Because of variations between the Oxycon systems at the two testing sites, V˙O2 values were increased by 7.8% at UTENN and decreased by 7.8% at UMASS. Further information on this correction can be found elsewhere (17). The GENEA was worn on the dominant waist (right waist) of all participants and was initialized to collect data at 80 Hz. After data collection, all data were downloaded onto a laboratory computer.

Measured V˙O2 values were converted to METs (1 MET = 3.5 mL·kg−1·min−1) (1). These MET values were then classified into an intensity category using the same thresholds as Esliger et al. (8) (sedentary (<1.5 MET), light (1.5–3.99 METs), moderate (4.0–6.99 METs), or vigorous (7+ METs)).

The GENEA data were analyzed using the GENEA postprocessing software (version 1.2.1) to convert the raw data into a gravity-subtracted signal vector magnitude (SVMgs) for each minute of activity. Three minutes (minutes 4–6) of each activity was averaged and classified into an intensity category using the hip cut-points of Esliger et al. (8) (sedentary (<77 counts per minute), light (77–219 counts per minute), moderate (220–2056 counts per minute), or vigorous (>2056 counts per minute).

Back to Top | Article Outline

Statistical analysis

Because of nonnormal distribution of the data, Spearman rank correlation coefficients between the METs and GENEA SVMgs values were computed. However, Pearson correlation coefficients were also computed to allow comparisons with previously published studies. Accuracy of intensity classification for each activity when using the cut-points of Esliger et al. (8) was assessed by a one-way chi-squared test. Cross-tabulation tables were constructed to provide GENEA under- and overestimations and accuracy of cut-points by intensity category. Sensitivity and specificity of the waist cut-points were computed to determine accuracy rate by intensity classification.

Back to Top | Article Outline

RESULTS

Table 1 shows the accuracy of intensity category agreement within each activity (one-way chi-squared analysis). The waist-worn GENEA accurately predicted intensity category classification 55.3% of the time. Two activities (walking at 4.8 km·h−1 at a 5% incline and running at 9.6 km·h−1) had perfect agreement between the GENEA estimated intensity categories and those based upon measured METs, and one activity (stationary cycling at 98 W) had 0% agreement. Thus, a chi-squared analysis could not be performed for these three activities.

TABLE 1

TABLE 1

Spearman rank order correlation coefficient between the GENEA SVMgs values and the measured MET values when all activities were combined was 0.730 (P < 0.001). The Pearson correlation coefficient was 0.734 (P < 0.001). Figure 1 shows the relation between energy expenditure (METs) and GENEA SVMgs values. Vertical lines indicate the proposed GENEA waist cut-points of Esliger et al. (8), and horizontal lines indicate the corresponding cut-points on the basis of measured energy expenditure.

FIGURE 1

FIGURE 1

Table 2 shows the cross-tabulation tables by actual intensity category as determined by the GENEA-estimated intensity category and the criterion intensity category. Intensity classification accuracy was calculated from the cross-tabulation tables. Shaded boxes represent accurate intensity classification. Overall intensity classification accuracy for all activities was 55.3% (Table 2a). When cycling was removed, intensity classification accuracy slightly increased to 58.3% (Table 2b). Light intensity activities were overestimated by the GENEA 60.6% of the time, and vigorous intensity activities were underestimated 43.2% of the time.

TABLE 2

TABLE 2

For all activities combined, the sensitivity of the cut-points for the four intensity categories ranged from 0.244 to 0.958 and specificity ranged from 0.576 to 0.943 (Table 3). Because of the frequent misclassification of cycling, when the two stationary cycling activities are removed from the analysis, sensitivity for the four intensity categories ranged from 0.148 to 0.958 and specificity ranged from 0.542 to 0.984 (Table 3).

Table 3

Table 3

Back to Top | Article Outline

DISCUSSION

Traditionally, accelerometer-based activity monitors have been worn on the waist to measure a person’s activity level (16). Results of the present study indicate that when the Esliger et al. (8) cut-points for the waist-worn GENEA activity monitor are used, they correctly classify intensity category 55.3% of the time. In contrast, Esliger et al. (8) reported an intensity classification accuracy of 95% at the waist site. Both studies included a large range of MET values, ranging from 0.94 to 13.61 METs for Esliger et al. (8) and 1.46 to 9.67 METs in the current study. However, the Pearson correlation coefficient between activity counts and energy expenditure in the current study (r = 0.73) was less than that seen in the study by Esliger et al. (r = 0.87). One factor that could have contributed to these differences is that the study of Esliger et al. (8) was a calibration study with a subsequent validation, whereas ours was a cross-validation study in a different group of individuals. Another factor that could have contributed to the higher rate of misclassification in the current study was the addition of sporting activities, which were not included in the calibration study of Esliger et al. Although basketball showed an accuracy rate of 77.4% (P = 0.719), the remaining sports activities (tennis and stationary cycling) were significantly below an 80% accuracy rate, with 0% agreement for cycling at 98 W.

Back to Top | Article Outline

Waist versus wrist site comparison

The positioning of the activity monitor on the body is an important consideration for physical activity researchers. The waist and wrist positions both have associated strengths and limitations. Although the waist-mounted accelerometers tend to underestimate upper body activities, the wrist site can underestimate lower body activities that do not involve upper body movement (13). Therefore, it is of interest to compare the current waist site placement with the left wrist cut-points developed by Esliger et al. (8), which were previously cross-validated by our group in the same group of subjects, performing the same activities (17). Previous research comparing accelerometer outputs between triaxial accelerometers at the wrist and waist has reported similar accuracy for these two wear locations (8). In addition, the use of triaxial accelerometers vastly increases the relation between energy expenditure and activity counts, whether accelerometers are worn at the wrist or waist sites (15). As previously discussed, Esliger et al. (8) reported high levels of explained variance at both the waist (r = 0.87) and left wrist (r = 0.86). Although our data showed a weaker relation between energy expenditure and activity counts than those of Esliger’s study, the relation between energy expenditure and activity counts for these two sites was similar in the current study (waist r = 0.73, left wrist r = 0.74) (17).

Rosenberger et al. (14) placed triaxial accelerometers (Wockets) on the dominant waist and dominant wrist during 20 different activities, including sedentary activities, cycling, ambulation, and lifestyle activities. In contrast to the two studies described above, their results demonstrated large differences in the variance in energy expenditure explained by an accelerometer at the waist site (R2 = 0.52) and wrist site (R2 = 0.13). However, this discrepancy could be due to differences in accelerometers’ dynamic range (Wockets, ±2g; GENEA, ±6g) or activities performed during validation. Chen et al. (5) combined the Tritrac R3D worn on the right waist and the ActiWatch worn on the dominant wrist during a 24-h stay in a room calorimeter. The Tritrac R3D reported a 90% accuracy rate and the ActiWatch an 86% accuracy rate when compared with the measured energy expenditure. However, because their study employed different monitor types at the wrist and the waist sites, it does not allow for a true comparison between these two different sites.

In addition, two studies have compared activity type (as opposed to intensity) classification accuracy at the waist and wrist site using machine-learning algorithms. These studies found that the waist and wrist sites showed close agreement, and both sites had high accuracy for classifying activity type (7,18). It should also be noted that the wrist site offers some important advantages compared with the waist site, including the ability to estimate sleep variables and increased participant compliance in wearing the monitors (3,9).

Recent research has highlighted the need for activity monitors that accurately classify sedentary behavior (3,11,12). The GENEA waist cut-points correctly classified sedentary activity 95.8% of the time, whereas the left wrist-worn GENEA only correctly classified sedentary activity 69.8% of the time (Table 4). However, at the opposite end of the intensity spectrum, the waist-worn GENEA activity monitor correctly classified vigorous intensity activity 56.8% of the time, but the wrist-worn activity monitor was able to correctly classify vigorous activity 70.7% of the time. Interestingly, basketball (mean METs, 8.5) was correctly classified 77.4% of the time with the waist-mounted GENEA, whereas the left-wrist worn GENEA correctly classified it 77.6% of the time (17). We had previously considered this to be a potential limitation to wrist-worn accelerometers because of the fact that most dribbling and shooting was performed with the dominant (right) arm, but the GENEA was worn on the nondominant wrist (6). Overall, the wrist did a better job of classifying the intensity of sports activities (i.e., tennis and basketball), whereas the waist-worn GENEA cut-points were better able to detect activities such as inclined walking.

TABLE 4

TABLE 4

Sensitivity and specificity measure the ability of the cut-points to correctly identify a particular activity classification (i.e., sedentary/not sedentary) by calculating the amount of true positives and true negatives. Esliger et al. (8) reported a range of sensitivity and specificity for the waist site from 0.73 to 0.99 and 0.80 to 0.99, respectively. Our waist site analysis showed a much lower overall range for both sensitivity (0.24–0.95) and specificity (0.58–0.94). When stationary cycling was removed from the analysis, the sensitivity of the light intensity cut-point actually decreased from 0.244 to 0.148 (Table 3). The decrease in sensitivity can be explained by a decrease in true positives when stationary cycling data were removed. With regard to the left wrist site, Esliger et al. (8) reported a sensitivity range of 0.78–0.97 and specificity range of 0.72–0.98. Our previous study, which cross-validated the Esliger et al. GENEA left wrist cut-points, resulted in a lower range of sensitivity (0.45–0.71) and specificity (0.74–0.91) when compared with the Esliger et al. reported values (17).

This study had several limitations. The participants’ age range in the current study was slightly younger (mean age, 39.1 ± 11.2 yr) than Esliger et al. (8) (mean age, 49.4 ± 6.5 yr). Also, we used the MET cut-points of 4 METs for moderate intensity activity and 7 METs for vigorous intensity activity. These cut-points were chosen to exactly replicate the method used by Esliger et al. (8). However, the use of these cut-points may limit comparability to other studies using other MET cut-points.

This study had several strengths worth noting. Participants with a wide range of ages (20–60 yr) were recruited, and we examined a wide array of activities within the different domains of physical activity. We were also able to compare intensity category classification accuracy for both the nondominant wrist site using our previously reported data and the dominant waist site, providing evidence relevant to positioning an activity monitor on the wrist versus the waist.

Back to Top | Article Outline

CONCLUSION

The intensity cut-points developed by Esliger et al. (8) for the waist-worn GENEA accelerometer have previously been shown to accurately estimate the classification of physical activity intensity. However, cross-validation of the waist cut-points in an independent sample, performing structured lifestyle activities, resulted in a 55.3% accuracy rate. In addition, the present study reported similar correlation coefficients between energy expenditure and activity counts for the waist-worn and wrist-worn devices (17), although these correlations were lower than those reported by Esliger et al. (8). This information is important to consider when deciding on which monitor to use and the placement of activity monitors in future studies.

The authors would like to thank Drs. Ann Swartz and Scott Strath for their contributions to the manuscript.

The authors acknowledge funding by the National Institutes of Health grant R01-1795-014 and U01-CA120783.

Patty Freedson and David Bassett are members of the Scientific Advisory Board for ActiGraph, LLC.

There are no other disclosures.

The results of the present study do not constitute endorsement by the American College of Sports Medicine.

Back to Top | Article Outline

REFERENCES

1. Ainsworth BE, Haskell WL, Whitt MC, et al. Compendium of physical activities: an update of activity codes and MET intensities. Med Sci Sports Exerc. 2000; 32 (9 Suppl): S498–516.
2. Atienza AA, Moser RP, Perna F, et al. Self-reported and objectively measured activity related to biomarkers using NHANES. Med Sci Sports Exerc. 2011; 43 (5): 815–21.
3. Bassett DR. Device-based monitoring in physical activity and public health research. Physiol Meas. 2012; 33 (11): 1769–83.
4. Butte NF, Ekelund U, Westerterp KR. Assessing physical activity using wearable monitors: measures of physical activity. Med Sci Sports Exerc. 2012; 44 (1 Suppl 1): S5–12.
5. Chen KY, Acra SA, Majchrzak K, et al. Predicting energy expenditure of physical activity using hip- and wrist-worn accelerometers. Diabetes Technol Ther. 2003; 5 (6): 1023–33.
6. Chen KY, Bassett DR Jr. The technology of accelerometry-based activity monitors: current and future. Med Sci Sports Exerc. 2005; 37 (11 Suppl): S490–500.
7. Choi L, Ward SC, Schnelle JF, Buchowski MS. Assessment of wear/nonwear time classification algorithms for triaxial accelerometer. Med Sci Sports Exerc. 2012; 44 (10): 2009–16.
8. Esliger DW, Rowlands AV, Hurst TL, Catt M, Murray P, Eston RG. Validation of the GENEA accelerometer. Med Sci Sports Exerc. 2011; 43 (6): 1085–93.
9. Freedson P, Bowles HR, Troiano R, Haskell W. Assessment of physical activity using wearable monitors: recommendations for monitor calibration and use in the field. Med Sci Sports Exerc. 2012; 44 (1 Suppl): S1–4.
10. Freedson PS, Lyden K, Kozey-Keadle S, Staudenmayer J. Evaluation of artificial neural network algorithms for predicting METs and activity type from accelerometer data: validation on an independent sample. J Appl Physiol. 2011; 111 (6): 1804–12.
11. Hart TL, Swartz AM, Cashin SE, Strath SJ. How many days of monitoring predict physical activity and sedentary behaviour in older adults? Int J Behav Nutr Phys Act. 2011; 8: 62.
12. Kozey-Keadle S, Libertine A, Lyden K, Staudenmayer J, Freedson PS. Validation of wearable monitors for assessing sedentary behavior. Med Sci Sports Exerc. 2011; 43 (8): 1561–7.
13. Matthews CE, Hagstromer M, Pober DM, Bowles HR. Best practices for using physical activity monitors in population-based research. Med Sci Sports Exerc. 2012; 44 (1 Suppl): S68–76.
14. Rosenberger ME, Haskell WL, Albinali F, Mota S, Nawyn J, Intille S. Estimating activity and sedentary behavior from an accelerometer on the hip or wrist. Med Sci Sports Exerc. 2013; 45 (5): 964–75.
15. Van Remoortel H, Giavedoni S, Raste Y, et al. Validity of activity monitors in health and chronic disease: a systematic review. Int J Behav Nutr Phys Act. 2012; 9: 84.
16. Ward DS, Evenson KR, Vaughn A, Rodgers AB, Troiano RP. Accelerometer use in physical activity: best practices and research recommendations. Med Sci Sports Exerc. 2005; 37 (11 Suppl): S582–8.
17. Welch WA, Bassett DR, Thompson DL, et al. Classification accuracy of the wrist-worn gravity estimator of normal everyday activity accelerometer. Med Sci Sports Exerc. 2013; 45 (10): 2012–9.
18. Zhang S, Rowlands AV, Murray P, Hurst TL. Physical activity classification using the GENEA wrist-worn accelerometer. Med Sci Sports Exerc. 2012; 44 (4): 742–8.
Keywords:

PHYSICAL ACTIVITY; ENERGY EXPENDITURE; ACTIVITY MONITOR; ACCELEROMETRY

© 2014 American College of Sports Medicine