Accurate measurement of physical activity (PA) in children is of critical importance to monitor prevalence and trends, establish associations with health outcomes, identify determinants, and evaluate the effectiveness of interventions to promote PA (1). Hip-mounted accelerometers have been commonly used to objectively quantify habitual PA in children (2). However, low participant compliance with accelerometry protocols has resulted in considerable non-wear time and, subsequently, loss of data (3). National biobanks, such as U.K. Biobank (4), and large population surveys (5), including the National Health and Nutrition Examination Study 2011–2014 (6) in the U.S. incorporated wrist-worn accelerometers’. Recent evidence indicates that wrist-placement results in increased wear time due to greater compliance (6–8), which has consequently caused a shift from hip-placement to wrist-placement.
Traditionally, accelerometer-based PA monitoring devices have provided proprietary units called “counts” from which cut points have been developed to classify moderate (MPA), vigorous (VPA), and moderate-to-vigorous PA (MVPA) and estimate time spent in MVPA. However, more recently, commonly used accelerometer-based motion sensors such as the GENEActiv (ActivInsights Ltd., Cambridge, UK) and ActiGraph GT3X+ and GT9X (ActiGraph Corporation, Pensacola Beach, FL) provide access to high-frequency triaxial acceleration data, and therefore cut points to define PA intensity have been developed for these data collected from wrist devices. The existence of multiple cut points makes comparisons of PA outcomes from studies that have used different cut points challenging, and inconsistencies between studies may affect conclusions about PA prevalence, health benefits, determinants, and the effectiveness of interventions. Therefore, studies are needed that simultaneously compare the validity of multiple cut points to provide evidence upon which consensus can be reached for consistent data reduction approaches, which could increase the comparability of PA outcomes between studies.
Recent laboratory-based calibration studies (9–11) have developed three sets of PA intensity thresholds for raw acceleration output from wrist-worn devices in 6- to 14-yr-old children using indirect calorimetry as the criterion measure. The cut points were cross validated and demonstrated acceptable classification accuracy. However, two studies (10,11) applied the leave-one-out cross-validation approach in the calibration sample and evaluated classification accuracy for the same MPA and VPA activities, which were predominantly ambulatory (e.g., treadmill walking and running). As such, generalizability to free living scenarios may be limited. One set of cut points (9) was cross validated in an independent sample of 5- to 8-yr-olds (12); however, the sample size was small (n = 15), the protocol included a limited range of activities, and the cut points were not cross validated in children older than 8 yr.
These independent calibration studies used different data processing methodologies and have resulted in different cut points, ranging from 192 mg (11) to 314 mg (10) and from 696 mg (11) to 998 mg (10) for MPA and VPA, respectively, thus providing different PA estimates that make it difficult to compare outcomes between studies. Therefore, additional studies are needed to adequately cross validate cut points. A recent study (13) validated various data processing approaches for the wrist-worn ActiGraph in children and concluded that differences in PA estimates were caused by the use of different methods. However, because Kim et al. (13) did not include a valid criterion measure, the most accurate approach could not be determined. In agreement with best practice recommendations from Welk et al. (14), the authors suggested that the validity of different methods, along with their corresponding cut points, should be evaluated simultaneously, relative to gold standard methods. Therefore, the aim of this study was to simultaneously evaluate the performance of three sets of wrist acceleration cut points for classifying MPA, VPA, and MVPA and estimating time spent in PA intensities, under consistent conditions, using portable indirect calorimetry as the criterion measure in 5- to 12-yr-old children.
METHODS
Participants
Fifty-seven children 5–12 yr of age who were without physical or health conditions that would affect participation in PA were recruited as part of an activity monitor validation study. The study was approved by the University of Wollongong Health and Medical Human Research Ethics Committee. Descriptive characteristics of participants are presented in Table 1. Written parental consent and participant assent were obtained before participation.
;)
TABLE 1: Participant characteristics.
Procedures
Participants were required to visit the laboratory on two occasions. Anthropometric measures were completed during the first visit using standardized procedures while children were wearing light clothing and with shoes removed. Body mass index (kg·m−2) was calculated to categorize participants as normal weight or overweight/obese, according to the 2000 CDC Growth Charts for the United States (15). Children completed a protocol of 15 semistructured activities (Table 2) from sedentary (lying down, TV viewing, handheld e-game, writing/coloring, and computer game), light-intensity PA (LPA: getting ready for school, standing class activity, slow walk, and dancing), and MVPA (tidy up, brisk walk, soccer, basketball, running, and locomotor course). Activities were equally divided over two visits and completed in a structured order of increasing intensity for 5 min (except for lying down, which was performed for 10 min).
TABLE 2: Activity protocol.
Instrumentation
At each visit, children were fitted with a portable respiratory gas analysis system (MetaMax® 3B, Cortex, Biophysics, Leipzig, Germany) to provide the criterion assessment of PA energy expenditure. Children were also fitted with a GENEActiv dorsally on the nondominant wrist.
Indirect calorimetry
Oxygen consumption (O2) was assessed using the MetaMax® 3B portable breath-by-breath respiratory gas analysis system to provide the criterion assessment of energy expenditure. The participants wore a facemask (Hans Rudolph, Kansas City, MO) covering their nose and mouth, which was held in place by a head harness. Before every measurement, the analyzer was calibrated according to the manufacturer’s guidelines. Breath-by-breath data from indirect calorimetry were downloaded and exported using MetaSoft (version 4.3.2).
Activity monitor
GENEActiv has a waterproof design and measures triaxial accelerations ranging in magnitude ±8g at a sample frequency ranging from 10 to 100 Hz. Acceleration values are digitized by a 12-bit analog-to-digital converter. Accelerometers were initialized with a sample frequency of 100 Hz.
Data Reduction
Energy expenditure
O2 uptake volume and CO2 production were averaged per 10 s for every entire activity bout of 5 min and converted into units of energy expenditure (kcal·min−1) using the Weir equation (16). For analytical purposes, and for consistency with the calibration studies of the cut points (9–11), the activities were categorized in the primary analyses as non-MVPA (<3 METs), MPA (≥3 to <6 METs), or VPA (≥6 METs) based on average measured energy expenditure values. MPA and VPA were subsequently combined and classified as MVPA (≥3 METs). The participants’ measured resting energy expenditure (REE) from the lying down trial was used to define 1 MET to calculate MET values for all activities. Breath-by-breath samples from the data collected between minutes 7.0 and 9.0 during the lying down trial were averaged to calculate mean REE. Metabolic data (10-s epochs) from the activities were scaled to the children’s REE and converted into youth METs using customized software. Although 3 METs has been widely used as an intensity threshold to distinguish MPA from LPA, there is considerable evidence that 4 METs is more accurate for classifying MPA in children and adolescents (17) and that brisk walking, a key behavioral indicator of MPA, is associated with an energy cost of approximately 4 METs (18). It should be noted that researchers have based these estimates on either predicted REE or measured REE. As such, studies have demonstrated that MET levels for walking and other activities are somewhat contingent on the choice of the denominator (19,20). In our sample, the larger value results in ~3 METs for brisk walking as the behavioral indicator, when based on measured REE (slow walking = 2.9 ± 0.5 METs; brisk walking = 3.4 ± 0.6 METs) (see Table, Supplemental Digital Content 1, Metabolic data by activities for indirect calorimetry, https://links.lww.com/MSS/B67). However, when based on predicted REE, the value was closer to 4 METs (slow walking = 4.0 ± 0.6 METs; brisk walking = 4.7 ± 0.7 METs), which was consistent with a previous study (comfortable walking = 3.9 ± 0.6 METs; brisk walking = 4.7 ± 0.6 METs) (21). Therefore, supplementary analyses were conducted testing the consistency of the findings using a threshold of 4 METs, for which METs were calculated by dividing mean energy expenditure values by REE predicted from the participant’s sex, age, body mass, and height using Schofield’s (22) equation for children 3–10 or 10–18 yr of age.
Accelerometry
Data reduction approaches were performed according to the methods reported in calibration studies by Hildebrand et al. (11), Phillips et al. (9), and Schaefer et al. (10) for the development of the three cut points evaluated. Raw wrist data were downloaded using the GENEActiv software version 2.2. Signal-processing codes from Hildebrand et al. (11) were downloaded and applied to convert raw acceleration data into 1-s epochs according to the Euclidian norm minus one (ENMO) approach. This method subtracted 1g from the Euclidian norm (EN = sqrt (x2 + y2 + z2)), after which negative values were rounded up to zero. According to the methods described by Phillips et al. (9), raw acceleration data were converted into 1-s epochs using the GENEActiv postprocessing software to create gravity-subtracted signal vector magnitude data. Customized software was developed using the statistical computing language R (V.3.1.2) to apply a band-pass filter to the raw acceleration data (fourth-order Butterworth filter with ω0 = 0.2–15 Hz) to remove the gravitational acceleration component as well as high-frequency sensor noise, as described by Schaefer et al. (10). EN was taken from the three resulting signals and averaged per 1-s epoch. This method is called band-pass filter followed by Euclidian norm (BFEN). The methods of the calibration studies resulted in sets of cut points in order of increasing acceleration magnitude and hereafter called as follows:
- Hildebrand et al. (11), ENMO192+: nondominant wrist; MPA, 192–695 mg; VPA, ≥696 mg.
- Phillips et al. (9), GENEA250+: right wrist; MPA, >275 to ≤700 mg; VPA, >700 mg, left wrist; MPA, >250 to ≤750 mg; VPA, >750 mg. Calibration procedures for these cut points were based on the cumulative sum of gravity-based accelerations measured with a sample frequency of 80 Hz, making the original cut points frequency dependent (11). For presentation purposes, the cut point values were converted from a time dependent unit (g·s−1) to the time independent unit milligrams to compare with values of other cut points.
- Schaefer et al. (10), BFEN314+: nondominant wrist; MPA, 314–998 mg; VPA, ≥998 mg.
The 1-s epochs for accelerometry data of all methods were averaged over 10-s windows to align with indirect calorimetry data.
Data synchronization
At the beginning of each laboratory visit, the activity monitors and indirect calorimetry were synchronized with an internal computer clock. After applying the cut points, predicted intensity classification for the wrist acceleration data was aligned with the ground truth energy expenditure data to examine classification accuracy. All valid epochs from each activity trial were included in analyses to reflect how activity monitors are applied under free-living conditions. Estimated time spent in each PA intensity using indirect calorimetry or wrist accelerometry was established by summing the 10-s epochs classified for each intensity.
Statistical Analyses
Normality of the data was confirmed before analyses. Classification accuracy for each set of cut points (MPA, VPA, non-MVPA) was examined by calculating weighted κ statistics. Kappa coefficients were interpreted using the ratings suggested by Landis and Koch (23): poor (0–0.20), fair (0.21–0.40), moderate (0.41–0.60), substantial (0.61–0.80), and almost perfect (0.81–1.0). Contingency tables were applied to summarize classification accuracy and percentage of misclassified epochs for each intensity. Because of the public health focus on MVPA, the intensities of MPA and VPA were combined as one dichotomous variable MVPA, and the classification accuracy was evaluated using sensitivity, specificity, and area under the receiver operating characteristic curve (ROC-AUC). ROC-AUC values were defined as excellent (≥0.90), good (0.80–0.89), fair (0.70–0.79), or poor (<0.70) (24). The equivalence of time estimates between the cut points and indirect calorimetry for each intensity was examined at the group level using the 95% paired equivalence test. To reject the null hypothesis of the equivalence test, the 90% confidence interval (CI) of time spent in the intensity predicted by the monitors should fall entirely within the predefined equivalence region of ±10% (25). Measurement agreement and systematic bias for estimated time spent in intensities were evaluated at the individual level using Bland–Altman procedures (26). Analyses were performed using the statistical computing language R V.3.1.2 (The R Foundation for Statistical Computing) and SPSS V.21.0 (IBM Corporation, Armonk, NY).
RESULTS
All participants completed the protocol. For one of the visits, wrist acceleration data were unavailable for three children. Data from one child were entirely excluded from the analyses, and data from three participants for a total of eight activities were excluded because of indirect calorimetry failure. A total of 25,452 PA intensity annotated 10-s epochs (94.4% of the total data) from 57 children were available for analyses.
Applying the contingency tables for classification accuracy (Table 3), ENMO192+ (κ = 0.72, 95% CI = 0.72 to 0.73), GENEA250+ (κ = 0.75, 95% CI = 0.74 to 0.76), and BFEN314+ (κ = 0.73, 95% CI = 0.72 to 0.74) exhibited substantial agreement. The proportion of correctly classified epochs for the BFEN314+ MPA and VPA cut points (52.0% and 93.6%, respectively) was higher than for the ENMO192+ cut points (46.5% and 70.0%, respectively) and the GENEA250+ cut points (45.4% and 79.9%, respectively). However, ENMO192+ and GENEA250+ classified non-MVPA (90.5% and 89.2%, respectively) more accurately than BFEN (81.7%). BFEN misclassified 19.7% of non-MVPA as MPA and 39.4% of MPA as VPA. The highest proportions of misclassification for ENMO192+ and GENEA250+ on the other hand were found for MPA misclassified as non-MVPA (ENMO192+: 32.6% epochs; GENEA250+: 26.5% epochs) and VPA misclassified as MPA (ENMO192+: 20.8% epochs; GENEA250+: 28.1% epochs). ENMO192+ and GENEA250+ misclassified 25.0% and 19.4% of VPA as MPA. Classification accuracy for MVPA was good for all cut points (ROC-AUC: ENMO192+ = 0.85, 95% CI = 0.85 to 0.86; GENEA250+ = 0.85, 95% CI = 0.85 to 0.86; BFEN314+ = 0.86, 95% CI = 0.86 to 0.87). Although the true-positive rate (sensitivity) for BFEN314+ (0.94) was higher than for ENMO192+ (0.80) and GENEA250+ (0.81), specificity for BFEN314+ was lower (0.78) compared with ENMO192+ (0.90) and GENEA250+ (0.89).
TABLE 3: Contingency tables for classification accuracy of raw wrist acceleration cut points.
At the group level, estimated time spent in MPA was equivalent (P < 0.01) to indirect calorimetry for BFEN314+ and estimated time spent in MVPA was equivalent for ENMO192+ and GENEA250+ (Fig. 1). Outcomes of the Bland–Altman analyses are presented in Table 4. BFEN314+ overestimated time spent in MPA by a small margin of 1.5% (limits of agreement [LoA] = −57.5% to 60.6%), whereas ENMO192+ and GENEA250+ overestimated time spent in MPA by 30.1% (LoA = −99.6% to 39.4%) and 31.0% (LoA = −104.1% to 42.0%), respectively. Overestimation of time spent in VPA was larger for BFEN314+ (92.2%, LoA = −54.6% to 238.9%) compared with ENMO192+ (58.5%, LoA = −127.4% to 244.5%) and GENEA250+ (75.2%, LoA = −91.8% to 242.2%). Mean bias for time spent in MVPA was small for ENMO192+ (−1.1%, LoA = −55.9% to 53.7%) and GENEA250+ (2.2%, LoA = −52.2% to 56.5%), whereas time spent MVPA was overestimated by BFEN314+ to a larger extent (29.3%, LoA = −25.3% to 83.9%). At the individual level, LoAs were wide for all cut points and for all intensities, especially for VPA estimates from all cut points and for MPA estimates from the ENMO192+ and GENEA250+. Systematic bias (P < 0.05) was found for time spent in all intensities estimated by all cut points, with the exceptions of time spent in MPA estimated by BFEN314+ and GENEA250+, indicating that errors increased with increasing time spent in the intensities.
TABLE 4: Agreement analysis of raw wrist acceleration-based estimations of physical activity intensities compared with indirect calorimetry.
FIGURE 1: The 95% equivalence test for raw wrist acceleration-based estimated time spent in physical activity intensities. Times estimated by wrist-worn cut points are equivalent to indirect calorimetry if 90% CI values lie entirely within the equivalence region of indirect calorimetry. ENMO: cut points developed using ENMO; GENEA: cut points developed using the GENEActiv postprocessing software; BFEN: cut points developed using BFEN.
Supplementary analyses (see Tables and Figure, Supplemental Digital Content 2, Supplementary analyses for the raw wrist acceleration cut points using a ≥4-MET MVPA definition, https://links.lww.com/MSS/B68) indicated that classification accuracy for MPA, VPA, and non-MVPA remained similar when 1 MET was defined using predicted REE and a 4-MET threshold for MPA was applied to the data (ENMO192+, κ = 0.65, 95% CI = 0.64 to 0.66; GENEA250+, κ = 0.71, 95% CI = 0.70 to 0.72; BFEN314+, κ = 0.75, 95% CI = 0.74 to 0.76). Although ROC-AUC values for MVPA (ENMO192+ = 0.85, 95% CI = 0.85 to 0.86; GENEA250+ = 0.86, 95% CI = 0.85 to 0.86; BFEN314+ = 0.87, 95% CI = 0.87 to 0.88) were similar to the primary analyses, slightly more non-MVPA epochs were correctly classified (see Table, Supplemental Digital Content 2, 2.1: Contingency tables for classification accuracy of raw wrist acceleration cut points using a ≥4-MET MVPA definition, https://links.lww.com/MSS/B68). Although time spent in MVPA estimated by ENMO192+ and GENEA250+ using the ≥4-MET MVPA definition was not equivalent to indirect calorimetry as they were in the primary analyses, the mean and/or 90% CI values for estimated time spent in MPA and MVPA for ENMO192+ and GENEA250+ overlapped the equivalence region and thus approached equivalence. BFEN314+ overestimated time spent in MVPA for both the 3-MET (1 MET = measured REE) approach (29.3%, LoA = −25.3% to 83.9%) and the 4-MET (1 MET = predicted REE) approach (18.3%, LoA = −13.5% to 50.2%) (see Table, Supplemental Digital Content 2, 2.2: Agreement analysis of raw wrist acceleration-based estimations of PA intensities compared with indirect calorimetry using a ≥4-MET MVPA definition, https://links.lww.com/MSS/B68). Time spent in MPA estimated by BFEN314+ was no longer equivalent to the criterion measure, whereas time spent in VPA was equivalent to the criterion measure (P < 0.01) (see Figure, Supplemental Digital Content 2, 2.3: 95% equivalence test for raw wrist acceleration-based estimated time spent in PA intensities using a ≥4-MET MVPA definition, https://links.lww.com/MSS/B68). By contrast, when defining MVPA as ≥4-METs, fewer MPA epochs were misclassified by BFEN314+ as VPA compared with the 3-MET approach; however, more VPA epochs were misclassified as MPA. The overestimation of time spent in VPA from BFEN314+ was small for the 4-MET approach (0.5%, LoA = −39.7% to 40.6%), whereas the overestimation of time spent in MPA for BFEN314+ was larger (34.4%, LoA = −20.4% to 89.1%). At the individual level, errors for all cut points were decreased for time spent in VPA when using the 4-MET approach but increased for time spent in MPA compared with outcomes from the 3-MET approach.
DISCUSSION
Current international PA guidelines specify that children should accumulate a minimum of 60 min·d−1 of MVPA (27). Therefore, the accurate measurement of MVPA is central to understanding the prevalence and patterns of PA, the dose of PA required to achieve health benefits, the determinants of PA, and the effect of PA interventions for children, which typically target MVPA. This study simultaneously cross validated three previously published wrist acceleration cut points for the classification of MVPA in children. ENMO192+, GENEA250+, and BFEN314+ demonstrated good classification accuracy for MVPA. However, although time spent in MVPA estimated by ENMO192+ and GENEA250+ was equivalent to indirect calorimetry, misclassification of non-MVPA as MVPA resulted in an overestimation of time spent in MVPA for BFEN314+. Although ENMO192+ and GENEA250+ classified non-MVPA more accurately than BFEN314+, these cut points still misclassified a significant proportion of MVPA epochs as non-MVPA (37.6% and 27.2%, respectively). Findings were relatively consistent in supplementary analyses, where predicted REE was used to define 1 MET and MVPA was defined as ≥4 METs. The classification accuracy of MPA, VPA, and MVPA remained relatively similar for all cut points compared with previous analyses, and although time spent in MVPA estimated by ENMO192+ and GENEA250+ was no longer equivalent to indirect calorimetry, estimates approached equivalence.
Findings from the current study were similar to findings in previous independent cross-validation studies, which demonstrated good classification accuracy for MVPA estimates from raw acceleration wrist cut points (10,12), and that classification for VPA is generally higher than for MPA (10–12). Although classification of MPA, VPA, and MVPA was most accurate for BFEN314+, ENMO192+ and GENEA250+ estimated time spent in MVPA more accurately than BFEN314+. Time spent in MVPA was overestimated by BFEN314+ because a relatively large proportion (19.7%) of non-MVPA was misclassified as MPA, which was in agreement with Schaefer et al.’s (10) application in free-living individuals. This misclassification could be explained by activities of light intensity that involve vigorous wrist movements. For example, BFEN314+ misclassified 66.4% of non-MVPA as MPA during the non-MVPA activity “Getting ready for school” (see Table, Supplemental Digital Content 3, Confusion matrices for the raw wrist acceleration cut points using a ≥3-MET MVPA definition, https://links.lww.com/MSS/B69), an activity of low intensity that involved relatively high wrist motion (e.g., while getting dressed, packing a schoolbag, brushing hair, etc.). The opposite effect may occur when MVPA activities involve limited wrist movement. As such, the ENMO192+ and GENEA250+ misclassified 82.3% and 77.1%, respectively, of MPA as non-MVPA during “Tidy up,” an activity of MPA intensity that may have involved limited upper body and wrist motions due to carrying objects while walking. Because of the public health focus on MVPA, misclassification by wrist cut points of MPA as VPA and vice versa may not represent a major measurement limitation. However, increased interest among researchers in the influence of sedentary behaviors, defined as any waking behaviors in a sitting or reclining position that require an energy expenditure of ≤1.5 METs (28) and LPA (1.5 to <3.0 METs), on health makes it critical to discriminate between these behaviors and MVPA. Previous studies indicate that accurate assessment of sedentary behaviors and the number of breaks in sedentary time based on a lack of wrist movement is challenging (11,29,30). The findings from this study confirm that the use of the magnitude of acceleration only might not be effective in distinguishing MVPA from non-MVPA. This finding is relatively consistent with previous studies using cut points based on proprietary activity “counts” (31–33). This is likely because the association between counts or raw acceleration and energy expenditure, whether on the hip or wrist, differs for different types of PA, resulting in cut points performing well for some activities and demonstrating considerable misclassification during other activities. It should be noted that the benefit of using raw acceleration-based cut points over using count-based cut points remains unclear, as in general cut points result in misclassification, which was also demonstrated by the results in this study for all cut points. Therefore, progress on alternative approaches, such as those using machine learning (29,33,34), may be required. However, similar to the inconsistencies that occur because of the existence of multiple cut points, the existence of different machine learning approaches and models, such as artificial neural networks (35), decision trees (36), and hidden Markov models (37), presents further challenges, and evidence to reach consensus on the most accurate approach for categorizing PA intensities in children is required.
An additional limitation of the wrist cut points validated in the current study is that calibration studies used different processing methodologies. Although Schaefer et al. (10) used a filtering approach to remove static accelerations from the triaxial data, Hildebrand et al. (11) and Phillips et al. (9) subtracted the value of gravity from the vector magnitude to focus the outcome variable on dynamic rather than static accelerations. Hildebrand et al. (11) used the ENMO method, which rounds negative values, resulting from subtracting the vector magnitude by 1g, up to zero. Phillips et al. (9), on the other hand, replaced the negative values with their absolute values and summed the resulting values, which creates a dependency on sample frequency, and thus the cut points should be converted when using different sample frequencies to compare results across studies. ENMO192+ and BFEN314+ were developed using averaged acceleration magnitudes and can be used for different sample frequencies and epoch lengths. The different processing methods also resulted in different units for the outcomes; Hildebrand et al. (11) and Schaefer et al. (10) used gravity units in g and mg, respectively, whereas Phillips et al. (9) used gravity-based acceleration seconds. Taking all of this into account makes it complicated to compare results from the different cut points, and as the field progresses, it is important that procedures are standardized based on evidence. Furthermore, some data indicate that raw acceleration output from the GENEActiv and ActiGraph may differ in children during common activities (11). This is likely because manufacturer-specific transformations (e.g., filtering) are applied to the raw acceleration data, resulting in different outputs from different devices that may not be a representation of the actual raw acceleration signals (38). As such, our findings may only apply to the GENEActiv monitor, and further evaluation across different monitor brands is required.
A strength of this study was that three recently developed sets of raw wrist acceleration cut points were evaluated simultaneously against a criterion measure. The study included a broad age range and an equal distribution of age and sex across the sample. In addition, a range of tasks, beyond treadmill-based ambulatory activities, that are likely to resemble children’s free-living behaviors were included in the protocol. Although these activities reflect daily activities that children typically engage in, the findings of the present study should be confirmed under free-living conditions. A potential limitation of this study is that validation focused on MVPA and did not include LPA or sedentary behavior. Our previous cross-validation study (29) of sedentary cut points demonstrated that hip-based cut points typically misclassify light activities (e.g., standing still) as sedentary postures, whereas wrist cut points exhibit some misclassification of nonsedentary behaviors as sedentary and vice versa. Therefore, it is essential to apply the most accurate intensity-specific cut points for accurate estimates of sedentary behaviors and LPA. However, to investigate the accuracy of cut points for distinguishing sedentary behaviors from LPA, postures such as sitting and standing should be evaluated. This is typically performed using alternative criterion measures, such as direct observation, as described in our previous work (39). Another potential limitation is that acceleration signals were not calibrated to local gravity before analysis to minimize sensor calibration errors, as described by van Hees et al. (40). Furthermore, body accelerations and metabolic rate during the exercise bouts may not have been aligned because of lags in oxygen consumption, and true classification accuracy may have been underestimated. However, this data reduction approach reflects how cut points are used in free-living population studies, and because the approach was applied consistently across cut points, one cut point was not biased over the other.
In conclusion, although raw acceleration wrist cut points exhibited good accuracy for classifying MVPA in children, all cut points misclassified a significant proportion of MVPA epochs as non-MVPA. Although the cut points demonstrated acceptable estimates of time spent in MPA, VPA, and MVPA at the group level, their application was less accurate for individual measures. When combined with the practical advantages of wrist worn placement, surveillance application of the raw wrist acceleration cut points would be acceptable for group-level estimates of MVPA, although alternative data processing approaches such as machine learning methods may be needed to achieve a generally higher accuracy for the assessment of PA intensities among individual children.
The authors thank all children and their parents for their participation. They also thank Melinda Smith for her assistance with recruitment and data collection. This study was funded by the National Heart Foundation of Australia (grant no. G11S5975). D. P. C. was supported by an Australian Research Council Discovery Early Career Researcher Award (grant no. DE140101588). A. D. O. was supported by a National Heart Foundation of Australia Career Development Fellowship (grant no. CR11S 6099). T. H. was funded by a National Health and Medical Research Council Early Career Fellowship (grant no. APP1070571). U. E. was funded by the Research Council of Norway (grant no. 249932/F20). The work of U. E. and S. B. was funded by the UK Medical Research Council (MC_UU_12015/3). S. T. was supported by the National Health and Medical Research Council Centre of Research Excellence on Sitting Time and Chronic Disease Prevention (grant no. APP1057608).
The authors have no conflict of interest to declare. The results of the present study do not constitute endorsement by the American College of Sports Medicine. The results are presented clearly, honestly, and without fabrication, falsification, or inappropriate data manipulation.
REFERENCES
1. Rennie KL, Wareham NJ. The validation of physical activity instruments for measuring energy expenditure: problems and pitfalls.
Public Health Nutr. 1998;1(4):265–71.
2. Trost SG. State of the art reviews: measurement of physical activity in children and adolescents.
Am J Lifestyle Med. 2007;1(4):299–314.
3. Freedson PS, John D. Comment on estimating activity and sedentary behavior from an accelerometer on the hip and wrist.
Med Sci Sports Exerc. 2013;45(5):962–3.
4. UK Biobank. Category 2 enhanced phenotyping at baseline assessment visit in last 100–150,000 participants [Internet]. 2009 [cited 2017 Jan 21]. Available from:
http://www.ukbiobank.ac.uk/wp-content/uploads/2011/06/Protocol_addendum:2.pdf.
5. da Silva IC, van Hees VT, Ramires VV, et al. Physical activity levels in three Brazilian birth cohorts as assessed with raw triaxial wrist accelerometry.
Int J Epidemiol. 2014;43(6):1959–68.
6. Troiano RP, McClain JJ, Brychta RJ, Chen KY. Evolution of accelerometer methods for physical activity research.
Br J Sports Med. 2014;48(13):1019–23.
7. Rowlands AV, Rennie K, Kozarski R, et al. Children’s physical activity assessed with wrist- and hip-worn accelerometers.
Med Sci Sports Exerc. 2014;46(12):2308–16.
8. Fairclough SJ, Noonan R, Rowlands AV, van Hees V, Knowles Z, Boddy LM. Wear compliance and activity in children wearing wrist- and hip-mounted accelerometers.
Med Sci Sports Exerc. 2016;48(2):245–53.
9. Phillips LR, Parfitt G, Rowlands AV. Calibration of the GENEA accelerometer for assessment of physical activity intensity in children.
J Sci Med Sport. 2013;16(2):124–8.
10. Schaefer CA, Nigg CR, Hill JO, Brink LA, Browning RC. Establishing and evaluating wrist cutpoints for the GENEActiv accelerometer in youth.
Med Sci Sports Exerc. 2014;46(4):826.
11. Hildebrand M, Van Hees VT, Hansen BH, Ekelund U. Age-group comparability of raw accelerometer output from wrist-and hip-worn monitors.
Med Sci Sports Exerc. 2014;46(9):1816–24.
12. Duncan MJ, Wilson S, Tallis J, Eyre E. Validation of the Phillips et al. GENEActiv accelerometer wrist cut-points in children aged 5–8 years old.
Eur J Pediatr. 2016;175(12):2019–21.
13. Kim Y, Hibbing P, Saint-Maurice PF, et al. Surveillance of youth physical activity and sedentary behavior with wrist accelerometry.
Am J Prev Med. 2017;52(6):872–9.
14. Welk GJ, McClain J, Ainsworth BE. Protocols for evaluating equivalency of accelerometry-based activity monitors.
Med Sci Sports Exerc. 2012;44(1 Suppl 1):S39–49.
15. Kuczmarski RJ, Ogden CL, Guo SS, et al. 2000 CDC Growth Charts for the United States: methods and development.
Vital Health Stat 11. 2002;246:1–190.
16. Weir JB. New methods for calculating metabolic rate with special reference to protein metabolism.
J Physiol. 1949;109(1–2):1–9.
17. Saint-Maurice PF, Kim Y, Welk GJ, Gaesser GA. Kids are not little adults: what MET threshold captures sedentary behavior in children?
Eur J Appl Physiol. 2016;116(1):29–38.
18. Trost SG, Loprinzi PD, Moore R, Pfeiffer KA. Comparison of accelerometer cut points for predicting activity intensity in youth.
Med Sci Sports Exerc. 2011;43(7):1360–8.
19. Schuna JM Jr, Barreira TV, Hsia DS, Johnson WD, Tudor-Locke C. Youth energy expenditure during common free-living activities and treadmill walking.
J Phys Act Health. 2016;13(6):S29–34.
20. Sasaki JE, Howe C, John D, et al. Energy expenditure for 70 activities in children and adolescents.
J Phys Act Health. 2016;13(6 Suppl 1):S24–8.
21. Trost SG, Drovandi CC, Pfeiffer K. Developmental trends in the energy cost of physical activities performed by youth.
J Phys Act Health. 2016;13(6):S35–40.
22. Schofield WN. Predicting basal metabolic rate, new standards and review of previous work.
Hum Nutr Clin Nutr. 1985;39:5–41.
23. Landis JR, Koch GG. The measurement of observer agreement for categorical data.
Biometrics. 1977;33(1):159–74.
24. Metz CE. Basic principles of ROC analysis.
Semin Nucl Med. 1978;8(4):283–98.
25. Batterham M, Van Loo CMT, Charlton K, Cliff D, Okely A. Improved interpretation of studies comparing methods of dietary assessment: combining equivalence testing with the limits of agreement.
Br J Nutr. 2016;115(7):1273–80.
26. Altman DG, Bland JM. Measurement in medicine: the analysis of method comparison studies.
The statistician. 1983;307–17.
27. WHO. Global Recommendations on Physical Activity for Health. World Health Organization [Internet]. 2010 [cited 2016 Dec 15]. Available from:
apps.who.int/iris/bitstream/10665/44399/1/9789241599979_eng.pdf.
28. Sedentary Behavior Research Network. Letter to the editor: standardized use of the terms “sedentary” and “sedentary behaviours.”
Appl Physiol Nutr Metab. 2012;37:540–2.
29. van Loo CMT, Okely AD, Batterham MJ, et al. Wrist accelerometer cut-points for classifying sedentary behavior in children.
Med Sci Sports Exerc. 2017;49(4):813–22.
30. Rosenberger ME, Haskell WL, Albinali F, Mota S, Nawyn J, Intille S. Estimating activity and sedentary behavior from an accelerometer on the hip or wrist.
Med Sci Sports Exerc. 2013;45(5):964.
31. Trost SG, Way R, Okely AD. Predictive validity of three
ActiGraph energy expenditure equations for children.
Med Sci Sports Exerc. 2006;38(2):380–7.
32. Janssen X, Cliff DP, Reilly JJ, et al. Predictive validity and classification accuracy of
ActiGraph energy expenditure equations and cut-points in young children.
PLoS One. 2013;8(11):e79124.
33. Trost SG, Wong W-K, Pfeiffer KA, Zheng Y. Artificial neural networks to predict activity type and energy expenditure in youth.
Med Sci Sports Exerc. 2012;44(9):1801–9.
34. Hagenbuchner M, Cliff DP, Trost SG, Van Tuc N, Peoples GE. Prediction of activity type in preschool children using machine learning techniques.
J Sci Med Sport. 2015;18(4):426–31.
35. Rothney MP, Neumann M, Béziat A, Chen KY. An artificial neural network model of energy expenditure using nonintegrated acceleration signals.
J Appl Physiol (1985). 2007;103(4):1419–27.
36. Bonomi AG, Plasqui G, Goris AH, Westerterp KR. Improving assessment of daily energy expenditure by identifying types of physical activity with a single accelerometer.
J Appl Physiol (1985). 2009;107(3):655–61.
37. Pober DM, Staudenmayer J, Raphael C, Freedson PS. Development of novel techniques to classify physical activity mode using accelerometers.
Med Sci Sports Exerc. 2006;38(9):1626–34.
38. John D, Freedson P.
ActiGraph and Actical physical activity monitors: a peek under the hood.
Med Sci Sports Exerc. 2012;44(1 Suppl 1):S86–9.
39. van Loo CMT, Okely AD, Batterham M, et al. Predictive validity of a thigh-worn accelerometer METs algorithm in 5-to 12-year-old children.
J Phys Act Health. 2016;13:S78–83.
40. van Hees VT, Fang Z, Langford J, et al. Autocalibration of accelerometer data for free-living physical activity assessment using local gravity and temperature: an evaluation on four continents.
J Appl Physiol (1985). 2014;117(7):738–44.
