Journal Logo

APPLIED SCIENCES

Wrist Accelerometer Cut Points for Classifying Sedentary Behavior in Children

VAN LOO, CHRISTIANA M. T.; OKELY, ANTHONY D.; BATTERHAM, MARIJKA J.; HINKLEY, TRINA; EKELUND, ULF; BRAGE, SØREN; REILLY, JOHN J.; TROST, STEWART G.; JONES, RACHEL A.; JANSSEN, XANNE; CLIFF, DYLAN P.

Author Information
Medicine & Science in Sports & Exercise: April 2017 - Volume 49 - Issue 4 - p 813-822
doi: 10.1249/MSS.0000000000001158

Abstract

Sedentary behaviors (SB) are defined as any waking behaviors in a sitting or reclining position that require an energy expenditure of ≤1.5 METs (30). Although some studies among children and adolescents suggest that the total volume or pattern of SB is associated with adverse health outcomes, independent of moderate- to vigorous-intensity physical activity (7,8,24), overall the evidence seems to be inconsistent (6,11). Accurate measures of SB are essential for both observational and experimental research to further investigate the influence of SB on health outcomes, as well as the prevalence and determinants of SB, and the effectiveness of interventions to reduce SB.

Accelerometry is the method of choice for objectively measuring the amount and patterning of SB in children (32), and various accelerometers are available for placement on different body locations (e.g., hip, wrist or thigh) (17). Hip-mounted accelerometers have commonly been used in children (32), with cut point approaches typically applied to define SB (17). For example, large population surveys, such as the National Health and Nutrition Examination Study 2003–2004, incorporated hip-worn ActiGraph accelerometers, and SB time was estimated using a threshold of <100 counts per minute (22). However, concerns about low participant compliance to accelerometry protocols and subsequent data loss have resulted in a shift from hip to wrist placement (14). The National Health and Nutrition Examination Study 2011–2014 (31) incorporated wrist-worn accelerometers, and the data from this study and other initial reports (13,28) indicate that wrist placement results in increased wear time because of greater compliance, which in turn leads to greater confidence that the data are representative of daily physical activity and SB. ActiGraph (ActiGraph LLC, Pensacola Beach, FL) and GENEActiv (ActivInsights Ltd., Cambridge, UK) are accelerometer-based motion sensors typically worn on the hip or wrist. Thresholds or cut points have been developed for the wrist-worn ActiGraph (5,9,19) and GENEActiv (26,29) to classify SB in children. The wrist cut points were developed using different age groups, sample sizes, and activity protocols, which results in variations in the cut points used to classify SB. For example, wrist cut points developed for ActiGraph's vertical axis (VA; x-axis) range from 35 counts per 5 s (9) to 202 counts per 5 s (Chandler et al., personal communication, 2016). Using different accelerometer models, placing them at different body locations, and applying different cut points results in considerable differences in estimates of SB (17,28), which makes it difficult to compare outcomes between studies and examine the epidemiology of SB. Therefore, comparison of these assessment methods is needed. Rowlands et al. (27) compared free-living SB estimates from a GENEActiv signal vector magnitude (SVM) wrist cut point (PhillipsSVM: right wrist, <6g·s−1; left wrist, <7g·s−1) with the widely used ActiGraph hip cut point for VA (Evenson: ≤25 counts per 15 s) (12) in a sample of free-living 10- to 12-yr-olds (28). This study reported that the outcomes from these monitors were highly correlated; however, sedentary time estimated by PhillipsSVM was significantly lower (9.6%) than estimates from the ActiGraph hip cut point. Because the study did not have a criterion measure of SB, the level of error from each measure is unknown. Furthermore, the relative validity of the range of GENEActiv and ActiGraph wrist cut points remains unknown because only one accelerometer model and one cut point for the wrist were evaluated.

It is also important to evaluate the validity of recent SB wrist cut points against alternative objective measures to understand the accuracy of newer approaches relative to other options for assessing SB. One alternative method is thigh-mounted accelerometry, such as the activPAL3TM (PAL Technology Ltd., Glasgow, UK) posture detection system, which classifies periods spent sitting/lying, standing, or stepping. Because of the monitor's placement on the thigh, it uses the orientation (angle to vertical) of the thigh to accurately estimate SB (34) rather than simply the movement intensity measures used in traditional hip-based cut point approaches, which have difficulties differentiating between standing and sitting (17,21). Whether or not wrist-based cut point approaches provide equally accurate estimates of SB relative to alternative approaches such as hip- or thigh-based accelerometry is unclear and requires further investigation. Furthermore, it is important to evaluate the accuracy of the wrist cut points to detect breaks in SB to understand their influence on health outcomes.

To our knowledge, no comprehensive validation studies have been conducted in children in which sedentary wrist cut points for the ActiGraph or GENEActiv have been evaluated simultaneously during a standardized activity protocol, against a criterion measure and alternative objective measures of SB. Therefore, the aims of this study were to examine the classification accuracy and the validity of sedentary wrist cut points for ActiGraph and GENEActiv, relative to the hip-mounted ActiGraph (Evenson: ≤25 counts per 15 s) and the thigh-mounted activPAL3TM, using direct observation as the criterion measure in 5- to 12-yr-olds. On the basis of evidence that the thigh-mounted activPAL3TM demonstrated acceptable accuracy for classifying SB in school-age children (34) and that traditional hip-based accelerometers tend to overestimate time spent in SB (17), and the assumption that wrist cut points might have similar difficulties as hip cut points in discriminating between standing and sitting, it was hypothesized that the most accurate wrist cut points would demonstrate similar accuracy as the hip cut point for assessing SB but lower accuracy than the thigh-mounted activPAL3TM.

METHODS

Participants

Fifty-seven children 5- to 12-yr-old who were without physical or health conditions that would affect participation in physical activity were recruited as part of an activity monitor validation study. The study was approved by the University of Wollongong Health and Medical Human Research Ethics Committee. Written parental consent and participant assent were obtained before participation.

Procedures

Participants were required to visit the laboratory on two occasions. Anthropometric measures were completed during the first visit using standardized procedures while children were wearing light clothing and with shoes removed. Body mass index (kg·m−2) and weight status were calculated (20). Children completed a protocol of 15 semistructured activities from sedentary (lying down, TV viewing, handheld e-game, writing/coloring, and computer game), light (getting ready for school, standing class activity, slow walk, and dancing), and moderate to vigorous (tidy up, brisk walk, soccer, basketball, running, and locomotor course) intensity (see Table, Supplemental Digital Content 1, http://links.lww.com/MSS/A804, Activity protocol). Activities were equally divided for two visits and completed in a structured order of increasing intensity for 5 min, except for lying down (10 min).

At each visit, children were fitted with an ActiGraph GT3X+ on the right hip (midaxilla line at the level of the iliac crest) with an elastic belt and an ActiGraph GT3X+ and a GENEActiv dorsally on each wrist. The distal and the proximal position of the ActiGraph and GENEActiv monitors on each wrist was alternated for each participant to avoid placement effects. An activPAL3TM was placed midanteriorly on the right thigh.

Activity monitors

The ActiGraph GT3X+ is a triaxial accelerometer that measures accelerations ranging in magnitude ±6g. Raw accelerometry data can be stored at a user-specified sample frequency ranging from 30 to 100 Hz. The GENEActiv has a waterproof design and measures triaxial accelerations ranging in magnitude ±8g at a sample frequency ranging from 10 to 100 Hz. The ActiGraph and the GENEActiv were initialized with a sample frequency of 100 Hz. Data reduction approaches were performed according to the methods used to develop each cut point (Table 1), as reported in original calibration studies (5,9,12,19,26,29). Raw ActiGraph data were downloaded using ActiLife version 6.12.1. ActiGraph hip and wrist data were converted to counts per 5 s (5,9), 15 s (12), or 60 s (19) corresponding to the epoch lengths used in their development. Output variables for ActiGraph monitors were VA, which is sensitive to movement only along the longitudinal axis of the lower arm or the dominant plane of the body (hip), and vector magnitude (VM), a three-dimensional measure of the acceleration, which is not sensitive to orientation and direction of movement. Raw GENEActiv wrist data were downloaded and converted into 1-s epochs using the GENEActiv software version 2.2 according to methods described by Phillips et al. (26) to create gravity-subtracted SVM data. Customized software was used to filter the raw GENEActiv data (band-pass filter, cutoff frequencies = 0.2 and 15 Hz) to remove the gravitational acceleration component as well as high-frequency sensor noise, as described by Schaefer et al. (29). An average gravity-subtracted SVM was then calculated for each second using a formula described by the authors.

TABLE 1
TABLE 1:
Sedentary wrist cut points.

The activPAL3TM is an activity monitor worn on the thigh that uses triaxial acceleration data (20 Hz) to assess the position and movement of the limb. The activPAL3TM software version 7.2.32 with proprietary algorithms was used to classify triaxial accelerometry data into periods spent sitting/lying, standing, or stepping. Event records created by the software were used to create 1-s epoch data files, which were used in the analyses to classify periods spent sedentary. The activPAL3TM was initialized with minimum sitting or upright period of 1 s.

Direct observation

Direct observation was used as criterion measure to establish the classification accuracy and validity of the cut points. Children were recorded on video completing the activities as well as during transitions between activities. A single observer coded all videos using Vitessa 0.1 (University of Leuven, Belgium), which generated a time stamp every time a change in posture or intensity was coded by the observer. Subsequently, a second-by-second classification system was generated. Every second after the time stamp inserted by the observer was classified as being the same posture as the one occurring at the time stamp itself until the next time stamp was created, indicating that a change in the child's posture had occurred. In the event of two postures occurring within the same second, this second was duplicated to label both postures. Labels for postures were sitting/lying (gluteus muscles resting on ground, feet, legs, or any other surface, or lying in prone position), standing (e.g., both feet touching the ground, squatting, standing on one foot, and kneeling on one or two knees), stepping (e.g., moving one leg in front of the other, including stepping with a flight phase, jumping, stepping, and sliding/side gallop), and “off screen” for direct observation using 1-s epochs. A dichotomous coding system was applied to recode postures into sedentary (sitting/lying: “1”) and nonsedentary (standing, stepping: “0”). Videos of five randomly selected participants were analyzed twice by the same observer and by a second observer to test inter- and intraobserver reliability. Inter- and intraobserver reliability were examined using Cohen's kappa and single measure intraclass correlation coefficients (ICC) from two-way mixed effect models (fixed-effects = observer, random effects = participants), using the consistency definition. Cohen's kappa coefficient for interobserver reliability was 0.941. Interobserver ICC was 0.974 (0.974–0.974) and intraobserver ICC was 0.963 (0.962–0.963).

Data synchronization

Monitors and direct observation were time synchronized using an internal computer clock. Second-by-second direct observation data were synchronized with 1-s epoch data from activPAL3TM and GENEActiv. Direct observation and activPAL3TM data files contained events of duplicated seconds when two postures were assigned to the same second. If this was the case for direct observation data, these seconds were duplicated at the corresponding time point for activPAL3TM and GENEActiv output. If this was the case for activPAL3TM data, the seconds were duplicated for direct observation and GENEActiv output. The second-by-second duplicates were not generated for ActiGraph output because these data were exported in 5-s, 15-s, and 60-s epochs. This method was applied for evaluation of classification accuracy and was in line with previous validation studies in preschool children (10,18). To align direct observation with ActiGraph epochs, new time frames were created for direct observation with steps of 5, 15, and 60 s. If >50% of the seconds within an epoch were classified as sedentary, the epoch was coded as sedentary (“1”), and if ≤50% of the epoch was classified as sedentary, the epoch was coded as nonsedentary (“0”). The synchronized direct observation and accelerometry data were excluded when direct observation epochs were coded as “off screen.” For estimates of time spent in different postures, codes of duplicated seconds for either direct observation (0.02% of total direct observation data) or accelerometer (0.04% of total activPAL3TM data) were assigned 0.5 s to avoid artificially inflating the total time observed. The absolute number of SB breaks for each method was defined as the number of transitions from SB to non-SB.

Statistical analyses

Before analyses, the total sample was divided into two age groups (5–8 yr, n = 25 and 9–12 yr, n = 32) because of the potential that younger and older children might engage in SB differently (17). Analyses included equivalence testing, Bland–Altman procedures, and calculating sensitivity, specificity, and area under the receiver operating curve (ROC-AUC) to evaluate and compare the accuracy and validity of different SB cut points for wrist-mounted ActiGraph and GENEActiv accelerometers, hip-worn ActiGraph accelerometer, and activPAL3TM. The equivalence of estimated sedentary time from different activity monitors, sites, and cut points and direct observation was examined at the group level of measurement using the 95% equivalence test. To reject the null hypothesis of the equivalence test, the 90% confidence interval (CI) of time spent sedentary predicted by the monitors should fall entirely within the predefined equivalence region of ±10% (2). The 90% CI values of the estimated sedentary time were bootstrapped because the sample sizes of the age groups were relatively small, and therefore, not all data were normally distributed. Agreement and systematic bias for estimated sedentary time were evaluated at the individual level using Bland–Altman procedures (17). For the ROC analyses, classification accuracy was rated as excellent (ROC-AUC ≥ 0.90), good (ROC-AUC = 0.80–0.89), fair (ROC-AUC = 0.70–0.79), or poor (ROC-AUC < 0.70) (23). The difference between the absolute number of SB breaks estimated by the monitors and direct observation was tested using paired sample t-tests.

RESULTS

Descriptive characteristics of participants are presented in Table 2. All participants completed the protocol and had valid activPAL3TM and ActiGraph wrist and hip data. For one of the visits, video data were unavailable for three children (age 5, 9, and 10 yr), and GENEActiv wrist data were unavailable for three different children (all 9–12 yr). Of the remaining 250,854 one-second epochs from 5 to 8 yr and 296,134 epochs from 9 to 12 yr, 27,983 and 23,513 epochs of direct observation were coded as “off screen” and excluded from analyses, respectively, leaving 222,872 (88.8%) valid epochs for 5–8 yr and 272,622 (92.1%) valid epochs for 9–12 yr. Mean direct observation time for 5–8 yr was 167.2 ± 21.9 min, of which 78.0 ± 11.8 min was coded as SB. Mean direct observation time for 9–12 yr was 154.2 ± 35.6 min, of which 69.5 ± 18.4 min was coded as SB. Results are presented for the nondominant wrist (unless stated otherwise) because placement on this wrist was recommended by the physical activity monitor protocol (4) released by the National Health and Nutrition Examination Survey, and previous studies have used the nondominant wrist for the development of wrist cut points (5,16,29). Results for the dominant wrist are presented in Supplemental Digital Content.

TABLE 2
TABLE 2:
Participant characteristics.

Validation of ActiGraph wrist cut points

Figures 1 (5–8 yr) and 2 (9–12 yr) present the 95% equivalence tests for accelerometry-based estimated time spent in SB from wrist-worn ActiGraph and GENEActiv cut points, the hip-worn ActiGraph cut point and activPAL3TM, and the equivalence region of direct observation. At the group level, estimates of SB time from the ActiGraph VM wrist cut point of Kim et al. (KimVM) were equivalent to direct observation (P = 0.02) in 5–8 yr, and estimates from the VA cut point (KimVA) approached equivalence (P = 0.08). Mean bias for estimated SB time from KimVM was 4.1% (limits of agreement [LoA] = −20.1% to 28.4%) (Table 3), whereas KimVA underestimated SB time by 6.5% (LoA = −33.1% to 20.2%). In 9–12 yr, CrouterVA/ROC and KimVA were equivalent to direct observation (P < 0.01), and CrouterVM/ROC approached equivalence (P = 0.05). These cut points underestimated SB time by 1.7% (LoA = −25.9% to 22.5%), 2.5% (LoA = −27.9% to 22.9%), and 5.3% (LoA = −27.9% to 22.9%), respectively. Estimates of SB time from other ActiGraph wrist cut points were not equivalent to direct observation in either age group. The mean bias varied from 7.2% (CrouterVA/ROC) to 20.5% (ChandlerVA/2016) in 5–8 yr and from 10.9% (CrouterVA/REG) to 29.6% (ChandlerVA/2016) in 9–12 yr. Good classification accuracy (Table 4) was found for KimVA (both age groups: ROC-AUC = 0.86) and KimVM (5–8 yr: ROC-AUC = 0.85; 9–12 yr: ROC-AUC = 0.82). Classification accuracy for other ActiGraph wrist cut points was fair (5–8 yr: ROC-AUC = 0.77–0.79; 9–12 yr: ROC-AUC = 0.72–0.75). At the individual level (Table 3), LoA values for all cut points, including the most accurate ActiGraph wrist cut points, were relatively wide (ChandlerVA/2016 in 5–8 yr: 0.0%–41.0%; ChandlerVA/2016 in 9–12 yr: −6.6% to 65.9%), which indicated large random error. No systematic bias (Table 3) was found for any of the ActiGraph wrist cut points (P > 0.05). Findings of the equivalence test, classification accuracy, and Bland–Altman analyses for ActiGraph wrist cut points for the dominant wrist (see Tables, Supplemental Digital Contents 2 and 3, http://links.lww.com/MSS/A805 and http://links.lww.com/MSS/A806, Agreement analysis and classification accuracy of accelerometry-based estimations of SB for the dominant wrist; see Figure, Supplemental Digital Content 4, http://links.lww.com/MSS/A807, Equivalence testing of accelerometry-based estimations of sedentary behavior for the dominant wrist) were consistent with findings for the nondominant wrist. Compared with direct observation, the absolute number of breaks was overestimated by all ActiGraph cut points in both age groups for both wrists (5–8 yr: mean difference range = 2.4–160.8, all P < 0.05; 9–12 yr: mean difference range = 1.8–138.6, all P < 0.05), except from KimVM for the nondominant wrist (5–8 yr: mean difference = 1.4 ± 5.7, P = 0.24; 9–12 yr: mean difference = 1.8, P = 0.05) (see Table, Supplemental Digital Content 5, http://links.lww.com/MSS/A808, Accelerometry-based estimations of breaks in sedentary behavior). Mean differences with direct observation were larger for wrist cut points developed with 5-s epochs (5–8 yr: 154.4 ± 4.1; 9–12 yr: 129.9 ± 5.2) compared with cut points developed with 60-s epochs (5–8 yr: 2.9 ± 1.2; 9–12 yr: 2.5 ± 0.8).

FIGURE 1
FIGURE 1:
The 95% equivalence test for accelerometry-based estimated time spent in SB in 5- to 8-yr-olds. Times estimated by wrist-worn ActiGraph and wrist-worn GENEActiv cut points are equivalent to direct observation if 90% CI lie entirely within the equivalence region of direct observation. VA, vertical axis; regression, developed using regression analysis.
FIGURE 2
FIGURE 2:
The 95% equivalence test for accelerometry-based estimated time spent in SB in 9- to 12-yr-olds. Times estimated by wrist-worn ActiGraph and wrist-worn GENEActiv cut points are equivalent to direct observation if 90% CI lie entirely within the equivalence region of direct observation. VA, vertical axis; ROC, developed using receiver operating curve analysis; regression, developed using regression analysis.
TABLE 3
TABLE 3:
Agreement analysis of accelerometry-based estimations of SB compared with direct observation.
TABLE 4
TABLE 4:
Classification accuracy of accelerometry-based estimations of SB.

Validation of GENEActiv wrist cut points

Estimates of SB time from GENEActiv wrist cut points PhillipsSVM and SchaeferSVM for the nondominant wrist were not equivalent to direct observation (Figs. 1 and 2). PhillipsSVM and SchaeferSVM overestimated SB time in 5–8 yr by 16.8% (LoA = −3.9% to 29.6%) and 9.6% (LoA = −13.8% to 33.0%), respectively, and in 9–12 yr by 17.8% (LoA = −11.6% to 47.3%) and 12.6% (LoA = −12.3% to 37.6%), respectively (Table 3). Although estimates from the GENEActiv wrist cut points for the dominant wrist were also not equivalent to direct observation in both age groups, the cut points performed slightly better for this wrist when estimating SB time at the group level (see Figure, Supplemental Digital Content 4, http://links.lww.com/MSS/A807, Equivalence testing of accelerometry-based estimations of sedentary behavior for the dominant wrist). For the dominant wrist, PhillipsSVM and SchaeferSVM overestimated SB time in 5–8 yr by 8.1% (LoA = −24.0% to 40.1%) and 6.5% (LoA = −16.1% to 29.1%), respectively, and in 9–12 yr by 8.2% (LoA = −18.6% to 35.0%) and 10.5% (LoA = −13.6% to 34.6%), respectively (see Table, Supplemental Digital Content 2, http://links.lww.com/MSS/A805, Agreement analysis of accelerometry-based estimations of sedentary behavior for the dominant wrist). Classification accuracy for all GENEActiv wrist cut points were fair to good in both age groups and for both wrists (ROC-AUC = 0.79–0.80). At the individual level, the LoA was smallest for PhillipsSVM (−3.9% to 29.6%), although all other LoA values for GENEActiv cut points were relatively wide, which indicated large random error (Table 3 and Table, Supplemental Digital Content 2, http://links.lww.com/MSS/A805, Agreement analysis of accelerometry-based estimations of sedentary behavior for the dominant wrist). No systematic bias was found for any of the GENEActiv wrist cut points (P > 0.05). All GENEActiv wrist cut points overestimated the absolute number of breaks compared with direct observation in both age groups (5–8 yr: mean difference range = 354.8–468.8, all P < 0.01; 9–12 yr: mean difference range = 313.2–398.1, all P < 0.01) (see Table, Supplemental Digital Content 5, http://links.lww.com/MSS/A808, Accelerometry-based estimations of breaks in sedentary behavior). Mean differences with direct observation were larger for the GENEActiv wrist cut points developed with 1-s epochs compared with the ActiGraph cut points developed with both 5-s epochs and 60-s epochs.

Comparison of validity of wrist cut points against ActiGraph hip cut point and activPAL3TM

In 5–8 yr, estimates of SB time by activPAL3TM (12.6%, LoA = −39.8% to 14.7%) and the hip-worn ActiGraph (15.8%, LoA = −5.7% to 37.2%) were not equivalent to direct observation, and the most accurate ActiGraph wrist cut points (KimVA and KimVM), GENEActiv wrist cut points for the dominant wrist, and SchaeferSVM for the nondominant wrist had smaller mean biases. Despite these differences, LoA values for the ActiGraph and GENEActiv wrist cut points were similarly wide to activPAL3TM and the hip-worn ActiGraph. In contrast to the group-level findings, classification accuracy for the Kim cut points was significantly lower than activPAL3TM (ROC-AUC = 0.92, 95% CI = 0.92–0.93) but similar to the hip-worn ActiGraph (ROC-AUC = 0.85, 95% CI = 0.84–0.85) in 5–8 yr. The classification accuracy of both GENEActiv wrist cut points for the nondominant and dominant wrist was significantly lower than activPAL3TM and the hip-worn ActiGraph.

In 9–12 yr, estimates of SB time by activPAL3TM were equivalent to DO (−1.4%, LoA = −13.95% to 11.0%) (P < 0.01), which was also the case for the most accurate ActiGraph wrist cut points (CrouterVA/ROC and KimVA). However, mean biases were larger and estimates of SB time were not equivalent to direct observation for the hip-worn ActiGraph (17.8%, LoA = −3.9% to 39.5%), and GENEActiv cut points for either wrist in 9–12 yr. LoA values for the ActiGraph and GENEActiv wrist cut points were wider than activPAL3TM, but similar to ActiGraph on the hip in 9–12 yr. The most accurate ActiGraph wrist cut point (KimVA) exhibited lower classification accuracy than activPAL3TM (ROC-AUC = 0.97, 95% CI = 0.97–0.97) but was similar to the hip-worn ActiGraph (ROC-AUC = 0.85, 95% CI = 0.84–0.85) in 9–12 yr. The classification accuracy of the GENEActiv cut points for both wrists was lower than activPAL3TM and the hip-worn ActiGraph, in 9–12 yr.

Mean differences with direct observation for SB breaks were larger for most ActiGraph and both GENEActiv wrist cut points compared with the activPAL3TM (5–8 yr: 8.5 ± 6.0, P < 0.01; 9–12: 3.2 ± 3.1, P < 0.01) and the hip-worn ActiGraph (5–8 yr: 33.2 ± 13.7, P < 0.01; 9–12: 29.3 ± 10.9, P < 0.01) in both age groups, except for the KimVM cut points where the differences were smaller.

DISCUSSION

This study examined the accuracy and validity of ActiGraph and GENEActiv wrist cut points for classifying SB in 5- to 12-yr-old children. The ActiGraph wrist cut points KimVM and KimVA accurately estimated SB time in 5–8 yr and 9–12 yr, respectively, at the group level, and exhibited good classification accuracy. These cut points provided more accurate estimates of SB time compared with the Evenson ActiGraph hip cut point (≤25 counts per 15 s). Although GENEActiv wrist cut points seemed to provide more accurate group-level estimates of SB time than the ActiGraph hip cut point for 5–8 yr and 9–12 yr, these cut points overestimated SB time, and classification accuracy was significantly lower than for the ActiGraph hip cut point and activPAL3TM in both age groups. Excluding an overestimation of SB time in 5–8 yr, activPAL3TM exhibited greater accuracy than the ActiGraph and GENEActiv wrist cut points and the ActiGraph hip cut point. Overall, the most accurate ActiGraph and GENEActiv wrist cut points estimated SB with similar accuracy as the ActiGraph hip cut point, although the accuracy of the thigh-mounted activPAL3TM was generally higher. The KIMVM cut point estimated the absolute number of breaks in SB more accurately than the ActiGraph hip cut point and activPAL3TM in both age groups, whereas the other ActiGraph and GENEActiv wrist cut points showed larger overestimations. To our knowledge, no previous studies have simultaneously evaluated the relative validity of multiple ActiGraph or GENEActiv wrist cut points developed in different studies among children. Crouter et al. (9) cross-validated their ActiGraph wrist cut points using indirect calorimetry in an independent sample of 11- to 14-yr-olds who completed 2 h of unstructured physical activity. The authors reported that the errors for estimated SB time were small (−8.6% to 2.5%) and not significantly different from the criterion measure. However, traditional analyses that fail to reject the null hypothesis of similarity do not necessarily demonstrate that the cut points meet an acceptable level of accuracy (2). Therefore, testing the equivalence could be beneficial when examining the clinical significance of potential errors. In our study, mean bias values for estimated SB time from the cut points of Crouter et al. (9) were slightly larger, ranging from −7.2% to 11.5% in 5–8 yr and from −1.7% to 16.8% in 9–12 yr. Equivalence testing indicated that only CrouterVA/ROC in 9–12 yr was equivalent to direct observation, although the classification accuracy for the cut points of Crouter et al. (9) across both age groups was only fair (ROC-AUC = 0.73–0.79). This suggests that, although errors may seem small, they might still be meaningful and misclassification of SB and non-SB may cancel each other out. Other methodological differences between our study and that of Crouter et al. (9), such as the younger age range of participants in our study, could have contributed to the differences in findings because younger and older children potentially engage in and move between sedentary and non-SB differently (17). Furthermore, the use of different criterion measures might have also contributed to the differences in measurement errors(17).

Kim et al. (19) used a protocol of 12 randomly selected semistructured activities to develop ActiGraph wrist cut points (KimVA and KimVM) in a subsample of 7- to 13-yr-olds (n = 49) and also provided results for the Evenson ActiGraph hip cut point (≤25 counts per 15 s, n = 125) against which wrist cut points could be compared. Although ROC-AUC values were not reported for the hip-worn ActiGraph, sensitivity (Se: true positive rate) for the wrist cut points (Se: 93.0%–94.3%) was similar to the hip cut point (Se = 93.7%), whereas specificity (Sp: true negative rate) for the wrist cut points (Sp: 79.9%–83.5%) was lower than the hip cut point (Sp = 92.5%) for classifying SB, suggesting that the hip-worn ActiGraph was slightly more accurate for classifying non-SB activities. However, the current study found that the classification accuracy for the ActiGraph wrist cut points of Kim et al. and the ActiGraph hip cut point was similar in both age groups. Cut point approaches for hip-mounted monitors cannot reliably distinguish between standing still and SB because SB is classified based on lack of movement, resulting in non-SB activities with minimal lower body movement being misclassified as SB. Because our study included transitions between activities, which likely involved standing with minimal movement, as well as a standing “classroom activity,” the likelihood of misclassifying non-SB as SB by the hip-worn ActiGraph was higher than that in the protocol of Kim et al. (19). By contrast, Kim et al. (19) indicated that most instances of misclassification of non-SB by the hip monitor occurred during a hand weight exercise involving minimal trunk and lower body movement. As such, our findings suggest that wrist cut points may have similar limitations to hip cut points in misclassifying standing still as SB.

In relation to wrist GENEActiv SB cut points, Rowlands et al. (28) compared PhillipsSVM for the nondominant wrist with the ActiGraph hip cut point (Evenson: ≤25 counts per 15 s) in a sample of free-living 10- to 12-yr-olds and reported that estimates of habitual SB time were 9.6% lower for the GENEActiv wrist cut point compared with the ActiGraph hip cut point, however, we found that the estimates of these cut points were similar. The difference in study designs may have contributed to these contrasting findings. However, our results showed larger misclassification of SB by PhillipsSVM compared with the hip-worn ActiGraph, and therefore precision for classifying SB and estimates at the individual level might be lower than group-level estimates.

Although some cut points in the current study seem to provide reasonably accurate estimates of SB time, the ROC-AUC values indicate that classification accuracy was only categorized as fair or good. For example, group-level estimates of SB time from KimVM and KimVA were equivalent or almost equivalent to direct observation and mean biases were smaller than that observed for the hip-worn ActiGraph and activPAL3TM, however ROC-AUC values were lower than activPAL3TM and similar to the ActiGraph hip cut point. In 9–12 yr, the cut points CrouterVA/ROC and KimVA were equivalent to DO and estimates of SB time were more accurate than the hip-worn ActiGraph and similar to activPAL3TM. However, although classification accuracy for KimVA was good, classification accuracy for CrouterVA/ROC was only fair and lower than both activPAL3TM and the hip-worn ActiGraph. A possible explanation is that SB as estimated by wrist cut points was misclassified as non-SB in some activities. For instance, the highest percentage of misclassified SB epochs (AG = 0.4%–7.3%, GA = 1.4%–5.7%) was found during the coloring activity in 5–8 yr, which requires the child to use the hand, and so wrist monitors might record counts high enough to be misclassified as non-SB. By contrast, standing still while writing on a white board resulted in the highest percentage of misclassified epochs during non-SB activities for the nondominant hand (5–8 yr: AG = 6.7%–9.7%, GA = 8.1%–8.6%; 9–12 yr: AG = 6.1%–9.0%, GA = 7.7%–8.3%), because the wrist monitors recorded low activity counts on this hand and misclassified epochs during the task as SB. Misclassification of SB and non-SB for wrist cut points may cancel each other out, resulting in seemingly accurate group-level estimates of SB time. Hip-placed monitors on the other hand seem to overestimate SB time at the group level because of the misclassification of standing still as SB. The results of this study suggest that although hip-based cut points typically misclassify standing still as SB, wrist cut points exhibit some misclassification of non-SB as SB and vice versa. Progress on alternative approaches, such as those using machine learning (15,27,33), is therefore required, but until such strategies are widely available, the use of the most accurate ActiGraph and GENEActiv wrist cut points for estimating SB is recommended.

ActiGraph wrist cut points developed with 60-s epochs seemed to perform better for estimating SB time at the group level and the absolute number of SB breaks and exhibited higher classification accuracy and compared with cut points developed with 5 s or 1-s epochs. This could be explained by a higher number of data points when using shorter epochs, resulting in a higher chance of misclassification. The lower classification accuracy with shorter epochs might have contributed to the lower performance of the GENEActiv wrist cut points as they were developed with 1 s data. This is in contrast to the common use of short epochs for accurately capturing sporadic and intermittent bursts of high-intensity physical activity in children (3). Previous studies have evaluated the effect of epoch length in free-living school-age children using ActiGraph hip data and showed that time spent in SB decreases when longer epochs are applied (1,25). A possible explanation is that very short periods (e.g., 1–5 s) of standing relatively still might be fairly common in children, resulting in non-SB being misclassified as SB using short epochs. By contrast, when using 60-s epochs, standing still would need to occur for almost all of a 60-s period for this to be misclassified as SB, and it is possible that this is less common than short periods of standing still among children. Although most ActiGraph wrist cut points designed for 5-s epochs overestimated SB in our analyses, CrouterVA/ROC and CrouterVM/ROC underestimated SB in 5–8 yr and exhibited similar accuracy as those for 60-s epochs in 9–12 yr, and so the combination of epoch and cut point is likely to be important. Nevertheless, our findings indicate that the most accurate SB wrist cut points were designed for 60-s epochs, which has implications for field-based applications. In studies of free-living children, estimates of both SB and physical activity are often desirable. If data are reduced using short epochs such as 5 s to estimate physical activity, the most accurate SB cut points for 5-s epochs could be applied, such as the CrouterVA/ROC or the CrouterVM/ROC of Crouter et al. (9) for ActiGraph and PhillipsSVM (26) or that of Schaefer et al. (29) for GENEActiv. Although these cut points exhibited lower classification accuracy than the most accurate 60 s wrist cut points and the ActiGraph hip cut point, group-level estimates of SB time were more accurate than the ActiGraph hip cut point.

A unique strength of the study was that several currently available wrist cut points for ActiGraph and GENEActiv were evaluated simultaneously, against a criterion measure and common alternative objective measures of SB. Another strength was that data from the entire activity protocol in our study were analyzed including transitions between activities, with the aim to also include data of behaviors outside of structured activities. In addition, the wide age range of the sample allowed for analyses across two age groups. However, because the study protocol predominantly included structured activities completed in a laboratory setting, the findings should be confirmed under free-living conditions.

In summary, the use of the most accurate ActiGraph and GENEActiv wrist-based activity monitor cut points for estimating SB can be applied in free-living children with similar confidence as the hip-based ActiGraph cut point (≤25 counts per 15 s), although alternative approaches may be needed to achieve the generally higher accuracy of thigh-based approaches such as activPAL3TM.

The authors thank all children and their parents for their participation. They also thank Melinda Smith for her assistance with recruitment and data collection and Woranart Maneenin for video analyses. This study was funded by the National Heart Foundation of Australia (G11S5975). D. P. C. was supported by an Australian Research Council Discovery Early Career Researcher Award (DE140101588). A. D. O. was supported by a National Heart Foundation of Australia Career Development Fellowship (CR11S 6099). T. H. was funded by a National Health and Medical Research Council Early Career Fellowship (APP1070571). The work of U. E. and S. B. was funded by the UK Medical Research Council (MC_UU_12015/3). S. T. was supported by the National Health and Medical Research Council Centre of Research Excellence on Sitting Time and Chronic Disease Prevention (APP1057608).

The authors have no conflict of interest to declare. The results of the present study do not constitute endorsement by the American College of Sports Medicine. The results are presented clearly, honestly, and without fabrication, falsification, or inappropriate data manipulation.

REFERENCES

1. Aibar A, Chanal J. Physical education: the effect of epoch lengths on children's physical activity in a structured context. PloS One. 2015;10(4):e0121238 doi: 10.1371/journal.pone.0121238.
2. Batterham MJ, Van Loo C, Charlton KE, Cliff DP, Okely AD. Improved interpretation of studies comparing methods of dietary assessment: Combining equivalence testing with the limits of agreement. Br J Nutr. 2016;115(7):1273–80.
3. Cain KL, Sallis JF, Conway TL, Van Dyck D, Calhoon L. Using accelerometers in youth physical activity studies: a review of methods. J Phys Act Health. 2013;10(3):437–50.
4. Center for Disease Control and Prevention [Internet]. Physical activity Monitor (PAM) Procedures Manual; [cited 2016 May 19]. Available from: http://www.cdc.gov/nchs/data/nhanes/nhanes_11_12/Physical_Activity_Monitor_Manual.pdf.
5. Chandler JL, Brazendale K, Beets MW, Mealing BA. Classification of physical activity intensities using a wrist-worn accelerometer in 8–12-year-old children. Pediatr Obes. 2016;11(2):120–7.
6. Cliff DP, Hesketh KD, Vella SA, et al. Objectively measured sedentary behaviour and health and development in children and adolescents: systematic review and meta-analysis. Obes Rev. 2016;17(4):330–44.
7. Cliff DP, Jones RA, Burrows TL, et al. Volumes and bouts of sedentary behavior and physical activity: associations with cardiometabolic health in obese children. Obesity (Silver Spring). 2014;22(5):E112–8.
8. Cliff DP, Okely AD, Burrows TL, et al. Objectively measured sedentary behavior, physical activity, and plasma lipids in overweight and obese children. Obesity (Silver Spring). 2013;21(2):382–5.
9. Crouter SE, Flynn JI, Bassett DR Jr. Estimating physical activity in youth using a wrist accelerometer. Med Sci Sports Exerc. 2015;47(5):944–51.
10. Davies G, Reilly JJ, McGowan AJ, Dall PM, Granat MH, Paton JY. Validity, practical utility, and reliability of the activPAL™ in preschool children. Med Sci Sports Exerc. 2012;44(4):761–8.
11. Ekelund U, Luan J, Sherar LB, et al. Moderate to vigorous physical activity and sedentary time and cardiometabolic risk factors in children and adolescents. JAMA. 2012;307(7):704–12.
12. Evenson KR, Catellier DJ, Gill K, Ondrak KS, McMurray RG. Calibration of two objective measures of physical activity for children. J Sports Sci. 2008;26(14):1557–65.
13. Fairclough SJ, Noonan R, Rowlands AV, van Hees V, Knowles Z, Boddy LM. Wear compliance and activity in children wearing wrist- and hip-mounted accelerometers. Med Sci Sports Exerc. 2016;48(2):245–53.
14. Freedson PS, John D. Comment on “estimating activity and sedentary behavior from an accelerometer on the hip and wrist.” Med Sci Sports Exerc. 2013;45(5):962–3.
15. Hagenbuchner M, Cliff DP, Trost SG, Van Tuc N, Peoples GE. Prediction of activity type in preschool children using machine learning techniques. J Sci Med Sport. 2015;18(4):426–31.
16. Hildebrand M, Van Hees VT, Hansen BH, Ekelund U. Age group comparability of raw accelerometer output from wrist- and hip-worn monitors. Med Sci Sports Exerc. 2014;46(9):1816–24.
17. Janssen X, Cliff DP. Issues related to measuring and interpreting objectively measured sedentary behavior data. Meas Phys Educ Exerc Sci. 2015;19(3):116–24.
18. Janssen X, Cliff DP, Reilly JJ, et al. Validation of activPAL defined sedentary time and breaks in sedentary time in 4- to 6-year-olds. Pediatr Exerc Sci. 2014;26(1):110–7.
19. Kim Y, Lee J-M, Peters BP, Gaesser GA, Welk GJ. Examination of different accelerometer cut-points for assessing sedentary behaviors in children. PloS One. 2014;9(4):e90630.
20. Kuczmarski RJ, Ogden CL, Guo SS, et al. 2000 CDC Growth charts for the United States: methods and development. Vital Health Stat 11. 2002;246:1–190.
21. Lubans DR, Hesketh K, Cliff D, et al. A systematic review of the validity and reliability of sedentary behaviour measures used with children and adolescents. Obes Rev. 2011;12(10):781–99.
22. Matthews CE, Chen KY, Freedson PS, et al. Amount of time spent in sedentary behaviors in the United States, 2003–2004. Am J Epidemiol. 2008;167(7):875–81.
23. Metz CE. Basic principles of ROC analysis. Semin Nucl Med. 1978;8(4):283–98.
24. Mitchell JA, Pate RR, Beets MW, Nader PR. Time spent in sedentary behavior and changes in childhood BMI: a longitudinal study from ages 9 to 15 years. Int J Obes (Lond). 2013;37(1):54–60.
25. Nettlefold L, Naylor P, Warburton DE, Bredin SS, Race D, McKay HA. The influence of epoch length on physical activity patterns varies by child's activity level. Res Q Exerc Sport. 2016;87(1):110–23.
26. Phillips LR, Parfitt G, Rowlands AV. Calibration of the GENEA accelerometer for assessment of physical activity intensity in children. J Sci Med Sport. 2013;16(2):124–8.
27. Rowlands AV, Olds TS, Hillsdon M, et al. Assessing sedentary behavior with the GENEActiv: introducing the sedentary sphere. Med Sci Sports Exerc. 2014;46(6):1235–47.
28. Rowlands AV, Rennie K, Kozarski R, et al. Children's physical activity assessed with wrist- and hip-worn accelerometers. Med Sci Sports Exerc. 2014;46(12):2308–16.
29. Schaefer CA, Nigg CR, Hill JO, Brink LA, Browning RC. Establishing and evaluating wrist cutpoints for the GENEActiv accelerometer in youth. Med Sci Sports Exerc. 2014;46(4):826–33.
30. Sedentary Behaviour Research Network. Letter to the editor: standardized use of the terms “sedentary” and “sedentary behaviours.” Appl Physiol Nutr Metab. 2012;37:540–2.
31. Troiano RP, McClain JJ, Brychta RJ, Chen KY. Evolution of accelerometer methods for physical activity research. Br J Sports Med. 2014;48(13):1019–23.
32. Trost SG. State of the art reviews: measurement of physical activity in children and adolescents. Am J Lifestyle Med. 2007;1(4):299–314.
33. Trost SG, Wong WK, Pfeiffer KA, Zheng Y. Artificial neural networks to predict activity type and energy expenditure in youth. Med Sci Sports Exerc. 2012;44(9):1801–9.
34. Van Loo CM, Okely AD, Batterham MJ, et al. Validation of thigh-based accelerometer estimates of postural allocation in 5–12 year-olds. J Sci Med Sport. 2016 [Internet]. 2016 [cited 2016 Aug 18]. Available from: http://www.sciencedirect.com/science/article/pii/S1440244016301517. doi: 10.1016/j.jsams.2016. 08.008.
Keywords:

ACTIVITY MONITOR; YOUTH; VALIDATION; PHYSICAL ACTIVITY; OBJECTIVE MEASUREMENT; SITTING

Supplemental Digital Content

© 2017 American College of Sports Medicine