Journal Logo

Original Articles

Assessment of Physical Activity Using Wearable Monitors: Recommendations for Monitor Calibration and Use in the Field

FREEDSON, PATTY1; BOWLES, HEATHER R.2; TROIANO, RICHARD2; HASKELL, WILLIAM3

Author Information
Medicine & Science in Sports & Exercise: January 2012 - Volume 44 - Issue 1S - p S1-S4
doi: 10.1249/MSS.0b013e3182399b7e
  • Free

Abstract

In recent years, physical activity assessment has increasingly relied on wearable monitors to provide a measure of exposure or dose in surveillance, intervention, and epidemiological research. The primary aim of the workshop “Objective Measurement of Physical Activity: Best Practices and Future Direction,” held on July 20–21, 2009, was to update best practice recommendations for the use of wearable monitors to assess physical activity. To achieve this goal, we critically reviewed the current state of knowledge and practices, identified strengths and weaknesses in these practices, and recommended standardized approaches for evaluating accelerometer-based physical activity monitors in the laboratory and for use by researchers in the field.

We are at a critical juncture in physical activity research, and activity monitors have become a key element in this research. Advances in technology offer the physical activity measurement community many device options to capture motion in one or multiple planes of movement, assess physiological markers such as HR and ventilation, and wirelessly collect, process, and transmit data by way of cellular telephones from multiple locations on the body. Data processing techniques also have advanced significantly and are beginning to offer researchers and end users alternatives to the traditional “cutpoint” or “count” approach for data interpretation. For example, certain accelerometers have sufficient data storage capacity to allow researchers to collect raw signals from the monitors during multiple days. Pattern recognition techniques develop algorithms by training with the accelerometer data for known activity types and energy expenditure and apply these algorithms to individual monitor data to identify activity type and estimate energy expenditure with reasonable accuracy (10,14,18).

Despite the increased use of these monitors in physical activity research and the rapid advances in technology and data processing techniques, the field of objective monitoring of physical activity lacks standards of practice for monitor calibration and validation and field application. In 2004, a conference entitled “Objective Monitoring of Physical Activity: Closing the Gaps in the Science of Technology” was held in Chapel Hill, NC. The conference was motivated by recognition of the importance of objective physical activity monitoring to allow researchers to more precisely quantify dose of activity exposure. At that time, objective monitoring of activity behavior was in the early stages of development, and the best practices in monitor selection, monitor use, monitor calibration, data analysis, and integration of monitor data with other sources of data were being developed. The collection of articles dealing with many key issues and a best practices summary article were published in 2005 (19).

Extraordinary advances in monitor technology have occurred in the years since that publication. We also have witnessed the publication of numerous activity measurement methods articles and an abundance of articles exploiting these methods for assessing physical activity behavior that contribute to a better understanding of the relationship between physical activity and health. However, numerous challenges regarding how to collect, calibrate, process, and use data from wearable monitors remain. On the basis of discoveries in objective physical activity measurement research, we are now able to advance the best practices to a new stage. This field will continue to evolve, and as further progress is made, the process of revising and updating these best practices will continue.

Several questions need to be addressed before prescribing specific protocols for monitor calibration and validation studies by the physical activity measurement researcher or for objective assessment of physical activity dose by the end user. These questions include the following: what are the specific study objectives? What is (are) the physical activity component(s) of interest? What is the feasibility of objective monitoring of physical activity in terms of study sample, cost, and logistics? Answers to these questions will inform the researcher about how to select a measurement tool, calibrate or validate an activity monitor, and quantify physical activity exposure or assess changes in activity behavior consequent to an intervention.

This article will be divided into three main sections presenting recommendations for 1) the physical activity measurement researcher; 2) the end user who will be using the devices for surveillance, epidemiological, or intervention research; and 3) sensor system developers and designers. The recommendations are not the only options for how physical activity monitors should be developed, calibrated, and used. These suggested practices are a guide for how to advance the field of physical activity measurement toward a goal of establishing standards for physical activity measurement. All recommendations were based not only on the consensus of the workshop participants but also on the translation of the most current research evidence.

BEST PRACTICE RECOMMENDATIONS

For the physical activity measurement researcher.

The physical activity measurement researcher should consider several factors in the design of calibration studies to maximize generalizability and provide information needed for interstudy comparisons. These factors include monitor properties and monitor output, study design, subject population, and validation analysis (7). Monitor data should be collected and saved as raw signals with postprocessing used for data transformation (2–4,20). For determining monitor equivalency, this approach will allow data from different monitors to be directly compared and avoids the uncertainty of the meaning of preprocessed data (e.g., counts) (20). Whenever possible, researchers should use multiple monitors within their study design (and select monitors that do not have proprietary algorithms) so that intermonitor comparisons can be made. To determine validity, the monitor output should be compared with one or more criterion measures (2,3). Criterion measures for energy expenditure include doubly labeled water, indirect or direct calorimetry, and direct observation. Doubly labeled water should be used as the criterion measure if total energy expenditure averaged over multiple days is the variable to be predicted from the activity monitor (3). If time spent in different intensities of activity is needed, then direct observation or calorimetry should be the criterion measurement. In using direct observation, the Compendium of Physical Activities for adults (1) or children and youth (12) can provide the MET values of observed activities. A broad age range of subjects should be recruited for monitor validation studies, including children, adolescents, adults, and older adults (including both sexes, with a range of body mass index and cardiorespiratory fitness) (2,20). Appropriate sample size calculations are needed to ensure adequate power for validation and calibration features of interest (15). A broad array of activities including sedentary, lifestyle, sport, and leisure activities representing a MET range of at least 1.1 to 10 METs should comprise the activity menu for monitor calibration (2,20). For data analysis, the criterion measure should be compared with the monitor data, with computation of mean differences, bias and root mean square errors, correlations, and individual differences (15). Monitor predictions should be tested on independent samples or holdout samples from the original data set (e.g., one holdout validation, 10% holdout validation [15]). Consideration also should be given to log scale analyses because estimates of physical activity energy expenditure are more variable as energy expenditure increases (15). Finally, for data interpretation, researchers should discontinue development and use of cutpoint methods to define intensity categories (16). Alternative analytic techniques, such as pattern recognition methods, that use more features of the raw acceleration signal and reduce the likelihood of extreme over- or underprediction of energy expenditure are recommended (2,3,10,13,14).

For the end user

For the end user using activity monitors in surveillance, intervention, and epidemiological research, the following practices are recommended. The rationale for the selection of a particular monitor should be provided (e.g., cost, logistics, major outcome(s)) along with data supporting the monitor reliability and validity in the target population. A description of how the monitor was positioned on the participant is essential and should be based on how monitors were positioned in calibration and validation studies (16). Careful attention to reporting the monitoring period is needed with details on hours per day, number of days, and weekday and weekend monitoring (7). A clear definition of wear time and what constitutes a valid day is recommended, and outcome variables should be clearly described (7,11). In addition, the criteria used to include and exclude data should be specified (11).

The features of many activity monitors are continuously being updated and modified. Therefore, it is essential that information about the monitor model and the versions of firmware and digital filtering be reported (4,7). Reporting the cost of using sensors is recommended. Other monitor information, such as the number of participants who refused monitoring, the number of monitors that malfunctioned, and the type of malfunction and number of lost devices, should be reported (11).

A description of lessons learned and problems encountered in specific research projects will benefit the engineers who will develop the next generation of activity monitors and inform researchers planning to use activity monitors in future studies. Concerns about length limits for articles that make it difficult to include methodological details may now be addressed with online appendices available in many journals.

For the monitor developer and designer

Numerous commercially manufactured accelerometer-based activity monitors are currently available. These monitors and associated algorithms are continuously being updated and refined. As the field moves toward collecting the raw signal for activity assessment and more advanced data processing, new monitors must have a large data storage capacity or the ability to transmit large volumes of data to remote servers. Such devices are becoming affordable and available. In addition, data processing algorithms should use open source code so researchers have access to the underlying equations. Transducer specifications and filters used also need to be available to researchers so that they can understand how and why monitors differ in their output (4). Monitor designers and engineers should maintain open lines of communication with researchers to gain a better understanding of the researchers’ outcomes of interest and data collection needs (8). The primary objective of new monitor development should be to provide advantages over existing monitors in terms of accuracy, reliability, new measures or outcomes, validity, ease of use, and cost.

NEW HORIZONS IN SENSOR DEVELOPMENT

The article by Intille et al. (8) identifies several new developments in the design of activity monitors. They anticipate improvement in overall device performance with different types of sensors located at multiple sites on the body and in the environment that will be fused into a single system with a large data storage capacity. Such a system will provide significantly more information that increases our ability to identify activity type, define the state of ambulation, characterize contextual information, and estimate energy expenditure for extended periods (e.g., months or even years). Use of mobile phones for physical activity measurement and/or data transmission will evolve rapidly, with built-in Internet access for data collection and participant compliance monitoring. These types of technological advances in activity monitoring will provide researchers with enormous capacity for innovation in intervention studies. For example, technology can provide a new mechanism for motivation to be physically active (e.g., real-time social networking) or provide real-time feedback to users and researchers.

FUTURE DIRECTIONS

As methods to calibrate activity monitors and measure physical activity using wearable monitors continue to be developed and refined, several factors should guide future investigations. Methods to classify the relative intensity of physical activity considering individual fitness and functional status need to be developed and validated. Traditionally, physical activity outcomes have been expressed as absolute intensity measures (e.g., 5 METs, 30 min in 3- to 6-MET range) with no consideration for the functional capacity of the individual. Incorporating a measure of an individual’s level of fitness into the physical activity metric will provide valuable information about the importance of relative physical activity intensity and health.

Assessing and understanding systematic and random error effects in accelerometer-based estimates of physical activity energy expenditure will improve the precision of these estimates and enhance our understanding of dose–response relationships between physical activity and health. Continued development of statistical methods that use more features of the accelerometer signal, such as signal distributions and frequency spectra, will provide the user with information about activity mode to complement information about duration and intensity of activity. Validation of activity mode recognition from accelerometer signal features will require the use of direct observation (in real time or through video analysis) as the criterion method to compare with the pattern recognition modeling methods.

Organizing multidisciplinary teams that include engineers, exercise physiologists, physical activity epidemiologists, interventionists, mathematicians, and statisticians to develop tools, process data, and perform calibration/validation studies will move the field of objective physical activity measurement forward in a manner that will provide end users with state-of-the-art data collection devices and processing tools. As new tools and data processing techniques become available, developers should organize and offer tutorials and conference presentations and design “how-to” Web sites to educate end users about the use of these tools. Development of a public access accelerometer data repository is recommended to provide researchers access to large accelerometer databases for processing and analysis. Finally, physical activity measurement researchers and end users should review alternative literature sources, such as those that include engineering articles (e.g., IEEE, Springer, Elsevier, Google Scholar).

CHALLENGES

We have made significant progress in the field of objective physical activity assessment. However, challenges remain, and for the advancement of the field, the following issues should be addressed. Validation of activity monitors in field settings remains elusive. However, use of devices, such as cellular telephones, in which much data can be relatively easily captured, processed, and transmitted, may help to address this challenge soon. Costs of devices must continue to be reduced if use in large-scale epidemiologic cohort studies is to be sustained. A recommended target price for a device is $100 per unit. Additional monitor calibration studies on older adults and individuals with functional limitations are needed to provide population-specific algorithms for data translation in these populations (16).

Researchers are increasingly focusing on sedentary behavior as an independent predictor of morbidity (6,17) and mortality (9). In addition to documenting total sedentary time, preliminary data indicate that it will be important to accurately determine time spent lying, sitting, and standing and frequency of breaks from sedentary time. Typically, this behavior is assessed using self-reported measures of media time. Recent evidence suggests that this measurement tool underestimates actual sedentary time and that the degree of underestimation is extremely variable between subjects (5). The validation of objective tools to assess sedentary behavior is needed to improve our understanding of the dose of sedentary time associated with increased risk of disease.

The authors thank two individuals for their important contribution to the preparation of this supplement: Anne Rodgers and Natalie Gay. Anne reviewed all articles and coedited the supplement. Natalie provided excellent support in managing the supplement drafts and reviews. Without Anne and Natalie, this work would not have been possible.

The authors report no conflicts of interest.

REFERENCES

1. Ainsworth BE, Haskell WL, Whitt MC, et al.. Compendium of physical activities: an update of activity codes and MET intensities. Med Sci Sports Exerc. 2000; 32 (9 suppl): S498–516.
2. Bassett DR Jr, Rowlands AV, Trost SG. Calibration and validation of wearable monitors. Med Sci Sports Exerc. 2012; 44 (1 suppl): S32–8.
3. Butte NF, Ekelund U, Westerterp KR. Assessing physical activity using wearable monitors: measures of physical activity. Med Sci Sports Exerc. 2012; 44 (1 suppl): S5–12.
4. Chen KY, Janz KF, Zhu W, Brychta RJ. Redefining the roles of sensors in objective physical activity monitoring. Med Sci Sports Exerc. 2012; 44 (1 suppl): S13–23.
5. Clark BK, Sugiyama T, Healy GN, Salmon J, Dunstan DW, Owen N. Validity and reliability of measures of television viewing time and other non-occupational sedentary behaviour of adults: a review. Obes Rev. 2009; 10 (1): 7–16.
6. Dunstan DW, Salmon J, Owen N, et al.. Associations of TV viewing and physical activity with the metabolic syndrome in Australian adults. Diabetologia. 2005; 48 (11): 2254–61.
7. Heil DP, Brage S, Rothney MP. Modeling physical activity outcomes from wearable monitors. Med Sci Sports Exerc. 2012; 44 (1 suppl): S50–60.
8. Intille SS, Lester J, Sallis JF, Duncan G. New horizons in sensor development. Med Sci Sports Exerc. 2012; 44 (1 suppl): S24–31.
9. Katzmarzyk PT, Church TS, Craig CL, Bouchard C. Sitting time and mortality from all causes, cardiovascular disease, and cancer. Med Sci Sports Exerc. 2009; 41 (5): 998–1005.
10. Kiani K, Snijders CJ, Gelsema ES. Recognition of daily life motor activity classes using an artificial neural network. Arch Phys Med Rehabil. 1998; 79 (2): 147–54.
11. Matthews CE, Hagströmer M, Pober DM, Bowles HR. Best practices for using physical activity monitors in population-based research. Med Sci Sports Exerc. 2012; 44 (1 suppl): S68–76.
12. Ridley K, Ainsworth BE, Olds TS. Development of a compendium of energy expenditures for youth. Int J Behav Nutr Phys Act. 2008; 5: 45.
13. Rothney MP, Neumann M, Beziat A, Chen KY. An artificial neural network model of energy expenditure using nonintegrated acceleration signals. J Appl Physiol. 2007; 103 (4): 1419–27.
14. Staudenmayer J, Pober D, Crouter S, Bassett DR Jr, Freedson PS. An artificial neural network to estimate physical activity energy expenditure and identify physical activity type from an accelerometer. J Appl Physiol. 2009; 107 (4): 1300–7.
15. Staudenmayer JS, Zhu W, Catellier DJ. Statistical considerations in the analysis of accelerometry-based activity monitor data. Med Sci Sports Exerc. 2012; 44 (1 suppl): S61–7.
16. Strath SJ, Pfeiffer KA, Whitt-Glover MC. Accelerometer use in children, older adults, and adults with functional limitations. Med Sci Sports Exerc. 2012; 44 (1 suppl): S77–85.
17. Sugiyama T, Healy GN, Dunstan DW, Salmon J, Owen N. Joint associations of multiple leisure-time sedentary behaviours and physical activity with obesity in Australian adults. Int J Behav Nutr Phys Act. 2008; 5: 35.
18. Tapia EM, Intille SS, Haskell W, et al.. Real-time recognition of physical activities and their intensities using wireless accelerometers and a heart rate monitor. In: Proceedings of the International Symposium on Wearable Computers. Boston (MA): IEEE; 2007. p. 37–40.
19. Ward DS, Evenson KR, Vaughn A, Rodgers AB, Troiano RP. Accelerometer use in physical activity: best practices and research recommendations. Med Sci Sports Exerc. 2005; 37 (11 suppl): S582–8.
20. Welk GJ, McClain J, Ainsworth BE. Protocols for evaluating equivalency of accelerometer-based activity monitors. Med Sci Sports Exerc. 2012; 44 (1 suppl): S39–49.
Keywords:

PHYSICAL ACTIVITY MEASUREMENT; WEARABLE ACCELEROMETERS; RECOMMENDATIONS FOR USING MONITORS

©2012The American College of Sports Medicine