Introduction
Enhanced technology over the last decade has resulted in an increase in the number of wearable devices developed for the collection of human biometric data to enable more appropriately informed decisions in health care (15,17), sports (7), and military environments (21). These wearable devices use technologies such as global positioning systems, heart rate (HR) sensors, blood pressure monitors, accelerometers, and plethysmographs to gather biometric data to remotely monitor human activity or to log such data for subsequent investigation.
In sports, software applications have been designed to display the biometric data collected to optimize training and performance (1). For example, HR variability, which is a useful objective measure for quantifying training load (16), can be examined using HR monitors or other wearable devices or the information can be downloaded into smartphone software applications (6) to give an indication of the athlete's training and recovery status.
A plethora of consumer health wearable devices are now available, which are also being marketed for use by researchers. Thus, investigations into the validity and reliability of such devices are required in various contexts before such devices can be recommended for laboratory use. A commercially available wearable and reusable sleeveless shirt (LifeShirt; VivoMetric, Ventura, CA) incorporating a respiratory inductive plethysmography system, along with electrocardiogram (ECG), pulse oximetry, and accelerometer sensors, has been shown to have an excellent agreement (r = 0.97) between the garment and gold standard spirometry for inspiratory minute ventilation (VI) at a mixture of low, intermediate, submaximal, and maximal treadmill running exercise intensities (2). Even when comparing VI at maximal exercise alone, the mean difference between the LifeShirt and the flowmeter was only ∼6% (2). Comparing the same garment (LifeShirt; VivoMetric) against laboratory-based monitoring systems, other researchers have also shown excellent agreement in HR and respiratory rate collected at either rest or during exercise (9,13). Similarly, other wearable devices such as the BioHarness (Zephyr Technology, Auckland, New Zealand) have been shown to be valid and reliable at determining respiratory rate during incremental treadmill testing to volitional exhaustion (8).
A relatively new wearable and reusable vest, the Hexoskin (Carré Technologies Inc., Montreal, Canada), produced reliable measures of HR at rest (lying, sitting, and standing) and during a submaximal walking test in healthy young volunteers (intraclass correlation between vest and laboratory-based equipment >0.96) (20). However, reliability and validity of HR and other physiological parameters measured by this vest at rest or during maximal exercise in elite athletes have not been conducted. Moreover, the unique posture and movements used during cycling could impact the reliability and validity of the Hexoskin in comparison with the more upright positioning during standing, sitting, and lying. The aim of this study was to investigate the validity and reliability of the Hexoskin vest by measuring respiration and HR in elite cyclists during a progressive exercise test to exhaustion.
Methods
Experimental Approach to the Problem
This study used a repeated-measures experiment to determine the validity and reliability of a number of physiological variables produced by the Hexoskin vest. In this study design, each participant performed a progressive cycling exercise test to exhaustion, which was repeated after a 3–4-day recovery period. Participants followed a strict testing protocol that was replicated exactly for each test.
Subjects
Ten participants from the Christchurch region in New Zealand participated in this study (18–52 years old), which was conducted over 1 week in May 2016. Participant inclusion criteria were uninjured elite male cyclists aged 18 or older currently or just completing training for national or international caliber cycling competitions within the previous 6 months. Participants were recruited from cycle clubs and coaches in the local area and represented elite club and national representatives (Table 1). Ethical approval for this research has been obtained by the Lincoln University Human Ethics Committee (reference 2016-09). All participants were older than 18 years, thus informed voluntary written consent was attained from all participants before their inclusion in the research.
;)
Table 1.: Participant characteristics.*
Procedures
Subject Preparation
Participants reported to the laboratory on 2 separate occasions over a 3–4-day period. Participants were instructed to present themselves for testing in a rested and hydrated state, having avoided the consumption of alcohol in the preceding 24 hours. Participants were instructed not to change their diet over the study period and to avoid consuming a heavy meal and caffeine-containing beverages for 4 hours before reporting to the laboratory. Researchers ensured that the aforementioned instructions were adhered to.
Equipment
The Hexoskin wearable garment was constructed from high-performance fabric (73% polyamide microfibers and 27% elastane) with flexible sensors sewn directly into the fabric. The vests are available in different sizes for both men and women to ensure the proper fit (3,4) based on the manufacturer's suggested chest and waist circumferences. In cases where participants were between the recommended vest sizes, waist circumference was used to determine the vest size. The supplied elastic straps were adjusted and positioned to hold both the chest and waist sensors firmly on all participants. The reinforcement of the elastic bands helped to attenuate the possibility of electrode displacement and reduced motion artifacts which could occur with upper body movements throughout the maximal cycle test.
Maximal Cycle Test
Before testing, body mass (in cycling gear without shoes) was determined. The athletes used their own pedals and cleated shoes for all testings. The cycle ergometer (Velotron; RacerMate Inc., Seattle, WA) was adjusted to the athletes' preferred position, which was then replicated in the subsequent trial. An oxygen mask and a Hexoskin vest were fitted and sizes were recorded for test-retest consistency. Hexoskin's instructions were followed to ensure a good fit of the garment and for verification purposes, Bluetooth connectivity was made, and real-time data were viewed on a portable electronic tablet device before the test. Athletes were informed about the testing protocol being power ramped, and they were instructed to continue cycling until exhaustion. After 10 minutes of resting data collection while seated stationary on the bike, athletes completed a 5-minute warm-up at a low work rate (100–125 W) before commencing the test at a work rate of 100 W. All stages were 60 seconds in duration with an increase in load being 25 W until exhaustion. Athletes were either mountain or road cyclists and to accommodate for individual cadence preferences, the range was set rather wide (∼80–100 rpm). Cyclists were instructed to choose a comfortable cadence and maintain that cadence throughout the test. Although not recorded with the other variables, cadence was visible to both participant and researchers to ensure that cyclists stayed near their chosen cadence ±5 rpm. The laboratory temperature was controlled at 15 ± 2° C. The test was terminated when the athlete had reached volitional exhaustion or was unable to maintain cadence. Throughout the test, ventilation and expired gases were collected and analyzed every 15 seconds using a metabolic cart (MetaMax 3B; Cortex Biophysik, Leipzig, Germany). Oxygen consumption (Vo2), expiratory minute ventilation (VE), and respiratory frequency (f) were calculated. Before testing, the gas analyzers were calibrated for volume (Hans Rudolph 5530 3-L syringe; Hans Rudolph, Inc., Kansas City, MO) and composition (15% O2 and 5% CO2). Maximal oxygen uptake Vo2max was taken as a 20-second average within which the highest O2 consumption was measured. Peak power output was defined as the highest work rate the athlete could complete for the full 60 seconds. Heart rate was measured continuously by ECG (ADInstruments, Colorado Springs, CO) and a Polar HR monitor (RS800CX; Polar, Kempele, Finland).
Data Analysis
Heart rate obtained from a standard 3-lead ECG was acquired continuously at 200 Hz using an analog-to-digital converter (Powerlab/16SP ML795; ADInstruments) interfaced with a computer. Breath-by-breath respiratory variables were gathered by the MetaMax 3B system. Heart rate, VE, and f were linearly interpolated at 1-second intervals either by the MetaMax (MetaSoft) or LabChart software (Human ECG, ADInstruments) to coincide with the frequency of data reported by the Hexoskin garment. Data acquisition using the Hexoskin vest was recorded at 128 Hz (analog dual-channel) during inspiration and expiration through 2 strain gauge bands woven into the garment around the chest and waist. Recordings were taken from these sensors to supply an estimation of f and VE, which was calculated as the average of the last 7 completed respiration cycles and output to the data file at 1 Hz. Analog 256 Hz ECG data were collected by the biometric vest and the respiration sensor measured VE in 128 Hz as raw data with 16 bits resolution. During all tests, Hexoskin data were stored locally in the Hexoskin device and then uploaded afterward using HxServices, Hexoskin's data synchronization desktop software for Windows 8.
With the exception of HR, all data were time-aligned in 1-second intervals and filtered for outliers, which included removing any values greater than 4 SDs from the average values for the 60-second period of each work rate and for the entire 5-minute rest period for f and VE data only. For the HR data, on average, 21.83 ± 14% comprised outliers (ranging from 2.5 to 52.79%), which were removed so as not to confound the maximal protocol filtering strategy. Work rate stages were selected to provide an overview of the changes at different work intensities (i.e., 25, 50, 75, and 100% of work rate maximum).
The progressive increase in HR with the maximal test, and the large range in erroneous values made an SD-based outlier-detection strategy unfeasible with HR data. Therefore, HR data were examined for suitability based on the previous appropriate data point, whereby any change in HR of >6 beats within 1 second was assumed to be physiologically unlikely (similar to assumptions by the Polar Heart Rate Monitor software filtering system). Where a HR data point was 6 beats higher or lower than the previous beat, the offending values were highlighted for manual examination and removal.
Statistical Analyses
For the validity analysis, variables gathered from the Hexoskin wearable vest including HR, minute ventilation, and breathing frequency were compared with those of standard laboratory devices (MetaMax 3B and laboratory ECG) during rest (sitting on the cycle ergometer not pedaling) and during an incremental cycling test to exhaustion. Data were log-transformed before analysis to avoid any effects of nonuniformity or error and used customized statistical analysis (11) based on previously published theory (10) to determine validity. The average of the 5-minute rest and 60-second exercise periods (25, 50, 75, and 100% work rate maximum in Watts) was used in the analysis. The best measure of validity is the SE of the estimate (also known as the typical error of the estimate) (19) which is reported along with other regression validity statistics (Pearson correlation coefficient). Cohen's smallest effect statistics were used to quantify magnitudes of change between variables (12). The standardized typical error (Cohen's d) was given, however, to interpret the magnitude of an error on Hopkin's scale of magnitudes; the error was doubled (18) to 0.2, 0.6, 1.2, 2.0, and 4.0 to indicate small, moderate, large, very large, and extremely large effects. A 2-sided post hoc power analysis using G*Power (5) revealed that with a sample size of 10, an effect size of 0.5, and an α of 0.05, the power (1 − β error probability) was estimated to be low at 29%. There was difficulty in finding a large number of high-performance cyclists, thus the sample size was small which impacted power. Future studies could help to verify findings using larger sample sizes.
We used a similar approach to analyze the reliability of the Hexoskin vest (11). Variables measured by the Hexoskin garment (HR, VE, and f) during moderate-intensity (i.e., 50 and 75% work rate maximum [Watts]) were averaged and compared with the same variables measured by the Hexoskin 3–4 days later. The within-subject variability (within-subject SD) is analogous to the SE of measurement which is reported along with the intraclass correlation coefficient. Data are reported as mean ± SD and, where appropriate, the mean effect along with the 90% confidence limits was given.
Results
Participants produced similar peak power outputs and Vo2max scores during the 2 incremental cycle tests (Table 1).
A typical response for the VE, f, and HR during the test is illustrated in Figure 1.
Figure 1.: Sample test response for (A) heart rate, (B) f, and (C) VE comparing criterion values (breath-by-breath system) and Hexoskin values during a maximal aerobic cycle ergometer test using a ramped protocol (starting at 100 W with 25 W increments each minute to failure).
Validity
Differences between the criterion (laboratory) and practical (Hexoskin) measures for HR and f tended to be small (Table 2). However, as evidenced by the larger typical (standard) error and reduced Pearson correlation coefficient, more error was found in the VE variable between the 2 devices. The HR recordings from the criterion measures aligned well with those from the Hexoskin for all work rate stages (r ≥ 0.98). The f showed good validity from rest through to 75%Wmax (r ≥ 0.95), but a moderate discrepancy was detected at 100%Wmax (typical error = 6.2%, r = 0.91). The breathing frequency data recorded by the Hexoskin (Table 2) indicate a mild damping impact, potentially as a result of the inherent tightness of the Hexoskin garment. Finally, large measurement discrepancies between the criterion and practical measures were found for VE during all stages of the test (r ≥ 0.69) except for 25%Wmax where the difference was moderate (typical error = 9.5%, r = 0.84).
Table 2.: Validity results using average of combined tests comparing the Hexoskin wearable vest against the MetaMax 3B and laboratory ECG during parallel data collection at rest and during incremental cycle exercise to exhaustion.*
Reliability
The test-retest reliability statistics of the Hexoskin are presented in Table 3. Based on the validity results, it was prudent to assess reliability with the 50 and 75%Wmax work rates which provided the most congruency between the criterion and practical measurement devices. The HR and f variables produced by the Hexoskin were reliable at these moderate-intensity exercise rates (r ≥ 0.94); however, the typical error was larger for the VE variable at both work rates (typical error ≥ 5.3%, r ≥ 0.81). Overall, the typical error between trials for the criterion measurement devices was all small indicating good reliability between trials.
Table 3.: Reliability results of the cardiorespiratory variables produced by the Hexoskin wearable vest and the criterion devices (MetaMax 3B and laboratory ECG) over 2 identical trials at the same submaximal work rate intensity.*
Discussion
The vast availability of wearable devices has expanded the potential impact of any research using them for data collection. However, completing reliability and validation studies in various contexts is critical before laboratory usage of such devices. For example, competitive cycling involves a unique postural positioning and cycling-specific movements which are in stark contrast to those of walking, lying, or sitting. This study added to the extant literature as it is the first study to investigate the validity and reliability of the Hexoskin wearable vest on elite cyclists during a maximal ramped cycle ergometer test. Moreover, the unique posture and movements used during cycling could impact the reliability and validity of the Hexoskin in comparison with the more upright positioning during standing, sitting, and lying.
Sensor data extracted from the Hexoskin vest were compared with the HR measured by a standard 3-lead ECG as well as the VE and f from the MetaMax 3B system. The MetaMax 3B system uses a bidirectional digital turbine to measure the volume and frequency of breathing and previous research has established that the MetaMax 3B has good reliability with intradevice technical error of <1.0% over a range of metabolic rates (14). Moreover, the MetaMax 3B collects breathing variables directly using a mask covering the nose and mouth and it is considered the gold standard device for such measurements. The Hexoskin garment does not provide a direct measurement of breathing variables, rather it derives measurements from an algorithm based on the movement sensor data collected at the chest and abdomen.
The Hexoskin was found to be valid in comparison with the laboratory ECG for HR and the MetaMax 3B system for f, but showed lower validity for the VE variable (typical error 9.5–14.9%), possibly because it was the variable requiring an algorithm for calculation. Although the Hexoskin was sufficient for measuring everyday activities in healthy adults (20), it was insufficiently reliable and valid when compared with the MetaMax 3B system for measuring VE in elite cyclists, particularly during high-intensity seated cycling. To achieve accurate VE and f measurements during recording, the Hexoskin vest functions best with little or no movement of the garment on the skin and requires continuous conductivity of both the chest and abdomen sensors, which can sometimes be difficult with large amounts of movement fore, aft, and laterally. Although body fat was not measured in this study, athlete's body mass index ranged from underweight to normal. When measuring people with higher body fat content, perhaps the Hexoskin would provide better contact between the skin and sensors, thus reducing measurement error and improving reliability and validity. Given the participants were elite cyclists, high aerobic work outputs were achieved during the progressive exercise test which led to high Vo2max values (Table 1). Increased aerobic activity led to an increase in upper body movement which is often used by cyclists while pedaling during bouts of high-intensity cycling (e.g., during the 100%Wmax stage of the test). This increased torso movement increased the likelihood of the Hexoskin sensors moving around on the skin, in turn, leading to potential noise signals in the measurements. The higher typical error found in all the variables at rest and in particular in the resting VE variable could be associated with the fact that the Hexoskin garment logs values at 1 Hz but then averages the last 7 completed respiratory cycles to the output. Sudden alterations to breathing characteristics which are recorded almost instantaneously with a breath-by-breath system (i.e., MetaMax 3B) are seemingly delayed with the Hexoskin garment. Such differences would be diminished during exercise as the subject quickly finds a new steady state to match the exercise workload. Caution should be exercised when comparing these results to those of a previous Hexoskin reliability and validity study. Previous research assessed young, healthy individuals rather than elite athletes (20). Also, the previous study measured participants in positions other than cycling (lying, sitting, standing, and walking) (20). Moreover, this study only measured 10 elite male cyclists on 2 repeated measures which limit the strength of the results.
To accurately measure reliability, participants are required to be working at the same relative intensity on both test days. To overcome the fact that the cycle test was a progressive test to exhaustion in which the lower and upper exercise intensities can be influenced by external factors such as physiological adjustments to the warm-up, tiredness from previous exercise etc., we chose to use exercise intensities that would be affected less by such extraneous factors (e.g., 50 and 75% of work rate maximum). Using these exercise intensities, we found that the Hexoskin produced low-to-moderate typical errors between tests suggesting the Hexoskin provides adequate reliability at moderate-intensity work rates.
The criterion devices commonly used in laboratory settings were also used for this study and they measured breathing directly at the nose and mouth, whereas HR was collected using sensors on the skin. Similarly, the Hexoskin relies on conductivity sensors touching the skin of the torso to collect HR. The similar methodology of HR data collection between criterion devices and the Hexoskin are elucidated by the good HR validity and reliability. To generate the VE and f variables the Hexoskin synthesizes data collected from chest sensors through an algorithm. These nondirect breathing measures lessen the Hexoskin's validity and reliability in assessing VE in elite cyclists during a ramped cycle test. Nevertheless, the Hexoskin would be well suited for HR and f, whereas laboratory devices would offer increased precision and reliability for VE.
The producers of the garment may consider developing separate algorithms for elite and nonelite subjects and for various sports. As suggested in previous research, more complex algorithms for higher intensity activities could aid in filtering noise signals during measurement (20). During the cycling tests, subjects at maximal exercise were breathing approximately 170–240 L of air per minute. The high VE indicated a relatively high error at rest and low- and high-intensity exercise yet the typical SE of the estimate increased with intensity. To help reduce measurement error, future research should consider increasing the sampling rate.
Practical Applications
Given the unique nature of these data collection methods, there are implications for practical applications. Field practitioners who train elite cyclists likely require accurate HR data and close estimates of breathing capabilities during high-intensity cycling. Such practitioners might consider using the Hexoskin wearable vest with its relative affordability, portability, and its versatility to collect real-time data on several athletes simultaneously. However, researchers or medical practitioners requiring valid and reliable VE are advised to use laboratory devices which are typically more expensive and less portable.
Acknowledgments
The authors are grateful to Jingji Wang for his technical assistance. This research was in collaboration with Carré Technologies Inc., Montreal, Quebec, Canada.
The authors have no conflicts of interest to declare.
References
1. Borresen J, Lambert MI. Autonomic control of heart rate during and after exercise. Sports Med 38: 633–646, 2008.
2. Clarenbach CF, Senn O, Brack T, Kohler M, Bloch KE. Monitoring of ventilation during exercise by a portable respiratory inductive plethysmograph. Chest 128: 1282–1290, 2005.
3. Di Rienzo M, Meriggi P, Rizzo F, Castiglioni P, Lombardi C, Ferratini M, Parati G. Textile technology for the vital signs monitoring in telemedicine and extreme environments. IEEE Trans Inf Technol Biomed 14: 711–717, 2010.
4. Di Rienzo M, Racca V, Rizzo F, Bordoni B, Parati G, Castiglioni P, Meriggi P, Ferratini M. Evaluation of a textile-based wearable system for the electrocardiogram monitoring in cardiac patients. Europace 15: 607–612, 2013.
5. Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 39: 175–191, 2007.
6. Flatt AA, Esco MR. Evaluating individual training adaptation with Smartphone-derived heart rate variability in a collegiate female soccer team. J Strength Cond Res 30: 378–385, 2016.
7. Gastin PB, McLean O, Spittle M, Breed RV. Quantification of tackling demands in professional Australian football using integrated wearable athlete tracking technology. J Sci Med Sport 16: 589–593, 2013.
8. Hailstone J, Kilding AE. Reliability and validity of the Zephyr™ BioHarness™ to measure respiratory responses to exercise. Meas Phys Educ Exerc Sci 15: 293–300, 2011.
9. Heilman KJ, Porges SW. Accuracy of the LifeShirt® (Vivometrics) in the detection of cardiac rhythms. Biol Psychol 75: 300–305, 2007.
10. Hopkins WG. Measure of reliability in sports medicine and science. Sports Med 30: 1–15, 2000.
11. Hopkins WG. Spreadsheets for analysis of validity and reliability. Sportscience 19: 36–42, 2015.
12. Hopkins WG, Marshall SW, Batterham AM, Hanin J. Progressive statistics for studies in sports medicine and exercise science. Med Sci Sports Exerc 41: 3–12, 2009.
13. Kent L, O'Neill B, Davison G, Nevill A, Elborn JS, Bradley JM. Validity and reliability of cardiorespiratory measurements recorded by the LifeShirt during exercise tests. Respir Physiol Neurobiol 167: 162–167, 2009.
14. Macfarlane D, Wong P. Validity, reliability and stability of the portable Cortex Metamax 3B gas analysis system. Eur J Appl Physiol 112: 2539–2547, 2012.
15. Park S, Jayaraman S. Enhancing the quality of life through wearable technology. IEEE Eng Med Biol Mag 22: 41–48, 2003.
16. Saboul D, Balducci P, Millet G, Pialoux V, Hautier C. A pilot study on quantification of training load: The use of HRV in training practice. Eur J Sport Sci 16: 172–181, 2016.
17. Scherr D, Kastner P, Kollmann A, Hallas A, Auer J, Krappinger H, Schuchlenz H, Stark G, Grander W, Jakl G. Effect of home-based telemonitoring using mobile phone technology on the outcome of heart failure patients after an episode of acute decompensation: Randomized controlled trial. J Med Internet Res 11: e34, 2009.
18. Smith TB, Hopkins WG. Variability and predictability of finals times of elite rowers. Med Sci Sports Exerc 43: 2155–2160, 2011.
19. Smith TB, Hopkins WG. Measures of rowing performance. Sports Med 42: 343–358, 2012.
20. Villar R, Beltrame T, Hughson RL. Validation of the Hexoskin® wearable vest during lying, sitting, standing, and walking activities. Appl Physiol Nutr Metab 40: 1019–1024, 2015.
21. Winterhalter CA, Teverovsky J, Wilson P, Slade J, Horowitz W, Tierney E, Sharma V. Development of electronic textiles to support networks, communications, and medical applications in future US Military protective clothing systems. IEEE Trans Inf Technol Biomed 9: 402–406, 2005.
