Secondary Logo

Journal Logo

Small Sample Sizes Confound Understanding of Cardiometabolic Responses to Exercise

Pescatello, Linda S.1,4; Corso, Lauren M.L.1,4; MacDonald, Hayley V.1,4,5; Thompson, Paul D.6; Taylor, Beth A.1,4,6; Panza, Gregory A.1,4,6; Zaleski, Amanda L.1,4,6; Johnson, Blair T.2,4; Chen, Ming-Hui3

Exercise and Sport Sciences Reviews: July 2017 - Volume 45 - Issue 3 - p 173–180
doi: 10.1249/JES.0000000000000115
Articles
Free
SDC

Considerable variability exists in the cardiometabolic disease biomarker response to exercise. We propose that a major contributor to this heterogeneity is underpowered studies due to small sample sizes. To test our hypothesis, we conducted a systematic review to identify meta-analyses/reviews of randomized controlled trials (RCT) and RCT that examined the cardiometabolic disease biomarker response to aerobic and resistance exercise.

We conducted a systematic review of meta-analyses/reviews and randomized controlled trials examining the cardiometabolic disease biomarker response to exercise.

1Departments of Kinesiology; 2Psychological Sciences; 3Statistics; 4Institute for Collaboration on Health, Intervention and Policy (InCHIP), University of Connecticut, Storrs CT; 5Department of Kinesiology, The University of Alabama, Tuscaloosa, AL; and 6Department of Cardiology, Henry Low Heart Center, Hartford Hospital, Hartford CT

Address for correspondence: Linda S. Pescatello, Ph.D., FACSM, FAHA, Department of Kinesiology & Human Performance Laboratory, College of Agriculture, Health and Natural Resources, University of Connecticut, Gampel Pavilion Room 206, 2095 Hillside Rd, U-1110, Storrs, CT 06269-1110 (E-mail: linda.pescatello@uconn.edu).

Accepted for publication: March 21, 2017.

Editor: Bo Fernhall, Ph.D., FACSM.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (www.acsm-essr.org).

Back to Top | Article Outline

Key Points

  • Considerable variability exists in the cardiometabolic disease biomarker response to exercise.
  • We conducted a systematic review to identify meta-analyses/reviews of randomized controlled trials (RCT) and RCT examining the cardiometabolic disease biomarker response to exercise.
  • Only 5 of 27 RCT were sufficiently powered for all cardiometabolic disease biomarkers reported.
  • Only one third to one half of the exercise interventions were adequately powered for at least one of the cardiometabolic disease biomarkers that they reported.
  • This literature was of high methodological quality indicating that investigators attempted to control some of the potential sources of heterogeneity. Yet, other sources of heterogeneity that exist are compounded by small sample sizes.
  • Investigators of future trials should minimize sample size shortcomings so that we can be better informed about the cardiometabolic health benefits of exercise.
Back to Top | Article Outline

INTRODUCTION

Regular exercise participation is important for the prevention, treatment, and management of some of the most prevalent, costly, and deadly chronic diseases and health conditions that include cardiovascular disease, diabetes mellitus, obesity, dyslipidemia, and hypertension (1,18,28). These chronic diseases and health conditions tend to cluster within a person and have been termed cardiometabolic disease (19). In general, greater volumes or intensities of exercise elicit greater cardiometabolic health benefits than do lesser amounts, as summarized by ourselves (7,8,23,24) and others (9,11,30). Yet, there is a wide range of interindividual variability in the response of cardiometabolic disease biomarkers to exercise that includes systolic blood pressure (SBP) and diastolic blood pressure (DBP) and fasting triglycerides, glucose, insulin, low-density lipoprotein cholesterol (LDL-C), and high-density lipoprotein cholesterol (HDL-C) (2). Indeed, the standard error of the change often exceeds the mean change of the cardiometabolic disease biomarker response to exercise.

Over 25 yr ago, we conducted a systematic review to gain insight into discrepant reports in the literature on the BP-lowering effects of aerobic exercise under ambulatory conditions (22). We found that ambulatory BP on average was reduced from prehypertensive to normotensive ranges after acute (i.e., short term, immediate) and chronic (i.e., long term, training) aerobic exercise. Yet, there was a considerable range in the response, with 35% and 38% of the subjects showing no change or an elevation in daytime ambulatory SBP and DBP, respectively, after versus before acute and chronic aerobic exercise.

From our review emerged important methodological considerations that provided insight into the discrepant findings of this literature, including not accounting for the circadian variation in BP with the inclusion of a control/comparison group and the acute last-bout effect when measuring the effects of exercise training on BP. At the root of the problem, however, seemed to be inadequate statistical power due to primary level studies with small sample sizes and inclusion of samples with predominately normal BP. Such studies would require large samples to be sufficiently powered per the law of initial values (29), by which individuals with the highest BP values (i.e., those with hypertension) would experience greater BP reductions than individuals with normal BP after versus before exercise.

More recently, we conducted a systematic review to determine the methodological quality of meta-analyses on the BP response to exercise (12). All 33 meta-analyses that met the inclusion criteria sampled randomized controlled trials (RCT), with most (72%) sampling only RCT. Nearly all meta-analyses (91%) found on average that exercise significantly lowered BP. Even so, no meta-analysis completely satisfied the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) contemporary methodological quality standards (16,17), and we found this literature to be of only fair methodological quality overall. Consistent with our earlier review (22), we noted that despite the volume of this literature, many of the studies in these meta-analyses had small sample sizes, most with normal BP, and, thus, they seemed to be underpowered (12).

Because cardiometabolic disease biomarkers cluster, the insights gained from these systematic reviews (12,22) and original research from our laboratory (7,10,21) have led us to postulate that sample size shortcomings not only apply to the BP response to exercise but to other major cardiometabolic disease biomarkers as well, including fasting triglycerides, glucose, insulin, LDL-C, and HDL-C. The purpose of this systematic review was to identify meta-analyses/reviews of RCT and RCT that have examined the response of these major cardiometabolic disease biomarkers to acute and chronic aerobic and dynamic resistance exercise. We hypothesized that the identified RCTs were underpowered due to small sample sizes, and, thus, sample size shortcomings are a major contributor to the heterogeneity in this literature (Fig. 1).

Figure 1

Figure 1

Back to Top | Article Outline

METHODS

This systematic review is reported consistent with the PRISMA Statement (16,17) and Assessment of Multiple Systematic Reviews Methodological Quality Scale (25,26).

Back to Top | Article Outline

Search Strategy and Selection Criteria

In consultation with a medical librarian from the University of Connecticut, comprehensive Boolean searches were run in PubMed (including Medline) from inception to September 1, 2016 to identify all potentially eligible studies. Reference lists of the included studies were searched manually, and relevant meta-analyses and reviews of RCT were disaggregated to ensure that we identified all potential qualifying studies examining the response of the cardiometabolic disease biomarkers (i.e., SBP and DBP and fasting triglycerides, glucose, insulin, LDL-C, and HDL-C) to acute and chronic aerobic and dynamic resistance exercise. The full search strategy appears in Supplemental Digital Content (SDC) 1, http://links.lww.com/ESSR/A27.

RCTs were included if they 1) involved healthy, sedentary adult populations 18 yr or older; 2) involved a nonexercise/nondiet control or comparison group; 3) had a sample size of more than 20 subjects to minimize bias toward small sample sizes and be consistent with the systematic review strategies of the 2008 Physical Activity Guidelines for Americans (28); 4) reported one or more of the following cardiometabolic disease biomarkers pre- and postexercise and control: SBP and DBP and fasting triglycerides, glucose, insulin, LDL-C, and HDL-C; 5) disclosed the frequency, intensity, and time of the acute or chronic aerobic or dynamic resistance exercise intervention; and 6) were published in English. Trials involving weight loss drugs, diet therapy, or diet modifications in addition to exercise, or populations with chronic disease(s) (e.g., cancer, coronary artery disease, human immunodeficiency virus/acquired immunodeficiency syndrome), were excluded.

Back to Top | Article Outline

Data Extraction

Coding

Coded variables were extracted using a standardized coding form and coder manual developed by our research team (12). Variables that were coded included study characteristics (e.g., sample size, method of randomization, and type of control group), sample characteristics (e.g., age, body mass index (BMI), and maximum oxygen consumption (V˙O2max)), features of the exercise intervention (e.g., frequency, intensity, time, and type (FITT)), and methodological study quality.

Back to Top | Article Outline

Study Methodological Quality

Study methodological quality was gauged using a modified version of the Downs and Black Checklist (6) (SDC 2, http://links.lww.com/ESSR/A28). Our team’s use of the modified Downs and Black Checklist is detailed elsewhere (13,15). Briefly, the Downs and Black Checklist addresses five subscales of quality (i.e., reporting, external validity, bias, confounding, and power) and is considered one of the most comprehensive tools available (5). We calculated the percentage of items satisfied out of a possible 29-point total and quantified the overall methodological study quality as low (≤49%), moderate (50%–69%), or high (≥70%) (4). Quality scores were calculated for each of the quality subscales and expressed as the percentage of items satisfied. In addition to quantifying the quality of the acute and chronic aerobic and dynamic resistance exercise literature, we examined correlations between dimensions of study quality with sample size and other coded variables.

Back to Top | Article Outline

Effect and Sample Size Calculations

For each trial, we calculated the effect size (ES) for each reported mean cardiometabolic disease biomarker response to exercise relative to control divided by the pooled standard deviation (SD) of the response. Using this ES, we then calculated the sample size (N) required to sufficiently power the trial. In some trials, we could not calculate the estimated N required to sufficiently power the trial because the response to control was better than for exercise. For each cardiometabolic disease biomarker, we then calculated the median N and N range (minimum, maximum) and the median ES and ES range (minimum, maximum). ES was calculated using R Project for Statistical Computing version 3.2.3 (https://www.r-project.org/). N estimates were generated using PASS 13 (University of Connecticut license), assuming a statistical power of 80% and a significance threshold of P < 0.05.

Back to Top | Article Outline

Statistical Analyses

Continuous variables are summarized as mean ± SD unless otherwise indicated, and categorical variables are presented as absolute values and percentages. For RCT that included independent exercise and control groups, differences in the baseline characteristics were tested using t-tests and one-way ANOVA. For each cardiometabolic disease biomarker, trials were considered sufficiently powered when the reported N met or exceeded the estimated N of the intervention (SDC 3, http://links.lww.com/ESSR/A29 and 4, http://links.lww.com/ESSR/A31 for acute and chronic aerobic exercise; SDC 5, http://links.lww.com/ESSR/A30 and 6, http://links.lww.com/ESSR/A32 for acute and chronic resistance exercise). Pearson correlations were calculated to examine relationships between RCT sample size (i.e., the reported N) and other coded features. Analyses were performed using Stata 13.1 (Stata Corp, College Station, Tex) (27). Significance was set at P < 0.05 for all analyses.

Back to Top | Article Outline

RESULTS

Of the 353 eligible articles, we identified 27 RCT (aerobic exercise = 13, dynamic resistance exercise = 14) that met our inclusion criteria (Fig. 2). Due to the paucity of dynamic resistance exercise trials that satisfied our inclusion criteria for a sample size of more than 20 subjects, we included the largest dynamic resistance exercise trials available. Details and supplemental references of the included acute and chronic aerobic and dynamic resistance exercise trials are presented in SDCs 3–7, http://links.lww.com/ESSR/A29, http://links.lww.com/ESSR/A30, http://links.lww.com/ESSR/A31, http://links.lww.com/ESSR/A32, http://links.lww.com/ESSR/A33, respectively.

Figure 2

Figure 2

Back to Top | Article Outline

Description of the Acute and Chronic Trials in the Sample

SDC 3, http://links.lww.com/ESSR/A29 and 4, http://links.lww.com/ESSR/A30 provide a general description of the included acute and chronic aerobic and dynamic resistance exercise trials that were published between 1993 and 2016. All eight of the acute trials and 2 of 19 chronic or training trials applied a randomized nonexercise control design, whereas the remaining chronic trials used an independent nonexercise or “wait-list” control comparison group. The BP response after acute aerobic exercise was assessed using 24-h ambulatory BP monitoring over an average of approximately 10 awake hours. All chronic aerobic exercise and acute and chronic dynamic resistance exercise trials assessed BP in the laboratory only. All metabolic outcomes were examined under fasting conditions and analyzed using manual or automated enzymatic/spectrophotometric methods. Acute aerobic exercise trials had significantly smaller sample sizes than chronic aerobic exercise trials (N = 25 vs N = 79, respectively; P < 0.05). Conversely, there was no statistical difference between the sample sizes of the acute and chronic dynamic resistance exercise studies (N = 19 vs N = 28, respectively; P > 0.05). Overall, both acute and chronic aerobic and dynamic resistance exercise trials achieved “high” methodological study quality (80.9% ± 8.2% of items satisfied; range, 58.6%–93.1%) on the modified version of the Downs and Black Checklist (5,7).

Back to Top | Article Outline

Aerobic Exercise

Sample Characteristics

The Table summarizes the preintervention sample characteristics for the acute and chronic aerobic exercise and control comparison groups. On average, the participants in the acute aerobic exercise trials (N = 100) were sedentary, young to middle aged (35.4 ± 7.0 yr), overweight (28.6 ± 2.7 kg·m−2) white (83%) men (96%) with prehypertension (131.9 ± 12.7/82.4 ± 3.2 mm Hg) and a fasting cardiometabolic profile within normal limits (20). On average, the participants in the chronic aerobic exercise trials (aerobic exercise, N = 1731; control, N = 566) were sedentary, middle aged (52.2 ± 8.9 yr), overweight (29.9 ± 2.5 kg·m−2) white (69%) women (86%) with prehypertension (129.6 ± 9.8/80.7 ± 1.1 mm Hg) and a fasting cardiometabolic profile within normal limits (20). Baseline subject characteristics were similar between the acute and chronic aerobic exercise and control comparison groups (P > 0.05). Participants in both the acute and chronic aerobic exercise trials were not taking medications and were free from any overt chronic diseases or health conditions other than their high BP.

TABLE

TABLE

Back to Top | Article Outline

Features of the Aerobic Exercise Interventions

The Table summarizes the features of the acute and chronic aerobic exercise interventions. On average, acute aerobic exercise was performed at moderate to vigorous intensity (55.0% ± 9.3% V˙O2max) for 24.3 ± 5.3 min per session on a cycle ergometer. On average, aerobic exercise training was performed 3.5 ± 1.1 d·wk−1 at moderate to vigorous intensity (60.0% ± 9.9% V˙O2max) for 40.2 ± 10.2 min per session for 28.1 ± 12.9 wk and involved a variety of modalities, including cycling (k = 2) or a combination of treadmill/cycling/walking (k = 7); four trials did not specify the type of aerobic exercise training modality. All aerobic exercise training trials were supervised, and adherence to the interventions was high (>80% of sessions attended/completed) with 80% of the trials reporting adherence rates.

Back to Top | Article Outline

Dynamic Resistance Exercise

Sample Characteristics

On average, the participants in the acute dynamic resistance exercise trials (N = 82) were sedentary, young to middle aged (34.8 ± 8.2 yr), overweight (26.0 ± 3.6 kg·m−2) men (63.4%) with normal BP (115.7 ± 3.2/74.7 ± 2.7 mm Hg) and a fasting cardiometabolic profile within normal limits (20). The race/ethnicity of the participants was not reported for any of the acute trials; however, the location where the study was performed suggested that most (71%) of the subjects were of Hispanic/Latino(a) descent (13,15). On average, the participants in the chronic dynamic resistance exercise trials (resistance exercise, N = 251; control, N = 226) were sedentary, young to middle aged (36.1 ± 12.6 yr), overweight (28.2 ± 3.5 kg·m−2) white (71%) women (55%) with normal BP (108.8 ± 40.9/78.2 ± 4.1 mm Hg) and a fasting metabolic profile within normal limits (20). Baseline subject characteristics were similar between the acute and chronic dynamic resistance exercise and control comparison groups (P > 0.05). Participants in the acute and chronic dynamic resistance exercise trials were not taking medications and were free from any overt chronic disease(s) or health condition(s).

Back to Top | Article Outline

Features of the Dynamic Resistance Exercise Interventions

The Table summarizes the features of the acute and chronic dynamic resistance exercise interventions. On average, acute dynamic resistance exercise was performed at low to moderate intensity (55.0% ± 16.3% 1-repetition maximum (1-RM)) for 45.0 ± 21.2 min. The acute resistance exercise session consisted of 2.6 ± 0.8 sets and 16.9 ± 6.0 repetitions of 8.6 ± 1.5 exercises that targeted the major muscles of the upper and lower body. All interventions incorporated machines and free weights. On average, dynamic resistance exercise training was performed 3.0 ± 0.0 sessions per week (54.2 ± 7.5 min per session) at vigorous intensity (70.1% ± 15.9% 1-RM) for 13.8 ± 5.6 wk. Weekly resistance exercise sessions consisted of 2.7 ± 0.4 sets and 9.7 ± 1.0 repetitions of 9.4 ± 3.1 exercises that targeted the major muscles of the upper and lower body. All dynamic resistance exercise training interventions incorporated machine and free weights. Resistance exercise training trials were supervised, and adherence to the intervention was high (>80% of sessions attended/completed); however, only 50% of trials disclosed this information.

Back to Top | Article Outline

The Cardiometabolic Disease Biomarker Response to Acute and Chronic Exercise

Aerobic Exercise

For each acute and chronic aerobic exercise trial, the reported cardiometabolic disease biomarker mean response ± SD to exercise relative to control and the estimated N required to sufficiently power the trial are shown in SDC 3, http://links.lww.com/ESSR/A29. In some trials denoted in blue, we could not calculate the estimated N required to sufficiently power the trial because the response to control was better than for exercise. For each cardiometabolic disease biomarker, the median N and N range (minimum, maximum) and median ES and ES range (minimum, maximum) also are shown in SDC 3, http://links.lww.com/ESSR/A29.

To summarize the range of the cardiometabolic disease biomarker response to acute aerobic exercise compared with control, the range (minimum, maximum) of the reported mean change ([INCREMENT]) and ES that we were able to calculate is as follows: SBP ([INCREMENT] = 0.4, −11.7 mm Hg; ES = −0.23, −0.81), DBP ([INCREMENT] = 0.1, −4.9 mm Hg; ES = −0.19, −0.52), glucose ([INCREMENT] = −0.2 ± 0.3 mmol·L−1, ES = −0.67), and insulin ([INCREMENT] = −3.0 ± 8.7 pmol·L−1, ES = −0.34). Of note, the estimates for glucose and insulin are from a single trial, and qualifying acute aerobic exercise trials did not report results for LDL-C and HDL-C. Only one (20%) acute aerobic exercise trial was sufficiently powered to detect a significant change for all the cardiometabolic disease biomarkers reported, and only two (40%) acute aerobic exercise trials were sufficiently powered for at least one of the cardiometabolic disease biomarkers that were reported (SDC 5, http://links.lww.com/ESSR/A31).

To summarize the range of the cardiometabolic disease biomarker response to chronic aerobic exercise compared with control, the range (minimum, maximum) of the reported mean [INCREMENT] and ES that we were able to calculate is as follows: SBP ([INCREMENT] = 0.2, −4.3 mm Hg; ES = −0.06, −0.34), DBP ([INCREMENT] = −2.0 mm Hg, ES = −0.15), triglycerides ([INCREMENT] = 0.3 mmol·L−1, −0.08 mmol·L−1; ES = −0.04, −0.38), glucose ([INCREMENT] = 0 mg·dL−1, −2.0 mg·dL−1; ES = −0.18, −0.20), insulin ([INCREMENT] = −0.8 pmol·L−1, −11.6 pmol·L−1; ES = -0.06, −0.67), LDL-C ([INCREMENT] = −0.1 mM, ES = −0.10), and HDL-C ([INCREMENT] = −0.01, 0.08 mmol; ES = 0.05, 0.40). Of note, the estimates for DBP and LDL-C are from a single trial. Only one (8%) chronic aerobic exercise trial was sufficiently powered to detect a significant change for all cardiometabolic disease biomarkers reported, and 54% of the chronic aerobic exercise trials were sufficiently powered for at least one of the cardiometabolic disease biomarkers that were reported (SDC 4, http://links.lww.com/ESSR/A31).

Back to Top | Article Outline

Dynamic Resistance Exercise

For each acute and chronic resistance exercise trial, the reported cardiometabolic disease biomarker mean response ± SD to exercise relative to control and the estimated N required to sufficiently power the trial are shown in SDC 4, http://links.lww.com/ESSR/A30. In some trials denoted in blue, we could not calculate the estimated N required to sufficiently power the trial because the response to control was better than for exercise. For each cardiometabolic disease biomarker, the median N and N range (minimum, maximum) and median ES and ES range (minimum, maximum) also are shown SDC 5, http://links.lww.com/ESSR/A30.

To summarize the range of the cardiometabolic disease biomarker response to acute dynamic exercise compared with control, the range (minimum, maximum) of the reported mean [INCREMENT] and ES that we were able to calculate is as follows: SBP ([INCREMENT] = 0.0, −8.0 mm Hg; ES = −0.13, −1.79), DBP ([INCREMENT] = −0.5, −2.6 mm Hg; ES = −0.10, −0.46), triglycerides ([INCREMENT] = −4.0 mg·dL−1, ES = −0.12), glucose ([INCREMENT] = −2.7, −3.0 mg·dL−1; ES = −0.34, −2.46), and insulin ([INCREMENT] = −0.2 μU·mL−1, −2.9 μU·mL−1; ES = −0.04, −1.00). Of note, the estimate for triglycerides is from a single trial, and qualifying acute dynamic resistance exercise trials did not report results for LDL-C and HDL-C. Only one (17%) acute dynamic resistance exercise intervention was sufficiently powered to detect a significant change for all cardiometabolic disease biomarkers reported, and only three (50%) acute resistance exercise interventions were sufficiently powered for at least one of the cardiometabolic disease biomarkers that were reported (SDC 6, http://links.lww.com/ESSR/A32).

To summarize the range of the cardiometabolic disease biomarker response to chronic dynamic exercise compared with control, the range (minimum, maximum) of the reported mean [INCREMENT] and ES that we were able to calculate is as follows: SBP ([INCREMENT] = 0.0, −10.0 mm Hg; ES = −0.40, −0.70), DBP ([INCREMENT] = −0.5, −9.0 mm Hg; ES = −0.05, −0.82), triglycerides ([INCREMENT] = −0.2 mmol·L−1, ES = −0.24), glucose ([INCREMENT] = −0.3 mg·dL−1, ES = −0.03), insulin ([INCREMENT] = −0.2 μU·mL−1, ES = −0.04), LDL-C ([INCREMENT] = −0.5, −0.9 mmol·L−1; ES = −0.03, −0.82), and HDL-C ([INCREMENT] = 0.01 mmol·L−1, ES = 0.02). Of note, the estimates for triglycerides, glucose, insulin, and HDL-C were from single trials. Only two (22%) chronic dynamic resistance exercise interventions were sufficiently powered to detect a significant change for all the cardiometabolic disease biomarkers reported, and only three (33%) chronic resistance exercise interventions were sufficiently powered for at least one of the cardiometabolic disease biomarkers that were reported (SDC 6, http://links.lww.com/ESSR/A32).

Surprisingly, we found only five RCT (aerobic exercise = 2 (acute = 1 + chronic = 1), SDC 3, http://links.lww.com/ESSR/A29 and 4, http://links.lww.com/ESSR/A31; resistance exercise = 3 (acute = 1 + chronic = 2), SDC 5, http://links.lww.com/ESSR/A30 and 6, http://links.lww.com/ESSR/A32) that were sufficiently powered to detect significant changes for all the cardiometabolic disease biomarkers reported. In addition, for a given cardiometabolic disease biomarker, no trial was sufficiently powered for LDL-C (0.0%, k = 0); 22% (k = 4) were for DBP, 27% (k = 7) were for SBP, 30% (k = 3) were for triglycerides, 33% (k = 3) were for glucose, 40% (k = 4) were for HDL-C, and 44% (k = 7) were for insulin (SDC 5, http://links.lww.com/ESSR/A31 and 6, http://links.lww.com/ESSR/A32). Last, there was a considerable range in the cardiometabolic disease biomarker response to acute and chronic aerobic and dynamic resistance exercise with an overall ES and N for exercise intervention type as follows (median (minimum, maximum)): 1) acute aerobic exercise, ES = −0.35 (−0.19, −0.81), N = 66 (14–230); 2) chronic aerobic exercise, ES = −0.18 (−0.04, −0.67), N = 250 (20–6408); 3) acute dynamic resistance exercise, ES = −0.43 (−0.04, −2.46), N = 72 (4–4280); and 4) chronic dynamic resistance exercise, ES = −0.41 (−0.03, −0.82), N = 50 (14–11,374).

Back to Top | Article Outline

Correlations Between Sample Size and Dimensions of Study Methodological Quality

Finally, we examined potential relationships among the actual sample size of the included trials and dimensions of their methodological quality. In general, larger sample sizes were associated with the following: higher study quality for the acute resistance exercise (r = 0.940, P = 0.001) and marginally higher study quality for the chronic aerobic exercise (r = 0.537, P = 0.08) interventions; more transparent reporting of study methods for the acute resistance exercise interventions (r = 0.979, P = 0.001); and disclosing power analyses for study outcomes for the chronic aerobic exercise interventions (r = 0.706, P = 0.02), of which only 4 of 10 of these trials did. Last, more recently published trials had larger sample sizes for acute resistance exercise (r = 0.930, P = 0.01) and trended toward significance for the aerobic exercise (r = 0.647, P = 0.08) interventions.

Back to Top | Article Outline

DISCUSSION

The purpose of this systematic review was to identify RCT with a sample size of more than 20 subjects that have examined the response of major cardiometabolic disease biomarkers (i.e., SBP and DBP and fasting triglycerides, glucose, insulin, LDL-C, and HDL-C) to acute and chronic aerobic and dynamic resistance exercise among healthy adults without chronic disease. In the 27 qualifying trials, we then calculated the mean ES for each cardiometabolic disease biomarker that was reported. Using this ES, we then back calculated the sample size, N, that was needed to adequately power the trial to detect for significant changes in the response of that cardiometabolic disease biomarker to exercise relative to control. Last, we then compared the estimated N with the reported sample size of the trial to determine if the trial was adequately powered to detect significant exercise-induced changes. Our major findings confirm our hypothesis that this literature is underpowered due to small sizes, and, thus, sample size shortcomings are a major contributor to the heterogeneity that exists in this literature (Fig. 1).

Indeed, we found only five of 27 RCT that were sufficiently powered for all cardiometabolic disease biomarkers reported in a study (SDC 5, http://links.lww.com/ESSR/A31 and 6, http://links.lww.com/ESSR/A32). Approximately 40% of the acute aerobic and 50% of the acute and 33% of the chronic resistance exercise interventions and one half of the chronic (54%) aerobic exercise interventions were adequately powered for at least one of the cardiometabolic disease biomarkers that were reported. Furthermore, for a given cardiometabolic disease biomarker, no exercise trial was sufficiently powered for LDL-C, and 22.2% were sufficiently powered for DBP, 26.9% for SBP, 30.0% for triglycerides, 33.3% for glucose, 40.0% for HDL-C, and 43.8% for insulin. It is also worthy to mention two other methodological concerns that emerged from our review: only 8 of 27 (i.e., 30%) of the trials reported a priori power calculations that included two acute aerobic, four chronic aerobic, and two chronic resistance exercise interventions and only 3 of 27 (i.e., 11.1%) of the trials performed an intention-to-treat analysis that included three chronic aerobic exercise interventions.

As we suspected, there was a considerable range of cardiometabolic disease biomarker response to acute and chronic aerobic and dynamic resistance exercise with a wide overall ES range (−0.03, −2.46) and estimated N range to adequately power the sample (4, 11,374) (SDC 3, http://links.lww.com/ESSR/A29 and 4, http://links.lww.com/ESSR/A30). It also should be noted that the median ES for acute aerobic exercise was −0.35, chronic aerobic exercise was −0.18, acute dynamic resistance exercise was −0.43, and chronic dynamic resistance exercise was −0.41, indicating a surprisingly small to moderate strength of effect of exercise on the cardiometabolic disease biomarker response to exercise. The extent of the sample size shortcomings and small to moderate ES that we found in this literature were unexpected, and collectively, they call into question what we truly know about the response of cardiometabolic disease biomarkers to exercise.

Another worrisome finding that emerged from our systematic review is that sample characteristics differed by exercise intervention type, which is likely another contributing factor to the heterogeneity of this literature. On average, participants in the 1) acute aerobic exercise trials (N = 100) were sedentary, young to middle aged, overweight, white men with prehypertension; 2) chronic aerobic exercise trials (N = 1731) were sedentary, middle aged, overweight, white women with prehypertension; 3) acute dynamic resistance exercise trials (N = 82) were sedentary, young to middle aged, overweight, Hispanic men with normal BP; and 4) chronic dynamic resistance exercise trials (N = 251) were sedentary, young to middle aged, overweight, white women with normal BP. In addition to having a fasting blood cardiometabolic profile within normal limits, the other common sample features across all exercise intervention types were that the subjects were sedentary and overweight. However, there were striking sample differences across exercise intervention types for sex, race/ethnicity, and BP status, all of which have been shown to be important moderators of the response of cardiometabolic disease biomarkers to exercise (3,12,14,15,22). Furthermore, the acute dynamic resistance exercise interventions were conducted at low to moderate intensity, whereas the chronic resistance training interventions were conducted at vigorous intensity. Given these noticeable differences in sample features across the exercise intervention types, the conclusions that can be drawn about the cardiometabolic health benefits of exercise are limited to the exercise intervention type and study sample being investigated, and they cannot be generalized to the overall cardiometabolic health benefits of exercise per se.

Several encouraging findings from our systematic review are important to document. Overall, the included acute and chronic aerobic and dynamic resistance exercise trials achieved high study methodological quality with more than 80% of the items satisfied on the modified version of the Downs and Black Checklist (5,7). The general high study methodological quality of this literature indicates that potential sources of heterogeneity on the Downs and Black Checklist, such as comprehensive data reporting and transparency of study methods and analyses, for the most part, are being properly addressed during study development (Fig. 1). Most (>80%) of the chronic aerobic and resistance exercise trials were supervised. Adherence to the intervention was high (>80% of sessions attended/completed), with 80% of the aerobic training trials but only 50% of the resistance training trials disclosing this information. We found that more recently published trials tended to have larger sample sizes, suggesting that study methods have steadily improved over time. In support of this premise, during the trial selection process, the studies with a sample size of less than 20 subjects that were excluded from our systematic review were published significantly earlier than those included in our review (2004 vs 2009, P = 0.003). These findings are consistent with temporal trends that we previously reported for the BP and exercise literature (12).

Our review is not without limitations. Our search strategy was thorough, but the possibility exists that some qualifying trials were not identified. Approximately 30% of trials that were excluded had sample sizes of 20 subjects or less, and it is unclear, other than publication year, how these trials differed from those that were included. Nonetheless, we choose studies with a sample size of more than 20 subjects to be exclusion criteria so that we included the largest RCT conducted to date not to bias our sample toward having small sample sizes and be consistent with the search strategies of the 2008 Physical Activity Guidelines for Americans (28). We conducted post hoc sample size and ES calculations. Therefore, the possibility exists that a trial falsely may seem underpowered when in fact the null hypothesis is true, meaning exercise does not affect the cardiometabolic biomarker in question. Last, we did not standardize our ES calculations for sample size. Nonetheless, we purposely included the largest RCT conducted to date in our sample to minimize this limitation by representing the trials that are most likely to be sufficiently powered for each cardiometabolic disease biomarker.

Our review fully satisfied contemporary methodological standards (25,26) and is the first to be conducted on this topic. We anticipate that our review will provide constructive guidance for researchers regarding the study design considerations depicted in Figure 1 to improve future trial methods and the science about the cardiometabolic health benefits of exercise. Improving the strength of this literature is important because existent scientific statements and clinical practice guidelines from leading professional organizations on the use of exercise in the prevention, treatment, and control of cardiometabolic diseases and conditions such as hypertension, dyslipidemia, and diabetes mellitus are based upon a largely underpowered literature that underestimates the effectiveness of exercise as lifestyle behavioral therapy.

Back to Top | Article Outline

CONCLUSIONS

We performed a systematic review to identify RCT with a sample size of more than 20 subjects that have examined the response of major cardiometabolic disease biomarkers to acute and chronic aerobic and dynamic resistance exercise among healthy adults without chronic disease to test our hypothesis that sample size shortcomings are at the root of much of the heterogeneity in this literature (Fig. 1). We found only 5 of 27 RCT were sufficiently powered for all cardiometabolic disease biomarkers reported, and only one third to one half of the exercise interventions were adequately powered for at least one of the cardiometabolic disease biomarkers reported. An encouraging finding was high study methodological quality of this literature, which is an evidence that investigators are attempting to address or control potential sources of heterogeneity as outlined in the modified Downs and Black Checklist. Nonetheless, the median ES of this literature was small to moderate, indicating there are other sources of heterogeneity such as the varied features of the study samples and lack of disclosure of exercise intervention features that exist that are further compounded by small sample sizes (Fig. 1). Our review speaks to the importance of investigators of future trials carefully attending to the study design considerations that can minimize sample size shortcomings, and consequently, better inform the science about the cardiometabolic health benefits of exercise.

Back to Top | Article Outline

Acknowledgments

This study was supported by funding from the Office of the Vice President for Research, Research Excellence Program, University of Connecticut, Storrs, CT: Institute for Collaboration on Healthy, Intervention and Policy (InCHIP), University of Connecticut, Storrs, CT. The authors declare no conflicts of interest.

Back to Top | Article Outline

References

1. Booth FW, Roberts CK, Laye MJ. Lack of exercise is a major cause of chronic diseases. Compr. Physiol. 2012; 2(2):1143–211.
2. Bouchard C, Blair SN, Church TS, et al. Adverse metabolic response to regular exercise: is it a rare or common occurrence? PLoS One. 2012; 7(5):e37887.
3. Casonatto J, Goessler KF, Cornelissen VA, Cardoso JR, Polito MD. The blood pressure-lowering effect of a single bout of resistance exercise: a systematic review and meta-analysis of randomised controlled trials. Eur. J. Prev. Cardiol. 2016; 23(16):1700–14.
4. Chudyk AM, Jutai JW, Petrella RJ, Speechley M. Systematic review of hip fracture rehabilitation practices in the elderly. Arch. Phys. Med. Rehabil. 2009; 90(2):246–62.
5. Deeks JJ, Dinnes J, D'Amico R, et al. Evaluating non-randomised intervention studies. Health Technol. Assess. 2003; 7(27: iii,x,):1–173.
6. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J. Epidemiol. Community Health. 1998; 52(6):377–84.
7. Eicher JD, Maresh CM, Tsongalis GJ, Thompson PD, Pescatello LS. The additive blood pressure lowering effects of exercise intensity on post-exercise hypotension. Am. Heart J. 2010; 160(3):513–20.
8. Eijsvogels TM, Molossi S, Lee DC, Emery MS, Thompson PD. Exercise at the extremes: the amount of exercise to reduce cardiovascular events. J. Am. Coll. Cardiol. 2016; 67(3):316–29.
9. Garber CE, Blissmer B, Deschenes MR, et al. American College of Sports Medicine position stand. Quantity and quality of exercise for developing and maintaining cardiorespiratory, musculoskeletal, and neuromotor fitness in apparently healthy adults: guidance for prescribing exercise. Med. Sci. Sports Exerc. 2011; 43(7):1334–59.
10. Guidry MA, Blanchard BE, Thompson PD, et al. The influence of short and long duration on the blood pressure response to an acute bout of dynamic exercise. Am. Heart J. 2006; 151(6):1322.e5–12.
11. Haskell WL. J.B. Wolffe Memorial Lecture. Health consequences of physical activity: understanding and challenges regarding dose-response. Med. Sci. Sports Exerc. 1994; 26(6):649–60.
12. Johnson BT, MacDonald HV, Bruneau ML Jr, et al. Methodological quality of meta-analyses on the blood pressure response to exercise: a review. J. Hypertens. 2014; 32(4):706–23.
13. Corso LM, MacDonald HV, Johnson BT, et al. Is concurrent training efficacious antihypertensive therapy? A meta-analysis. Med. Sci. Sports Exerc. 2016; 48(12):2398–406.
14. Lin X, Zhang X, Guo J, et al. Effects of exercise training on cardiorespiratory fitness and biomarkers of cardiometabolic health: a systematic review and meta-analysis of randomized controlled trials. J. Am. Heart Assoc. 2015; 4(7):pii: e002014.
15. MacDonald HV, Johnson BT, Huedo-Medina TB, et al. Dynamic resistance training as stand-alone antihypertensive lifestyle therapy: a meta-analysis. J. Am. Heart Assoc. 2016; 5(10):pii: e003231.
16. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J. Clin. Epidemiol. 2009; 62(10):1006–12.
17. Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst. Rev. 2015; 4:1.
18. Pedersen BK, Saltin B. Exercise as medicine — evidence for prescribing exercise as therapy in 26 different chronic diseases. Scand. J. Med. Sci. Sports. 2015; 25(Suppl. 3):1–72.
19. Pescatello L. How much physical activity for cardiometabolic health benefit? Am. J. Med. Sports. 2000; (2):368–70, 373–382.
20. Pescatello LS, Arena R, Riebe D, Thompson PD. ACSM's guidelines for exercise testing and prescription. 9th Edition. Baltimore (MD): Lippincott Williams and Wilkins; 2013. p. 40–59.
21. Pescatello LS, Guidry MA, Blanchard BE, et al. Exercise intensity alters postexercise hypotension. J. Hypertens. 2004; 22(10):1881–8.
22. Pescatello LS, Kulikowich JM. The aftereffects of dynamic exercise on ambulatory blood pressure. Med. Sci. Sports Exerc. 2001; 33(11): 1855–61.
23. Pescatello LS, MacDonald HV, Ash GI, et al. Assessing the existing professional exercise recommendations for hypertension: a review and recommendations for future research priorities. Mayo Clin. Proc. 2015; 90(6):801–12.
24. Pescatello LS, MacDonald HV, Lamberti L, Johnson BT. Exercise for hypertension: a prescription update integrating existing recommendations with emerging research. Curr. Hypertens. Rep. 2015; 17(11):87, 015-0600-y.
25. Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med. Res. Methodol. 2007; 7:10.
26. Shea BJ, Hamel C, Wells GA, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J. Clin. Epidemiol. 2009; 62(10):1013–20.
27. StataCorp. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP, 2013.
28. U.S. Department of Health and Human Services. 2008 Physical Activity Guidelines for Americans [Internet]. [cited 2011 January 6]. Available from: http://www.health.gov/paguidelines/pdf/paguide.pdf.
29. Wilder J. The law of initial value in neurology and psychiatry; facts and problems. J. Nerv. Ment. Dis. 1957; 125(1):73–86.
30. Williams PT. Physical fitness and activity as separate heart disease risk factors: a meta-analysis. Med. Sci. Sports Exerc. 2001; 33(5):754–61.
Keywords:

aerobic exercise; blood pressure; glucose; insulin; lipids-lipoproteins; resistance exercise; systematic review

Supplemental Digital Content

Back to Top | Article Outline
© 2017 American College of Sports Medicine