Journal Logo

APPLIED SCIENCES

Nonplacebo Controls to Determine the Magnitude of Ergogenic Interventions: A Systematic Review and Meta-analysis

MARTICORENA, FELIPE MIGUEL1; CARVALHO, ARTHUR1; OLIVEIRA, LUANA FARIAS DE1; DOLAN, EIMEAR1; GUALANO, BRUNO1,2; SWINTON, PAUL3; SAUNDERS, BRYAN1,4

Author Information
Medicine & Science in Sports & Exercise: August 2021 - Volume 53 - Issue 8 - p 1766-1777
doi: 10.1249/MSS.0000000000002635

Abstract

Historically in research, placebos have been used as a control treatment that is indistinguishable from the active intervention but has no active component. This allows the intervention arm to be compared with the nonactive placebo treatment, which, theoretically, should allow for the determination of the “true” effect of the intervention, by reducing the expectancy or belief about the intervention which may artificially inflate the estimated response to that intervention (i.e., placebo effect). These placebo effects are induced by administering an inert intervention (e.g., sham drugs, nutrients, or equipment), which can elicit a neurobiological response due to verbal suggestions, explicit expectations, or implicit experience (1). Placebo effects in sport and exercise nutrition are well reported with a systematic review showing small to moderate positive effects on exercise outcomes (d = 0.35 (confidence interval, 0.20–0.51)) (2) when participants are administered an inert substance believed to be a beneficial treatment. In the real world, the total effect of interventions is likely to comprise both the active physiological component and the placebo effect. This means that studies could underestimate the overall real-world effect of a given intervention if they do not include a nonplacebo control arm. Accordingly, studies that comprise three conditions, namely, active treatment, placebo, and a no-treatment control, are required to determine the total effect of the intervention (active treatment plus placebo effects) and to estimate the relative contribution of these component parts.

Caffeine and buffering supplements, such as sodium bicarbonate, are ergogenic supplements considered effective to improve exercise capacity and performance (3). However, placebo effects may occur with some nutritional supplements. Specifically, individuals who believe they have ingested caffeine (4–6) or sodium bicarbonate (7), but have actually ingested placebo, can improve their exercise performance. On the contrary, performance improvements might be blunted when individuals consume caffeine but believe they have received placebo (8). Thus, in double-blind placebo-controlled designs, there may be a variable placebo or nocebo effect related to the psychosocial context of ingesting a supplement (active or inactive), which can influence the exercise outcome due to expectancy (5). Studies investigating the efficacy of supplements commonly include an active treatment arm and a placebo comparator in a double-blind crossover design, but do not always use a nonplacebo control session. Those studies that do incorporate both a placebo and a nonplacebo control session provide an excellent platform to determine the proportion and variability of ergogenic effects that can be attributed to placebo effects. Moreover, knowing the resulting net effect of a given active intervention plus its placebo effect would help inform prescription for athletes.

The aims of this study were to estimate the size of the placebo effects associated with caffeine and buffering supplements and to determine the proportion of the overall ergogenic effect that is explained by placebo. This was achieved through a meta-analysis of aggregate data from studies including both placebo and nonplacebo control sessions. A secondary aim was to determine which factors might modify the size of these placebo effects, including exercise, population, and supplement characteristics. Caffeine and buffering supplements were chosen as a model for this study because they are considered effective ergogenic aids, and both have been shown to incur placebo effects.

METHODS

Study eligibility

The study protocol was designed in accordance with PRISMA guidelines (9) (Table, Supplemental Digital Content 1, PRISMA guidelines checklist, https://links.lww.com/MSS/C276) and the inclusion criteria defined according to PICOS criteria (Population, Intervention, Comparator, Outcomes and Study design). Only English-language peer-reviewed, original human studies were included within this review. The population included healthy human males and females of any age, but studies conducted with diseased-state participants were excluded. Recreationally active and trained individuals and professional athletes were considered for inclusion. The intervention required a supplementation protocol comprising any dose of caffeine or sodium bicarbonate, sodium citrate, calcium lactate, or sodium lactate before performing an exercise test. These supplements (i.e., caffeine and extracellular buffers) were selected to test our hypotheses, as there is a substantial body of placebo-controlled studies estimating ergogenic and placebo effects. In relation to the comparator, the aims of this study determined that studies necessarily used both a placebo (inert substance) and a nonplacebo control (no treatment) trial. Studies that reported on outcomes based on exercise performance or capacity tests (e.g., total work done and mean power output) were considered for inclusion. Study design included any randomized and blinded, crossover, or parallel-group design. Studies that used balanced placebo designs, in which participants were informed (correctly or deceptively) what supplement they had received, were not included, as this has been reviewed extensively elsewhere (2). The study was not preregistered.

Search strategy and quality assessment

An electronic search of the literature was undertaken by F.M.M. using three databases (MedLine, Embase, and SPORTDiscus) to identify relevant articles. Caffeine studies were searched using the term “caffeine” concatenated with “exercise,” “performance,” “physical performance,” and “training.” Extracellular buffer studies were searched using the search terms “sodium bicarbonate,” “sodium citrate,” “calcium lactate,” “sodium lactate,” and “alkalosis” concatenated with “supplementation,” “exercise,” “training,” “athlete,” and “performance.” The extracellular buffers search was originally conducted to inform a systematic review and meta-analysis on the use of extracellular buffers on exercise outcomes.

Duplicates were removed before a two-phase search strategy was performed independently by two reviewers for buffering supplements (L.F.O. and E.D.) and caffeine (F.M.M. and A.C.) using Rayyan software for systematic reviews (10). Phase 1 assessed the eligibility of the title and abstract of every article generated from the search terms against the inclusion/exclusion criteria. Studies with uncertain suitability were included at this stage, and a final decision was reached at the next phase, namely, phase 2 in which full articles were retrieved and assessed against the eligibility criteria. Reference lists of all included studies and review articles were screened to ensure all relevant studies were included. Any differences of opinion relating to study eligibility were resolved through discussion. The original searches were conducted in November 2019 and these searches were updated in April 2020, to identify any eligible studies published in the interim. No date limit was applied to the search.

Data extraction and variable categorization

Data extraction was conducted by F.M.M. using a standardized and prepiloted extraction spreadsheet created using Microsoft excel. Extracted information included authors, location and year of publication, population characteristics (age, sex, and training status), supplementation protocol (dose, timing, and form of administration), exercise protocol, type (exercise capacity or performance) and duration (<30 s, 30 s–10 min, >10 min) of exercise, and the exercise outcomes. All extracted data are available (Table, Supplemental Digital Content 2, Extracted Study Data, https://links.lww.com/MSS/C277). To avoid duplication bias, a solitary outcome measure from each exercise protocol was extracted based on an a priori hierarchy agreed upon by all authors to ensure consistency in data extraction. Data were extracted according to availability, prioritizing exercise measures over physiological measures, according to the following hierarchical profile (11):

  1. Total work done
  2. Mean output throughout the test (i.e., mean power output, mean velocity, mean height)
  3. Time to completion (performance test)/time to exhaustion (capacity test)

After data extraction, an a posteriori decision was made to determine the contribution of factors that might modify the placebo response to supplementation. As such, data were categorized according to the following factors for analysis:

  • i) Exercise protocols were separated by exercise duration [exercise duration] according to the approach of Saunders et al. (11), namely, 0–0.5, 0.5–10, and >10 min, which were chosen considering energy system contribution to tests that differ in duration.
  • ii) Exercise protocols were also categorized according to whether they measured exercise capacity or performance [exercise type], as these different exercise types have been shown to modify the effects of supplements (12).
  • iii) Studies were separated according to the sample population recruited [training status] because trained athletes may have greater belief in the power of placebos to enhance sporting performance (13), which might differ from nontrained individuals. Trained individuals were considered those engaged in a structured training program with a training plan relevant to the exercise task used in the study, whereas remaining populations that did not fit these criteria (i.e., recreationally active, nontrained, sedentary) were categorized as nontrained.
  • iv) Supplement characteristics may modify the placebo effects of an intervention (14–16). Supplementation protocols were thus separated according to delivery method [supplement delivery], namely, whether they provided the placebo and active interventions as capsules or dissolved in solution.
  • v) Finally, studies were categorized according to year of publication [study year], separating those published prior to 2000, and after 2000.

Risk of bias and quality assessment

Risk of bias was assessed using the most recent Cochrane tool for assessing risk of bias in randomized crossover trials (17), and study quality was evaluated using an adapted Downs and Black questionnaire (18) (Table, Supplemental Digital Content 3, Adapted Downs and Black questionnaire and scores, https://links.lww.com/MSS/C278). Using these two tools allowed greater certainty and reliability to identify the quality of the included studies. Evaluation of risk of bias and study quality was performed in a blinded fashion by two independent reviewers (F.M.M. and A.C.), and any disagreements were resolved between these two reviewers via discussion and, when necessary, with the help of a third reviewer (B.S.). We adapted the Downs and Black questionnaire checklist because some questions were not relevant for the purpose of this review, resulting in a 14-point adapted questionnaire. According to each question, the answers were classified with points from 0 to 1 or 2 that were summed to provide an overall score. The quality score was ranked according to the following intervals: high (12–14), moderate (9–11), low (6–8), and very low (≤5).

Data analysis

Extracted data were transformed into pairwise effect sizes (intervention vs placebo, intervention vs control, and placebo vs control) by calculating Hedges’ g standardized mean difference. Calculation of standardized effect sizes enabled results from exercise tests conducted on different scales to be pooled in the meta-analysis. Standard distributional assumptions were used to calculate effect size standard errors (19). Most previous meta-analyses have been conducted within a frequentist framework where parameters such as the pooled effect size are estimated, and the uncertainty expressed with a 95% confidence interval (i.e. the values that would not be rejected by P < 0.05 [20]). However, confidence intervals contain no distributional information, such that there is no direct sense by which the parameter values in the middle of the interval are more probable that the ends (15). In contrast, Bayesian frameworks combine prior beliefs regarding the most plausible values with data to provide values that can be directly interpreted as probabilities. Results can therefore be interpreted easily, and more applied questions such as the probability that parameters of interest exceed relevant thresholds can be addressed. In the present meta-analysis, the Bayesian framework was implemented through three-level hierarchical models with random effects to account for variation in the mean effect and covariance where multiple outcomes were reported in the same study (21). To investigate potential moderating effects of factors such as [supplement type], [supplement delivery], [exercise type] and [exercise duration], [training status], and [study year], meta-regressions were performed. To investigate the relative proportions of the placebo and intervention effect, models combining both intervention and placebo study effect sizes were conducted in a meta-regression. Inferences from all analyses were performed on posterior samples generated by Hamiltonian Markov Chain Monte Carlo with Bayesian 95% credible intervals (CrI). Interpretations were based on visual inspection of the posterior sample, the median value (ES0.5: 0.5-quantile) and 95% CrIs. Threshold values of 0.01, 0.2, 0.5, and 0.8 were used to describe pooled effect sizes as very small, small, medium, and large, respectively (22). Interpretations of meta-regressions were based on location and spread of posterior distributions of regression coefficients (β0.5; Reference : Comparison). Expression of placebo as a proportion was achieved by taking the ratio of the two posterior samples. Analyses were performed using the R wrapper package brms, which interfaced with Stan to perform sampling (23). Weakly informative Student-t prior and half-t priors with 3 degrees of freedom and scale parameter equal to 2.5 were used for intercept and variance parameters (24). Convergence of parameter estimates was obtained for all models with Gelman–Rubin R-hat values below 1.1 (25). Where outliers (g ≥ 2.0, 15 of 168, calculated effect sizes) were present, sensitivity analyses were conducted by performing robust hierarchical models using a t-distribution for the likelihood. No substantive differences in findings were obtained for any sensitivity analysis conducted.

RESULTS

Study Search, Characteristics, and Quality Appraisal

The primary search resulted in 7824 articles for caffeine and 3621 for buffers (Fig. 1). After the removal of duplicates (caffeine, 2193; buffers, 334), Phase 1 resulted in the exclusion of 5143 caffeine and 2994 buffer articles. The remaining articles were screened in their entirety for suitability, and after the removal of studies without a nonplacebo control session, 34 published studies met the criteria for inclusion in the analyses (Table 1), containing a total of 56 exercise outcomes from 363 participants. These comprised 10 caffeine studies yielding 11 outcomes, and 24 articles for buffering supplements yielding 45 outcomes. All studies were randomized crossover study designs.

F1
FIGURE 1:
Flowchart of the search strategy and study selection.
TABLE 1 - Studies included in the meta-analysis.
Authors and Location Participants Supplementation Protocol Exercise Protocol(s)
Caffeine studies
 Bridge and Jones (26), United Kingdom Trained male distance runners(n = 8) Capsule containing 3 mg·kg−1 of caffeine or placebo (glucose) 60 min before exercise 8-km running TT
 Clarke et al. (27),
United Kingdom
Recreationally active males(n = 12) Beverage containing 3 mg·kg−1 of caffeine or placebo (flavored water) 45 min before exercise 18 × 4-s cycling sprints separated by 116-s recovery
 Dolan et al. (28),
United States
Male lacrosse players(n = 14) Caffeine solution or placebo (flavored water) mouth rinse 10 s before exercise YoYo (level 1)
 Flinn et al. (29), Australia Recreational male cyclists(n = 9) Beverage containing 10 mg·kg−1 of caffeine or placebo (p-flour) 180 min before exercise Incremental cycle test
 Karayiğit et al. (30), Turkey Physically active males(n = 10) 8 × 25-mL mouth rinse with caffeine solution (2%) or placebo (water) at 30-s intervals during 5-min warm-up 30-s cycling Wingate anaerobic test
 McNaughton et al. (31), United Kingdom Trained male cyclists(n = 6) Beverage containing 6 mg·kg−1 of caffeine or placebo (flavored water) 60 min before exercise 1-h cycling TT
 Wiles et al. (32),
United Kingdom
Trained male cyclists(n = 8) Beverage containing 5 mg·kg−1 of caffeine or placebo (flavored water) 60 min before exercise 1-km cycling TT
 Saunders et al. (5), Brazil Trained male cyclists(n = 42) Capsule containing 6 mg·kg−1 of caffeine or placebo (dextrose) 60 min before exercise 30-min cycling TT
 Rezaei et al. (33), Iran Trained males and females karatekas(n = 8) Capsule containing 6 mg·kg−1 of caffeine or placebo (cellulose) 50 min before exercise Karate Specific Aerobic Test
 Grgic et al. (34), Australia Recreationally trained males(n = 26) Capsule containing 6 mg·kg−1 of caffeine or placebo (dextrose) 60 min before exercise Countermovement jump
Buffering supplement studies
 Bird et al. (35), United Kingdom Male distance runners(n = 10) 0.3 mg·kg−1 of sodium bicarbonate solution or placebo (sodium chloride and calcium carbonate) 1500-m running
 Carr et al. (36), Australia Well-trained rowers(n = 8) 0.3 mg·kg−1 of sodium bicarbonate in gelatine capsules or placebo (corn flour) 90 min before exercise 2000-m rowing ergometer TT
 Coombes and McNaughton (37), Australia Healthy physical education university students(n = 9) 0.3 mg·kg−1 of sodium bicarbonate solution or placebo (calcium carbonate) 90 min before exercise Isokinetic leg extension/flexion exercise
 Coppoolse et al. (38), United States Healthy(n = 5) 0.3 mg·kg−1 of sodium bicarbonate solution or placebo 60 min before exercise Cycling test with a work rate increment of 25 or 30 W·min−1
 Goldfinch et al. (39), Australia Athletes(n = 6) 0.4 mg·kg−1 of sodium bicarbonate solution or placebo (calcium carbonate) 60 min before exercise 400-m run
 Griffen et al. (40), United Kingdom Well-trained(n = 9) Chronic supplementation of 0.3 mg·kg−1 of sodium bicarbonate solution or placebo (maltodextrin) 6 × 10-s cycling sprints 7.5% BM
 Lindh et al. (41), United Kingdom Elite-standard swimmers(n = 9) 0.3 mg·kg−1 of sodium bicarbonate in gelatine capsules or placebo (calcium carbonate) 90 min before exercise 200-m freestyle swim
 Materko et al. (42), Brazil Strength trained(n = 11) 0.3 mg·kg−1 of sodium bicarbonate solution or placebo (sodium chloride) 120 min before exercise Bench press test
Pull press test
 McLellan et al. (43), Canada Healthy(n = 4) Chronic supplementation of 0.2 mg·kg−1 of sodium bicarbonate in gelatine capsules or placebo (calcium carbonate) Cycling: 10 min at 50% and 70% and 90% of V˙O2max until exhaustion
 McNaughton (44), Australia Healthy(n = 11) 0.1, 0.2, 0.3, 0.4, and 0.5 mg·kg−1 of sodium citrate solution or placebo (calcium carbonate) 90 min before exercise Maximal 1-min cycle effort
 McNaughton et al. (45), Australia Cyclists(n = 8) 0.4 mg·kg−1 of sodium bicarbonate solution or placebo (calcium carbonate) 60 min before exercise Maximal 1-min cycle effort
 McNaughton (46), Australia Healthy(n = 9) 0.1, 0.2, 0.3, 0.4, and 0.5 mg·kg−1 of sodium bicarbonate solution or placebo (calcium carbonate) 90 min before exercise Maximal 1-min cycle effort
 McNaughton (47), Australia Healthy(n = 8) 0.3 mg·kg−1 of sodium bicarbonate solution or placebo (calcium carbonate) 90 min before exercise Maximal 10-s cycle effort
Maximal 30-s cycle effort
Maximal 120-s cycle effort
Maximal 240-s cycle effort
 McNaughton and Cedaro (48), Australia Healthy(n = 10) 0.5 mg·kg−1 of sodium citrate solution or placebo (calcium carbonate) 90 min before exercise Maximal 10-s cycle effort
Maximal 30-s cycle effort
Maximal 120-s cycle effort
Maximal 240-s cycle effort
 McNaughton et al. (49), Australia Physical active women(n = 10) 0.3 mg·kg−1 of sodium bicarbonate solution or placebo 90 min before exercise Maximal 1-min cycle effort
 McNaughton et al. (50), United Kingdom Cyclists(n = 10) 0.3 mg·kg−1 of sodium bicarbonate solution or placebo (sodium chloride) 90 min before exercise 60-min cycling
 Miller et al. (51), United Kingdom Active team and individual sports (n = 11) 0.3 mg·kg−1 of sodium bicarbonate solution or placebo (sodium chloride) 10 × 6-s cycle sprints with 60-min recovery
 Morris et al. (52), United States Competitive cyclists(n = 11) 0.1 mg·kg−1 of lactate in gelatine capsules or placebo (aspartame) 90 min before exercise Cycling test until exhaustion starting at 3 W/BM and increases of 0.3 W/BM
 Oliveira et al. (53), Brazil Athletes of rugby, judo, and jiu-jitsu at university level (n = 18) Chronic supplementation of 0.5 mg·kg−1·d−1 of sodium bicarbonate or calcium lactate in gelatine capsules or placebo (calcium carbonate) 4 bouts of the 30-s Wingate upper body anaerobic test with 3-min recovery
 Painelli et al. (54), Brazil Junior-standard swimmers(n = 7) 0.3 mg·kg−1 of sodium bicarbonate in gelatine capsules or placebo (dextrose) 90 min before exercise 100-m swimming
200-m swimming
 Pierce et al. (55), United States Varsity swimmers(n = 7) 0.2 mg·kg−1 of sodium bicarbonate solution or placebo (sodium chloride) 60 min before exercise 100-yard (91.4 m) swim freestyle
Individual 200-yard swims
 Rezaei et al. (33), Iran Karatekas(n = 8) Chronic supplementation of 0.3 mg·kg−1 of sodium bicarbonate in gelatine capsules or placebo (cellulose) Karate Specific Aerobic Test
 Tiryaki and Atterbom (56), Turkey Track athletes and nonathletes (n = 15) 0.3 mg·kg−1 of sodium bicarbonate, sodium citrate or placebo solution 120 min before exercise 600-m running test
 Wilkes et al. (57), Canada Varsity track athletes(n = 6) 0.3 mg·kg−1 of sodium bicarbonate solution or placebo (calcium carbonate) 120 min before exercise 800-m run race
TT, time trial.

Risk of Bias and Quality Assessment

Almost all studies included in the meta-analysis were classified as having “some concerns” according to ROB2, except one, which was categorized as “high” risk of bias due to issues in the randomization process (Fig. 2). Although all studies were randomized, 20 studies had some concerns in domain 1 (randomization process) because of a lack of detail, whereas all studies were classified as having some concerns owing to a lack of a prespecified analysis plan (as outlined in domain 5). The Downs and Black quality appraisal showed all studies attained a high score of between 12 and 14 (Table, Supplemental Digital Content 3, Adapted Downs and Black questionnaire and scores, https://links.lww.com/MSS/C278).

F2
FIGURE 2:
Risk of bias presented as percentages across all included studies for the five main domains of evaluation. (Figure was created using robvis (58) and is in a color blind–friendly color scheme).

Meta-analysis

Absolute placebo effects (placebo vs nonplacebo control)

An initial assessment of the placebo effect was obtained by pooling pairwise effect sizes (placebo vs nonplacebo control) across studies investigating caffeine and/or buffering supplements. The pooled effect size indicated a very small effect (ES0.5 = 0.09 (95% CrI, 0.01 to 0.17); τ0.5 = 0.04 (0.00–0.14); ICC = 0.38 (0.10–0.69); Fig. 3A) of placebo compared with nonplacebo control (Figure, Supplemental Digital Content 4, Plot including effect sizes before creation of shrunken estimates, https://links.lww.com/MSS/C279). The probability that the pooled effect size represented at least a small effect was P = 0.005. There was some evidence that the placebo effect was moderated by [supplement type] and [supplement delivery], with greater effects obtained with buffers (β0.5; Caffeine : Buffers = 0.06 (95% CrI, −0.12 to 0.24)) and with solution (β0.5; Capsule : Solution = 0.10 (95% CrI, −0.07 to 0.28)). However, when the meta-regression was performed with both factors included in the model, the analysis indicated that [supplement delivery] was more likely to act as a moderator (β0.5; Intercept : Solution = 0.11 (95% CrI, −0.08 to 0.31); β0.5; Intercept : Buffers = −0.01 (95% CrI, −0.21 to 0.20)). There was no evidence of a moderating effect of [training status] (β0.5; Trained : Nontrained = 0.00 (95% CrI, −0.17 to 0.17)), [exercise type] (β0.5; Performance : Capacity = 0.00 (95% CrI, −0.21 to 0.20)), or [exercise duration] (β0.5; < 30s : 30s − 10min = 0.01 (95% CrI, −0.21 to 0.22); β0.5; < 30s : + 10min = −0.01 (95% CrI, −0.26 to 0.25)). Finally, there was some evidence of an effect of year of publication, with greater effects reported in studies published before 2000 than after (β0.5; < 2000 : > 2000 = −0.10 (95% CrI, −0.27 to 0.05)). All coefficients for meta-regressions are presented in Table 2.

F3
FIGURE 3:
Forest plot of the effect sizes for the included buffering supplements and caffeine studies. Panel A shows effects sizes of placebo vs control, and panel B shows effect sizes of the active intervention vs control. Results from individual studies represent shrunken estimates based on the random-effects model fitting and borrowing of information across studies to reduce uncertainty. Circles represent the pooled estimate from individual studies and across studies (average), generated with Bayesian inference along with the 95% CrI. Positive values favor the noncontrol condition. Included studies investigating caffeine are indicated with an asterisk (*).
TABLE 2 - Moderator analyses conducted for placebo and supplement effect sizes.
Moderator Placebo vs Nonplacebo, Parameter Estimate (95% CrI) Supplement vs Nonplacebo, Parameter Estimate (95% CrI)
[Supplement type]
 Caffeine (n = 11) 0.06 [−0.11 to 0.23] 0.13 (−0.14 to 0.44]
 Buffers (n = 45) 0.12 (0.01 to 0.20) 0.46 (0.26 to 0.68)
[Supplement delivery]
 Capsules (n = 13) 0.02 (−0.12 to 0.16) 0.33 (0.04 to 0.66)
 Solution (n = 41) 0.13 (0.03 to 0.23) 0.43 (0.23 to 0.70)
[Training status]
 Untrained (n = 25) 0.09 (−0.04 to 0.22) 0.46 (0.16 to 0.81)
 Trained (n = 31) 0.09 (0.03 to 0.23) 0.33 (0.13 to 0.57)
[Exercise duration]
 <30 s (n = 8) 0.08 (−0.12 to 0.28) 0.15 (−0.20 to 0.52)
 30 s–10min (n = 34) 0.09 (−0.02 to 0.20) 0.46 (0.25 to 0.20)
 +10 min (n = 14) 0.07 (−0.08 to 0.21) 0.31 (−0.01 to 0.68)
[Exercise type]
 Capacity (n = 10) 0.09 (−0.09 to 0.27) 0.33 (0.01 to 0.71)
 Performance (n = 46) 0.09 (0.00 to 0.17) 0.38 (0.19 to 0.61)
[Study year]
 Before 2000 (n = 34) 0.14 (0.02 to 0.25) 0.58 (0.35 to 0.89)
 After 2000 (n = 22) 0.03 (−0.08 to 0.15) 0.18 (−0.02 to 0.40)
n, number of outcomes for factor level.

Intervention effects (active intervention vs placebo and vs nonplacebo control)

The intervention effect was initially investigated by pooling the pairwise nonplacebo control effect sizes across studies investigating both caffeine and buffering supplements. The pooled effect size indicated a small to medium effect (ES0.5 = 0.37 (95% CrI, 0.20 to 0.56); τ0.5 = 0.25 (0.04 to 0.48); ICC = 0.33 (0 to 0.69)); Fig. 3B) compared with nonplacebo control (Figure, Supplemental Digital Content 4, Plot including effect sizes prior to creation of shrunken estimates, https://links.lww.com/MSS/C279). The probability that the pooled effect size represented at least a small effect was P = 0.976, with the probability of at least a medium or large effect equal to P = 0.076 and P < 0.001, respectively. There was strong evidence of a differential effect between supplements (β0.5; Caffeine : Buffers = 0.33 (95% CrI, −0.03 to 0.68)), with buffering supplements (ES0.5;Buffers = 0.46 (95% CrI, 0.26 to 0.68)) producing substantially larger effects than caffeine (ES0.5;Caffeine = 0.13 (95% CrI, −0.14 to 0.44)). The median effect of intervention was reduced when compared with placebo (as opposed to vs nonplacebo control), although it remained between small and medium (ES0.5 = 0.30 (0.13 to 0.48)). All coefficients for meta-regressions are presented in Table 2.

Proportion of total effect due to placebo effects

Given the differences in placebo effects and ergogenic effects, further analyses were completed with each supplement separately to estimate the percentage of each supplement effect that could be explained by placebo. Separating the effect size estimates according to supplement resulted in positive skews between supplementation type. Although larger absolute placebo effects were shown for buffers, the percentage of the effect that could be explained by placebo was estimated to be 25% (75% CrI, 16% to 35%). For the smaller supplement effect of caffeine, the percentage of the effect that could be explained by placebo was estimated to be 59% (75% CrI, 34% to 94%). Plots to visualize the relative placebo and intervention effects for each study are contained within supplemental figures (see Figures, Supplemental Digital Content 5 for sodium bicarbonate, https://links.lww.com/MSS/C280, and Supplemental Digital Content 6 for caffeine, https://links.lww.com/MSS/C281). A final sensitivity analysis was completed by removing the caffeine mouth rinse data and completing for ingested caffeine effects alone. The estimated effect for caffeine increased slightly (ES0.5caffeine = 0.27 (95% CrI, −0.15 to 0.83)), and as a result, the estimated percentage of the effect that could be explained by placebo reduced to 49% (75% CrI, 30% to 77%). Visual inspection of funnel plots from both placebo and intervention effect sizes identified no clear signs of asymmetry and therefore of small study effects such as publication bias (Fig. 4).

F4
FIGURE 4:
Funnel plot of intervention (A) and placebo (B) effect sizes for the included buffering supplements and caffeine studies.

DISCUSSION

The results of this meta-analysis of studies involving caffeine and buffering supplements showed that there is a very small but significant placebo effect on exercise performance, which means that the real-world effect of these supplements is likely to be somewhat larger than commonly indicated by the scientific literature. There was evidence that the magnitude of the placebo effect may be influenced by supplement form with greater effects obtained when the placebo was presented as a solution compared with a capsule. The size of the placebo effect was very consistent across the ergogenic agents investigated, and although the absolute magnitude of the reported effect was very small, it was substantive when expressed relative to the overall ergogenic effect of the supplements. This was particularly apparent for the lower ergogenic effect of caffeine. These data confirm the notion that the use of a nonplacebo control arm is influential in describing the total effect of a given ergogenic aid, which, in the real world, is the net addition of the active intervention and its placebo effects.

The current novel data showed that double-blind placebo-controlled studies with ergogenic supplements such as caffeine and buffers result in very small (ES0.5 = 0.09 (95% CrI, 0.01 to 0.17)) but real placebo effects on exercise outcomes. There was some evidence that the absolute magnitude of the placebo effect (placebo trial vs nonplacebo control trial) was different between studies investigating buffering supplements and caffeine, which confirms previous research showing that the size of placebo effects differs between ergogenic supplements (2); however, the form in which the supplement was provided seems more important. Not all placebos and placebo effects are created equally, and different characteristics such as color, taste, and administration form may interfere with the placebo effects attributed to an intervention (14–16). Placebo effects here were greater when supplements were provided in a solution as opposed to in capsules. To the authors’ knowledge, these are the first data to suggest that placebo effects in sport may depend on the form in which the supplements are administered. The reasons for these disparate effects are unclear but may relate to their taste because buffering and caffeine supplements dissolved in solution have distinct and strong flavors. Their respective placebo solutions are generally taste-matched to render indistinguishable any differences in flavor between the two solutions; however, the property of taste itself may inadvertently lead to ergogenic effects because of the interaction with sweet and bitter taste receptors found in the mouth (59). The current results showed that placebo effects did not differ according to exercise capacity or performance tests, or between different exercise durations. Participants were separated according to training status because we speculated that differences in personality traits and placebo expectations between trained and nontrained individuals might lead to differential placebo effects. However, the magnitude of the placebo effect did not differ between trained and nontrained individuals. Had more detailed information relating to specific personality traits been collected, this might have rendered different and clearer insights. There was also some evidence that studies published before the year 2000 had larger placebo effects, this perhaps being indicative of improved blinding methods in more recent studies.

The size of the absolute placebo effects shown here is smaller than the small to moderate placebo effects on exercise outcomes reported previously (2). It is important to highlight, however, that the previous systematic review examined the magnitude of the placebo effect elicited by direct experimental manipulation wherein individuals were specifically led to believe that they were provided with an active treatment (2). All studies included herein consisted of double-blind randomized crossover studies in which individuals unknowingly received an active or placebo intervention, meaning any prior expectation of the participants will have been due to their own individual beliefs, and given these double-blinded conditions, it seems likely that mixed expectations (positive, negative, or neutral) will have been present (as per Saunders et al. [5]). An important limitation here is that we cannot make generalizations regarding what participants believed they were ingesting during placebo or active trials. This is important because the psychosocial context of ingesting a substance, as well as verbal cues, expectancy, positivity, beliefs, and preconditioning may elicit neurobiological effects, activating specific brain pathways that can affect subsequent exercise performance (1,60,61). We have previously shown that knowingly taking an active supplement can lead to further exercise improvements above the mean of the intervention, whereas knowingly taking a placebo can impair performance (5). Interestingly, openly provided placebos can improve exercise performance (62), although the beneficial effects seem related to the psychosocial context in which the placebo is provided (63). It is likely that when individuals believe they are taking a placebo in a double-blind context, the connotations are more negative because they know that they could have been taking the active supplement. Similarly, an individual who believes he/she is taking a substance that can improve performance will likely have more positive reinforcement. Overall, we showed very small placebo effects associated with double-blind crossover supplementation studies, but manipulating an individual’s expectation of the supplement may increase or decrease this effect, causing further placebo (i.e., beneficial) or even nocebo (i.e., detrimental) effects (2).

The percentage of the overall effect of the intervention (active intervention vs nonplacebo control) that can be attributed to placebo effects was high and was estimated to be higher for caffeine than for the extracellular buffers (59% vs 25%). Although the absolute magnitude of the placebo effect with caffeine was slightly lower than for buffers, the larger contribution to the total effect was due to the substantially lower overall effect of caffeine supplementation in the studies included in the current meta-analysis. Removing caffeine mouth rinse studies increased the overall effect of caffeine and subsequently reduced the point estimate of the percentage explained by placebo effects from 59% to 49%. Effect sizes estimated for caffeine research on aerobic exercise (0.22–0.61) are generally higher than those for anaerobic exercise (0.16–0.20) (64). Because placebo responses were consistent across different exercise modes and duration, it could be speculated that the proportion of the overall response attributed to placebo effects may be lower for endurance versus anaerobic exercise with caffeine ingestion. This would be due to caffeine’s greater overall effect on endurance performance (64), whereas placebo effects would accordingly likely comprise a very substantial part of caffeine’s ergogenic effect on short-duration anaerobic exercise because the overall effects are smaller. This also means that these meta-analytical estimates (64), which were compared with placebo, are likely to be even higher had they been compared with a nonplacebo control.

Our searches revealed a paucity of studies that incorporated nonplacebo controls (11 caffeine and 24 buffer studies from over 200 original articles for each). Although the absolute placebo effects shown here were very small, they could potentially lead to substantial differences upon repeat exposure, such as during a training plus supplementation study. Considering the current data, where feasible we recommend that double-blind placebo-controlled studies should include a nonplacebo control session to quantify the placebo effect associated with the treatment under examination and calculate the entire effect associated with the active intervention plus placebo effects. This recommendation is in line with the consensus statement on placebo effects in sports and exercise to adopt methods that aim to quantify placebo effects that could explain some of the interindividual variability seen in response to interventions (65). However, it is acknowledged that this may not always be feasible, as the inclusion of an additional nonplacebo control may substantially add to the complexity and cost of designs, in particular those involving chronic supplementation strategies (e.g., creatine or β-alanine), which would require an entire extra nonplacebo control group. Participants should be questioned as to what intervention they believed they had taken to account for any effects of expectation on performance (5), and this is of particular importance in studies where a three-arm trial is not possible.

Caffeine and buffering supplementation studies were chosen here because of the well-known ergogenic and placebo effects associated with their ingestion (4,5,7), but these only served as a model, and results could be extrapolated to other effective nutritional supplements (3) or any alternative intervention or interaction that might elicit placebo effects (1). For example, the largest placebo effects of nutritional ergogenic aids on exercise performance were shown when individuals believed they were receiving banned performance enhancers like anabolic steroids (2). Placebo effects could also differ depending on the general and recognized efficacy of the supplement under investigation (e.g., greater effects with known effective supplements) or with supplements with clear and obvious side-effects (e.g., easier to determine when ingesting the placebo). It is unclear if placebo effects would be different for supplements that are considered less or ineffective, but the contribution of placebo effects to their overall effects would likely be greater than those shown here and may even represent their entire effect, although this is somewhat speculative. Thus, the proportion of the placebo contribution to the overall effect will likely depend on a multitude of factors including the exercise protocol, belief, expectation, preconditioning, and the intervention itself. Also, studies have shown that certain personality traits might influence the placebo response (24,25), including supplement use and beliefs. Further work should elucidate how much each of these factors can contribute to the placebo effect associated with double-blind placebo-controlled research designs.

The current meta-analysis showed that there is a real placebo effect associated with double-blind crossover studies involving caffeine and extracellular buffers, and these effects are greater when supplements are provided in solution compared with capsules. Although it does not change the statistically significant effect of these ergogenic aids (i.e. their effects vs placebo remain significant), these placebo effects modify the overall size attributed to the intervention and accounts for a large proportion of their total effects. Coaches and practitioners should be aware that actual ergogenic effects with these supplements are likely greater than those shown in placebo-controlled studies because in the real world, the net effect of the physiological effects of active interventions and their placebo effects are additive. Practitioners should also consider taking advantage of these placebo effects inherent to effective interventions by instilling positive belief in them to optimize training adaptations and performance outcomes. Nutritional supplementation studies should strive to use a nonplacebo control arm where possible as a comparator to measure the overall effect of the intervention (physiological and psychobiological) and identify the proportion of this effect that can be estimated to come from placebo effects. This approach will allow for the determination of the net additive effect of a given active intervention and its placebo effects, allowing practitioners and coaches to base their prescription on the totality of evidence for a real-world scenario.

No specific funding was received for writing this review. Felipe Miguel Marticorena (2019/20614-0), Eimear Dolan (2019/05616 and 2019/26899-6), and Bryan Saunders (2016/50438-0) are financially supported by Fundação de Amparo à Pesquisa do Estado de São Paulo. Bryan Saunders has received a grant from Faculdade de Medicina da Universidade de São Paulo (2020.1.362.5.2). This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior—Brasil (CAPES)—Finance Code 001.

The authors declare no conflict of interest. The results of the study are presented clearly, honestly, and without fabrication, falsification, or inappropriate data manipulation. The present study does not constitute endorsement by the American College of Sports Medicine.

B. S. is responsible for the conception of the work. F. M. M., L. F. O., and E. D. performed the searches. F. M. M., A. C., L. F. O., and E. D. performed the screening, and F. M. M. and L. F. O. performed the data extraction. P. S. performed the meta-analysis. F. M. M. and B. S. are responsible for the initial writing of the manuscript. P. S. and B. G. helped interpret the data and revised the manuscript. All authors approved the final version of the manuscript.

REFERENCES

1. Davis AJ, Hettinga F, Beedie C. You don’t need to administer a placebo to elicit a placebo effect: social factors trigger neurobiological pathways to enhance sports performance. Eur J Sport Sci. 2020;20:302–12.
2. Hurst P, Schipof-Godart L, Szabo A, et al. The placebo and nocebo effect on sports performance: a systematic review. Eur J Sport Sci. 2020;20:279–92.
3. Maughan RJ, Burke LM, Dvorak J, et al. IOC consensus statement: dietary supplements and the high-performance athlete. Br J Sports Med. 2018;52(7):439–55.
4. Beedie CJ, Stuart EM, Coleman DA, Foad AJ. Placebo effects of caffeine on cycling performance. Med Sci Sports Exerc. 2006;38(12):2159–64.
5. Saunders B, de Oliveira LF, da Silva RP, et al. Placebo in sports nutrition: a proof-of-principle study involving caffeine supplementation. Scand J Med Sci Sports. 2017;27(11):1240–7.
6. Foad AJ, Beedie CJ, Coleman DA. Pharmacological and psychological effects of caffeine ingestion in 40-km cycling performance. Med Sci Sports Exerc. 2008;40(1):158–65.
7. McClung M, Collins D. “Because I know it will!”: placebo effects of an ergogenic aid on athletic performance. J Sport Exerc Psychol. 2007;29(3):382–94.
8. Hurst P, Schipof-Godart L, Hettinga F, Roelands B, Beedie C. Improved 1000-m running performance and pacing strategy with caffeine and placebo: a balanced placebo design study. Int J Sports Physiol Perform. 2019;1–6.
9. Moher D, Liberati A, Tetzlaff J, Altman DG; Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Open Med. 2009;3(3):e123–30.
10. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):210.
11. Saunders B, Elliott-Sale K, Artioli GG, et al. β-Alanine supplementation to improve exercise capacity and performance: a systematic review and meta-analysis. Br J Sports Med. 2017;51(8):658–69.
12. Hobson RM, Saunders B, Ball G, Harris R, Sale C. Effects of beta-alanine supplementation on exercise performance: a meta-analysis. Med Sci Sport Exer. 2012;44:446.
13. Bérdi M, Köteles F, Hevesi K, Bárdos G, Szabo A. Elite athletes’ attitudes towards the use of placebo-induced performance enhancement in sports. Eur J Sport Sci. 2015;15(4):315–21.
14. Kong J, Spaeth R, Cook A, et al. Are all placebo effects equal? Placebo pills, sham acupuncture, cue conditioning and their association. PloS one. 2013;8(7):e67485.
15. de Craen AJ, Tijssen JG, de Gans J, Kleijnen J. Placebo effect in the acute treatment of migraine: subcutaneous placebos are better than oral placebos. J Neurol. 2000;247(3):183–8.
16. de Craen AJ, Roos PJ, de Vries AL, Kleijnen J. Effect of colour of drugs: systematic review of perceived effect of drugs and of their effectiveness. BMJ. 1996;313(7072):1624–6.
17. Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898.
18. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998;52(6):377–84.
19. Morris SB, DeShon RP. Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychol Methods. 2002;7(1):105–25.
20. Cox DR. Principles of Statistical Inference. Cambridge: Cambridge University Press; 2006.
21. Fernandez-Castilla B, Jamshidi L, Declercq L, Beretvas SN, Onghena P, Van den Noortgate W. The application of meta-analytic (multi-level) models with multiple random effects: a systematic review. Behav Res Methods. 2020;52:2031–52.
22. Sawilowsky SS. New effect size rules of thumb. J Mod Appl Stat Method. 2009;8(2):597–9.
23. Bürkner PC. brms: an R package for Bayesian multilevel models using Stan. J Stat Softw. 2017;80(1):28.
24. Gelman A. Prior distributions for variance parameters in hierarchical models. Bayesian Analysis. 2006;1(3):515–34.
25. Gelman A, Carlin JB, Stern HS, Rubin DB. Bayesian Data Analysis. Abingdon (UK): Taylor & Francis; 2014.
26. Bridge CA, Jones MA. The effect of caffeine ingestion on 8 km run performance in a field setting. J Sports Sci. 2006;24(4):433–9.
27. Clarke N, Baxter H, Fajemilua E. Coffee and caffeine ingestion have little effect on repeated sprint cycling in relatively untrained males. Sports (Basel). 2016;4(3):45.
    28. Dolan P, Witherbee KE, Peterson KM, Kerksick CM. Effect of carbohydrate, caffeine, and carbohydrate + caffeine mouth rinsing on intermittent running performance in collegiate male lacrosse athletes. J Strength Cond Res. 2017;31(9):2473–9.
    29. Flinn S, Gregory J, McNaughton LR, Tristram S, Davies P. Caffeine ingestion prior to incremental cycling to exhaustion in recreational cyclists. Int J Sports Med. 1990;11(3):188–93.
    30. Karayiğit R, Yaşli BÇ, Karabiyik H, Koz M, Ersöz G. Effect of serial caffeine mouth rinse on Wingate anaerobic performance. SPORMETRE. 2017;15(4):191–6.
      31. Mc Naughton LR, Lovell RJ, Siegler JC, Midgley AW, Sandstrom M, Bentley DJ. The effects of caffeine ingestion on time trial cycling performance. J Sports Med Phys Fitness. 2008;48(3):320–5.
      32. Wiles JD, Coleman D, Tegerdine M, Swaine IL. The effects of caffeine ingestion on performance time, speed and power during a laboratory-based 1 km cycling time-trial. J Sports Sci. 2006;24(11):1165–71.
      33. Rezaei S, Akbari K, Gahreman DE, et al. Caffeine and sodium bicarbonate supplementation alone or together improve karate performance. J Int Soc Sports Nutr. 2019;16(1):44.
        34. Grgic J, Venier S, Mikulic P. Both caffeine and placebo improve vertical jump performance compared with a nonsupplemented control condition. Int J Sports Physiol Perform. 2020;1–4.
          35. Bird SR, Wiles J, Robbins J. The effect of sodium bicarbonate ingestion on 1500-m racing time. J Sports Sci. 1995;13(5):399–403.
          36. Carr AJ, Slater GJ, Gore CJ, Dawson B, Burke LM. Reliability and effect of sodium bicarbonate: buffering and 2000-m rowing performance. Int J Sports Physiol Perform. 2012;7(2):152–60.
            37. Coombes J, McNaughton LR. Effects of bicarbonate ingestion on leg strength and power during isokinetic knee flexion and extension. J Strength Cond Res. 1993;7(4):241–9.
            38. Coppoolse R, Barstow TJ, Stringer WW, Carithers E, Casaburi R. Effect of acute bicarbonate administration on exercise responses of COPD patients. Med Sci Sports Exerc. 1997;29(6):725–32.
            39. Goldfinch J, Mc Naughton L, Davies P. Induced metabolic alkalosis and its effects on 400-m racing time. Eur J Appl Physiol Occup Physiol. 1988;57(1):45–8.
            40. Griffen C, Rogerson D, Ranchordas M, Ruddock A. Effects of creatine and sodium bicarbonate coingestion on multiple indices of mechanical power output during repeated Wingate tests in trained men. Int J Sport Nutr Exerc Metab. 2015;25(3):298–306.
            41. Lindh AM, Peyrebrune MC, Ingham SA, Bailey DM, Folland JP. Sodium bicarbonate improves swimming performance. Int J Sports Med. 2008;29(6):519–23.
            42. Materko W, Santos E, Novaes JS. Effect of bicarbonate supplementation on the muscular strength. Med Sci Sport Exer. 2008;11(5):25–33.
              43. McLellan T, Jacobs I, Lewis W. Acute altitude exposure and altered acid–base states. II. Effects on exercise performance and muscle and blood lactate. Eur J Appl Physiol Occup Physiol. 1988;57(4):445–51.
              44. McNaughton LR. Sodium citrate and anaerobic performance: implications of dosage. Eur J Appl Physiol Occup Physiol. 1990;61(5–6):392–7.
              45. McNaughton L, Curtin R, Goodman G, Perry D, Turner B, Showell C. Anaerobic work and power output during cycle ergometer exercise: effects of bicarbonate loading. J Sports Sci. 1991;9(2):151–60.
              46. McNaughton LR. Bicarbonate ingestion: effects of dosage on 60 s cycle ergometry. J Sports Sci. 1992;10(5):415–23.
              47. McNaughton LR. Sodium bicarbonate ingestion and its effects on anaerobic exercise of various durations. J Sports Sci. 1992;10(5):425–35.
              48. McNaughton L, Cedaro R. Sodium citrate ingestion and its effects on maximal anaerobic exercise of different durations. Eur J Appl Physiol Occup Physiol. 1992;64(1):36–41.
              49. McNaughton LR, Ford S, Newbold C. Effect of sodium bicarbonate ingestion on high intensity exercise in moderately trained women. J Strength Cond Res. 1997;11(2):98–102.
              50. McNaughton L, Dalton B, Palmer G. Sodium bicarbonate can be used as an ergogenic aid in high-intensity, competitive cycle ergometry of 1 h duration. Eur J Appl Physiol Occup Physiol. 1999;80(1):64–9.
              51. Miller P, Robinson AL, Sparks SA, Bridge CA, Bentley DJ, McNaughton LR. The effects of novel ingestion of sodium bicarbonate on repeated sprint ability. J Strength Cond Res. 2016;30(2):561–8.
              52. Morris DM, Shafer RS, Fairbrother KR, Woodall MW. Effects of lactate consumption on blood bicarbonate levels and performance during high-intensity exercise. Int J Sport Nutr Exerc Metab. 2011;21(4):311–7.
              53. Oliveira LF, de Salles Painelli V, Nemezio K, et al. Chronic lactate supplementation does not improve blood buffering capacity and repeated high-intensity exercise. Scand J Med Sci Sports. 2017;27(11):1231–9.
              54. Painelli Vde S, Roschel H, Jesus Fd, et al. The ergogenic effect of beta-alanine combined with sodium bicarbonate on high-intensity swimming performance. Appl Physiol Nutr Metab. 2013;38(5):525–32.
                55. Pierce EF, Eastman NW, Hammer WH, Lynn TD. Effect of induced alkalosis on swimming time trials. J Sports Sci. 1992;10(3):255–9.
                56. Tiryaki GR, Atterbom HA. The effects of sodium bicarbonate and sodium citrate on 600 m running time of trained females. J Sports Med Phys Fitness. 1995;35(3):194–8.
                57. Wilkes D, Gledhill N, Smyth R. Effect of acute induced metabolic alkalosis on 800-m racing time. Med Sci Sports Exerc. 1983;15(4):277–80.
                58. McGuinness LA, Higgins JPT. Risk-of-bias VISualization (robvis): an R package and shiny web app for visualizing risk-of-bias assessments. Res Synth Methods. 2021;12:55–61.
                  59. Best R, McDonald K, Hurst P, Pickering C. Can taste be ergogenic?Eur J Nutr. 2020;60:45–54.
                  60. Lundh LG. Placebo, belief, and health. A cognitive–emotion model. Scand J Psychol. 1987;28(2):128–43.
                  61. Price DD, Finniss DG, Benedetti F. A comprehensive review of the placebo effect: recent advances and current thought. Annu Rev Psychol. 2008;59:565–90.
                  62. Saunders B, Saito T, Klosterhoff R, et al. “I put it in my head that the supplement would help me”: open-placebo improves exercise performance in female cyclists. PLoS One. 2019;14(9):e0222982.
                  63. Saito T, Barreto G, Saunders B, Gualano B. Is open-label placebo a new ergogenic aid? A commentary on existing studies and guidelines for future research. Sports Med. 2020;50:1225–9.
                  64. Grgic J, Grgic I, Pickering C, Schoenfeld BJ, Bishop DJ, Pedisic Z. Wake up and smell the coffee: caffeine supplementation and exercise performance-an umbrella review of 21 published meta-analyses. Br J Sports Med. 2020;54:681–8.
                  65. Beedie C, Benedetti F, Barbiani D, et al. Consensus statement on placebo effects in sports and exercise: the need for conceptual clarity, methodological rigour, and the elucidation of neurobiological mechanisms. Eur J Sport Sci. 2018;18(10):1383–9.
                  Keywords:

                  EXERCISE; NUTRITION; PHYSIOLOGY; BELIEF; EXPECTATION; NEUROBIOLOGICAL

                  Supplemental Digital Content

                  Copyright © 2021 by the American College of Sports Medicine