The primary aim of sport scientists working with elite athletes is to assess the effects of training and nutritional or other treatments on performance. Several mathematical models have been suggested for analyzing effects of treatments on performance (2,4,11,17,28,29). Repeated-measures ANOVA is a commonly used method, but it can lead to loss of power when there are missing values in a series of repeated measurements: either the entire trial with a missing value has to be deleted or the entire series of values of each subject with a missing value has to be deleted. A better approach is mixed modeling, which overcomes the missing value problem and, in addition, allows specification and estimation of different sources of variation or error (30). For example, in tracking performance using training and competition time trials, performance could be more variable in training.
In this article, we report an analysis with mixed modeling in which we have devised a novel coding method to account for various factors that could affect performance. We monitored performance in a squad of elite swimmers preparing for Olympic-qualifying trials and assessed changes in performance arising from training versus competition, morning versus evening swims, and with use of caffeine or placebo.
Nine highly trained swimmers (age range = 21-26 yr) competing at the international level and specializing in 400-m freestyle (n = 1), 100-m backstroke (n = 1), 200-m backstroke (n = 2), 100-m butterfly (n = 1), 200-m butterfly (n = 2), 100-m breaststroke (n = 1), or 400-m individual medley (n = 1) took part in this study. Subjects' characteristics are shown in Table 1. The swimmers were performing a similar training program consisting of two 2-h swim sessions each day, except for Wednesday and Saturday (morning session only) and Sunday (no session). Sessions started at 6:30 a.m. and 4 p.m. All subjects gave written informed consent as required by the AUT University Ethics Committee, which approved this study.
The swimmers performed 2-8 time trials in training and 2-7 in competition in the 9 wk before and including Olympic-qualifying trials; 0-4 time trials were performed in the morning and 4-10 were performed in the evening. Morning time trials were performed between 9 and 11:30 a.m., and evening time trials were performed between 5 and 8 p.m. All trials were performed in the stroke and distance of each swimmer's main event, with a standardized individualized competition warm-up, leading-in music, and cheering from fellow athletes. The swimmers used a race swimsuit of the same size and brand in all trials. The warm-up consisted of stretching, 1-2 km of swimming, and mental preparation. We monitored performance times and use of caffeine in these time trials. All trials were performed in a 50-m pool with a water temperature of 27°C and an ambient temperature of 24°C. Data for one elite swimmer are shown in Figure 1.
The athletes consumed caffeine of varying doses in some time trials and a fixed dose (5 mg·kg−1 body mass) or placebo in a double-blind, diet-controlled crossover manner in two training time trials ≤2 wk apart. The order of treatment in the crossover was randomized to balance gender and distance of the main competitive event. The caffeine was consumed in tablet form except for the crossover intervention, in which the swimmers ingested capsules containing either placebo (custard powder; Hansells, Auckland, New Zealand) or crushed 100-mg caffeine tablets (No-Doz; Key Pharmaceuticals, Sydney, Australia). These capsules were ingested with lemon-flavored water and were consumed blindfolded, even though the custard powder and caffeine had similar color and texture. The caffeine and placebo were always consumed 75 min before the time trial.
We standardized diet only for the two time trials in the caffeine crossover intervention. All subjects completed a 36-h diet diary before the first time trial of this intervention and repeated this diet for the second time trial. The athletes were also instructed to refrain from caffeine and alcohol for 3 d before these two time trials. Only one swimmer consumed caffeine on a regular basis.
Focus, sleep quality, and perception of having consumed caffeine.
The athletes completed a questionnaire the morning after each time trial part of the caffeine crossover intervention. They were asked to rate pre-time-trial focus and sleep quality between 0 and 100 on a 21-point scale anchored throughout with descriptors ("0, no focus at all" through "100, extremely focused" and "0, no sleep" through "100, perfect sleep," respectively). This scale was adopted from Ritchie and Hopkins (26). The questionnaire was also used to record the time that the athletes went to bed, the time required to fall asleep, the duration of sleep, and the number of times of waking up. The swimmers were also asked to rate their perception of having consumed caffeine or placebo as follows: "almost certainly not caffeine," "probably not caffeine," "unsure," "probably caffeine," or "almost certainly caffeine"; we coded this variable from 0 to 4, respectively. We also assigned this variable a value of 0 for all swims where no caffeine was consumed knowingly and a value of 4 for swims when caffeine was consumed knowingly. We then calculated the placebo effect as the difference between the effect on performance of a perception of 4 and 0 (all other effects held constant).
We used the mixed-model procedure (Proc Mixed) in the Statistical Analysis System (Version 9.2; SAS Institute, Cary, NC) to quantify the changes in performance that occurred in training versus competition, in morning versus evening swims, and with use of caffeine and placebo. Predictor variables were coded as 0 or 1 to represent absence or presence of a condition and were included in the model as fixed effects. Caffeine dose was included in the model as the number of capsules ingested. We did not include an interaction between caffeine dose and gender because a meta-analysis of the effects of caffeine on performance showed a trivial effect of gender (Hopkins, unpublished observations). Date of each performance test and caffeine (present or absent) were included as numeric linear fixed effects to estimate their mean effects. We also interacted these variables with the random effect for athlete to estimate individual differences in their effects. One residual variance was specified for the training time trials and another for the competition time trials, and the square roots of these variances were interpreted as the typical errors of the performance test to interpret the reliability of the test under the conditions of the study. We allowed for negative variance to estimate these individual differences because treatment effects, training effects, and sampling variation could reduce variability in performance. Confidence limits produced by the mixed-model procedure for the SD representing individual differences are only approximate with our small sample size. A plot of residuals from the analysis versus date was examined to assess the appropriateness of the use of a linear effect for date.
To investigate the effect of caffeine on longer versus shorter swims, the duration of the time trial was included in the model as a log-transformed value interacted with caffeine (presence or absence) and the number of caffeine capsules. To avoid a problem with collinearity, the duration chosen was that for a swim that would merit 900 points on the Point Scoring of the Fédération Internationale de Natation. The effect of time-trial duration on the effect of caffeine dose was estimated as the change in the effect of caffeine for a doubling of time-trial time. We opted for a factor effect of time-trial duration rather than a linear effect because a linear effect would imply the same change in the effect of caffeine on performance per minute of exercise, whether the exercise lasted 5 min or 1 h. A doubling of time-trial time implies the same change in the effect of caffeine with an increase in time-trial duration from 5 to 10 min as for 1 to 2 h, which is more realistic.
We used an Excel spreadsheet for crossovers (19) to determine the changes in measures derived from the focus and sleep questionnaire. For these variables, means and between-subject SD were derived from the raw values of the measures; and errors of measurement were calculated in the spreadsheet as the SD of change scores divided by √2. An intraclass (retest) correlation was calculated from errors of measurement using the following formula: (SD2 − error2)/SD2. For the reliability of measurement in the performance test, we calculated the typical error expressed as a percentage of the subject's mean score.
To make inferences about true (population) values of an effect, the uncertainty in the effect was expressed as 90% confidence limits and as likelihoods that the true value of the effect represents substantial change (harm or benefit) (5). An estimate of the smallest substantial change in a given dependent variable was required to make these inferences. The threshold change in performance for benefit and harm was established as 0.24% (0.3× within-athlete race-to-race variability in performance of 0.8%); the thresholds for moderate and large effects were 0.72% and 1.3%, respectively (20). For the dependent variables derived from the questionnaire, we assumed that the smallest substantial change was a standardized change of 0.2 of the between-subject SD of all (placebo and caffeine) values; the thresholds for moderate and large effects were assumed to be 0.6 and 1.2 of the between-subject SD, respectively (20).
Magnitude-based inferences were categorized as clinical for performance measures and mechanistic for other measures (20). With clinical inferences, an effect with possible benefit (>25% chance) was clear if harm was very unlikely (odds ratio of benefit/harm > 66) and unclear; otherwise, other effects were clearly not beneficial. With mechanistic inferences, an effect was deemed unclear if its 90% confidence interval (90% CI) overlapped thresholds for substantiveness (i.e., if the effect could be substantially positive and negative); other effects were clear.
Performance times were highly reliable in training and competition (typical errors of 0.9% and 0.8%, respectively). The plot of residuals versus date of the performance test showed a uniform scatter that justified the use of a linear effect of date on performance. Performance time improved by 0.8% (qualitative outcome moderate; 90% CI = 0.4%-1.2%) per 4 wk of training, with individual differences (SD) in the trend of 0.3% (unclear; 90% CI = −0.6% to 0.4%) per 4 wk. The swimmers performed better in evenings versus mornings by 0.6% (small; 90% CI = 0.1%-1.0%) and in competition versus training by 1.4% (large; 90% CI = 0.9%-1.9%).
A 100-mg dose of caffeine enhanced performance in 1-min training and competition time trials by 1.3% (large; 90% CI = −0.3% to 2.8% and 0.1% to 2.6%, respectively); each additional 100 mg reduced the benefit slightly by 0.1% (unclear; 90% CI = −0.5% to 0.3%). Only one swimmer (the habitual caffeine user) experienced an increase in swim time with caffeine in the crossover intervention, which is consistent with observed small individual differences (SD) in the effect of caffeine of 0.8% (unclear; 90% CI = −1.4% to 0.7%). The effect of a doubling of time-trial time on the effect of caffeine was −0.3% (unclear; 90% CI = −1.5% to 0.9%). The placebo effect was a slight improvement of 0.2% (unclear; 90% CI = −1.0% to 1.4%).
In the double-blind crossover intervention, most swimmers were able to identify consumption of caffeine: four swimmers were almost certain they had consumed caffeine, four swimmers perceived that they probably had consumed caffeine, and the one athlete who was a regular user of caffeine was almost certain of not having consumed caffeine. Reliability in measures derived from the questionnaire expressed as error of measurement (and retest correlation) was as follows: focus (0-100 scale) = ±6.7 (0.53), bedtime = ±67 min (−0.03), sleep quality (0-100 scale) = ±11 (0.52), duration of sleep = ±63 min (0.16), time required to fall asleep = ±17 min (0.49), and number of times of waking up = ±0.77 (0.47). Despite this poor reliability of measurement, we were able to identify some small effects with caffeine intake: an increase in focus (change in mean = 4.4, 90% confidence limits = ±8.5), a decrease in duration of sleep (change in mean = 47, 90% confidence limits = ±56), and an increase in time required to fall asleep (change in mean = 16, 90% confidence limits = ±15). Other measures were unclear.
We successfully used mixed linear modeling of elite swimmers' performance times to demonstrate and quantify individual trends in performance with training, better performance in evenings versus mornings, better performance in competition versus training time trials, and better performance with caffeine. We were able to quantify these changes in performance despite the small sample size because some of the effects were large and because there were multiple observations in each subject. The high reliability in the time trials used to monitor performance was another important reason: test-retest reliability matched measures in previously published findings on the reliability in competition performance in top athletes (25).
We modeled a linear effect for the date of the test because the data showed linear rather than nonlinear trends. This linear effect provided the coach with a measure of change in performance with training that was valuable to assess the training program overall and to identify individual responses with training. We have not attempted to identify which of the many outcomes of training (fitness, pacing, fatigue, technique, psychological state, etc.) were responsible for the improvement in performance. Wearing a new race swimsuit may have contributed to the performance increase in competition time trials. Tapering undoubtedly contributed to the substantial improvement in performance in competition time trials performed in the last week (7).
The performance increase with swimming in the evening versus the morning was substantial but smaller in magnitude than that in previous research (for reviews, see Atkinson and Reilly (3) and Drust et al. (15)). Baxter and Reilly (6) found improvements of 1.9% and 0.8% in the 100- and 400-m swim times, respectively, when the swims were performed at 5 p.m. versus those performed at 9 a.m. Deschodt and Arsac (13) reported a 3.6% improvement in swim time in 50-m time trials performed at 6 p.m. versus those performed at 9 a.m.. The diurnal effect on swim performance may have been smaller in the present study either because of the extensive warm-up (13) or because of the nature of the performance tests (competitions and competition simulations) or because the morning time trials were performed between 9 and 11:30 a.m. Indeed, Kline et al. (22) observed little change in performance in 200-m swim time trials performed at 5 p.m. versus those performed at 11 a.m. In any case, it would be sensible to include strategies for using this change in performance to the athletes' advantage; for example, training might be more effective in the afternoon, and using strategies for resynchronizing the circadian rhythm after transmeridional travel might be worth considering (15,22).
The swimmers performed better in competition than in training, although the training time trials were competition simulations. It is already well known that athletes perform better in competition because of higher arousal, competitive stress, less fatigue, and/or tapering for the competition event. The novelty of this project is that we were able to quantify the effect, which may be an aid for the coach to predict competition performance.
We found that caffeine ingested before a swim time trial substantially enhanced performance. Effects of caffeine on endurance performance are well established, but its effects on short-term endurance and sprinting in well-trained subjects are less clear (1,8,10,12,31). In particular, a recent conference abstract showed little effect of a low dose of caffeine (2 mg·kg−1 body mass) on 100-m swim performance in elite athletes (10). One of the reasons for this discrepancy with our findings may be the timing of caffeine intake: the swimmers in the present study ingested the caffeine 75 min before the time trials, whereas in the previous study, it was ingested 60 min before the time trials.
There have been few studies of the dose-response of caffeine on high-intensity, short-term performance in well-trained subjects (1,8). In comparing the effects of doses of 6 and 9 mg·kg−1 caffeine in a 2000-m rowing time trial lasting ∼7 min, Bruce et al. (8) found similar ergogenic effects in competitive oarsmen, whereas Anderson et al. (1) reported a greater effect with the higher dose of caffeine in competitive oarswomen. We observed that an intake of caffeine greater than 100 mg did not improve performance further, although this finding was unclear. Pending further research on the effect of dose of caffeine on swimming performance, swimmers should use a dose of ∼100 mg or ∼1.3 mg·kg−1.
The effect of a doubling of time-trial time on the effect of caffeine was unclear. We observed that the effect of caffeine was slightly decreased when the duration of the time trial was doubled, but we would need more observations to be confident.
Although the measures of focus and sleep were not obtained with a validated questionnaire, 5 mg caffeine per kilogram body mass increased focus before the time trial, which is consistent with previous research (18,27). We also measured the effect of caffeine on sleep after the time trial, a concern for elite athletes (9). Studies have shown that even low doses of caffeine decrease sleep quality (10,23). The present study confirms that 5 mg caffeine per kilogram body mass consumed ∼7 h before bedtime had a harmful effect on sleep. The implications for subsequent competition performance need further investigation. On the basis of our findings, swimmers and presumably other athletes in sports with similar performance times should use the relatively low dose of caffeine (100 mg) that we have found to be effective.
Caffeine improved performance in the crossover intervention in all but one athlete, who consumed caffeine on a regular basis. The effects of caffeine on performance may be attenuated in caffeine-habituated individuals (16). The athletes were instructed not to consume caffeine in the 3 d before each time trial, so either a 3-d abstinence is not sufficient to resensitize to caffeine or the athlete failed to comply. A check on compliance would have required testing for metabolites of caffeine in urine samples, which was beyond the scope of this study. Whether 3 d of abstinence is sufficient to resensitize to caffeine is unclear from published work, although a meta-analysis might provide an answer. It may therefore be worthwhile to investigate how long athletes need to refrain from caffeine for making maximum use of its ergogenic potential. Similarly, studies examining the interaction of caffeine with other proven ergogenic aids, such as carbohydrate (21), creatine (14), and bicarbonate (24), are important because this scenario is common in athletes.
There was considerable uncertainty in the estimates of individual responses to training and caffeine. Such uncertainty will be a problem in future applications using mixed modeling with a small number of subjects unless it is possible to obtain more observations for each subject. There are several other issues with the application of mixed modeling to monitor athletic performance. If the error of measurement is uniform between trials within athletes and between subgroups of athletes, it is possible to use the general linear model (or simply multiple linear regression). With either mixed or general linear modeling, the crucial step is the coding of the presence or absence of treatments or other factors affecting performance as predictors with values of 1 or 0. Mixed linear modeling is still required to obtain confidence limits for estimates of SD representing individual responses and individual differences, although, as previously noted, a considerable number of observations and/or large sample size are needed. Mixed and general linear modeling also allow modeling of trends as polynomials, when data show nonlinear trends. More complex curvature requires nonlinear mixed modeling, which, in the Statistical Analysis System, is a challenging procedure.
In summary, we successfully used mixed linear modeling of elite swimmers' performance times and a novel coding method to demonstrate and quantify:
- individual trends in performance with training and use of caffeine;
- a substantial performance increase in evenings versus mornings;
- a substantial performance increase in competition versus training time trials;
- a substantial performance increase with caffeine.
We were able to detect and quantify these changes in performance owing to the high reliability of the performance test, multiple observations in the season, and that some effects were large in magnitude.
The authors thank the swimmers and coaches for their contributions.
The authors have no professional relationship with a for-profit organization that would benefit from this study.
Publication does not constitute endorsement by the American College of Sports Medicine.
1. Anderson ME, Bruce CR, Fraser SF, et al. Improved 2000-meter rowing performance in competitive oarswomen after caffeine ingestion. Int J Sport Nutr Exerc Metab
2. Anderson ME, Hopkins WG, Roberts AD, Pyne DB. Monitoring seasonal and long-term changes in test performance in elite swimmers. Eur J Sport Sci
3. Atkinson G, Reilly T. Circadian variation in sports performance. Sports Med
4. Avalos M, Hellard P, Chatard JC. Modeling the training-performance relationship using a mixed model in elite swimmers. Med Sci Sports Exerc
5. Batterham AM, Hopkins WG. Making meaningful inferences about magnitudes. Int J Sports Physiol Perform
6. Baxter C, Reilly T. Influence of time of day on all-out swimming. Br J Sports Med
7. Bosquet L, Montpetit J, Arvisais D, Mujika I. Effects of tapering on performance: a meta-analysis. Med Sci Sports Exerc
8. Bruce CR, Anderson ME, Fraser SF, et al. Enhancement of 2000-m rowing performance after caffeine ingestion. Med Sci Sports Exerc
9. Burke LM. Caffeine and sports performance. Appl Physiol Nutr Metabol
10. Burke LM, Anderson ME, Pyne DB. Low dose caffeine intake and sprint performance in swimmers. Med Sci Sports Exerc
. 2006;38(5 suppl):S174.
11. Busso T, Thomas L. Using mathematical modeling in training planning. Int J Sports Physiol Perform
12. Collomp K, Ahmaidi S, Chatard JC, Audran M, Prefaut C. Benefits of caffeine ingestion on sprint performance in trained and untrained swimmers. Eur J Appl Physiol
13. Deschodt VJ, Arsac LM. Morning vs. evening maximal cycle power and technical swimming ability. J Strength Cond Res
14. Doherty M, Smith PM, Richard Davison RC, Hughes MG. Caffeine is ergogenic after supplementation of oral creatine monohydrate. Med Sci Sports Exerc
15. Drust B, Waterhouse J, Atkinson G, Edwards B, Reilly T. Circadian rhythms in sports performance-an update. Chronobiol Int
16. Fisher SM, McMurray RG, Berry M, Mar MH, Forsythe WA. Influence of caffeine on exercise performance in habitual caffeine users. Int J Sports Med
17. Hellard P, Avalos M, Lacoste L, Barale F, Chatard JC, Millet GP. Assessing the limitations of the banister model in monitoring training. J Sports Sci
18. Hoffman JR, Kang J, Ratamess NA, Hoffman MW, Tranchina CP, Faigenbaum AD. Examination of a pre-exercise, high energy supplement on exercise performance. J Int Soc Sports Nutr
19. Hopkins WG. Spreadsheets for analysis of controlled trials, with adjustment for a subject characteristic. Sportscience
20. Hopkins WG, Marshall SW, Batterham AM, Hanin J. Progressive statistics for studies in sports medicine and exercise science. Med Sci Sports Exerc
21. Hulston CJ, Jeukendrup AE. Substrate metabolism and exercise performance with caffeine and carbohydrate intake. Med Sci Sports Exerc
22. Kline CE, Durstine JL, Davis JM, et al. Circadian variation in swim performance. J Appl Physiol
23. Landolt HP, Werth E, Borbély AA, Dijk DJ. Caffeine intake (200 mg) in the morning affects human sleep and EEG power spectra at night. Brain Res
24. Pruscino CL, Ross ML, Gregory JR, Savage B, Flanagan TR. Effects of sodium bicarbonate, caffeine, and their combination on repeated 200-m freestyle performance. Int J Sport Nutr Exerc Metab
25. Pyne D, Trewin C, Hopkins W. Progression and variability of competitive performance of Olympic swimmers. J Sports Sci
26. Ritchie SE, Hopkins WG. The intensity of exercise in deep-water running. Int J Sports Med
27. Smith A. Effects of caffeine on human behavior. Food Chem Toxicol
28. Taha T, Thomas SG. Systems modelling of the relationship between training and performance. Sports Med
29. Thomas L, Mujika I, Busso T. A model study of optimal training reduction during pre-event taper in elite swimmers. J Sports Sci
30. Ugrinowitsch C, Fellingham GW, Ricard MD. Limitations of ordinary least squares models in analyzing repeated measures data. Med Sci Sports Exerc
31. Wiles JD, Coleman D, Tegerdine M, Swaine IL. The effects of caffeine ingestion on performance time, speed and power during a laboratory-based 1 km cycling time-trial. J Sports Sci