Heat illness prevention guidelines based on the hourly wet-bulb globe temperature (WBGT) index were first implemented by the U.S. military during the mid 1950s as a consequence of the high rates of heat illness that occurred during basic training at Marine Corps Recruit Depot, Parris Island (MCRD-PI), SC. These guidelines were founded on clinical research studies and are still the current standard used in the U.S. military today, with only minor modifications made over the past five decades (16,21 ). The WBGT index for the outdoor environment is computed from readings of natural wet-bulb (Twb ), black-globe (Tbg ), and dry-bulb ambient (Tdb ) temperatures according to the following formula (WBGT = 0.7Twb + 0.2Tbg + 0.1Tdb ). It was developed to help minimize the occurrence of heat casualties among military trainees; its use has been expanded to include both occupational environments and athletic events (2,10,18,19 ).
The current Marine Corps training guidelines state that continued exercise is permitted with a WBGT index from 80 to 84.9°F (26.7–29.4°C) (green flag). At 90°F (32.2°C) and above (black flag), all activities are to be stopped (16,21 ). Kark et al. (11 ) found that many exertional heat illness (EHI) episodes occurred with WBGT values well below the green flag condition range of 80–84.9°F (26.7–29.4°C). In addition, a majority of the EHI cases in Kark’s study were exposed to a WBGT above 80°F (26.7°C) on the day before becoming a case. These findings suggested a cumulative exposure effect that continues to the next day of training.
The objective of this study was to determine whether the daily average (7 a.m. to 4 p.m.) WBGT index for 1 or 2 d preceding an EHI incident is a better measure for predicting cases of EHI than the same day average WBGT. In light of the changes in training activity, a secondary goal was to investigate the components of the WBGT index that were most strongly predictive of EHI and determine whether an alternative heat index could predict EHI risk better than the WBGT index.
A case-crossover design was applied to observe the overall cumulative effect of heat exposure, as measured by the WBGT index, using U.S. Marine Corps recruit EHI cases as their own controls. The case-crossover design is used to study the transient effects of intermittent exposure on the risk of acute events in close temporal proximity to exposure (13,15 ). This design is essentially a special type of case-control study in which each case is his/her own control. Exposures for each case during an “at risk” or hazard period are compared with the distribution of exposure for follow-up times that occur before and after the hazard period and do not result in an event. The comparison period needs to be chosen carefully because selection bias and confounding can arise from multiple competing biases such as long-term time trends, seasonal patterns, autocorrelation in exposures, and day of week effects (3,4,8 ). This study design requires no additional control subjects to be sampled and can control for individual characteristics by making comparisons within subject.
METHODS
The case-crossover design with symmetrical bidirectional control sampling was applied to evaluate the association between EHI and WBGT index. Exposure during and immediately before time of EHI event was compared with pre- and postcontrol periods. Cases of EHI provided their own control periods (2 per case) for estimation of exposure.
Study population.
All U.S. Marine Corps Recruit EHI cases were included if they occurred during basic training at the Marine Corps Recruit Depot, Paris Island, SC, from 1979 to 1997. Investigators have adhered to the policies for protection of human subjects as prescribed in Army Regulation 70–25, and the research was conducted in adherence with the provisions of 45 CFR 46. Cases were defined as those recruits in basic training who made an emergency room hospital or clinic visit for exercise-induced heat illness, defined as heat stroke, heat injury, heat exhaustion , heat cramps, exertional dehydration, and/or rhabdomyolysis (11,12 ). Cases were identified by NAVMED 6500 forms and (for some years), additional cases were identified by review of all acute care clinic records. Additional clinical information was obtained from this review also, but for the earlier years (<1986), we had only the NAVMED 6500 forms. Cases were excluded when a WBGT temperature was not available at the approximate hour of EHI episode. (See “Exposure and other covariates” for source of temperature data.)
Cases served as their own controls defined by reference days excluding an appropriate lag period before and after EHI case day. The period from which control days were chosen was defined in the following way. Two symmetrical periods on control days, one before and one after each case day, were defined by determining: 1) a “black-out period” of 7 d around the case day during which control days would not be chosen; and 2) an outer limit of 21 d for the control period, that is, the maximum number of days away from the case day that a control day could be. Description of this procedure and detailed results are published elsewhere (22 ).
Once the blackout period and outer limit of the control period were defined, the resulting sampling period was between 8 and 21 d before and after case day. Two control days were randomly assigned to each case, which were j days before and after each event where j was a random number between 8 and 21. The number of randomly selected controls, in some instances, was not twice that of cases since WBGT and components were not available for every day of each year. Control days were selected using a program developed in the Agilent Visual Engineering Environment (VEE) to accomplish this process (1 ).
Exposure and other covariates.
The measures of exposure were WBGT temperature on the case or reference day, previous daily average (7 a.m. to 4 p.m.) WBGT temperature, previous 2-d daily average WBGT temperature, previous peak (maximum) day WBGT temperature and 2-d peak WBGT temperature before case episode (i.e., 2 d leading up to the case). The four WBGT measures on previous days were also treated as covariates to WBGT temperature at the time of EHI incident.
Individual recordings of hourly WBGT, Tdb , Twb , Tbg , and percent relative humidity (%RH) from the Marine Corps Air Station were electronically collected from the Air Force Combat Climatology Center archived historical weather database in Asheville, NC, onto a Microsoft Excel spreadsheet. For selected periods during several years, these WBGT measurements were compared to those collected by training personnel at the training sites. These comparisons showed close correlation of the training site WBGT measurements with those made at the Air Station. Therefore we used the Air Station records, because they covered the entire study period. These data were used to assign hourly, mean, and peak temperatures and %RH to cases and prepost controls and to calculate other alternative heat index measurements using the individual temperatures and %RH.
Statistical analysis.
The conditional logistic regression models using the PHREG procedure in SAS (20 ) were applied to the cohort of cases and matched reference periods to estimate cumulative incidence over previous 1 and 2 d of exposure. Conditional logistic regression was also used to identify the index most strongly predictive of heat illness (9 ). The likelihood ratio test was used as a measure of “goodness of fit” for model comparisons (14 ). Pearson’s product–moment correlation coefficients were used to identify highly correlated pairs of variables so that the chance of multicollinearity could be reduced in these regression models.
RESULTS
Characteristics of EHI cases and hot temperature conditions at Parris Island.
During the 19-yr study period, there were 2069 cases of EHI, of which 11% involved women and 89% men. The majority of the recruits were Caucasian (73%) with 24.5% African American and 2.5% other (Hispanic, Asian, etc.).
The WBGT at the hour immediately preceding a heat illness episode was on average about 2°F warmer than on control days (76.6°F and 77.3°F for pre- and postcontrol days vs 79.4°F for case days). A similar difference of 1–2°F was observed for previous day average and peak WBGT, and previous second day average (2 d before the event day) and peak WBGT (Table 1 ). The same approximate relations held as well for air and radiant temperatures: Twb , Tbg , and Tdb . The difference in %RH between case and control days was small unlike the differences noted for WBGT and its components (Table 1 ).
TABLE 1: Mean WBGT and component temperatures for case and control days.
Male cases experienced somewhat higher mean WBGT temperatures at time of EHI incident than female cases (males 79.5 ± 6.9°F; females 78.8 ± 9.0°F). Also, Caucasian cases experienced slightly higher mean WBGT temperatures than African American cases (Caucasians 79.6 ± 7.1°F; African Americans 78.7 ± 7.0°F).
The temperature components that make up the hourly WBGT were generally highly correlated (r ≥ 0.83) with one another with the exception of Twb and Tbg (r = 0.61), which were less strongly correlated. Percent relative humidity was not as highly correlated with the WBGT components (r ≤ 0.44). Correlations among hourly WBGT and mean/peak previous and second day previous WBGT were all moderately high (r ≥ 0.65), as expected.
Prediction of Marine Corps recruit EHI risk by heat index and components.
We first estimated EHI risk during the 12 wk of basic training from a conditional logistic regression model containing only WBGT as a continuous variable (Table 2 , model 1). This initial model included all study recruits (2062 cases and 3912 pre- and postcontrols). Hourly WBGT was a significant predictor of EHI risk (P < 0.0001) for all Marine Corps recruits at MRDC-PI during the 12 wk of basic training. The model predicted an 11% increase in risk per degree Fahrenheit (95% CI, 1.10–1.13).
TABLE 2: EHI risk and cumulative heat effects, estimated from case-crossover analyses using both pre- and postevent control days.
Next, a categorical model was fit comparing risk among eight categories of WBGT (the four established flag condition ranges for green, yellow, red, and black and the four lower 5° intervals). Using the reference category WBGT < 60°F (15.6°C), hourly WBGT was associated (OR = 1.5 to 5.6 °F−1 ; 95% CI, 1.3–8.5) with risk of EHI rising from WBGT 75°F (Fig. 1 ). Risk of EHI was more than double (OR = 2.2 °F−1 ; 95% CI, 1.7–2.9) at the presumed safe green flag level of 80 to 84.9°F (26.7 to 29.4°C).
The percentages of all EHI cases occurring in each WBGT category are shown in Figure 1 ; 83% of EHI cases occurred at and below the assumed safe flag conditions.
The possibility of a cumulative effect of WBGT on EHI risk was investigated by adding additional WBGT parameters for the previous day and second day previous to the above continuous conditional logistic regression model (Table 2 ). By including previous day average in the model (model 2), the fit was significantly improved (the improvement in the −2 log likelihood statistic comparing models 2 and 1 had a P value of < 0.001). An alternative model substituted an interaction term (WBGT*previous day average WBGT) in place of previous day average WBGT, which further improved model fit (model 3, Table 2 ). The interaction term was positive, suggesting that at high hourly WBGT and previous day average WBGT, the effects of these two variables on risk were no longer independent but that their combined effect was somewhat higher than would be predicted on the basis of the main effect of WBGT and previous day average WBGT alone. This suggests that there was a cumulative effect of previous day and current day heat exposure in these Marine Corps recruits. When previous second day average WBGT alone was entered into the model with hourly WBGT, the model fit was improved (model 4, Table 2 ) although not as strongly as when previous day average was used. The remaining models (models 5 through 7, Table 2 ) also did not improve model fit. All logistic regression models had excellent discrimination between case (EHI) days and noncase days (area under ROC curve: 0.80 to 0.82) (10 ).
The components of WBGT (and %RH) were then entered into logistic regression models to determine whether an alternative index might predict EHI risk better than WBGT itself. Two basic sets of predictors were investigated: the WBGT components Twb , Tbg , and Tdb ; and the alternative set which substituted %RH for Twb . An accessible measure of water vapor (%RH) was substituted for Twb to determine whether an alternative heat index was as effective in predicting risk of EHI as an index with Twb . The component temperatures were not available for all hours of each day, reducing the number of cases (1872) and controls (3334) available for analysis.
Alternative weights for the component measures in WBGT were evaluated by evaluating the three WBGT components as independent predictors of EHI. The results suggest a different set of weights would improve prediction of EHI when compared to the model using WBGT (Table 3 , models 1 and 5).
TABLE 3: EHI risk, WBGT, component temperatures, and alternative heat indices.
Both Twb and Tdb were clearly associated with risk of EHI, whereas Tbg contributed very little to the model. Eliminating Tbg (model 2, Table 3 ) resulted in an overall fit that was nearly as good as the model using all three components and better fitting than the WBGT model.
As an alternative components model, we substituted %RH for Twb (model 3, Table 3 ). Dry-bulb temperature was clearly associated with risk of EHI, and there was only modest contribution to EHI risk from Tbg and %RH. Excluding Tbg from this model caused only a small reduction in goodness of fit (models 3 and 4, Table 3 ). The simple model using just %RH and Tdb fit the data substantially better than WBGT.
Using the results of these models, we created two new measures of heat stress risk, modeled on the WBGT. The first, called the wet-bulb dry temperature (WBDT), was based on model 2 in Table 3 . To get appropriate weightings of each component in the new heat indices, β coefficients from the previous component models were used to calculate the weights. The coefficients of Twb and Tdb (βwb and βdb , respectively) from that model were reexpressed as fractions according to the following equation:
Then, WBDT was calculated as:
The weights of the WBDT were Twb 40% and Tdb 60%, whereas the weights of the current WBGT were Twb 70%, Tbg 20%, and Tdb 10%. A second new measure was created in a similar way, from model 4, called the relative humidity dry temperature (RHDT), having weights of %RH 10% and Tdb 90%.
The new, simpler indices were about as strongly associated with risk of EHI as was WBGT, as evidenced by their odds ratios (compare model 5 with models 6 and 7, Table 3 ), but they fit the EHI risk data better, as evidenced by the log likelihood statistics. The percent improvement in predicting risk of EHI was approximately 2–3% when we compared the new heat indices to the current standard (WBGT). This was estimated by calculating the difference between the two indices and dividing by the standard (WBGT), and then multiplying the whole expression by 100.
These models were fit to data stratified by race (African American and Caucasian) and gender to determine if prediction of EHI using these heat indices differed for male and female Marine Corps recruits. The results were substantially similar and not reported here. Detailed analyses are available in USARIEM Technical Report T03-14 (22 ).
To apply what we have learned from the above cumulative exposure models to predicting EHI risks under a variety of outdoor training conditions. We used model 3 in Table 2 as a summary presentation of the cumulative effect to generate odds ratios representing the relative risk of EHI for various combinations of current and previous day WBGT (Table 4 ).
TABLE 4: Cumulative effects of heat on EHI risk: odds ratios for combinations of current hour and previous day average WBGT.
DISCUSSION
In this study, case events were compared with control periods with respect to WBGT on the days preceding case events. WBGT was measured every hour each day during training periods at the Marine Corps Air Station, Beaufort, SC. In addition, alternative indices of heat exposure were computed from the component variables of Tdb , %RH, and solar radiation, measured by Tbg .
Overall, current WBGT was confirmed as an important predictor for risk of EHI during the 12 wk of basic training at Marine Corps Recruit Depot, Parris Island for all Marine Corps recruits. Despite long-standing awareness of this association and the existence of a flag system to limit heat exposure, the majority of EHI cases occurred at and below established “safe” green flag conditions of 80.0–84.9°F (26.7–29.4°C). Kark et al. (11 ) reported similar findings in a previous study using a smaller cohort of these same Marine Corps recruits. This high number of EHI cases occurring below established “safe” flag conditions led us to investigate a possible cumulative effect from the previous day’s WBGT.
Risk of EHI was associated not only with current temperatures but with those of the previous day as well. Including the previous day average WBGT in our models improved prediction of EHI risk, and substituting an interaction term (current WBGT*previous day average WBGT) improved prediction of EHI risk even more. These results suggest that the combined effect of current day and previous day WBGT was more important in predicting risk of EHI than current day WBGT alone. The interaction term suggests the importance of a cumulative effect of previous day and current day WBGT in predicting risk of EHI in these Marine Corps recruits. Adding information on heat 2 d before event did not further improve risk prediction in this cohort of Marine Corps recruits. This suggests that no additional information on risk is gained by going further back than the day before training. A similar increase in prediction of EHI risk was also observed using previous day peak WBGT as well, but the model did not fit quite as well, so the previous day average was preferred. These are initial findings and it is recommended that a validation study be performed before these models are put into use.
Our second goal was to determine whether we could improve upon the outdoor WBGT index as a heat index for predicting EHI risk to reduce the number of EHI events. The outdoor WBGT index is a weighted mean of the Twb , Tbg , and Tdb , with weights of Twb 70%, Tbg 20%, and Tdb 10%. We used the epidemiologic model of EHI risk to estimate alternative weights for those same temperature components. Alternative weights were found, which predicted risk of EHI somewhat better than the WBGT. One alternative, the WBDT was simpler than WBGT in that it did not include the Tbg . Its weights were Twb 40% and Tdb 60%. A second new index substituted relative humidity for the Twb . This index, the RHDT, had weights of %RH 10% and Tdb 90%. It fit the EHI data about as well as the WBDT and better than WBGT. This RHDT index empirically describing RH also reflects the explicit, global effects of rising ambient water vapor pressure during heat waves (as was shown last summer in France) where it is unusual that ambient vapor pressure (i.e., absolute humidity) rises above 15–20 mm Hg (5–7 ).
In these new indices, the weighting of heat load is attributed more to the dry-bulb temperature (60 and 90%) than to either the percent relative humidity or the natural wet bulb. These are relative weightings essentially opposite of those in the WBGT index where the dry bulb only contributes 10% of the heat load to the index, which weights more heavily the Twb (70%). It is desirable not to collect Tbg because this eliminates the need for having to measure radiant temperature, which contributed very little to the risk of EHI in our models. Also, Tdb and percent relative humidity, components of the RHDT, are easily attainable from local weather forecasts. This RHDT heat index, due to its easily accessible components, would also be very beneficial for training situations outside the military. Professional, collegiate, and high school athletic programs could use such an index to adjust their training to hot weather conditions and reduce risk of EHI.
Guidelines for Marine Corps recruit training based on the WBGT index were developed during the mid 1950s on the assumption that soldiers were engaged only in calisthenics and marching. Also, clothing insulation values of uniforms have decreased with the introduction of new clothing technology replacing older nonbreathable fabrics (17 ). Current training now includes better uniforms for dissipating heat but involves more intense activities requiring higher activity levels that increase body core temperature. This modification in training activities most likely changes the interaction between environmental conditions and training. It may be that the increased importance of Tdb in predicting risk (compared with its relatively modest contribution to WBGT) for all recruits result from these changes in training methods. A heat index that accounts for this strong association with Tdb would appear to be a better tool for predicting risk of EHI in present day recruit training than the currently used empirical WBGT index.
One way to reduce EHI events for Marine Corps recruit training would be to simply lower the maximum limit of all colored flag WBGT temperatures, thereby lessening the amount of heat exposure during consecutive hot days of training. If there is resistance to re-adjustment of the flag warning system, then another option might be to recalibrate the WBGT component weightings (Twb 40%, Tbg 10%, and Tdb 50%) to account for the stronger relationship between dry-bulb temperature and EHI risk in these Marine Corps recruits. This recalibration would reduce WBGT values (e.g., 80.4 to 79.1°F) resulting in lower flag warning thresholds.
Advantages of this study were: 1) a large number of EHI events, with nearly complete temperature data; 2) the ability to control for the primary risk factors for EHI: anthropometric measures, training conditions, and clothing by using a case-crossover design; 3) consistent medical testing and nutritional regimen and no permitted alcohol during training period, which eliminated potential confounding by these variables; and 4) the ability to study both genders.
At the same time that the case-crossover design controlled for conditioning and other personal risk factors, it limited our ability to study interactions between environmental factors and conditioning or physical activities at the time of heat illness. The fact that so many of the EHI cases occurred under relatively mild environmental conditions suggests strongly that an effective EHI strategy will have to include some form of physiological monitoring, which is currently in development using noninvasive sensor technology. Another potential limitation of this study is that physical training on green flag days may be more intense than on red flag days. If this were the case, lowering the safety threshold might have less effect on EHI risk than these data suggest.
Table 4 and Figure 1 show that limiting physical activity during hot periods can substantially reduce EHI risk, but there is clearly no “threshold” below which there is no risk. Identifying such safe conditions for heavy physical activity will require study of the combined effects of heat, physical conditioning, physical activity, acclimation, clothing, and perhaps other factors as well.
CONCLUSIONS
Combining what we have learned from developing new heat indices with what we know of the effects of cumulative heat exposure will allow us to better predict risk of EHI. This can be accomplished by applying the new heat indices of WBDT and RHDT formulated in this study to models containing current and previous day average to better predict the risk of EHI in these Marine Corps recruits.
We recommend that consideration be given to applying one of the heat indices, WBDT or RHDT, in hot weather training conditions at MCRD-PI. Whether these indices are relevant to locations and weather patterns in other locales besides Parris Island is not known. Additional data needs to be collected to determine whether these new indices are also applicable to other geographical locations and occupational indoor working environments. In light of recent military activity in environments characterized by dry heat, further examination of EHI in dry environments is essential. We also recommend that consideration be given to the importance of cumulative heat effects. Table 4 illustrates these effects by showing the risk (odds ratio) of EHI for combinations of current day and previous day average WBGT. Using Table 4 as a tool for scheduling training activities could assist health care professionals, drill instructors and sports coaches in understanding how they can anticipate a problem of EHI by keeping track of the current day’s temperature and therefore monitor the next day closely to know of new or added risk for EHI. Such a table might also be used in a new warning system to identify hazardous training conditions.
This manuscript is dedicated to the memory of Dr. C. Bruce Wenger due to his untimely death on November 22, 2002. Dr. Wenger was a good friend and committed colleague. We will forever be indebted to Bruce’s generosity and his knowledge of medicine.
The authors wish to thank Laurie Blanchard for her programming expertise in creating the case-crossover data sets. We also wish to thank Dr. John Kark of Howard University Medical School for assistance with data collection and in the preparation of this manuscript.
This study was supported in part by the Henry M. Jackson Foundation for the Advancement of Military Medicine grant no. DAMD17-95-1-5052. The opinions expressed in this paper are those of the individual authors and are not to be construed as official or reflecting the views of the United States Department of Defense.
REFERENCES
1. Agilent.
VEE Pro 6.0 Graphical Programming Environment. Palo Alto, CA: Agilent Technologies Inc, 2002.
2. American College of Sports Medicine. Position statement on prevention of heat injuries during distance running.
Med. Sci. Sports 7:vii–viii, 1975.
3. Bateson, T. F., and J. Schwartz. Selection bias and confounding in case-crossover analyses of environmental time-series data.
Epidemiology 12:654–661, 2001.
4. Bateson, T. F., and J. Schwartz. Control for seasonal variation and time trend in case-crossover studies of acute effects of environmental exposures.
Epidemiology 10:539–544, 1999.
5. Climate Visualization (CLIMVIS) System. NOAA/National Climatic Data Center, Asheville, NC, 2001.
6. Crabbe, C. France caught cold by heatwave.
Bull. WHO 81:773–774, 2003.
7. Gagge, A. P., and R. R. Gonzalez. Mechanisms of heat exchange: biophysics and physiology. In:
Handbook of Physiology: Environmental Physiology, M. J. Fregly and C. M. Blatteis (Eds.). New York: Oxford University Press, 1996, pp. 45–84.
8. Greenland, S. Confounding and exposure trends in case-crossover and case-time-control designs.
Epidemiology 7:231–239, 1996.
9. Hosmer, D. W., and S. Lemeshow.
Applied Logistic Regression, 2nd Ed. New York: John Wiley, 2000, pp. 243–259.
10. ISO 7243. Hot environments: estimation on the heat stress of working man, based on the WBGT-Index (Wet Bulb Globe Temperature). Geneva: International Organization of Standardization, 1989, pp. 1–3.
11. Kark, J. A., P. Q. Burr, C. B. Wenger, E. Gastaldo, and J. W. Gardner. Exertional heat illness in Marine Corps recruit training.
Aviat. Space Environ. Med. 67:354–360, 1996.
12. Knochel, J. P. Heat stroke and related heat stress disorders.
Dis. Mon. 35:301–378, 1989.
13. Levy, D., L. Sheppard, H. Checkoway, et al. A case-crossover analysis of particulate matte air pollution and out-of-hospital primary cardiac arrest.
Epidemiology 12:193–199, 2001.
14. Loomis, D., A. Salvan, H. Kromhout, and D. Kriebel. Selecting indices of occupational exposure for epidemiologic studies.
Occup. Hygiene 5:73–91, 1997.
15. Maclure, M. The case-crossover design: a method for studying transient effects on the risk of acute events.
Am. J. Epidemiol. 133:144–153, 1991.
16. Minard, D. Prevention of heat casualties in Marine Corps Recruits.
Mil. Med. 126:261–272, 1961.
17. NATO.
Handbook on Clothing: Biomedical Effects of Military Clothing and Equipment Systems . Research Study Group 7 on Bio-Medical Research Aspects of Military Protective Clothing, 1988, Ch. 2, p. 4.
18. NIOSH.
Criteria for a Recommended Standard. Occupational Exposure to Hot Environments. Washington, DC: U.S. Government Printing Office, 1986, pp. 105–106.
19. Office of the Surgeon General, U.S. Army.
Textbooks of Military Medicine. Medical Aspects of Harsh Environments: Clinical Diagnosis, Management, and Surveillance of Exertional Heat Illness, Vol. 1, 2001, pp. 231–279.
20. SAS Institute.
SAS/STAT Software: The PHREG Procedure, Version 8, SAS OnlineDoc. Cary, NC: SAS Institute Inc., 1999.
21. U.S. Department of Defense.
Occupational and Environmental Health: Prevention, Treatment and Control of Heat Injuries. Washington, DC: Departments of the Army, Navy and Air Force; TB MED 507, NAVMED P-5052-5, AFP 160-161, 1980.
22. Wallace, R. F.
Risk Factors and Mortality in Relation to Heat Illness Severity. U.S. Army Research Institute of Environmental Medicine; USARIEM Technical Report T03-14, 2003.