This issue of Epidemiology includes a case-control study by Mueller et al1 addressing whether helmets are effective in preventing head injuries in skiers. The cases were skiers who fell and had head injuries, and controls were skiers who fell and had other injuries. This commentary discusses the assumptions inherent in using other injuries as controls in injury case-control studies.
Pre-Event Phase and Event-Phase Risk Factors
A seminal development in injury epidemiology was the proposal by William Haddon2 to separate injury interventions into 3 groups: pre-event, event, and post-event. Haddon further categorized interventions along a second axis comprising host, vehicle, and social and physical environment. The resulting “Haddon matrix” is a cornerstone of modern injury epidemiology.
In Haddon's schema, an injury “event” is an exchange of energy that damages tissue (eg, a motor vehicle crash). An example of a pre-event intervention would be the rumble strips at the side of the highway, which alert drivers who are beginning to drift off the roadway. Airbags are an event-phase intervention: they deploy during a vehicle crash. Post-event factors occur after a crash has occurred, eg, the time it takes for emergency help to arrive. These 3 steps in the causal chain are common to all acute injuries.
It is recommended that injury epidemiologists separate risk factors into pre-event phase and event-phase.3,4 The study by Mueller et al,1 addresses ski helmets as an event-phase factor. The research question is: “if a skier falls, will a helmet prevent head injury?”
Event-Phase Case-Control Studies
A second important development in injury epidemiology occurred in the late 1980s, when researchers pioneered the case-control design used by Mueller et al for studying event-phase interventions. In this design, cases are injuries at a specific body site (such as the head), and controls are other injuries.5,6
A helmet cannot protect the head until a fall occurs. Therefore, in evaluating whether ski helmets are effective, it is logical to restrict the study population to skiers who fell. In other words, when studying event-phase risk factors, the study population should be defined as those who sustained the event.
An entirely different (and equally important) research question is whether wearing a ski helmet influences the incidence of falls. A study addressing that research question might use the pre-event source population (all skiers); cases would be skiers who fell.
Using Other Injuries as Controls
The study design used by Mueller et al1 is shown in Figure 1. The event-phase source population of skiers who fell (Group B), is a subset of all skiers (Group A, the pre-event source population). The cases are skiers who fell and had a head injury (Group D). The ideal controls would be a random sample of all skiers who fell (Group B), but there is no simple means of identifying skiers who fell but were not injured, and so injured skiers without head injuries (Group C′, defined as Group C minus Group D) were used as controls.7
Is this surrogate population a valid substitution?8 It is, under the assumption that the prevalence of helmet-wearing in Group C′ is a good proxy for the prevalence of helmet-wearing in Group B.7,9 I refer to this assumption as the “substitution assumption.”
This design is very similar to case-control studies that use hospital controls. In fact, the failure of the substitution assumption is essentially equivalent to Berkson's bias.10
A concern alluded to in the Mueller et al study1 is that skiers who choose to wear helmets may be inherently more safety-conscious (perhaps skiing at slower speeds or on less-challenging slopes) compared with unhelmeted skiers. If this is true, they might be less likely to be injured in a fall, not because of the protective effect of the helmet, but simply because there were going slower on slopes with fewer hazards. This bias would tend to depress the prevalence of helmet-wearing in the study's controls (Group C′) relative to the source population (Group B). Such a study would tend to over-estimate the protective effect of helmets.
If the converse were true, and helmet-wearing skiers were risk-takers who are more likely to be injured following a fall than nonhelmeted skiers because they were going faster or on more hazardous slopes, then the study's controls (Group C′) would over-represent the prevalence of helmet-wearing in the event-phase source population (Group B). Such a study would tend to under-estimate the protective effect of helmets.
Any factor that is associated with both helmet-wearing and injury given a fall creates a violation of the substitution assumption and could potentially generate major bias. In addition to risk-taking/safety-conscious behavior, other factors that might create bias could include being a novice skier (if inexperienced skiers are less likely to wear a helmet and less likely to be injured given a fall), or previous history of head injury (if those with a previous head injury are more likely to be reinjured given a fall and are also more likely to wear a protective helmet). Furthermore, the injured skier had to receive assistance from the ski patrol to be included in the study. If helmet-wearers were less (or more) likely than nonwearers to seek out ski patrol assistance following a fall (aside from any protective effect of the helmet on head injury) this would create a type of “referral” bias, again violating the substitution assumption.
Because the validity of the design depends on the substitution assumption, all investigators using other injuries as controls should verify, by empirical means, that the exposure prevalence in their other-injury control group (Group C′) is a reasonable proxy for the exposure prevalence in the event-phase source population (Group B, all people who sustain the event).
What can be done to check the validity of the substitution assumption? Mueller et al1 explore the assumption with DAGs. They also perform a sensitivity analysis restricting the control group to those with severe injuries, with the idea of removing any “referral” bias. Additional strategies could include a validation substudy of the prevalence of helmet-wearing in a sample of skiers who fell (Group B), perhaps by placing video cameras on random sample of ski slopes (or skiers).
Other Limitations of Using Other Injuries as Controls
Even if the substitution assumption is unequivocal, there is still an unappealing element to studies using other injuries as controls. Most of the event-phase source population is typically uninjured, whereas all the controls are injured. Noting this, critics charge that the design is simply a comparison of 2 “case” groups: a case series of skiers who fell and had head injuries (Group B) and a case series of skiers who fell and had nonhead injuries (Group C′). Critics question whether this is a sound basis for inferences about helmet-wearing in all skiers (Group A). Indeed, it is not: studies in which the controls are randomly sampled from the overall population of skiers (Group A, pre-event controls)11 can better address that research question. However, studies using other injuries as controls (Group C′, event-phase controls)1,12 do provide a basis for valid inferences about the effectiveness of helmets given a fall (provided the substitution assumption is true).
The convenient aspect of this study design is that it makes efficient use of routinely collected injury data (such as ski patrol logs or Emergency Department records). However, if the additional work required to investigate the validity of the substitution assumption is time-consuming or expensive, this advantage is greatly reduced.
Finally, the extent to which this design can be extended to include all injuries (not just injuries at a specific body site) is unclear.7
Case-control designs in which other injuries are used as controls are valid if the substitution assumption holds true. The design is attractive because it allows investigators to make use of easily-obtained data. However, the design is biased if the incidence of injury in the event of interest is associated with the exposure (other than through the effect of the exposure). All investigators using other injuries as controls should verify, by empirical means, the substitution assumption, ie, the assumption that the exposure prevalence in the other injury group is a good proxy for the exposure prevalence in all people who sustain the event.
The larger issue, of course, is “what have we learned about the benefits of ski helmets?” There is growing evidence1,11,12 that ski helmets are effective in preventing head injury and consideration should be given to mandating their use on ski slopes. Ski helmet manufacturers should agree to a common and stringent manufacturing standard (currently there are 3 standards, none mandatory). Prudent individuals should consider the protection that ski helmets appear to offer against head injuries, and the growing evidence of long-term damage from brain trauma (which includes amnesia, anxiety, depression, and possibly Alzheimer Disease).13–15 On the slopes as well as in the rest of life, an ounce of prevention is worth a pound of cure.
Thanks to Katherine Hoggatt for providing debate and intellectual stimulation.
ABOUT THE AUTHOR
STEPHEN MARSHALL is an Associate Professor of Epidemiology at the University of North Carolina at Chapel Hill. He has worked in injury research for nearly two decades. His main research interest is prevention of sports injuries, and he is currently carrying out a prospective cohort study of risk factors for injury of the anterior cruciate ligament.
1. Mueller BA, Cummings P, Rivara FP, et al. Case-injuries of the head, face, and neck injury in relation to ski helmet use among skiers who fell. Epidemiology
2. Haddon W. Options for the prevention of motor vehicle crash injury. Israeli Med J
3. Li G, Baker SP. Exploring the male-female discrepancy in death rates from bicycling injury: the decomposition method. Accid Anal Prev
4. Li G, Baker SP, Langlois JA, et al. Are female drivers safer? An application of the decomposition method. Epidemiology
5. Thompson RS, Rivara FP, Thompson DC. A case-control study of the effectiveness of bicycle safety helmets. N Engl J Med
6. Thompson DC, Rivara FP, Thompson RS. Effectiveness of bicycle safety helmets in preventing head injuries. A case-control study. JAMA
7. Cummings P, Koepsell TD, Roberts I. Case-control studies in injury research. In: Rivara FP, Cummings P, Koepsell TD, et al, eds. Injury Control: A Guide to Research and Program Evaluation
. New York, NY: Cambridge University Press; 2001:139–156.
8. Maldonado G, Greenland S. Estimating causal effects. Int J Epidemiol
9. Cummings P, Rivara FP, Thompson DC, et al. Misconceptions regarding case-control studies of bicycle helmets and head injury. Accid Anal Prev
10. Berkson J. Limitations of the application of fourfold table analysis to hospital data. Biometrics
11. Sulheim S, Holme I, Ekeland A, et al. Helmet use and risk of head injuries in alpine skiers and snowboarders. JAMA
12. Hagel BE, Pless IB, Goulet C, et al. Effectiveness of helmets in skiers and snowboarders: case-control and case crossover study. BMJ
13. Haboubi NH, Long J, Koshy M, et al. Short-term sequelae of minor head injury (6 years experience of minor head injury clinic). Disabil Rehabil
14. Powell TJ, Collin C, Sutton K. A follow-up study of patients hospitalized after minor head injury. Disabil Rehabil
15. Guskiewicz KM, Marshall SW, Bailes J, et al. Association between recurrent concussion, mild cognitive impairment, and Alzheimer's disease in retired professional football players. Neurosurgery