Secondary Logo

Improving Severe Traumatic Brain Injury Observational Research

Not All Questions Need an RCT

Krishnamoorthy, Vijay, MD, PhD*; Vavilala, Monica S., MD

Journal of Neurosurgical Anesthesiology: April 2019 - Volume 31 - Issue 2 - p 173–174
doi: 10.1097/ANA.0000000000000577

*Department of Anesthesiology, Duke University, Durham, NC

Department of Anesthesiology and Pain Medicine, University of Washington, Seattle, WA

The authors have no funding or conflicts of interest to disclose.

Severe traumatic brain injury (TBI) is a major public health problem and a leading cause of mortality and major disability worldwide.1 Among survivors, severe TBI results in a significant burden to patients and their families and often necessitates lifelong care. To help improve TBI outcomes, promising preclinical therapeutic strategies have been developed at the bench, but these have largely failed to translate into improved outcomes when tested in randomized-controlled trials (RCTs). This has led to frustration for all stakeholders, including clinicians, patients, and families. Because of the time and high costs associated with conducting well-powered RCTs in the acute care of severe TBI, there remains a place for observational studies to fill knowledge gaps. In addition, health care has concurrently seen an explosion in the data available to researchers, including multi-institutional electronic medical record sources, large administrative claims data, and health care registries.2 Unfortunately, observational research in severe TBI has often been criticized for lack of rigor, bias, and limited generalizability. Therefore, it is important to harness advanced methodological knowledge, statistical approaches, and computing abilities in order to make progress in the rigorous conduct of observational research. We call for a paradigm shift in the conduct of severe TBI observational research in the acute care setting and provide suggestions about how data can be better used to improve patient outcomes.

The interpretation of observational research in severe TBI is affected by a variety of biases, although solutions exist to help address these. Observational comparative effectiveness research in TBI is often hampered by the bias of confounding by indication. Patients receiving an intervention outside of clinical trials are often systematically different from those who do not receive the intervention; they are usually more critically ill or injured and, therefore, more likely to experience a poor outcome. This limitation can make an intervention appear to result in inferior outcomes, despite potentially improved efficacy. Fortunately, time-tested epidemiologic methods, such as improved cohort restriction, stratification, and adequate control for residual confounding through multivariable modeling strategies, can be utilized to limit this bias. These must be consistently applied in severe TBI observational studies in order to adequately compare “apples to apples.” In addition, the application of modern epidemiological methods, such as propensity score matching, inverse-probability weighting, and coarsened exact matching,3 can further help overcome bias and improve the validity of severe TBI observational studies.

Although observational studies suggest correlations between exposures and outcomes, a lack of definite causality, as well as relevant mechanisms, represents another limitation of TBI epidemiologic studies. We urge that investigators create a strong conceptual model based on supportive data and scientific knowledge, possibly in the form of directed acyclic graphs which are visual representations of causal assumptions and confounding variables.4 In addition, variables and statistical procedures to analyze the data must be identified a priori. With a strong and scientifically based conceptual model, as well as transparency in sharing of all aspects of data and analysis plans, additional investigators could better reproduce study findings in varied patient cohorts. Furthermore, the practice of advanced methods for causal inference, such as mediation analyses or instrumental variables, can help test the underlying mechanisms of observed exposure-outcome relationships.5 Lastly, associations discovered in severe TBI observational research must be consistently interpreted in the greater context of criteria for causality, including rigorous examination of biological plausibility, strength of association, and consistency (replication of findings) among multiple studies in diverse severe TBI cohorts.

Many changes to clinical care in populations have nothing to do with research interventions, but result from implementation of new interventions, policy changes, or resource constraints. However, such changes in clinical care can create conditions for natural experiments in clinical medicine. By utilizing external influences as randomizing effects, investigators can reduce selection bias in epidemiologic studies. For example, a drug shortage may provide the ideal condition to conduct a comparative effectiveness study of critical care interventions.6 As patient characteristics would be expected to be largely similar before and after a practice change, there may be a greater balance of measured and unmeasured characteristics, providing conditions for “as if” randomization. Such natural experiments remain a poorly explored method to improve causal inference in severe TBI observational research and should be added to our “toolkit” of study designs.

Lastly, the emergence of large quantities of data, in association with computing power to detect patterns in these data, has allowed the growth of the field of data science in clinical medicine.7 Exploiting these opportunities, there is an emerging body of literature in the prediction of disease at an earlier stage than would be recognized using traditional methods.8 The possible application of data science methods, such as machine learning, in severe TBI research, is understudied but potentially valuable for predicting the risk of secondary brain injuries as well as to the incorporation of evolving clinical data over time to enhance functional outcome prediction. These advanced methods are not future “science fiction,” but are already being practiced in health care research. For example, deep machine learning algorithms that can predict postinduction hypotension9 and postoperative mortality10 have already been developed. Harnessing this exciting and growing field may allow the development of predictive capabilities in the care of severe TBI patients that were not possible even just a few years ago, if the complexities of this approach, such as regulatory standards and product reliability, are adequately addressed.11

New strategies are rapidly becoming available to address bias, analyze data, and improve the validity of observational research in severe TBI. Greater and consistent practice of these methods could significantly enhance severe TBI research and translate into improved outcomes in this vulnerable patient population. Given the cost and logistic burdens of conducting RCTS, the research community should attempt to prioritize which research methods are important for answering the myriad of research questions in severe TBI. Not all questions need an RCT.

Vijay Krishnamoorthy, MD, PhD*

Monica S. Vavilala, MD†

*Department of Anesthesiology, Duke University, Durham, NC

†Department of Anesthesiology and Pain Medicine, University of Washington, Seattle, WA

Back to Top | Article Outline


1. Rutland-Brown W, Langlois JA, Thomas KE, et al. Incidence of traumatic brain injury in the United States, 2003. J Head Trauma Rehabil. 2006;21:544–548.
2. Lee CH, Yoon HJ. Medical big data: promise and challenges. Kidney Res Clin Pract. 2017;36:3–11.
3. Sceats LA, Trickey AW, Morris AM, et al. Nonoperative management of uncomplicated appendicitis among privately insured patients. JAMA Surg. 2018. Doi: 10.1001/jamasurg.2018.4282. [Epub ahead of print].
4. Williamson EJ, Aitken Z, Lawrie J, et al. Introduction to causal diagrams for confounder selection. Respirology. 2014;19:303–311.
5. Mascha EJ, Dalton JE, Kurz A, et al. Statistical grand rounds: understanding the mechanism: mediation analysis in randomized and nonrandomized studies. Anesth Analg. 2013;117:980–994.
6. Vail E, Gershengorn HB, Hua M, et al. Association between US Norepinephrine shortage and mortality among patients with septic shock. JAMA. 2017;317:1433–1442.
7. Zhu L, Zheng WJ. Informatics, data science, and artificial intelligence. JAMA. 2018;320:1103–1104.
8. Nemati S, Holder A, Razmi F, et al. An interpretable machine learning model for accurate prediction of sepsis in the ICU. Crit Care Med. 2018;46:547–553.
9. Kendale S, Kulkarni P, Rosenberg AD, et al. Supervised machine-learning predictive analytics for prediction of postinduction hypotension. Anesthesiology. 2018;129:675–688.
10. Lee CK, Hofer I, Gabel E, et al. Development and validation of a deep neural network model for prediction of postoperative in-hospital mortality. Anesthesiology. 2018;129:649–662.
11. Shortliffe EH, Sepulveda MJ. Clinical decision support in the era of artificial intelligence. JAMA. 2018;320:2199–2200.
Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved