Skip Navigation LinksHome > January 2012 - Volume 13 - Issue > Chapter 2. Methods
Pediatric Critical Care Medicine:
doi: 10.1097/PCC.0b013e31823f43df
Articles

Chapter 2. Methods

Kochanek, Patrick M. MD, FCCM; Carney, Nancy PhD; Adelson, P. David MD, FACS, FAAP; Ashwal, Stephen MD; Bell, Michael J. MD; Bratton, Susan MD, MPH, FAAP; Carson, Susan MPH; Chesnut, Randall M. MD, FCCM, FACS; Ghajar, Jamshid MD, PhD, FACS; Goldstein, Brahm MD, FAAP, FCCM; Grant, Gerald A. MD; Kissoon, Niranjan MD, FAAP, FCCM; Peterson, Kimberly BSc; Selden, Nathan R. MD, PhD, FACS, FAAP; Tong, Karen A. MD; Tasker, Robert C. MBBS, MD, FRCP; Vavilala, Monica S. MD; Wainwright, Mark S. MD, PhD; Warden, Craig R. MD, MPH, FAAP, FACEP

Free Access
Article Outline
Collapse Box

Author Information

From Critical Care Medicine (PMK, MJB), University of Pittsburgh School of Medicine, Pittsburgh, PA; Department of Medical Informatics and Clinical Epidemiology (NC, SC, KP), Oregon Health & Science University, Portland, OR; Barrow Neurological Institute at Phoenix Children's Hospital (PDA), and Pediatric Neurosurgery/ Children' Neurosciences (PDA), Phoenix, AZ; Division of Child Neurology, Department of Pediatrics (SA) and Section of Neuroradiology (KAT), Loma Linda University School of Medicine, Loma Linda, CA; Pediatric Critical Care Medicine (SB), University of Utah School of Medicine, Salt Lake City, UT; Department of Neurological Surgery (NRS), Oregon Health & Science University, Portland, OR; Orthopedics and Sports Medicine (RMC), University of Washington School of Medicine, Seattle, WA; Neurological Surgery (JG), Weill Cornell Medical College; President of the Brain Trauma Foundation (JG), New York, NY; Translational Science (BG), Ikaria, Inc., Clinton, NJ; Pediatrics (BG), University of Medicine and Dentistry of New Jersey, Robert Wood Johnson Medical School, New Brunswick, NJ; Surgery and Pediatrics (GAG), Duke University School of Medicine, Durham, NC; Pediatrics and Emergency Medicine (NK), British Columbia's Children's Hospital, University of British Columbia, Vancouver, BC; Neurocritical Care (RCT), Children's Hospital Boston; Neurology and Anesthesia (RCT), Harvard Medical School, Boston, MA; Anesthesiology and Pediatrics (MSV), University of Washington School of Medicine, Seattle, WA; Molecular Pharmacology and Biological Chemistry (MSW), Northwestern University Feinberg School of Medicine, Chicago, IL; Emergency Medicine and Pediatrics (CRW), and Pediatric Emergency Services (CRW), Oregon Health & Science University/Doernbercher Children's Hospital, Portland, OR.

Funding provided by the Brain Trauma Foundation and partially funded by the Charles Maddock Foundation.

The authors have not disclosed any potential conflicts of interest.

For information regarding this article, E-mail: carneyn@ohsu.edu

Back to Top | Article Outline

I. TOPIC REFINEMENT

The Brain Trauma Foundation (BTF) and BTF Center for Guidelines Management (Center) convened a virtual meeting of previous guidelines authors and colleagues new to the project. The panel consisted of 15 clinicians and three methodologists. They specified which previous topics would be maintained and agreed on new topics to include. Topics were not included in the second edition if they were adequately addressed in other guidelines documents (e.g., prehospital management of pediatric patients with traumatic brain injury is addressed in the Guidelines for Prehospital Management of Severe Traumatic Brain Injury [1]) or if there was no literature meeting inclusion criteria to support any level of recommendation. Specification of new topics of interest was determined by panel consensus. Previous topics that were updated are Indications for Intracranial Pressure Monitoring, Intracranial Pressure Treatment Threshold, Cerebral Perfusion Pressure, Antiseizure Prophylaxis, Hyperventilation, Cerebrospinal Fluid Drainage, Hyperosmolar Therapy, Decompressive Craniectomy, Barbiturates, Analgesics–Sedatives–Neuromuscular Blockades, and Steroids. Topics from the first edition not included in this update are Trauma Systems and Pediatric Trauma Centers, Prehospital Airway Management, Resuscitation of Blood Pressure and Oxygenation, Intracranial Pressure Monitoring Technology, and the Critical Pathway. New topics are Advanced Neuromonitoring and Neuroimaging. The previous topic of Temperature Control was expanded to Hypothermia and Temperature Control, and the previous topic of Nutrition was expanded to Glucose and Nutrition.

Back to Top | Article Outline

II. LITERATURE SEARCH AND RETRIEVAL

Center staff worked with a doctoral-level research librarian to construct electronic search strategies for each topic (Appendix B). For new topics, the literature was searched from 1950 to 2009 and for previous topics from 1996 to 2009. A second search was conducted for 2009–2010 to capture any new relevant literature. Strategies with the highest likelihood of capturing most of the targeted literature were used, which resulted in the acquisition of a large proportion of nonrelevant citations.

Two contributing authors (coauthors) were assigned to each topic, and a set of abstracts was sent to each coauthor. Blinded to each other's work, they read the abstracts and eliminated citations using the prespecified inclusion/exclusion criteria. Center staff compared the coauthors' selections and identified and resolved discrepancies either through consensus or through use of a third reviewer. A set of full-text publications was then sent to each coauthor. Again blinded to each other's work, they read the publications and selected those that met the inclusion criteria.

Results of the electronic searches were supplemented by recommendations of peers and by reading reference lists of included studies. Relevant publications were added to those from the original search, constituting the final library of studies that were used as evidence in this document. The yield of literature from each phase of the search is presented in Appendix C.

Back to Top | Article Outline

III. STUDY SELECTION

Inclusion Criteria

Inclusion criteria consisted of severe traumatic brain injury (Glasgow Coma Scale score <9); human subjects; English language publications; pediatric patients (age ≤18 yrs); randomized controlled trials (N ≥25)a; cohort studies, prospective or retrospective (N ≥25)b; case–control studies (N ≥25); and case series (N ≥5).

The intervention (independent variable) must be specific to the topic.

The outcome must be a relevant health outcome (morbidity or mortality) or a surrogate outcome that associates with a health outcome.

Minimum sample sizes were identified to circumscribe the body of literature and manage the scope of the project. There is no evidence that the selected cutoffs associate with levels of confidence in the reported results.

Back to Top | Article Outline
Exclusion Criteria

Exclusion criteria consisted of penetrating brain injury; animal studies; cadaver studies; non-English language publications; and adult patients (age >18 yrs).

Also excluded were studies in which the sample contained >15% of adult patients or >15% of patients with pathologies other than traumatic brain injury without separate analysis (Appendix D).c

Case studies/editorials/comments/letters were excluded.

For each topic, relevant information from the Guidelines for the Management of Severe Traumatic Brain Injury (2) is reviewed. The panel agreed that data from the adult guidelines would not be used to contribute to recommendations for this document.

Back to Top | Article Outline
Inclusion of Direct and Indirect Evidence

Figures 1 and 2 illustrate different links in a “causal pathway” that represent either direct or indirect evidence. In Figure 1, arc A represents direct evidence, derived from a comparative study, of the influence of an intervention on an important health outcome (like functional status). Arc B represents direct evidence of the influence of an intervention on a surrogate outcome (like partial pressure of brain tissue oxygen), and arc C represents a correlation between measures on the surrogate outcome and the important health outcome. Taken together, arcs B and C represent indirect evidence of the influence of the intervention on an important health outcome. Studies were included if they contained direct evidence or if they contained both components of the indirect evidence illustrated in Figure 1.

Figure 1
Figure 1
Image Tools
Figure 2
Figure 2
Image Tools

Figure 2 illustrates a second kind of indirect evidence that we included. In some studies, an intervention was introduced to the entire study sample (without a comparison group). Change in an important health outcome was measured, and then authors looked for associations between surrogate measures and the health outcome. For example, in the chapter on Cerebral Perfusion Pressure, Downard et al (3) conducted a retrospective analysis of 118 patients who were all treated for severe traumatic brain injury and assessed for Glasgow Outcome Scale score at ≥3 months, dichotomized as “good” or “poor.” Then, using a logistic regression analysis, they looked for significant associations between cerebral perfusion pressure, a surrogate measure, and outcome. Lower cerebral perfusion pressure was associated with poorer outcomes. This association was used as weak class III evidence for the chapter's recommendations. In Figure 2, arc A′ represents an uncontrolled association between an intervention and an important health outcome, and arc B′ represents a correlation between measures on the surrogate outcome and the important health outcome. Studies were included if they contained both components of the indirect evidence illustrated in Figure 2.

Back to Top | Article Outline

IV. DATA ABSTRACTION AND SYNTHESIS

Remaining blinded, coauthors read each publication and abstracted data using an evidence table template (Appendix E). They compared results of their data abstraction and through consensus finalized the data tables that constitute the evidence on which the recommendations are based. As a result of heterogeneity of studies within topics, and the lack of literature of adequate quality, data were not combined quantitatively.

Coauthors drafted manuscripts for each topic. The entire team gathered for a 2-day work session to discuss the literature base and craft the recommendations. Manuscripts were revised. Virtual meetings were held with a subset of the coauthors to complete the editing process. The final draft manuscript was circulated to the peer review panel and was revised incorporating selected peer review input.

Back to Top | Article Outline

V. QUALITY ASSESSMENT OF INDIVIDUAL STUDIES AND CLASSIFICATION OF EVIDENCE

In April of 2004, the BTF established a formal collaboration with the Evidence-Based Practice Center from Oregon Health & Science University. Center staff worked with two Evidence-Based Practice Center epidemiologists to develop criteria and procedures for the quality assessment of individual studies and classification of level of evidence provided by each included study. These criteria are designed to assess risk of bias for individual studies based on study design and conduct. Criteria for classification of evidence are in Table 1 and are derived from criteria developed by the U.S. Preventive Services Task Force (4), the National Health Service Centre for Reviews and Dissemination (U.K.) (5), and the Cochrane Collaboration (6). These criteria were used to assess the literature.

Table 1
Table 1
Image Tools

Three members of the Center staff, two of whom are Evidence-Based Practice Center epidemiologists, conducted all of the quality assessments. Two assessors, blinded to each other's work and to publication identification, read the selected studies and classified them as class I, II, or III based on the criteria in Table 2. Discrepancies were resolved through consensus or through a third person's review.

Class I evidence is derived from randomized controlled trials. However, some randomized controlled trials may be poorly designed, lack sufficient patient numbers, or suffer from other methodologic inadequacies.

Class II evidence is derived from clinical studies in which data were collected prospectively and retrospective analyses that were based on clearly reliable data. Comparison of two or more groups must be clearly distinguished. Types of studies include observational, cohort, prevalence, and case–control. Class II evidence may also be derived from flawed randomized controlled trials.

Class III evidence is derived from prospectively collected data that are purely observational and retrospectively collected data. Types of studies include case series, databases, or registries. Class III evidence may also be derived from flawed randomized controlled trials or flawed observational, cohort, prevalence, or case–control studies.

Back to Top | Article Outline

VI. QUALITY OF BODY OF EVIDENCE AND STRENGTH OF RECOMMENDATIONS

At the beginning of each recommendation section in this document, the recommendations are categorized in terms of strength and quality of evidence. The strength of the recommendation is derived from the overall quality of the body of evidence used to assess the topic.

Back to Top | Article Outline
Quality of Body of Evidence

The underlying methods for assessing risk of bias for individual studies are represented in Table 1. However, ultimately the individual studies must be considered in aggregate, whether through meta-analyses or through qualitative assessment. Thus, the strength of recommendations must be derived from the quality of the overall body of evidence used to address the topic.

Consistent with recommendations for grading a body of evidence adopted by the Agency for Healthcare Research and Quality (7), we assessed the overall quality of the body of evidence considering the domains of 1) risk of bias from individual studies; 2) consistency of findings across studies; 3) directness of evidence; and 4) precision of estimates of effect. The quality of the overall body of evidence for each recommendation in this document is classified as high, moderate, or low. Factors that may decrease the quality include potential bias, differing findings across studies, the use of indirect evidence, or lack or precision. For example, if two or more class I studies demonstrate contradictory findings for a particular topic, the overall quality most probably will be low because there is uncertainty about the effect. Similarly, class I or II studies that provide indirect evidence may only constitute low-quality evidence, overall.

Back to Top | Article Outline
Strength of Recommendations

Consistent with methods generated by the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) Working Group, recommendations in this document are categorized as either strong or weak. As stated in the American Thoracic Society's official statement (8), in which they endorsed the GRADE methods for their guidelines endeavors, “The strength of a recommendation reflects the degree of confidence that the desirable effects of adherence to a recommendation outweigh the undesirable effects.”

Strong recommendations are derived from high-quality evidence that provides precise estimates of the benefits or downsides of the topic being assessed. With weak recommendations, 1) there is lack of confidence that the benefits outweigh the downsides; 2) the benefits and downsides may be equal; and/or 3) there is uncertainty about the degree of benefits and downsides.

a One randomized controlled trial had a sample of 24 patients (Kloti, 1987) and one a sample of 18 (Fisher, 1992). Cited Here...

b One retrospective review had a sample of 24 patients (Pfenninger, 1983). Cited Here...

c One study included 16% of patients with moderate traumatic brain injury (Downard, 2000). Cited Here...

Back to Top | Article Outline

REFERENCES

1. Badjatia N, Carney N, Crocco T, et al.: Guidelines for the prehospital management of severe traumatic brain injury, 2nd edition. Prehosp Emerg Care 2007; 12:S1–S52

2. Bratton S, Bullock R, Carney N, et al.: Guidelines for the management of severe traumatic brain injury, 3rd edition. J Neurotrauma 2007; 24(Suppl 1):S1–S105

3. Downard C, Hulka F, Mullins RJ, et al.: Relationship of cerebral perfusion pressure and survival in pediatric brain-injured patients. J Trauma 2000; 49:654–658; discussion 658–659

4. Harris RP, Helfand M, Woolf SH, et al.: Current methods of the third U.S. Preventive Services Task Force. Am J Prev Med 2001; 20:21–35

5. Anonymous: Undertaking Systematic Reviews of Research on Effectiveness: CRD's Guidance for Those Carrying Out or Commissioning Reviews. CRD Report Number 4. Second Edition). York, UK, NHS Centre for Reviews and Dissemination, 2001, p 4

6. Mulrow CD, Oxman AD: How to Conduct a Cochrane Systematic Review. Version 3.0.2. Presented at Cochrane Collaboration, San Antonio, TX, 1997

7. Owens D, Lohr K, Atkins D, et al.: Grading the strength of a body of evidence when comparing medical interventions—Agency for Healthcare Research and Quality and the Effective Health Care Program. J Clin Epidemiol 2010; 63:513–523

8. Schunemann H, Jaeschke R, Cook D, et al.: An official ATS statement: grading the quality of evidence and strength of recommendations in ATS guidelines and recommendations. Am J Respir Crit Care Med 2006; 174:605–614

©2012The Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies

Login

Article Tools

Images

Share