Secondary Logo

Journal Logo


Examining the Evidence in Anesthesia Literature: A Critical Appraisal of Systematic Reviews

T-L. Choi, Peter MD, FRCPC*,; Halpern, Stephen H. MD, FRCPC‡,; Malik, Nariman BSc‡,; Jadad, Alejandro R. MD, DPhil, FRCPC†‡§,; Tramèr, Martin R. MD, DPhil∥,; Walder, Bernhard MD∥,

Author Information
doi: 10.1213/00000539-200103000-00029
  • Free


Review articles provide summaries of pertinent information. However, like primary articles, review articles can vary in their quality with respect to validity and clinical relevance. There are two types of review articles: narrative reviews and systematic reviews. Narrative reviews are easy to read, summarize lots of information on a subject, and often provide recommendations from clinical experts. The methods for data collection, synthesis, and interpretation are not usually reported; therefore, the completeness of information and the conclusions must be taken at face value. Unfortunately, narrative reviews can delay identification of effective or harmful interventions by 10 to 15 yr (for example, in treatment of myocardial infarction) when compared with systematic reviews (1).

In contrast, systematic reviews, by definition, are reviews that use scientific strategies to reduce bias in the collection, appraisal, and interpretation of relevant studies (2). Systematic reviews have been used in a number of health care specialties to summarize existing information on a specific topic, to provide estimates of effects of established interventions before the planning of clinical trials, and to provide supporting evidence for practice guidelines. Evidence from systematic reviews may also play a role in health policy development (3).

In anesthesia, a number of challenges face the clinician who wishes to use systematic reviews as a tool for acquiring knowledge. First, in contrast to specialties such as internal medicine and obstetrics, few systematic reviews have been published in peer-reviewed journals on topics related to anesthesia (4). Second, management issues in anesthesia often overlap with other specialties (for example, fluid resuscitation) and may be published in nonanesthesia journals. Thus, reviews with important information could be missed. Third, there is little information relating to the quality of published systematic reviews. The quality of such reviews has been evaluated in chronic pain research but no information is available on other areas of anesthesia (5).

To examine these challenges, from April to June 1999, we undertook a systematic search for systematic reviews published on management issues relevant to anesthesia with the following goals in mind:

  • 1. To review the subject content of systematic reviews published on topics relevant to anesthesia and highlight areas they have not covered.
  • 2. To evaluate the quality of these systematic reviews with respect to type of journal (anesthesia or nonanesthesia) and, for reviews of interventions, direction of clinical conclusions (effective or not effective) using a validated scale.
  • 3. To highlight some methodological and reporting issues affecting the systematic reviews.

Our hope is that this paper will enable readers to understand the structure of systematic reviews and be aware of the topics that have been reviewed in this fashion thus far. As well, for readers with methodological or editorial interests, we provide an evaluation of the quality of systematic reviews in anesthesia and identify priorities for methodological or reporting improvement.


Identification of Relevant Reviews

Systematic reviews published on management issues relevant to anesthesia were identified by literature search of MEDLINE (1966 to June 1999), EMBASE (1988 to June 1999), CINAHL (1982 to June 1999), and the Cochrane Library (1999 issue 2). The initial search strategy was designed for MEDLINE (Table 1) and then tailored to the other bibliographic databases. The reference lists of all articles were reviewed and our personal files were searched to identify additional citations. Finally, the January 1999 to June 1999 issues of the top five general anesthesia journals based on citation impact factor (Anesthesiology, Anesthesia & Analgesia, British Journal of Anesthesia, Anesthesia, and Canadian Journal of Anesthesia) were hand searched for additional citations. Authors of systematic reviews were not contacted.

Table 1
Table 1:
Search Strategy to Identify Systematic Reviews in Computerized Bibliographic Databases

Selection of Relevant Reviews

All citations were reviewed independently by three of the authors (PTC, SHH, NM) for inclusion in this review. Disagreements were resolved by consensus. A citation was considered relevant for this review if it met the following inclusion criteria.

  • The article had to review issues relating to the diagnosis, causation, harm, prognosis, prevention, or treatment of conditions relating to anesthesia. We included articles that reviewed issues in fluid therapy, invasive monitoring, and mechanical ventilation if the issue was relevant to perioperative care. For example, reviews on techniques of central venous cannulation would be included but reviews on frequency of central venous catheter changes would be excluded. We excluded nonacute pain reviews as this area has been reviewed previously (5). Reviews of methodological or biostatistical issues in anesthesia were excluded. When duplicate publication occurred (for example, publication in the Cochrane Library and in a peer-reviewed journal), the article providing the most comprehensive review of the literature was used.
  • The article had to be described as a “systematic review,” “systematic overview,” “quantitative review,” or “meta-analysis.”
  • The article had to have been published in a peer-reviewed publication. Unpublished work, abstracts, letters, and conference proceedings were excluded. No language exclusion was applied.

Evaluation of Systematic Reviews

The primary outcome of interest was the overall quality of each systematic review. Each included review was appraised independently by the same three authors using the Overview Quality Assessment Questionnaire (OQAQ, Fig. 1) (5). The OQAQ is a validated checklist that asks nine questions on the methodological quality of the review. Each question can be answered as “yes,” “partially/can’t tell,” or “no.” Based on the replies to the nine ques-tions, the overall quality is graded using a 7-point Likhert scale. Differences among assessors were resolved by consensus. In addition to evaluating methodology, the clinical topics, the conclusions, and publication information were also examined for each review.

Figure 1
Figure 1:
Overview Quality Assessment Questionnaire used in this study. Instructions for use of this index may be found in the appendix of Jadad and McQuay’s 5. The index is reprinted with permission.

Statistical Analysis

Given the use of ranks in the OQAQ and the likelihood that the assumption of normal distribution could not be made, the median was used as the statistic for central tendency and nonparametric tests were used for comparisons regarding methodological quality. We compared quality scores between anesthesia and nonanesthesia journals and between individual anesthesia journals using the Mann-Whitney U-test and the Kruskal-Wallis nonparametric analysis of variance test respectively. For reviews of interventions, the distribution of quality scores of review articles reporting “effective” and “not effective” were compared using the χ2 statistic with Yates’ correction.


We identified 153 citations. Seventy-one citations were excluded (three duplicate publications, eight abstracts or letters, 17 narrative reviews, 13 citations of nonacute pain topics, 21 citations of nonanesthesia topics, and nine articles on research methodology or informatics in anesthesia). Eighty-two citations met inclusion criteria


Clinical Findings

Systematic reviews have been used to examine many topics in anesthesia (Table 2). The areas most frequently reviewed have been interventions for treatment of acute pain (22.0%), interventions for prevention or treatment of postoperative nausea or vomiting (19.5%), choice of crystalloids or colloids for fluid therapy (9.8%), alternatives to homologous blood transfusion (8.5%), and regional anesthesia for labor or Cesarean delivery (8.5%). The majority of reviews evaluated interventions for prevention or treatment. Diagnosis or harm was the subject of only eight reviews (6,12,44,49,55–57,76). Incidence, prognosis, patient or economic preferences have not been subject to systematic review.

Table 2
Table 2:
Topics Reviewed in Anesthesia Systematic Reviews

Methodological Quality

Inter-rater reliability for the OQAQ was 0.89. Thirty-four reviews (41.5%) had minor or minimal flaws (OQAQ score of 5 or more). Analysis of the component scores of the OQAQ revealed that 73.2% stated search methods to find evidence, 59.8% had a reasonably comprehensive search, 80.5% reported inclusion criteria, 56.1% used measures to avoid selection bias, 48.8% reported criteria used to assess validity of included studies, 47.6% used appropriate criteria for validity assessment, 81.7% reported the methods used to combine findings of relevant studies, and 70.7% had conclusions supported by the data or analysis reported in the review.

In addition, no language restriction was used in 84.1% of all reviews. Of the reviews that reported methods used to aggregate data from primary studies, 17.9% used qualitative descriptive techniques and 82.0% used quantitative techniques of meta-analysis. The majority of meta-analyses used fixed effect (29.7%) or random effects (45.3%) models. Heterogeneity testing, sensitivity analysis, and assessment for publication bias were reported in only 35.4%, 19.5%, and 4.9% of all reviews respectively.

Table 3 summarizes the component scores of the OQAQ and other considerations related to methodology by type of journal. No significant differences were seen between reviews published in anesthesia and nonanesthesia journals. Table 4 enumerates the number of reviews by the overall OQAQ score and type of journal. Median OQAQ score for reviews published in anesthesia and nonanesthesia journals were 4 and 6 respectively and the difference was not statistically significant (P = 0.089). There was no statistically significant difference in the quality scores of reviews published between the 11 anesthesia journals (P = 0.435); however, the number of reviews published in some journals was small. Of the 77 reviews of interventions, there was no statistically significant difference in the proportion of reviews that concluded an intervention was effective when reviews with minimal or minor flaws were compared with those with major or extensive flaws (17/33 versus 26/44, P = 0.643).

Table 3
Table 3:
Overview Quality Assessment Questionnaire (OQAQ) Component Scores and Other Methodological Considerations by Type of Journal
Table 4
Table 4:
Number of Anesthesia Systematic Reviews by Overview Quality Assissment Questionaire (OQAQ) Scores and Type of Journal

Publication Details

The number of systematic reviews published in the anesthesiology literature has increased yearly; the majority were published in the past five years (Fig. 2). We did not find anesthesia systematic reviews published before 1989. Anesthesia systematic reviews were published in anesthesia journals (61%), nonanesthesia medical journals (30.5%), nursing journals (2.4%), and other publications such as the Cochrane Library (6.1%). Half of all reviews were published in the British Journal of Anesthesia (14.6%), Anesthesia & Analgesia (13.4%), Anesthesiology (8.5%), British Medical Journal (7.3%), or Acta Anaesthesiologica Scandinavica (6.1%).

Figure 2
Figure 2:
Bar chart of the number of anesthesia systematic reviews published yearly. The bar for 1999 represents the number of reviews published from January to June 1999.


In theory, systematic reviews are “application[s] of scientific strategies that limit bias to the systematic assembly, critical appraisal, and synthesis of all relevant studies on a specific topic”(3). When the process is conducted in a methodologically rigorous fashion and the results are interpreted in a clinically relevant manner, systematic reviews can summarize existing information on a specific topic in a succinct and comprehensive fashion, provide estimates of treatment effects, and pinpoint areas for future research by explicitly highlighting lack of high quality evidence.

The systematic review is a fairly new type of publication in anesthesia. The first anesthesia-related systematic review was published in 1989 (82); the first systematic review published in an anesthesia journal occurred one year later (57). Since then, the number of systematic reviews has increased steadily. This trend is consistent with publication trends observed in other medical specialties (5,88). The findings of our appraisal highlight the content, challenges, and areas of strength of this type of literature.

Content in Anesthesia Systematic Reviews

A wide variety of topics in anesthesia have been systematically reviewed but the list is by no means comprehensive. Relatively few topics have been reviewed in most subspecialties or in areas of assessment, monitoring, and complications. Few pharmacological and no physiological topics have been the subject of systematic reviews. Reasons for these gaps may include the lack of relevant and valid data from original trials or lack of statistical methods for synthesizing the outcomes used in these areas. The majority of systematic reviews focused on discrete outcomes (incidence of postoperative nausea and vomiting) or continuous outcomes in which statistical methods exist for combining values (pain scores). As a result, most reviews examined topics relating to prevention or treatment.

Challenges in Evaluating and Interpreting Systematic Reviews

Although there is room for improvement in systematic reviews in anesthesia, we were encouraged to find that nearly half of all systematic reviews had minor or minimal flaws (OQAQ score of five or more) considering the relatively recent use of this type of literature in anesthesia. There were no statistically significant differences in quality between the type of journal or between individual anesthesia journals. The direction of the clinical conclusions (effective or not effective) in systematic reviews of prevention or treatment was also unrelated to the quality of the review.

In general, differentiation between flaws in methodology (failure to perform steps to limit bias) and flaws in reporting (failure to disclose procedures or criteria) could not be distinguished. At the time of this study, we used the OQAQ as a measure of overall quality as it was the only validated checklist of the methodological quality of a systematic review. However, the conclusions derived from the OQAQ are limited in that the checklist is unable to distinguish between the two types of flaws. It is possible that a methodologically sound systematic review could receive a low OQAQ score if methodological details were not reported. Therefore, we do not recommend that the reader dismiss reviews that received OQAQ scores less than five in this study; however, caution is advised when interpreting their conclusions.

Some of the challenges of evaluating the quality of systematic reviews could be rectified if peer-reviewed journals provide authors and peer reviewers with clear reporting criteria. The QUOROM statement is a checklist developed for reporting of systematic reviews and has recently been published (89). The QUORUM statement recommends a structure of reporting details so that the methods are transparent; thus, the ability of evaluating methodological quality of a systematic review should be less hampered by reporting issues.

Currently, for readers just beginning to use systematic reviews, the OQAQ is a helpful checklist but it will be insufficient for readers who desire a greater appreciation of the methodological issues of systematic reviews. Basic instructions on interpretation and application of systematic reviews can be obtained from the Users’ Guides to the Medical Literature series published in the Journal of the American Medical Association(90). For more detailed discussions of systematic reviews, other resources are available (3).

Challenges in Performing Systematic Reviews

For individuals performing systematic reviews, how can the quality be improved? First, we need to pay close attention to the methodology of the primary trials that are conducted. A number of methodologists have demonstrated the concerns with validity, statistical analysis, and reporting observed in anesthesia randomized clinical trials (4,91–93). Inadequately reported trials tend to be biased in favor of experimental groups and result in inflated estimates of treatment effects (94). In turn, inclusion of such trials can weaken inferences made in systematic reviews (95).

Second, reviewers need to improve the method in which systematic reviews are conducted. As one group has stated, “statistical tests cannot compensate for lack of common sense, clinical acumen, and biological plausibility in the design of the protocol of a meta-analysis”(96). Methodological guidelines have been published to assist reviewers in performing systematic reviews (2). Initiatives such as the Cochrane Collaboration also provide reviewers opportunities to collaborate with each other in producing methodologically sound systematic reviews. The Cochrane Anesthesia Review Group was established recently to review anesthesia-related topics in a systematic fashion. (Further details can be found on their website

Based on the results from this study, we suggest that the methodological quality of systematic reviews in anesthesia could be improved by: 1) expanding the search strategies used to find relevant studies (for example, use of more than one computerized bibliographic database plus citation review plus hand searching of journals); 2) using methods to avoid selection bias (for example, two reviewers to assess every study for inclusion in a systematic review); 3) using validated methods to evaluate the validity of each included study; and 4) assessment of potential sources of bias (for example, publication bias). For systematic reviews that perform quantitative analyses, quality of reviews and confidence in the conclusions could be improved by performance of heterogeneity testing and sensitivity analyses. Further details have been published elsewhere (3).

Third, further advances in design and statistical methods are needed to enable reviewers to appraise and combine studies of natural history, diagnosis, prognosis, or harm. Recently, models are being developed to estimate rare adverse events in anesthesia and may be applied to systematic reviews (97). Quantitative summaries of information relating to these issues would enable clinicians to weigh risks versus benefits and to better judge the effectiveness of a particular course of action.

Why Bother?

With the challenges in interpreting and using information from systematic reviews, why should a reader bother? Why not abandon systematic reviews altogether? We would contend that such an action would be analogous to shooting the messenger (98). Despite the challenges discussed above, systematic reviews have already provided important insights to the anesthetic community. With the methodical and comprehensive collation of primary studies, systematic reviews help to clarify the limits of information from current anesthesia research. For example, we know much about the incidence of postoperative nausea and vomiting from systematic reviews but still know little about patient preferences for its treatment. Similarly, our belief (based on research data and clinical impressions) that incidence of postdural puncture headache is affected by shape and size of the spinal needle is confirmed by systematic review (37), yet relatively little information is available on the clinical course of this complication. In some areas, systematic reviews have forced us to reconsider our pattern of practice. For example, reviews on fluid therapy (21,64) have resulted in renewed (and sometimes heated) discussion on fluid choices for resuscitation. Systematic reviews can show us the quality of our research efforts, demonstrate areas of bias (32), and provide the impetus for improvement in the manner in which we conduct clinical trials (99).

Additionally, systematic reviews can help unearth lack of evidence and assist in defining the research agenda. Increasingly, institutional review boards and funding agencies are requesting evidence to justify the rationale for clinical research projects. Systematic reviews are an efficient way to summarize preexisting work and highlight lack of evidence in an arena of research.

We are not advocating that readers, clinicians, researchers, editors, or policy makers should abandon the use of narrative reviews in their acquisition of knowledge. Narrative reviews can provide readers with a panorama of a broad subject whereas systematic reviews permit readers to scrutinize all the information relating to a specific question (98). Each has a complementary role in the dissemination of knowledge in anesthesia. To use the review literature effectively, readers will need to become familiar with the strengths and limitations of both. As the rate of publication of systematic reviews continues to escalate, we believe that collaborative efforts between clinicians, researchers, methodologists, peer reviewers, and editors will be necessary (98) as we refine the systematic review as one of our tools for acquiring information and knowledge in the 21st century.


1. Antman EM, Lau J, Kupelnick B, et al. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts: treatments for myocardial infarction. JAMA 1992; 268: 240–8.
2. Cook DJ, Sackett DL, Spitzer WO. Methodologic guidelines for systematic reviews of randomized control trials in health care from the Potsdam Consultation on meta-analysis. J Clin Epidemiol 1995; 48: 167–71.
3. Mulrow CD, Cook DJ, eds. Systematic reviews: Synthesis of best evidence for health care decisions. Philadelphia: American College of Physicians, 1998.
4. Halpern SH, Jadad AR, Choi PT-L. Evidence based practice in anaesthesia–how good is the evidence? In: Tramèr MR, ed. Evidence based resource in anaesthesia and analgesia. London: BMJ Books, 2000: 27–44.
5. Jadad AR, McQuay HJ. Meta-analyses to evaluate analgesic interventions: a systematic qualitative review of their methodology. J Clin Epidemiol 1996; 49: 235–43.
6. Archer C, Levy A, McGregor M. Value of routine preoperative chest x-rays: a meta-analysis. Can J Anaesth 1993; 40: 1022–7.
7. Armand S, Langlade A, Boutros A, et al. Meta-analysis of the efficacy of extradural clonidine to relieve postoperative pain: an impossible task. Br J Anaesth 1998; 81: 126–34.
8. Ballantyne JC, Carr DB, Chalmers TC, et al. Postoperative patient-controlled analgesia: meta-analyses of initial randomized control trials. J Clin Anesth 1993; 5: 182–93.
9. Ballantyne JC, Carr DB, deFerranti S, et al. The comparative effects of postoperative analgesic therapies on pulmonary outcome: cumulative meta-analyses of randomized, controlled trials. Anesth Analg 1998; 86: 598–612.
10. Beuvoir C, Peray P, Daures JP, et al. Pharmacodynamics of vecuronium in patients with and without renal failure: a meta-analysis. Can J Anaesth 1993; 40: 696–702.
11. Bisonni RS, Holtgrave DR, Lawler F, Marley DS. Colloids versus crystalloids in fluid resuscitation: an analysis of randomized controlled trials. J Fam Pract 1991; 32: 387–90.
12. Boivin JF. Risk of spontaneous abortion in women occupationally exposed to anaesthetic gases: a meta-analysis. Occup Environ Med 1997; 54: 541–8:
13. Brimacombe J. The advantages of the LMA over the tracheal tube or facemask: a meta-analysis. Can J Anaesth 1995; 42: 1017–23.
14. Brimacombe JR, Berry A. The incidence of aspiration associated with the laryngeal mask airway: a meta-analysis of published literature. J Clin Anesth 1995; 7: 297–305.
15. Bryson GL, Laupacis A, Wells GA. Does acute normovolemic hemodilution reduce perioperative allogeneic transfusion? A meta-analysis: the international study of perioperative transfusion. Anesth Analg 1998; 86: 9–15.
16. Bunn F, Alderson P, Hawkins V. Colloid solutions for fluid resuscitation. The Cochrane Library [serial online]. Oxford: Update Software, 1999. Document No. CD001319.
17. Carroll D, Tramèr M, McQuay H, et al. Randomization is important in studies with pain outcomes: systematic review of transcutaneous electrical nerve stimulation in acute postoperative pain. Br J Anaesth 1996; 77: 798–803.
18. Carroll D, Tramèr M, McQuay H, et al. Transcutaneous electrical nerve stimulation in labour pain: a systematic review. Br J Obstet Gynaecol 1997; 104: 169–75.
19. Cattaneo M, Harris AS, Stromberg U, Mannucci PM. The effect of desmopressin on reducing blood loss in cardiac surgery–a meta-analysis of double-blind, placebo-controlled trials. Thromb Haemost 1995; 74: 1064–70.
20. Choi PT, Yip G, Quinonez LG, Cook DJ. Crystalloids vs. colloids in fluid resuscitation: a systematic review. Crit Care Med 1999; 27: 200–10.
21. Cochrane Injuries Group Albumin Reviewers. Human albumin administration in critically ill patients: systematic review of randomised controlled trials. BMJ 1998; 317: 235–40.
22. Collins SL, Edwards JE, Moore RA, McQuay HJ. Single-dose dextropropoxyphene in post-operative pain: a quantitative systematic review. Eur J Clin Pharmacol 1998; 54: 107–12.
23. Cote CJ, Zaslavsky A, Downes JJ, et al. Postoperative apnea in former preterm infants after inguinal herniorrhaphy: a combined analysis. Anesthesiology 1995; 82: 809–22.
24. Curatolo M, Petersen-Felix S, Scaramozzino P, Zbinden AM. Epidural fentanyl, adrenaline and clonidine as adjuvants to local anaesthetics for surgical analgesia: meta-analyses of analgesia and side-effects. Acta Anaesthesiol Scand 1998; 42: 910–20.
25. de Craen AJ, Di Giulio G, Lampe-Schoenmaeckers JE, et al. Analgesic efficacy and safety of paracetamol-codeine combinations versus paracetamol alone: a systematic review. BMJ 1996; 313: 321–5.
26. Dexter F, Tinker JH. Comparisons between desflurane and isoflurane or propofol on time to following commands and time to discharge: a metaanalysis. Anesthesiology 1995; 83: 77–82.
27. Dexter F. Research synthesis of controlled studies evaluating the effect of hypocapnia and airway protection on cerebral outcome. J Neurosurg Anesthesiol 1997; 9: 217–22.
28. Dexter F. Regional anesthesia does not significantly change surgical time versus general anesthesia–a meta-analysis of randomized studies. Reg Anesth Pain Med 1998; 23: 439–43.
29. Divatia JV, Vaidya JS, Badwe RA, Hawaldar RW. Omission of nitrous oxide during anesthesia reduces the incidence of postoperative nausea and vomiting: a meta-analysis. Anesthesiology 1996; 85: 1055–62.
30. Domino KB, Anderson EA, Polissar NL, Posner KL. Comparative efficacy and safety of ondansetron, droperidol, and metoclopramide for preventing postoperative nausea and vomiting: a meta-analysis. Anesth Analg 1999; 88: 1370–79.
31. Faura CC, Collins SL, Moore RA, McQuay HJ. Systematic review of factors affecting the ratios of morphine and its major metabolites. Pain 1998; 74: 43–53.
32. Figueredo ED, Canosa LG. Ondansetron in the prophylaxis of postoperative vomiting: a meta-analysis. J Clin Anesth 1998; 10: 211–21.
33. Flordal PA. Pharmacological prophylaxis of bleeding in surgical patients treated with aspirin. Eur J Anaesthesiol Suppl 1997; 14: 38–41.
34. Forgie MA, Wells PS, Laupacis A, Fergusson D. Preoperative autologous donation decreases allogeneic transfusion but increases exposure to all red blood cell transfusion: results of a meta-analysis. International Study of Perioperative Transfusion (ISPOT) Investigators. Arch Intern Med 1998; 158: 610–6.
35. Fremes SE, Wong BI, Lee E, et al. Metaanalysis of prophylactic drug treatment in the prevention of postoperative bleeding. Ann Thorac Surg 1994; 58: 1580–8.
36. Haigh CG, Kaplan LA, Durham JM, et al. Nausea and vomiting after gynaecological surgery: a meta-analysis of factors affecting their incidence. Br J Anaesth 1993; 71: 517–22.
37. Halpern S, Preston R. Postdural puncture headache and spinal needle design: metaanalyses. Anesthesiology 1994; 81: 1376–83.
38. Halpern SH, Leighton BL, Ohlsson A, et al. Effect of epidural vs parenteral opioid analgesia on the progress of labor: a meta-analysis. JAMA 1998; 280: 2105–10.
39. Hartung J. Twenty-four of twenty-seven studies show a greater incidence of emesis associated with nitrous oxide than with alternative anesthetics. Anesth Analg 1996; 83: 114–6.
40. Hofmeyr GJ. Prophylactic intravenous preloading for regional analgesia in labour. The Cochrane Library [serial online]. Oxford: Update Software, 1999. Document No. CD000175.
41. Howell CJ, Chalmers I. A review of prospectively controlled comparisons of epidural with non-epidural forms of pain relief during labour. Int J Obstet Anesth 1992; 1: 93–110.
42. Kalso E, Tramèr MR, Carroll D, et al. Pain relief from intra-articular morphine after knee surgery: a qualitative systematic review. Pain 1997; 71: 127–34.
43. Kavanagh BP, Katz J, Sandler AN. Pain control after thoracic surgery: a review of current techniques. Anesthesiology 1994; 81: 737–59.
44. Kellen M, Aronson S, Roizen MF, et al. Predictive and diagnostic tests of renal failure: a review. Anesth Analg 1994; 78: 134–42.
45. Kindler CH, Seeberger MD, Staender SE. Epidural abscess complicating epidural anesthesia and analgesia: an analysis of the literature. Acta Anaesthesiol Scand 1998; 42: 614–20.
46. Laupacis A, Fergusson D, et al. Drugs to minimize perioperative blood loss in cardiac surgery: meta-analyses using perioperative blood transfusion as the outcome. Anesth Analg 1999; 85: 1258–67.
47. Lee A, Done ML. The use of nonpharmacologic techniques to prevent postoperative nausea and vomiting: a meta-analysis. Anesth Analg 1999; 88: 1362–9.
48. Li Wan PA, Zhang WY. Systematic overview of co-proxamol to assess analgesic effects of addition of dextropropoxyphene to paracetamol. BMJ 1997; 315: 1565–71.
49. Mantha S, Roizen MF, Barnard J, et al. Relative effectiveness of four preoperative tests for predicting adverse cardiac outcomes after vascular surgery: a meta-analysis. Anesth Analg 1994; 79: 422–33.
50. Mapleson WW. Effect of age on MAC in humans: a meta-analysis. Br J Anaesth 1996; 76: 179–85.
51. Merikle PM, Daneman M. Memory for unconsciously perceived events: evidence from anesthetized patients. Conscious Cogn 1996; 5: 525–41.
52. Moiniche S, Mikkelsen S, Wetterslev J, Dahl JB. A qualitative systematic review of incisional local anaesthesia for postoperative pain relief after abdominal operations. Br J Anaesth 1998; 81: 377–83.
53. Moore A, Collins S, Carroll D, McQuay H. Paracetamol with and without codeine in acute pain: a quantitative systematic review. Pain 1997; 70: 193–201.
54. Morton SC, Williams MS, Keeler EB, et al. Effect of epidural analgesia for labor on the cesarean delivery rate. Obstet Gynecol 1994; 83: 1045–52.
55. Munro J, Booth A, Nicholl J. Routine preoperative testing: a systematic review of the evidence. Health Technol Assess 1997; 1: 1–76.
56. Novis BK, Roizen MF, Aronson S, Thisted RA. Association of preoperative risk factors with postoperative acute renal failure. Anesth Analg 1994; 78: 143–9.
57. Pace NL. Prevention of succinylcholine myalgias: a meta-analysis. Anesth Analg 1990; 70: 477–83.
58. Picard PR, Tramèr MR, McQuay HJ, Moore RA. Analgesic efficacy of peripheral opioids (all except intra-articular): a qualitative systematic review of randomised controlled trials. Pain 1997; 72: 309–18.
59. Randolph AG, Cook DJ, Gonzales CA, Pribble CG. Ultrasound guidance for placement of central venous catheters: a meta-analysis of the literature. Crit Care Med 1996; 24: 2053–8.
60. Randolph AG, Cook DJ, Gonzales CA, Andrew M. Benefit of heparin in peripheral venous and arterial catheters: systematic review and meta-analysis of randomised controlled trials. BMJ 1998; 316: 969–75.
61. Randolph AG, Cook DJ, Gonzales CA, Andrew M. Benefit of heparin in central venous and pulmonary artery catheters: a meta-analysis of randomized controlled trials. Chest 1998; 113: 165–71.
62. Robinson BJ, Uhrich TD, Ebert TJ. A review of recovery from sevoflurane anaesthesia: comparisons with isoflurane and propofol including meta-analysis. Acta Anaesthesiol Scand 1999; 43: 185–90.
63. Rubens FD, Fergusson D, Wells PS, et al. Platelet-rich plasmapheresis in cardiac surgery: a meta-analysis of the effect on transfusion requirements. J Thorac Cardiovasc Surg 1998; 116: 641–7.
64. Schierhout G, Roberts I. Fluid resuscitation with colloid or crystalloid solutions in critically ill patients: a systematic review of randomised trials. BMJ 1998; 316: 961–4.
65. Seers K, Carroll D. Relaxation techniques for acute pain management: a systematic review. J Adv Nurs 1998; 27: 466–75.
66. Sneyd JR, Carr A, Byrom WD, Bilski AJ. A meta-analysis of nausea and vomiting following maintenance of anaesthesia with propofol or inhalational agents. Eur J Anaesthesiol 1998; 15: 433–45.
67. Sorenson RM, Pace NL. Anesthetic techniques during surgical repair of femoral neck fractures: a meta-analysis. Anesthesiology 1992; 77: 1095–104.
68. Taddio A, Ohlsson A, Einarson TR, et al. A systematic review of lidocaine-prilocaine cream (EMLA) in the treatment of acute pain in neonates. Pediatrics 1998; 101: E1.
69. Taddio A, Ohlsson A. Lidocaine-prilocaine cream (EMLA) to reduce pain in male neonates undergoing circumcision. The Cochrane Library [serial online]. Oxford: Update Software, 1999. Document No. CD000496.
70. Tangkanakul C, Counsell C, Warlow C. Local versus general anaesthesia for carotid endarterectomy. The Cochrane Library [serial online]. Oxford: Update Software, 1999. Document No. CD000126.
71. Thorp JA, Breedlove G. Epidural analgesia in labor: an evaluation of risks and benefits. Birth 1996; 23: 63–83.
72. Tramèr M, Moore A, McQuay H. Prevention of vomiting after paediatric strabismus surgery: a systematic review using the numbers-needed-to-treat method. Br J Anaesth 1995; 75: 556–61.
73. Tramèr M, Moore A, McQuay H. Omitting nitrous oxide in general anaesthesia: meta-analysis of intraoperative awareness and postoperative emesis in randomized controlled trials. Br J Anaesth 1996; 76: 186–93.
74. Tramèr M, Moore A, McQuay H. Meta-analytic comparison of prophylactic antiemetic efficacy for postoperative nausea and vomiting: propofol anaesthesia vs omitting nitrous oxide vs total i.v. anaesthesia with propofol. Br J Anaesth 1997; 78: 256–59.
75. Tramèr M, Moore A, McQuay H. Propofol anaesthesia and postoperative nausea and vomiting: quantitative systematic review of randomized controlled studies. Br J Anaesth 1997; 78: 247–55.
76. Tramèr MR, Moore RA, McQuay HJ. Propofol and bradycardia: Causation, frequency and severity. Br J Anaesth 1997; 78: 642–51.
77. Tramèr MR, Reynolds DJ, Moore RA, McQuay HJ. Efficacy, dose-response, and safety of ondansetron in prevention of postoperative nausea and vomiting: a quantitative systematic review of randomized placebo-controlled trials. Anesthesiology 1997; 87: 1277–89.
78. Tramèr MR, Moore RA, Reynolds DJ, McQuay HJ. A quantitative systematic review of ondansetron in treatment of established postoperative nausea and vomiting. BMJ 1997; 314: 1088–92.
79. Tramèr MR, Williams JE, Carroll D, et al. Comparing analgesic efficacy of non-steroidal anti-inflammatory drugs given by different routes in acute and chronic pain: a qualitative systematic review. Acta Anaesthesiol Scand 1998; 42: 71–9.
80. Tramèr MR, Fuchs-Buder T. Omitting antagonism of neuromuscular block: Effect on postoperative nausea and vomiting and risk of residual paralysis: a systematic review. Br J Anaesth 1999; 82: 379–86.
81. Tramèr MR, Walder B. Efficacy and adverse effects of prophylactic antiemetics during patient-controlled analgesia therapy: a quantitative systematic review. Anesth Analg 1999; 88: 1354–61.
82. Velanovich V. Crystalloid versus colloid fluid resuscitation: a meta-analysis of mortality. Surgery 1989; 105: 65–71.
83. Vickers AJ. Can acupuncture have specific effects on health? A systematic review of acupuncture antiemesis trials J R Soc Med 1996; 89: 303–11.
84. Wade C, Grady J, Kramer G. Efficacy of hypertonic saline dextran (HSD) in patients with traumatic hypotension: meta-analysis of individual patient data. Acta Anaesthesiol Scand Suppl 1997; 110: 77–9.
85. Wade CE, Kramer GC, Grady JJ, et al. Efficacy of hypertonic 7.5% saline and 6% dextran-70 in treating trauma: a meta-analysis of controlled clinical studies. Surgery 1997; 122: 609–16.
86. Writer WDR, Stienstra R, Eddleston JM, et al. Neonatal outcome and mode of delivery after epidural analgesia for labour with ropivacaine and bupivacaine: A prospective meta-analysis. Br J Anaesth 1998; 81: 713–7.
87. Zhang WY, Li Wan PA. Analgesic efficacy of paracetamol and its combination with codeine and caffeine in surgical pain–a meta-analysis. J Clin Pharm Ther 1996; 21: 261–82.
88. Jadad AR, Moher M, Browman GP, et al. Systematic reviews and meta-analysis on the treatment of asthma: a critical evaluation. BMJ 2000; 320: 537–40.
89. Moher D, Cook DJ, Eastwood S, et al. for the QUORUM Group. Improving the quality of reports of meta-analyses of randomized controlled trials: the QUORUM statement. Lancet 1999; 354: 1896–900.
90. Oxman AD, Cook DJ, Guyatt GH, and the Evidence-Based Med Working Group. Users’ guides to the medical literature: VI. How to use an overview. JAMA 1994; 272: 1367–71.
91. Bender JS, Halpern SH, Thangaroopan M, et al. Quality and retrieval of obstetrical anaesthesia randomized controlled trials. Can J Anaesth 1997; 44: 14–8.
92. Avram MJ, Shanks CA, Dykes MHM, et al. Statistical methods in anesthesia articles: an evaluation of two American journals during two six-month periods. Anesth Analg 1985; 64: 607–11.
93. Duncan PG, Cohen MM. The literature of anaesthesia: what are we learning? Can J Anaesth 1988; 35: 494–9.
94. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995; 273: 408–12.
95. Moher D, Jones A, Cook DJ, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998; 352: 609–13.
96. Lau J, Ioannidis JP, Schmid CH. Quantitative synthesis in systematic reviews. Ann Intern Med 1997; 127: 820–6.
97. Tramèr MR, Moore RA, Reynolds DJM, McQuay HJ. Quantitative estimation of rare adverse events which follow a biological progression–a new model applied to chronic NSAID use. Pain 2000; 85: 169–82.
98. Choi PT-L, Jadad AR. Systematic reviews in anesthesia: should we embrace them or shoot the messenger? Can J Anaesth 2000; 47: 486–93.
99. Chalmers TC, Lau J. Meta-analytic stimulus for changes in clinical trials. Stat Meth Med Res 1993; 2: 161–72.
© 2001 International Anesthesia Research Society