To compare the results of randomized controlled trials versus observational studies in meta-analyses of digestive surgical topics.
While randomized controlled trials have been recognized as providing the highest standard of evidence, claims have been made that observational studies may overestimate treatment benefits. This debate has recently been renewed, particularly with regard to pharmacotherapies.
The PubMed (1966 to April 2004), EMBASE (1986 to April 2004) and Cochrane databases (Issue 2, 2004) were searched to identify meta-analyses of randomized controlled trials in digestive surgery. Fifty-two outcomes of 18 topics were identified from 276 original articles (96 randomized trials, 180 observational studies) and included in meta-analyses. All available binary data and study characteristics were extracted and combined separately for randomized and observational studies. In each selected digestive surgical topic, summary odds ratios or relative risks from randomized controlled trials were compared with observational studies using an equivalent calculation method.
Significant between-study heterogeneity was seen more often among observational studies (5 of 12 topics) than among randomized trials (1 of 9 topics). In 4 of the 16 primary outcomes compared (10 of 52 total outcomes), summary estimates of treatment effects showed significant discrepancies between the two designs.
One fourth of observational studies gave different results than randomized trials, and between-study heterogeneity was more common in observational studies in the field of digestive surgery.
Comparison of the results of randomized controlled trials versus observational studies in meta-analyses of digestive surgical topics was performed. One fourth of observational studies gave different results than randomized trials, and between-study heterogeneity was more common in observational studies in the field of digestive surgery.
From the *Department of Digestive Surgery, Kyoto Prefectural University of Medicine, Kyoto, Japan; †Department of Clinical Epidemiology, Kyoto University Graduate School of Medicine, Kyoto, Japan; ‡Department of Health Informatics, Kyoto University School of Public Health, Kyoto, Japan; and §Department of Medicine, Fujita Health University School of Medicine, Aichi, Japan.
Supported by a Health and Labour Sciences Research Grant (Health Technology Assessment) from the Ministry of Health, Labour and Welfare, Japan.
Reprints: Takeo Nakayama, MD, PhD, Department of Health Informatics, Kyoto University School of Public Health, Konoe-cho, Yoshida, Sakyo-ku, Kyoto 606-8501, Japan. E-mail: firstname.lastname@example.org.