Journal Logo

Design, Conduct, and Interpretation of Nonrandomized Orthopaedic Studies: A Practical Approach

(All) Evidence Matters

Bhandari, Mohit MD, MSc, FRCSC1; Morshed, Saam MD, MPH2; Tornetta, Paul III MD3; Schemitsch, Emil H. MD, FRCSC4

doi: 10.2106/JBJS.H.01747
Design, Conduct, and Interpretation of Nonrandomized Orthopaedic Studies--A Practical Approach

1Division of Orthopaedic Surgery, Department of Surgery, McMaster University, 293 Wellington Street North, Suite 110, Hamilton, ON L8L 2X2, Canada. E-mail address:

2Department of Orthopaedic Surgery, University of California at San Francisco, Orthopaedic Trauma Institute at San Francisco General Hospital, 1001 Potrero Avenue, Room 3A-36, San Francisco, CA 94110

3Department of Orthopaedic Surgery, Boston Medical Center, 850 Harrison Avenue, D2N, Boston, MA 02118

4Division of Orthopaedic Surgery, St. Michael's Hospital, 55 Queen Street East, #800, Toronto, ON M5C 1R6, Canada

The most sophisticated practice of evidence-based orthopaedics requires a clear delineation of relevant clinical questions, a thorough search of the literature relating to the questions, a critical appraisal of available evidence and its applicability to the clinical situation, and a balanced application of the conclusions to the clinical problem. The balanced application of the evidence (i.e., clinical decision-making) is the central point of practicing evidence-based medicine and involves an integration of our clinical expertise and judgment with patients' and societal values and with the best available research evidence.

While emphasis in orthopaedics is placed on the randomized trial as the optimal research design to evaluate a surgical intervention, randomized controlled trials are not always practical or feasible. At the time of this writing, almost 90% of the orthopaedic literature represents research findings from nonrandomized study designs, ranging from the more popular case series to prospective cohort studies1.

Studies in which randomization is not employed can be referred to as nonrandomized, or observational, study designs. There has been considerable debate about whether the results of nonrandomized studies are consistent with the results of randomized controlled trials2,3. Nonrandomized studies have been reported to both overestimate and underestimate treatment effects2,3. However, these findings have not been supported in two published papers that identified nonsignificant differences in results between randomized controlled trials and observational studies4,5.

Although surgeons may perceive that evidence-based medicine mandates a strict adherence to evidence from randomized trials, it more accurately involves informed and effective use of all types of evidence (from meta-analysis of randomized trials to the results reported in individual case series and case reports). This supplement highlights critical issues in the design, conduct, and interpretation of observational study designs in orthopaedic surgery. Using contributions from experienced surgeons and researchers, we provide both basic and advanced topics, with the primary objective of offering a balanced discussion that is relevant to the spectrum of clinical research experience of The Journal's readership.

Disclosure: The authors did not receive any outside funding or grants in support of their research for or preparation of this work. Neither they nor a member of their immediate families received payments or other benefits or a commitment or agreement to provide such benefits from a commercial entity.

Back to Top | Article Outline


1. Obremskey WT, Pappas N, Attallah-Wasif E, Tornetta P 3rd, Bhandari M. Level of evidence in orthopaedic journals. J Bone Joint Surg Am. 2005;87:2632-8.
2. Ioannidis JP, Haidich AB, Pappa M, Pantazis N, Kokori SI, Tektonidou MG, Contopoulos-Ioannidis DG, Lau J. Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA. 2001;286:821-30.
3. Kunz R, Oxman AD. The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials. BMJ. 1998;317:1185-90.
4. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000;342:1887-92.
5. Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. N Engl J Med. 2000;342:1878-86.
Copyright © 2009 by The Journal of Bone and Joint Surgery, Incorporated