Journal Logo

Institutional members access full text with Ovid®

Evaluating Agreement: Conducting a Reliability Study

Karanicolas, Paul J. MD, PhD1; Bhandari, Mohit MD, MSc, FRCSC1; Kreder, Hans MD, FRCSC2; Moroni, Antonio MD3; Richardson, Martin MD4; Walter, Stephen D. PhD1; Norman, Geoff R. PhD1; Guyatt, Gordon H. MD, MSc1; on Behalf of the Collaboration for Outcome Assessment in Surgical Trials (COAST) Musculoskeletal Group*

doi: 10.2106/JBJS.H.01624
Design, Conduct, and Interpretation of Nonrandomized Orthopaedic Studies--A Practical Approach

Instruments that are useful in clinical or research practice will, when the object of measurement is stable, yield similar results when applied at different times, in different situations, or by different users. Studies that measure the relation of differences between patients or subjects and measurement error (reliability studies) are becoming increasingly common in the orthopaedic literature. In this paper, we identify common aspects of reliability studies and suggest features that improve the reader's confidence in the results. One concept serves as the foundation for all further consideration: in order for a reliability study to be relevant, the patients, raters, and test administration in the study must be similar to the clinical or research context in which the instrument will be used. We introduce the statistical measures that readers will most commonly encounter in reliability studies, and we suggest an approach to sample-size estimation. Readers interested in critically appraising reliability studies or in developing their own reliability studies may find this review helpful.

1Department of Clinical Epidemiology and Biostatistics, McMaster University, 293 Wellington Street North, Suite 110, Hamilton, ON L8L 2X2, Canada. E-mail address for M. Bhandari:

2Department of Surgery, University of Toronto, Sunnybrook Campus, MG365, 2075 Bayview Avenue, Toronto, ON M4N 3M5, Canada

3Rizzoli Orthopaedic Institute, University of Bologna, Via G.C. Pupilli 1, Bologna 40136, Italy

4Department of Orthopaedic Surgery, Royal Melbourne Hospital, University of Melbourne, Grattan Street, Parkville 3050, Melbourne, Victoria, Australia

Copyright © 2009 by The Journal of Bone and Joint Surgery, Incorporated
You currently do not have access to this article

To access this article: