Secondary Logo

Journal Logo

The Importance of Outcome Measurement in Orthopaedics

Ayers, David, C., MD1; Bozic, Kevin, J., MD, MBA2, a

Clinical Orthopaedics and Related Research: November 2013 - Volume 471 - Issue 11 - p 3409–3411
doi: 10.1007/s11999-013-3224-z
Orthopaedic Healthcare Worldwide

1 Department of Orthopaedic Surgery, University of Massachusetts School of Medicine, 01605, Worcester, MA, USA

2 Department of Orthopaedic Surgery, University of California San Francisco, 500 Parnassus Ave. MU320W, 94143, San Francisco, CA, USA

a e-mail;

Received: July 26, 2013 / Accepted: July 31, 2013 / Published online: August 10, 2013

Note from the Editor-in-Chief:

We are pleased to present to readers of Clinical Orthopaedics and Related Research®the latest Orthopaedic Healthcare Worldwide column. This section explores the political, social, and economic issues associated with delivering musculoskeletal care in the many environments in which our specialty is practiced, both in the United States and around the world. We welcome reader feedback on all of our columns and articles; please send your comments

Each author certifies that he or she, or a member of his or her immediate family, has no commercial associations (eg, consultancies, stock ownership, equity interest, patent/licensing arrangements, etc) that might pose a conflict of interest in connection with the submitted article.

All ICMJE Conflict of Interest Forms for authors and Clinical Orthopaedics and Related Research® editors and board members are on file with the publication and can be viewed on request.

The opinions expressed are those of the writers, and do not reflect the opinion or policy of CORR® or the Association of Bone and Joint Surgeons®.

Value in healthcare can be defined as the ratio of the benefits that accrue to patients from treatment received to the dollars spent providing that treatment. It is widely believed that healthcare in the United States is moving from a volume-driven paradigm to a system that increasingly rewards value in healthcare delivery. How can orthopaedic surgeons measure the value and the quality of care they deliver to their patients? The American Academy of Orthopedic Surgeons (AAOS), in its position paper Public Reporting of Provider Performance [1] states: “The AAOS believes that systems for measuring and reporting quality in health care should continue to evolve and expand. The current generation of quality measures, which primarily rely on process measures and administrative data, have not yet been proven to accurately correlate with improved functional outcomes, which are the primary outcomes of interest to patients who undergo orthopedic procedures.” Most patients elect to undergo orthopaedic surgery to relieve pain and improve physical function. In order to demonstrate value, orthopaedic surgeons must assess the results of their surgical interventions by measuring the degree of pain relief and improved physical function the patient experiences after surgery.

Orthopaedic joint replacement registries have been established in many countries around the world. These registries typically focus on the implant as the primary unit of comparison, based on the idea that if the implant was not revised, the joint replacement was a success. In the case of revisions, the number of years to revision is recorded. While such registries have provided valuable data to compare the longevity of various implants, it is time to move beyond implant-centric registries to include other outcomes of interest to patients. Registries should include pain and physical function as important patient reported outcome measures before and after joint replacement surgery. This concept is a change in the traditional paradigm of a joint replacement registry from an implant-centric registry to a patient-centered registry [3].

Patient reported outcomes (PROs) measures rely on validated patient questionnaires that have been used extensively in clinical orthopaedic research, as they accurately assess the patient’s pain, stiffness, and function. PROs also provide the patient and the care team with summary scores delineating the patient’s physical and emotional health and function. Some practices have developed systems to collect and incorporate PROs into routine clinical practice [5]. It is critical that the PRO data collection process does not impact the office workflow or interfere with the patient-clinician interaction. PROs should preferably be collected outside the exam room before the encounter with the physician, either on a tablet in the waiting room, or at home online before the visit. PROs should be scored and available at the time of the encounter with the orthopaedic surgeon. The data provides added value to the patient and the surgeon to track patient pain and function through time, assess treatment and surgical outcome, and assist with shared medical decision making [2]. While some academic practices have successfully established a system to routinely collect PROs in a busy clinical environment, most orthopaedic practices in the United States view the collection of PROs as time consuming, cumbersome, and costly. Many providers still believe collecting this data is extremely labor intensive, involving the use of pencil and paper to collect information. We suspect these providers are not familiar with the major advances that have been made in PROs, including data collection using computerized adaptive technology (CAT), and Internet-supported software platforms.

Despite the challenges involved in collecting PRO data, orthopaedic surgeons, multispecialty group practices, hospitals, and insurers understand the value of PROs when assessing the degree to which a particular healthcare intervention improves quality of life and function. These data have proven valuable to orthopaedic surgeons in discussions and negotiations with, payers, hospitals, accountable care organizations, and referring physicians.

Because the steps and resources necessary to establish a mechanism to routinely collect PROs can be seen as daunting, another option is to join an established registry that has a successful program and software platform already in place. A patient-centered registry can provide participating surgeons and hospitals with the software platform and clinical pathway to successfully collect PROs during routine clinical practice without interfering with practice efficiency or patient or surgeon satisfaction [3-5]. Successful registries employ trained data collection specialists that will work to ensure the success of the PROs collection process. Data collection is Internet-based via a computer or a tablet in the waiting room. The registry also provides real-time scoring of the PROs that provide actionable data available at the time of the patient office encounter, and can be used in shared medical decision making. A successful program also provides a national comparator group of patients so the surgeon/practice/hospital can compare their patients’ outcomes with national data. Comparisons should include PROs on pain and function as well as complications, readmissions, and reoperations. In order to facilitate accurate comparisons of patient outcomes, risk adjustment of the data is important and should include risk adjustment for both medical co-morbid conditions and orthopedic co-morbid conditions. The Agency for Healthcare Research and Quality-funded Functional Outcome Research and Comparative Effectiveness in Total Joint Replacement (FORCE-TJR) joint replacement registry is an example of a nationally-representative joint replacement registry that currently provides such a software platform, data collection capabilities, and data analysis to its members [3, 4]. FORCE-TJR is a consortium of a national network of orthopaedic surgeons supported by a team of researchers at the University of Massachusetts Medical School. This consortium has developed a software platform that is patient-centered and user-friendly and has been able to collect patient-reported clinical data through the Internet or in the waiting room using CAT technology. The FORCE-TJR national data set is providing valuable baseline PRO data on patients undergoing total hip and knee replacement surgery. The consortium is also providing information on the influence of patient demographics (age, gender, race, socio-economic status) and patient clinical factors (BMI, diabetes, and other medical comorbid conditions and orthopaedic comorbid conditions) on the change in PROs following joint replacement surgery. Other registries in the United States are in the process of developing or currently have similar capabilities to collect and report outcomes, including the American Joint Replacement Registry, California Joint Replacement Registry, Kaiser Permanente Joint Replacement Registry, The Michigan Arthroplasty Registry Collaborative Quality Initiative, and The Virginia Joint Replacement Registry. Each registry has different software platforms and differences in patient data collection pathways and patient enrollment. Providers who are interested in measuring and comparing their PROs and other relevant outcomes data with similar data from other providers should consider joining a registry as one way of gaining access to state-of-the-art technology for data collection.

Regardless of the endpoint used in clinical data registries, whether it be revision surgery (implant longevity) or patient reported outcomes, the importance of maximizing compliance with followup (either to confirm implant survivorship or to assess pain and function through PRO surveys) should not be underestimated. Registries that rely on patients to return to the same practice for followup run the risk of inflating the apparent benefits of treatment, as patients with worse functional outcomes or failed joint replacements necessitating revision may be more likely to seek care from a different surgeon. Therefore, all clinical data registries should have a system in place for tracking patients to ensure complete followup in order to minimize the risk of selection bias in outcomes reporting. One strategy to improve compliance with PRO measurement is for the registry to collect data directly from the patient through the Internet. This separates the data collection process from the surgeon’s office.

As orthopaedic surgery enters the era of public reporting of outcome data, and reimbursement is increasingly based on the value of the care provided to patients, critically evaluating one’s own patient outcomes provides a significant advantage in successfully managing the transition from volume to value.

Back to Top | Article Outline


1. American Academy of Orthopaedic Surgeons (AAOS). Position statement: public reporting of provider performance. 2012. Available at: Accessed July 26, 2013.
2. Ayers DC, Zheng H, Franklin PD. Integrating patient-reported outcomes (PROs) into orthopaedic clinical practice: proof of concept from FORCE-TJR. Clin Orthop Relat Res. DOI 10.1007/s11999-013-3143-z.
3. Franklin, PD., Allison, JJ. and Ayers, DC. Beyond joint implant registries: a patient-centered research consortium for comparative effectiveness in total joint replacement. JAMA. 2012; 308: 1217-1218. 10.1001/jama.2012.12568
4. Franklin PD, Harrold LR, Ayers DC. Incorporating PROs in the FORCE total joint replacement registry: challenges and opportunities. Clin Orthop Relat Res. DOI 10.1007/s11999-013-3193-2.
5. QualityMetric. UMass Memorial Medical Center. Providing comprehensive care with the DYNHA SF-36 health survey. Available at: Accessed July 26, 2013.
© 2013 Lippincott Williams & Wilkins LWW