Where Are We Now?
The current trend toward value-based health care requires a focus on the quality of our outcomes and the costs paid to achieve those outcomes. Measuring quality is complex but remains the foundation for improvement. Numerous quality measures are currently being used in attempts to improve care, but also to hold clinicians and hospitals accountable via payment programs or ratings. Unfortunately, many measures may not be as meaningful as hoped, with their validity being scrutinized . In addition to being evidence-based, valid, and reliable, a meaningful measure must also have appropriate risk adjustment to account for differences in baseline patient characteristics [9, 11].
Large-database investigations previously identified variables such as age or cardiac disease that were associated with mortality [4, 13] in patients undergoing total shoulder arthroplasty (TSA). Johnson and colleagues  demonstrated that a more complex comorbidity index, such as the American Society of Anesthesiologists classification, was also associated with complications after TSA. The present study by Fu and colleagues , using the National Surgical Quality Improvement Program (NSQIP) database, suggests neither demographic variables, such as age or BMI nor comorbidity indices, such as the modified Charlson Comorbidity Index or American Society of Anesthesiologists classification, reach even fair discriminative ability for adverse events after TSA.
Despite clinical data that are longitudinally maintained by medical record reviewers, the NSQIP database can suffer from data quality issues, particularly missing data [5, 10]. A possible result of missing data (and other quality-control problems) is that some studies [1, 3, 4, 8, 13] suggest correlations between some factors and outcomes, whereas the present study suggests poor discrimination. Another study  suggested that obesity was not associated with differences in early outcomes. This divergence should cause us to consider the limitations both of small clinical studies, as well as large databases to adequately inform and quantify risk-adjustment strategies for quality metrics.
Where Do We Need To Go?
Perhaps MacLean and colleagues  are right to call for a “time out” in performance measurement. The science of measuring quality and adequately adjusting for risk will need to catch up with the enthusiasm for value-based care. Research measuring clinical outcomes is heavily dependent on large databases, particularly for rare outcomes, such as surgical complications and adverse events. The use of electronic medical records (EMRs) is rapidly evolving, and our ability to ensure the quality of the data used in this research will be critical to the development of appropriate indices to stratify risk and therefore inform patients and surgeons. This information can also be used to determine not only what type of care we offer and to whom, but also the most cost-effective environment in which to perform it, balancing known risks with patient safety.
The implications of this area of research are tremendous and, if not carefully considered and applied, can lead to unintended consequences. One example is the use of strict cutoffs for known risk factors, such as BMI or hemoglobin A1c levels. Wang and colleagues  demonstrated that applying inflexible thresholds in these variables may worsen racial or socioeconomic disparities in joint arthroplasty.
Many questions remain as to how we can harness large databases to measure outcomes that are appropriately risk adjusted. Can statistical methods for handling missing data sufficiently improve our ability to make inferences? Should NSQIP-specific comorbidity indices be developed to use the database in its current form as the authors of the present study suggest? How can we improve the quality of various databases and registries to allow for more accurate outcomes assessments?
How Do We Get There?
Although it may be attractive to adapt an index measure to an existing database, this is unlikely to further our goal of accurate outcomes assessment. Even with complete clinical data and advanced statistical methods, we may not capture the necessary elements for precise risk adjustment of surgical outcomes in our current databases. Ultimately, we need a diverse set of inputs, including identifiable demographic data and comorbidities, as well as other pertinent details from domains, such as surgical indications, pathologic severity, socioeconomic factors, environmental factors, and even educational background.
The collection of these types of data in these diverse domains, though burdensome, is necessary to elevate the precision of quality measurement after surgery. Because of our increasing ability to manage large volumes of data and the widespread use of EMRs, multicenter registries are becoming more common and feasible, even in non-nationalized healthcare systems. Prospective, large trials organized by subspecialty societies or private networks have helped initiate these efforts and maintained them across diverse geographic areas by offering financial and administrative support [6, 11]. Individual surgeons and practices can join these efforts with fewer and fewer hurdles.
Enhancing the capabilities of our EMRs can also help the cause. By aligning documentation requirements for billing with that of clinical work, and therefore clinical outcomes research, we can improve the quality of our databases and registries, allowing us to truly harness their “big data” potential. Several initiatives are underway, such as the formation of specialty and subspecialty-specific steering committees by EMR vendors. These groups, consisting of active clinicians and EMR developers, continue to work on the development of EMR tools to allow for efficient clinical documentation and synchronous research data collection. Orthopaedic societies are also helping to organize multicenter groups, many of which share a common EMR, to align efforts to answer our most complex and challenging quality questions.
1. Fu MC, Boddapati V, Dines DM, Warren RF, Dines JS, Gulotta LV. The impact of insulin dependence on short-term postoperative complications in diabetic patients undergoing total shoulder arthroplasty. J Shoulder Elbow Surg. 2017;26:2091-2096.
2. Fu MC, Ondeck NT, Nwachukwu BU, Garcia GH, Gulotta LV, Verma NN, Grauer JN. What associations exist between comorbidity indices and postoperative adverse events after total shoulder arthroplasty? Clin Orthop Relat Res. [Published online ahead of print]. DOI: 10.1097/CORR.0000000000000624
3. Garcia GH, Fu MC, Dines DM, Craig EV, Gulotta LV. Malnutrition: A marker for increased complications, mortality, and length of stay after total shoulder arthroplasty. J Shoulder Elbow Surg.
4. Griffin JW, Hadeed MM, Novicoff WM, Browne JA, Brockmeier SF. Patient age is a factor in early outcomes after shoulder arthroplasty. J Shoulder Elbow Surg. 2014;23:1867-1871.
5. Hamilton BH, Ko CY, Richards K, Hall BL. Missing data in the American College of Surgeons National Surgical Quality Improvement Program are not missing at random: Implications and potential impact on quality assessments. J Am Coll Surg. 2010;210:125-139.
6. Harris JD, Pedroza A, Jones GL, MOON (Multicenter Orthopedic Outcomes Network) Shoulder Group. Predictors of pain and function in patients with symptomatic, atraumatic full-thickness rotator cuff tears: A time-zero analysis of a prospective patient cohort enrolled in a structured physical therapy program. Am J Sports Med. 2012;40:359-366.
7. Jiang JJ, Somogyi JR, Patel PB, Koh JL, Dirschl DR, Shi LL. Obesity is not associated with increased short-term complications after primary total shoulder arthroplasty. Clin Orthop Relat Res. 2016;474:787-795.
8. Johnson CC, Sodha S, Garzon-Muvdi J, Petersen SA, McFarland EG. Does preoperative American Society of Anesthesiologists score relate to complications after total shoulder arthroplasty? Clin Orthop Relat Res. 2014;472:1589-1596.
9. MacLean CH, Kerr EA, Qaseem A. Time out—Charting a path for improving performance measurement. N Engl J Med. 2018;378:1757-1761.
10. Parsons HM, Henderson WG, Ziegenfuss JY, Davern M, Al-Refaie WB. Missing data and interpretation of cancer surgery outcomes at the American College of Surgeons National Surgical Quality Improvement Program. J Am Coll Surg. 2011;213:379-391.
11. Thigpen CA, Floyd SB, Chapman C, Tokish JM, Kissenberth MJ, Hawkins RJ, Brooks JM. Comparison of surgeon performance of rotator cuff repair: Risk adjustment toward a more accurate performance measure. J Bone Joint Surg Am. 2018;100:2110-2117.
12. Wang AY, Wong MS, Humbyrd CJ. Eligibility criteria for lower extremity joint replacement may worsen racial and socioeconomic disparities. Clin Orthop Relat Res. 2018;476:2301-2308.
13. Waterman BR, Dunn JC, Bader J, Urrea L, Schoenfeld AJ, Belmont PJ Jr. Thirty-day morbidity and mortality after elective total shoulder arthroplasty: Patient-based and surgical risk factors. J Shoulder Elbow Surg. 2015;24:24-30.