Meaningful Thresholds for the Volume-Outcome Relationship in Total Knee Arthroplasty

Wilson, Sean BA; Marx, Robert G. MD; Pan, Ting-Jung MPH; Lyman, Stephen PhD

Journal of Bone & Joint Surgery - American Volume:
doi: 10.2106/JBJS.15.01365
Scientific Articles
Abstract

Background: Increasing evidence supports the finding that patients undergoing a total knee arthroplasty with high-volume physicians and hospitals achieve better outcomes. Unfortunately, the existing definitions for high-volume surgeons and hospitals are highly variable and entirely arbitrary. The aim of this study was to identify a set of meaningful hospital and surgeon total knee arthroplasty volume thresholds.

Methods: Using 289,976 patients undergoing primary total knee arthroplasty from an administrative database, we applied stratum-specific likelihood ratio (SSLR) analysis of a receiver operating characteristic (ROC) curve to generate sets of volume thresholds most predictive of adverse outcomes. The outcomes considered for surgeon volume included 90-day complication and 2-year revision. For hospital volume, we considered 90-day complications and 90-day mortality.

Results: SSLR analysis of the ROC curves for 90-day complication and 2-year revision rates by surgeon volume identified four volume categories: 0 to 12, 13 to 59, 60 to 145, and ≥146 total knee arthroplasties per year. Complication rates decreased significantly (p < 0.05) in progressively higher-volume categories. Revision rates followed a similar pattern, but did not decrease between surgeons performing 60 to 145 arthroplasties per year and those performing ≥146 arthroplasties per year. SSLR analysis of 90-day complication and 90-day mortality rates by hospital volume also identified four volume categories: 0 to 89, 90 to 235, 236 to 644, and ≥645 total knee arthroplasties per year. Complication rates decreased significantly (p < 0.05) in progressively higher-volume categories, but the rates did not decrease between hospitals performing 236 to 644 arthroplasties per year and those performing ≥645 arthroplasties per year. Mortality rates for hospitals with ≥645 total knee arthroplasties per year were significantly lower (p < 0.05) than those below the threshold.

Conclusions: Our study supports the use of SSLR analysis of ROC curves for risk-based volume stratification in total knee arthroplasty volume-outcomes research. SSLR analysis established meaningful volume definitions for low, medium, high, and very high-volume total knee arthroplasty surgeons and hospitals. This should help patients, surgeons, hospitals, and policymakers to make decisions with regard to the optimal delivery of total knee arthroplasty.

Level of Evidence: Therapeutic Level III. See Instructions for Authors for a complete description of levels of evidence.

Author Information

1Weill Cornell Medical College, New York, NY

2Healthcare Research Institute, Hospital for Special Surgery, New York, NY

E-mail address for S. Lyman: lymans@hss.edu

Article Outline

Increasing evidence supports the finding that patients undergoing a variety of procedures and treatments with high-volume providers, meaning physicians and hospitals that undertake the procedure with relatively high frequency, achieve better outcomes1-3. In total knee arthroplasty, studies have associated higher surgeon volume with lower mortality, infection, and transfusion rates; shorter procedure times; decreased lengths of stay; and superior patient-reported outcomes4-8. High hospital volume has been associated with lower rates of venous thromboembolism, pneumonia, and mortality8,9. Unfortunately, the existing definitions for high-volume surgeons and hospitals are highly variable and entirely arbitrary5.

Indeed, the common feature of volume-outcomes research across surgical specialties is this arbitrariness. The stratification of hospital and surgeon volume from a continuous variable into categories (for example, low, medium, high volume) is useful for ease of interpretation and the implementation of interventions, but there is no consensus on how to define each category. In the absence of an accepted rigorous methodology to identify volume thresholds, volume outcomes studies have defaulted to assigning arbitrary cutoffs or ranking patients by volume and splitting them into quantiles for analysis10. The result is potential information loss and the invention of volume categories that have limited rational basis11,12. Consequently, published volume categories are remarkably inconsistent between studies. For example, a systematic review by Lau et al. on the total knee arthroplasty surgeon volume-outcomes relationship found definitions for a “low-volume surgeon” ranging among studies from surgeons performing <3 to <52 surgical procedures per year, with a “high-volume surgeon” variably defined as a surgeon performing >5 to >70 surgical procedures per year5.

The arbitrary and variable nature of volume thresholds becomes problematic when they are appropriated as structural measures of quality, a common practice in the health-care quality realm13-16. For example, the Leapfrog Group for patient safety has been using volume standards as an important quality measure for high-risk procedures for more than a decade13. More recently, three major hospital systems announced that they plan to adopt minimum hospital volume standards for certain procedures17. Their proposed volume standards for total knee arthroplasty are 25 total knee arthroplasties per year for surgeons and 50 total knee arthroplasties per year for hospitals18. Accumulating evidence from volume-outcomes research supports these interventions but supplies little information about where exactly the thresholds should fall. A responsible and clinically useful implementation of volume-outcomes research hinges on establishing meaningful volume thresholds.

In this study, we applied stratum-specific likelihood ratio (SSLR) analysis, a method of analyzing receiver operating characteristic (ROC) curves, to identify volume thresholds associated with different risks of adverse outcomes. Peirce and Cornell first described SSLR in the context of diagnostic testing research as a method of establishing numerous thresholds on an ROC curve19. Applied to volume-outcomes research, SSLR analysis can establish multiple volume strata with different associated risks, as Russo et al. demonstrated in the risk stratification of hospitals performing heart transplants20. SSLR analysis offers an enhanced analysis of ROC curves, but has been used only sporadically and mostly in diagnostic testing research19-23. Applied to volume-outcomes research, SSLR identifies the volume thresholds where the risk of adverse events drops most abruptly19. Using SSLR, we sought to answer the following two questions about the volume effect in total knee arthroplasty: (1) What surgeon-volume thresholds are most predictive of 90-day complication and 2-year revision? (2) What hospital-volume thresholds are most predictive of 90-day complication and 90-day mortality?

Back to Top | Article Outline

Materials and Methods

Using SSLR, we generated two sets of surgeon-volume categories using two end points: revision within 2 years and complications within 90 days. The 2-year revision end point was defined by revision total knee arthroplasty procedure codes (International Classification of Diseases, Ninth Revision, Clinical Modification [ICD-9-CM] codes 00.80-00.84, 81.55) during the index total knee arthroplasty admission or in a subsequent admission within 2 years. The 90-day complications included 19 categories of postoperative complications, identified using ICD-9-CM codes (Table I).

The two end points used to generate SSLR thresholds for hospital volume included the same 90-day complications and 90-day all-cause, in-hospital mortality.

For the purpose of sensitivity analyses, we also generated volume strata from quantile analysis, a common stratification approach, and area under the curve (AUC) analyses, as described by Ravi et al.24.

Back to Top | Article Outline
Data Sources and Study Population

The New York State Department of Health’s Statewide Planning and Research Cooperative System (SPARCS) is a comprehensive data reporting system that collects patient-level detail on all discharges from nonfederal acute-care hospitals. Our data set included patient-specific data from 1997 through 2011, as patient identifiers were not available before then.

The cohort for the 2-year revision model included New York residents undergoing a unilateral primary total knee arthroplasty (ICD-9-CM procedure code 81.54) from 1997 to 2009. Because ICD-9-CM coding does not establish laterality, we excluded patients with a diagnosis code indicating a prior total knee arthroplasty (ICD-9-CM code V43.65) to ensure that revisions were assigned to the correct primary procedure. A total of 233,859 patients met these criteria and were included. The analyses of 90-day complication and 90-day mortality included patients with surgical procedure dates before October 1, 2011. There were 289,976 patients who met these criteria. We defined annual volume as the number of total knee arthroplasties (from ICD-9-CM procedure codes) performed in the 365 days prior to the surgical procedure of interest.

Back to Top | Article Outline
Predictors

Multivariable analyses included patient-level factors associated with adverse outcomes in univariate analyses: age, race, sex, diagnosis, insurance type, and comorbidity burden. The categories and definitions for each variable are detailed in an earlier study using a similar data set25. Measures of annual hospital volume and surgeon experience (calculated from years since licensure) were included in each model. Although we report the Deyo modification of the Charlson Comorbidity Index in Table II, we used the Elixhauser Comorbidity Algorithm in the multivariable analyses, as it is more strongly associated with outcomes commonly available in administrative data sets26-29.

Back to Top | Article Outline
Statistical Analysis

A detailed technical description more fully describes SSLR analysis (see Appendix). SSLR is a modified approach to analyzing ROC curves, commonly used in diagnostic testing research. The first step is splitting the sample into as many strata as possible by the variable of interest. In this study, the initial number of strata is equal to the number of distinct annual volumes (i.e., 1, 2, 3… total knee arthroplasties per year). Next, an SSLR is calculated with 95% confidence intervals (95% CIs) using this equation, in which “abnormal” refers to the outcome of interest: SSLR = (Number of abnormal in nth stratum/Total number abnormal)/(Number normal in nth stratum/Total number normal).

If neighboring strata have overlapping 95% CIs, they are merged. This process continues until a significant difference in SSLRs (95% CIs that do not overlap) is observed between adjacent strata. A threshold is established at this gap. Merging of overlapping strata continues above and below the threshold, with additional thresholds established using the same approach.

For surgeon volume, we applied SSLR analysis to ROC curves for 2-year revision and 90-day complication rates to generate two sets of volume strata19. We repeated the process for hospital volume, using 90-day complications and 90-day mortality. Although all of these outcomes are likely influenced by a combination of surgeon and hospital level factors, we reasoned that 90-day mortality is more dependent on postoperative care in the hospital and 2-year revision is more surgeon-dependent, as early implant failure is often attributed to technical error, although, to our knowledge, there have been limited data supporting this30,31. We included 90-day complications for both. A Cox proportional hazards analysis measured the effect of surgeon volume strata on the 2-year revision risk. Multivariable logistic regression analyses measured the effect of volume strata on 90-day complications and 90-day mortality. A critical p value of <0.05 was set as the threshold for significance. SSLR analyses were performed in Excel 2010 (Microsoft), and all other statistical analyses were performed using SAS 9.3 (SAS Institute).

Back to Top | Article Outline

Results

The mean age (and standard deviation) for patients undergoing primary total knee arthroplasty was 67.4 ± 10.5 years. The cohort was 67% female, 76% white, and 9% black (Table II). Medicare (58%) and private insurance (33%) were the most common payer types. The most common diagnoses were osteoarthritis (93%) and inflammatory arthritis (5%). With regard to the Charlson Comorbidity Index score, 65% of patients had a score of 0, 26% of patients had a score of 1, and 8% of patients had a score of ≥2. Surgeon volume ranged from 0 to 565 total knee arthroplasties per year. Hospital volume ranged from 0 to 3,606 total knee arthroplasties per year. The overall rates were 2.4% for 2-year revision, 7.5% for 90-day complications, and 0.28% for 90-day mortality.

Back to Top | Article Outline
Surgeon-Volume Strata and 2-Year Revision Risk

SSLR analysis of the ROC curve for 2-year revision rates by surgeon volume identified two thresholds at 12 and 59 surgical procedures per year, yielding three volume strata: 0 to 12 arthroplasties (low-volume surgeons), 13 to 59 arthroplasties (medium-volume surgeons), and ≥60 arthroplasties (high-volume surgeons) (Fig. 1).

Unadjusted revision rates were 3.3% for low-volume surgeons, 2.6% for medium-volume surgeons, and 2.0% for high-volume surgeons. The volume strata were significantly associated with 2-year revision in a Cox proportional hazards analysis, with higher rates of revision for low-volume surgeons (hazard ratio, 1.56 [95% CI, 1.44 to 1.69]) and medium-volume surgeons (hazard ratio, 1.26 [95% CI, 1.19 to 1.34]) compared with their high-volume counterparts (Fig. 2).

Back to Top | Article Outline
Surgeon-Volume Strata and 90-Day Complication Risk

SSLR analysis of the ROC curve for 90-day complication rates by surgeon volume identified three thresholds at 11, 64, and 145 surgical procedures per year, yielding four volume strata: 0 to 11 total knee arthroplasties (low-volume surgeons), 12 to 64 total knee arthroplasties (medium-volume surgeons), 65 to 145 total knee arthroplasties (high-volume surgeons), and ≥146 total knee arthroplasties (very high-volume surgeons).

Unadjusted complication rates were 9.8% for low-volume surgeons, 7.9% for medium-volume surgeons, 6.9% for high-volume surgeons, and 5.7% for very high-volume surgeons. The volume strata from the SSLR analysis of complication risk were strongly associated with 90-day complication rates. Low-volume surgeons (odds ratio, 1.85 [95% CI, 1.75 to 1.97]), medium-volume surgeons (odds ratio, 1.49 [95% CI, 1.41 to 1.56]), and high-volume surgeons (odds ratio, 1.27 [95% CI, 1.20 to 1.34]) had significantly higher complication rates compared with very high-volume surgeons (Fig. 2).

As evident from these results, the number and location of SSLR-generated thresholds can vary depending on the outcome of interest. To combine the slightly divergent sets of thresholds, we favored the 2-year revision cutpoints, reasoning that total knee arthroplasty revision is a more serious and surgeon-dependent outcome. The very high-volume threshold from the 90-day complications analysis demarcates the highest strata (Table III).

Back to Top | Article Outline
Hospital-Volume Strata and 90-Day Complication Risk

SSLR analysis of the ROC curve for 90-day complication rates by hospital volume identified two significant thresholds, 89 and 235 total knee arthroplasties per year, yielding three volume strata: 0 to 89 arthroplasties (low-volume hospitals), 90 to 235 arthroplasties (medium-volume hospitals), and ≥236 arthroplasties (high-volume hospitals).

Complication rates were 9.2% for low-volume hospitals, 7.6% for medium-volume hospitals, and 6.8% for high-volume hospitals. Low-volume hospitals (odds ratio, 1.37 [95% CI, 1.32 to 1.42]) and medium-volume hospitals (odds ratio, 1.13 [95% CI, 1.10 to 1.17]) had significantly higher complication rates compared with high-volume hospitals (Fig. 3).

To investigate the simultaneous effect of hospital and surgeon volume on 90-day complications, we generated volume categories that combined the surgeon and hospital strata (i.e., low-volume surgeon and low-volume hospital, low-volume surgeon and medium-volume hospital, and so forth) from the complications SSLR analyses (Table IV). Although there is overlap of the odds ratios between many of the volume categories, likely because of smaller sample sizes, there is a clear trend toward lower 90-day complication rates at higher surgeon volumes at a fixed hospital volume (and vice versa).

Back to Top | Article Outline
Hospital-Volume Strata and 90-Day Mortality Risk

SSLR analysis of the ROC curve for 90-day mortality rates by hospital volume identified one significant threshold: 644. This yielded two volume strata: 0 to 644 arthroplasties (low-volume hospitals) and ≥645 arthroplasties (high-volume hospitals).

Mortality rates were 0.31% for low-volume hospitals and 0.11% for high-volume hospitals. Low-volume hospitals (odds ratio, 2.21 [95% CI, 1.45 to 3.41]) had significantly higher mortality rates compared with high-volume hospitals (Fig. 3).

Back to Top | Article Outline
Sensitivity Analyses

AUC analysis generated a surgeon threshold of 65 total knee arthroplasties per year and a hospital threshold of 200 total knee arthroplasties per year, with low-volume surgeons (odds ratio, 1.39 [95% CI, 1.35 to 1.43]) and low-volume hospitals (odds ratio, 1.12 [95% CI, 1.09 to 1.16]) having significantly increased risks of 90-day complications. Comparing the results of multivariable models from quartile, AUC, and SSLR methods demonstrated that SSLR maximized effect sizes in all cases (results not shown).

Back to Top | Article Outline

Discussion

The association between higher surgeon and hospital volumes and improved patient outcomes has become evident for numerous orthopaedic procedures, including total knee arthroplasty5. Whereas previous studies have defined relative volumes in an arbitrary manner, we used SSLR analysis of 2-year revision, 90-day complication, and 90-day mortality risks to establish meaningful surgeon and hospital volume thresholds. An association between annual surgeon volume and postoperative complications has been established, but, to our knowledge, total knee arthroplasty volume-outcomes studies have not detected a volume effect on early revision, potentially because of insufficient sample sizes in previous analyses4-8,32.

Surgeons performing 0 to 12 total knee arthroplasties annually had significantly higher rates of 2-year revision and 90-day complications compared with their medium-volume and higher-volume counterparts. The “practice makes perfect” interpretation of the volume-outcomes relationship, which posits that higher-volume surgeons have better outcomes because they perform the task more frequently, would suggest that these low-volume surgeons could benefit from performing more total knee arthroplasties or by abandoning total knee arthroplasties and focusing on the procedures that they perform more frequently1,33,34.

Rates of revision and complications continued to decline past the initial threshold, with another threshold at 59 total knee arthroplasties per year. Surgeons above this threshold had lower rates of revisions and complications. Beyond this threshold, the volume effect for revision risk was muted, with no measurable difference between high-volume surgeons and very high-volume surgeons. However, the rate of 90-day complications was significantly lower for surgeons performing ≥145 total knee arthroplasties annually compared with those performing 60 to 144 total knee arthroplasties annually, suggesting that patients continue to obtain benefit from very high-volume surgeons, even when compared with high-volume surgeons.

Hospitals performing fewer than 90 total knee arthroplasties annually had significantly higher rates of 90-day complications compared with their medium-volume and higher-volume counterparts. Complication rates continued to decline past the initial threshold, with another threshold at 235 total knee arthroplasties per year. For mortality risk, we established an additional threshold at 644, with patients at hospitals operating above this volume having a significantly reduced mortality rate. Although complication rates were not significantly lower for hospitals above this threshold, the reduced mortality risk suggests that patients continue to obtain benefit from very high-volume hospitals.

The clinical utility of volume-outcomes research depends on establishing meaningful thresholds, and these SSLR-generated thresholds represent an improvement on previously established thresholds in that they are not arbitrary. Avoiding arbitrary thresholds is especially critical considering the increasing dependence of regulatory bodies and patient safety organizations on procedural volume as a measure of quality. Volume is a convenient measure because it is simple to monitor and makes intuitive sense: practice makes perfect35. A prominent example of this is the Leapfrog Group, a health-care system improvement initiative focused on high-risk elective procedures13. Critics of Leapfrog and similar center-of-excellence initiatives contend that the thresholds used to direct care to high-volume providers are frequently arbitrary and overly restrictive, potentially limiting access to care without improving quality14,15. Our methods address these concerns.

Because of the small differences in risk between volume strata, a productive SSLR analysis requires a very large sample size, making administrative databases ideal. However, these databases have inherent limitations, including a restricted set of patient and surgeon factors available for analysis36-38. Patient expectations, activity levels, intraoperative data (including prosthesis type and surgical approach), case complexity, and other variables are unavailable. Likewise, detailed surgeon-level data are only available for a fraction of patients, making it difficult to control for career volume and training. The outcomes available for study are also limited and do not include measures such as patient-reported pain, function, or mobility. This is important as SSLR thresholds are outcome-dependent. We reasoned that 2-year revision was the most surgeon-dependent outcome variable in our data set, but we also analyzed 90-day complications for comparison. Interestingly, the thresholds were similar. This gives us confidence that the outcomes that we chose are not only clinically relevant, but also accurately measurable. For the hospital analysis, we used complications and mortality, two important measures of quality.

Another limitation to using the database was that patients who sought follow-up care outside of New York State were not captured by SPARCS. As a result, revision and complication rates are likely underestimated. In addition, because patients preferentially regionalize to high-volume centers (and surgeons), the high-volume surgeons in our sample were about 1.7 times more likely to operate on patients from out of state, creating potential for bias in loss to follow-up39. To limit this effect, we restricted our study cohort to New York State residents.

It is important to note that our definition of surgeon and hospital volume is narrow: total knee arthroplasties per year. For example, an alternative (or additional) definition of surgeon volume could have been all knee surgical procedures in the past year. For hospitals, volume could be defined as arthroplasties (of any joint) per year or orthopaedic surgical procedures per year. These alternative definitions of volume could be used in future studies to address more nuanced questions about volume and patient outcomes.

This study adds to the already mounting evidence that “busier is better.” However, more importantly, it offers a novel and more rigorous approach to answering the question of how much busier. The next step is to determine the generalizability and reproducibility of this approach, which is beyond the scope of this study. This will require future studies that apply SSLR analysis to similar administrative data sets in different geographical locations.

Finally, our findings only associated volume and quality. They supported neither the practice-makes-perfect nor the selective-referral theory (that is, patients seek out more skilled providers, pushing up their volume) of the volume-outcomes effect.

In conclusion, our study supports the use of SSLR analysis of ROC curves for risk-based volume stratification in total knee arthroplasty volume-outcomes research. SSLR analysis established meaningful volume definitions for low-volume, medium-volume, high-volume, and very high-volume total knee arthroplasty surgeons and hospitals. This should help patients, surgeons, hospitals, and policymakers to make decisions with regard to optimal delivery of total knee arthroplasty.

Back to Top | Article Outline

Appendix Cited Here...

A description of the SSLR analysis used in this study is available with the online version of this article as a data supplement at jbjs.org.

Investigation performed at the Healthcare Research Institute, Hospital for Special Surgery, New York, NY

A commentary by Nicholas J. Giori, MD, PhD, is linked to the online version of this article at jbjs.org.

Disclosure: There was no external funding used for this study. On the Disclosure of Potential Conflicts of Interest forms, which are provided with the online version of the article, one or more of the authors checked “yes” to indicate that the author had a relevant financial relationship in the biomedical arena outside the submitted work.

Back to Top | Article Outline

References

1. Luft HS, Bunker JP, Enthoven AC. Should operations be regionalized? The empirical relation between surgical volume and mortality. N Engl J Med. 1979 ;301(25):1364–9.
2. Halm EA, Lee C, Chassin MR. Is volume related to outcome in health care? A systematic review and methodologic critique of the literature. Ann Intern Med. 2002 ;137(6):511–20.
3. Chowdhury MM, Dagash H, Pierro A. A systematic review of the impact of volume of surgery and specialization on patient outcome. Br J Surg. 2007 ;94(2):145–61.
4. Lavernia CJ, Guzman JF. Relationship of surgical volume to short-term mortality, morbidity, and hospital charges in arthroplasty. J Arthroplasty. 1995 ;10(2):133–40.
5. Lau RL, Perruccio AV, Gandhi R, Mahomed NN. The role of surgeon volume on patient outcome in total knee arthroplasty: a systematic review of the literature. BMC Musculoskelet Disord. 2012 ;13:250.
6. Kreder HJ, Grosso P, Williams JI, Jaglal S, Axcell T, Wal EK, Stephen DJ. Provider volume and other predictors of outcome after total knee arthroplasty: a population study in Ontario. Can J Surg. 2003 ;46(1):15–22.
7. Manley M, Ong K, Lau E, Kurtz SM. Total knee arthroplasty survivorship in the United States Medicare population: effect of hospital and surgeon procedure volume. J Arthroplasty. 2009 ;24(7):1061–7. Epub 2008 Nov 1.
8. Katz JN, Barrett J, Mahomed NN, Baron JA, Wright RJ, Losina E. Association between hospital and surgeon procedure volume and the outcomes of total knee replacement. J Bone Joint Surg Am. 2004 ;86(9):1909–16.
9. Singh JA, Kwoh CK, Boudreau RM, Lee GC, Ibrahim SA. Hospital volume and surgical outcomes after elective hip/knee arthroplasty: a risk-adjusted analysis of a large regional database. Arthritis Rheum. 2011 ;63(8):2531–9.
10. Birkmeyer JD, Siewers AE, Finlayson EVA, Stukel TA, Lucas FL, Batista I, Welch HG, Wennberg DE. Hospital volume and surgical mortality in the United States. N Engl J Med. 2002 ;346(15):1128–37.
11. Altman DG, Lausen B, Sauerbrei W, Schumacher M. Dangers of using “optimal” cutpoints in the evaluation of prognostic factors. J Natl Cancer Inst. 1994 ;86(11):829–35.
12. Shahian DM, Normand SLT. The volume-outcome relationship: from Luft to Leapfrog. Ann Thorac Surg. 2003 ;75(3):1048–58.
13. Milstein A, Galvin RS, Delbanco SF, Salber P, Buck CR Jr. Improving the safety of health care: the Leapfrog Initiative. Eff Clin Pract. 2000 ;3(6):313–6.
14. Livingston EH, Elliott AC, Hynan LS, Engel E. When policy meets statistics: the very real effect that questionable statistical analysis has on limiting health care access for bariatric surgery. Arch Surg. 2007 ;142(10):979–87.
15. Christian CK, Gustafson ML, Betensky RA, Daley J, Zinner MJ. The Leapfrog volume criteria may fall short in identifying high-quality surgical centers. Ann Surg. 2003 ;238(4):447–55; discussion 455-7.
16. Dimick JB, Osborne NH, Nicholas L, Birkmeyer JD. Identifying high-quality bariatric surgery centers: hospital volume or risk-adjusted outcomes? J Am Coll Surg. 2009 ;209(6):702–6.
17. Sternberg S. Hospitals move to limit low-volume surgeries. 2015 May 19. http://www.usnews.com/news/articles/2015/05/19/hospitals-move-to-limit-low-volume-surgeries. Accessed 2016 Jun 30.
18. Clark C. Limits urged on surgeries by low-volume providers. 2015 May 20. http://healthleadersmedia.com/content/QUA-316542/Limits-Urged-on-Surgeries-by-LowVolume-Providers. Accessed 2016 Jun 30.
19. Peirce JC, Cornell RG. Integrating stratum-specific likelihood ratios with the analysis of ROC curves. Med Decis Making. 1993 ;13(2):141–51.
20. Russo MJ, Iribarne A, Easterwood R, Ibrahimiye AN, Davies R, Hong KN, Ascheim DD, Gelijns AC, Naka Y. Post-heart transplant survival is inferior at low-volume centers across all risk strata. Circulation. 2010 ;122(11)(Suppl):S85–91.
21. Schmitz N, Kruse J, Tress W. Application of stratum-specific likelihood ratios in mental health screening. Soc Psychiatry Psychiatr Epidemiol. 2000 ;35(8):375–9.
22. Wada K, Tanaka K, Theriault G, Moriyama M, Satoh T, Aizawa Y. Application of the stratum-specific likelihood ratio (SSLR) analysis to results of a depressive symptoms screening survey among Japanese workers. Soc Psychiatry Psychiatr Epidemiol. 2007 ;42(5):410–3. Epub 2007 Mar 29.
23. Furukawa TA, Andrews G, Goldberg DP. Stratum-specific likelihood ratios of the general health questionnaire in the community: help-seeking and physical co-morbidity affect the test characteristics. Psychol Med. 2002 ;32(4):743–8.
24. Ravi B, Jenkinson R, Austin PC, Croxford R, Wasserstein D, Escott B, Paterson JM, Kreder H, Hawker GA. Relation between surgeon volume and risk of complications after total hip arthroplasty: propensity score matched cohort study. BMJ. 2014 ;348:g3284–3284.
25. Dy CJ, Marx RG, Bozic KJ, Pan TJ, Padgett DE, Lyman S. Risk factors for revision within 10 years of total knee arthroplasty. Clin Orthop Relat Res. 2014 ;472(4):1198–207. Epub 2013 Dec 18.
26. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998 ;36(1):8–27.
27. Southern DA, Quan H, Ghali WA. Comparison of the Elixhauser and Charlson/Deyo methods of comorbidity measurement in administrative data. Med Care. 2004 ;42(4):355–60.
28. Gordon M, Stark A, Sköldenberg OG, Kärrholm J, Garellick G. The influence of comorbidity scores on re-operations following primary total hip replacement: comparison and validation of three comorbidity measures. Bone Joint J. 2013 ;95-B(9):1184–91.
29. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992 ;45(6):613–9.
30. Cuckler JM. Top “10” ways to ruin a perfectly good TKA. Bone Jt J Orthop Proceedings. 2013;95-B:92.
31. Windsor RE, Scuderi GR, Moran MC, Insall JN. Mechanisms of failure of the femoral and tibial components in total knee arthroplasty. Clin Orthop Relat Res. 1989 ;248:15–9; discussion 19-20.
32. Paterson JM, Williams JI, Kreder HJ, Mahomed NN, Gunraj N, Wang X, Laupacis A. Provider volumes and early outcomes of primary total joint replacement in Ontario. Can J Surg. 2010 ;53(3):175–83.
33. Hughes RG, Hunt SS, Luft HS. Effects of surgeon volume and hospital volume on quality of care in hospitals. Med Care. 1987 ;25(6):489–503.
34. Gowrisankaran G, Olin JM, Ho V, Town RJ. Causality, learning and forgetting in surgery. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.183.4869. Accessed 2016 Jun 14.
35. Marjoua Y, Bozic KJ. Brief history of quality movement in US healthcare. Curr Rev Musculoskelet Med. 2012 ;5(4):265–73.
36. Michaëlsson K. Surgeon volume and early complications after primary total hip arthroplasty. BMJ. 2014 ;348:g3433.
37. Haut ER, Pronovost PJ, Schneider EB. Limitations of administrative databases. JAMA. 2012 ;307(24):2589; author reply 2589-90.
38. Sarrazin MSV, Rosenthal GE. Finding pure and simple truths with administrative data. JAMA. 2012 ;307(13):1433–5.
39. Lyman SL, Dy C, Marx RG, Pan TJ. Regionalization of primary total knee arthroplasty: predictors and implications. Arthroscopy. 2013;29:e141–2.
Copyright 2016 by The Journal of Bone and Joint Surgery, Incorporated