Intuitively, the quality of cancer care is important to the consumer, health care insurance organizations, health care administrators and professionals, and government agencies as a mechanism to ensure maximum health care value and cost-effectiveness of care. Until recently, quality control has been a neglected aspect of most types of cancer care. Ovarian cancer is the fifth leading cause of cancer-related death among women in the United States and accounts for more deaths than all other gynecologic cancers combined.1 As a result, there is a pressing need to determine appropriate measures of ovarian cancer care quality. Previous research has been fragmentary in approaching the continuum of ovarian cancer care and has failed to yield a consensus on the most appropriate quality measures.2 A useful framework for assessing health care quality is the Donabedian paradigm for quality measurement, which considers three domains: structure; process; and outcomes.3 Although each approach has unique advantages, they each have conceptual and practical limitations. Consequently, quality improvement efforts should ideally incorporate all three domains, with the balance dictated by the specific clinical situation.4
The National Comprehensive Cancer Network Clinical Practice Guidelines are provided for 40 different disease sites, and each represents a consensus statement of evidence from a panel of disease site–specific experts regarding their views of currently accepted approaches to cancer treatment.5 Surprisingly, the National Comprehensive Cancer Network guidelines for ovarian cancer, although widely acknowledged as defining the standard of care, have not been rigorously validated as correlating with improved patient outcomes. The overarching objective of the current study was to collectively examine the three quality measurement domains (structure, process, and outcomes) in ovarian cancer care from a population-based perspective. More specifically, we aimed to validate National Comprehensive Cancer Network ovarian cancer guideline adherence as a quality process measure associated with improved survival and to identify structural health care characteristics predictive of adherence to National Comprehensive Cancer Network guideline care.
MATERIALS AND METHODS
This is a retrospective, population-based, case-only study of invasive epithelial ovarian cancer reported to California Cancer Registry between 1 January 1999 and 31 December 2006, and it received exempt status by the Institutional Review Board of the University of California, Irvine (HS#2011-8317). California Cancer Registry case reporting is estimated to be 99% for the entire state of California, with follow-up completion rates exceeding 95%.6 International Classification of Diseases Codes for Oncology based on World Health Organization's criteria were used for tumor location and histology. Cases were identified using ovarian Surveillance, Epidemiology, and End Results (SEER) primary site code (C569).
Case selection criteria included age 18 years or older and a first or only invasive epithelial ovarian cancer diagnosed (Fig. 1). Age at diagnosis was treated either as a continuous variable or as a categorical variable with four groups (younger than 45 years, 45–54 years, 55–69 years, and 70 years or older). Tumor characteristic included International Federation of Gynecology and Obstetrics (FIGO) stage, tumor grade, and histology. Hospital volume was calculated based on the average annual number of ovarian cancer cases that were admitted in that hospital. Hospitals with 20 or more cases per year were classified as high-volume and hospitals with fewer than 20 cases per year were considered low-volume. Physician volume was derived from the average annual number of cases from each patient's physician (surgeon, medical oncologist, or attending physician, whichever had higher volume). Physicians with 10 or more cases per year were categorized as high-volume and those with fewer than 10 cases per year were considered low-volume.7–10 Cause of death was recorded according to International Classification of Diseases criteria in effect at the time of death.11 The last date of follow-up was either the date of death or the date of last contact. Ovarian cancer–specific mortality was defined as death caused by ovarian cancer. Patients who died from other causes were treated as censored cases at the time of the event.
For the purposes of this study, adherence to National Comprehensive Cancer Network guideline therapy was selected as the process measure of quality of cancer care and considered the therapeutic standard that the majority of ovarian cancer patients should be provided. Adherence to treatment guidelines for ovarian cancer was based on National Comprehensive Cancer Network recommendations for surgery and chemotherapy according to the time period of diagnosis (1997–2005).12–16 For FIGO stages I–IIIB, surgical treatment was considered adherent to National Comprehensive Cancer Network guidelines if it included a minimum of oophorectomy (with or without hysterectomy), pelvic lymph node biopsy, para-aortic lymph node biopsy, or both, and omentectomy. A minimum of oophorectomy (with or without hysterectomy) and omentectomy was considered adherent surgical care for FIGO stages IIIC–IV disease. For cases of stages IA–IB, grades 1–2 disease, no adjuvant treatment was considered adherent to National Comprehensive Cancer Network guidelines. Administration of multiagent chemotherapy was considered appropriate for cases of stages IC–IV or grade 3 disease. Surgery must have preceded chemotherapy for stages I–IIIB to be considered adherent to National Comprehensive Cancer Network guidelines, whereas for stages IIIC–IV either initial surgery or chemotherapy was characterized as appropriate care. Dichotomous variables, adherence or nonadherence, were created for adherence to surgical guidelines, chemotherapy guidelines, and the overall treatment plan (both surgery and chemotherapy). For analysis of adherence or nonadherence of the overall treatment plan, cases of discordance between providers (eg, high-volume hospital for surgery and low-volume hospital for chemotherapy) were assigned as high-volume.
Descriptive statistics for demographic and clinical characteristics were analyzed with χ2 test or Fisher exact test for categorical variables. The first main outcome variable was adherence to National Comprehensive Cancer Network guidelines for treatment of ovarian cancer. Multivariable logistic regression analysis was performed to estimate the probability of nonadherence to National Comprehensive Cancer Network guidelines. The second main outcome variable was disease-specific survival. Survival analysis was performed using the Kaplan-Meier estimate of survival probability and log-rank test. After verifying the proportionality assumption, a Cox proportional hazards model was fitted to evaluate the independent effect on survival of each predictor, with FIGO stage treated as a strata rather than as a covariate. The model compared time to death among patients with the same stage and generated a single-weighted coefficient for other factors in the model. Possible interaction terms of main effects were tested, and statistically insignificant factors were removed from the final model using forward selection. Adjusted hazard ratios (HR) and 95% confidence intervals (CI) were generated. All statistical analysis was performed using SAS 9.2.
A total of 13,321 patients with complete clinical and pathologic information were identified. The median age at diagnosis was 61 years (range 18–104 years). The patient and health care provider characteristics are shown in Table 1. FIGO stages III–IV disease accounted for 69.9% of patients, and serous tumors were the most common histologic subtype (41.5%). Analysis of hospital annual ovarian cancer case volume revealed that the majority (81.2%) of patients underwent surgery at low-volume hospitals, with just 18.8% of surgeries performed at high-volume centers. There was no recorded operating surgeon or treating physician identified (missing data) in 21.5% of cases (2,857 patients). Among all cases, a high-volume physician treated 16.4% of patients, whereas low-volume health care providers cared for 62.1%. Of the 10,464 patients with an operating surgeon or treating physician of record, high-volume health care providers accounted for 20.9% of cases.
Overall, 4,952 patients (37.2%) received National Comprehensive Cancer Network guideline–adherent care. Appropriate surgery was performed in 54.1% of cases, whereas appropriate chemotherapy was administered to 60.7% of patients. Both patient demographic and disease characteristics were significantly associated with the likelihood of receiving National Comprehensive Cancer Network guideline–adherent care (Table 1). Age 70 years or older, early-stage disease, and atypical histologic subtypes were associated with an increased likelihood of nonadherent care. Among health care provider characteristics, the most notable differences in administration of National Comprehensive Cancer Network guideline–adherent care were associated with individual physician annual ovarian cancer case volume. Compared with low-volume providers, high-volume physicians were significantly more likely to perform proper surgery (69.6% compared with 49.7%; P<.001), administer proper chemotherapy (65.8% compared with 58.0%; P<.001), and deliver appropriate overall treatment (47.6% compared with 34.5%; P<.001). High-volume hospitals were more likely to administer National Comprehensive Cancer Network guideline–adherent care (50.8% compared with 34.1%; P<.001), as were hospitals with an American College of Surgeons–approved cancer program (38.6% compared with 36.0%; P=.002).
A multivariable logistic regression model was developed to determine the independent effect of each variable on the likelihood of National Comprehensive Cancer Network guideline–adherent care (Table 2). Advanced FIGO stage of disease was significantly associated with a higher likelihood of receiving proper surgery, chemotherapy, and overall treatment. Although higher tumor grade was associated with an increased risk of receiving chemotherapy nonadherent to National Comprehensive Cancer Network guidelines, there was a significant positive association between increasing tumor grade and a higher likelihood of proper surgery and overall treatment. Increasing age at diagnosis was associated with a marginal increase in the likelihood of receiving nonadherent care. Provider characteristics were significantly and independently predictive of National Comprehensive Cancer Network guideline–adherent care. Low-volume hospitals were significantly more likely to deliver overall ovarian cancer treatment that was nonadherent to National Comprehensive Cancer Network guidelines (OR 1.83, 95% CI 1.66–2.01) compared with high-volume hospitals. The presence of an American College of Surgeons–approved cancer program was associated with a higher likelihood of proper overall treatment (OR 1.09, 95% CI 1.00–1.18). Compared with high-volume ovarian cancer physicians, low-volume physician status was independently predictive of treatment that was nonadherent to National Comprehensive Cancer Network guidelines for surgery (OR 1.62, 95% CI 1.45–1.81) and overall treatment (OR 1.19, 95% CI 1.07–1.32).
Univariable survival analysis revealed a statistically significant difference in disease-specific survival between patients receiving adherent and nonadherent National Comprehensive Cancer Network guideline care for stages I–II (Fig. 2) and stages III–IV (Fig. 3) disease. The 5-year disease-specific survival rates for patients with early-stage disease receiving overall treatment adherent and nonadherent to National Comprehensive Cancer Network guideline care were 86.1% and 81.3% (P<.001), respectively. The 5-year disease-specific survival rates for patients with advanced-stage disease receiving overall treatment adherent and nonadherent to National Comprehensive Cancer Network guidelines were 34.9% and 25.6% (P<.001), respectively. Multivariable survival analysis confirmed the known negative prognostic effects of increasing age, higher FIGO stage, increasing tumor grade, and atypical histologic subtypes (Table 3). After controlling for other variables, adherence of the overall treatment plan to National Comprehensive Cancer Network guidelines was a statistically significant and independent predictor of improved disease-specific survival. Compared with patients treated according to National Comprehensive Cancer Network guidelines, patients receiving substandard care experienced more than a 30% increase in the risk of ovarian cancer–related death (HR 1.33, 95% CI 1.26–1.41). Among provider characteristics, both low-volume hospitals (HR 1.08, 95% CI 1.01–1.16) and low-volume physicians (HR 1.18, 95% CI 1.09–1.28) also were significantly negatively associated with survival, independent of overall treatment adherence to National Comprehensive Cancer Network guidelines.
Quality of care has been defined by the Institute of Medicine as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.”17 Monitoring health care quality is impossible without the use of clinical indicators, which create the basis for quality improvement and prioritization in the health care system.18 The issue of quality in ovarian cancer care is not new. However, previous research has consisted of single-institution studies with limited applicability to large-scale implementation or population-based studies with narrowly defined or untested quality criteria not reflective of the entire continuum of care.2,19–23 A unique feature of the current analysis is the integration of the three domains of the Donabedian paradigm for health care quality improvement (structure, process, and outcomes) within a single population-based analysis. Specifically, our objectives were to validate National Comprehensive Cancer Network ovarian cancer guideline adherence as a quality process measure associated with improved outcome (survival) and to identify structural health care characteristics predictive of adherence to National Comprehensive Cancer Network guideline care.
The most obvious way to assess the quality of cancer care is through direct outcome measures (eg, morbidity and mortality rates, patient satisfaction, and survival). However, direct outcomes are also the most difficult and expensive to generate because of the length of follow-up and detail of data required. As a result, attention has generally focused on structural and process measures of health care quality that can be correlated with improved outcomes. In ovarian cancer, process measures that detail the specific care received include performance of comprehensive staging, maximal surgical cytoreduction, and administration of recommended chemotherapy.24 Individually, these clinical indicators have been correlated with improved outcomes; however, consolidation of both surgical and chemotherapeutic indicators into a single comprehensive quality measure of the overall treatment program is uncommon. In fact, only a few studies have evaluated the degree to which overall treatment for ovarian cancer in the United States adheres to contemporary national standards of recommended care. For example, in 2003, Harlan et al25 described 1,167 ovarian cancer patients from the SEER database in 1991 and 1996. These investigators examined trends in surgery and chemotherapy according to recommendations from the 1994 National Institutes of Health Consensus Development Conference on ovarian cancer and found that in 1996, 56.2% of ovarian cancer patients were treated according to National Institutes of Health guidelines. The only study specifically examining National Comprehensive Cancer Network guidelines for ovarian cancer was published in 2005 by Harlan et al26 and included 504 patients from the SEER Patterns of Care database and found that the proportion of patients receiving guideline-adherent care ranged from 23.9% to 35.2%, depending on insurance status. This study did not, however, correlate receipt of guideline therapy with survival outcome.
Adherence to National Comprehensive Cancer Network guidelines is attractive as a quality process measure because it can provide a straightforward comparison between what should be accomplished in administering care and what is actually performed. The current data represent a large-scale, population-based study that consolidates both surgical and chemotherapeutic components of National Comprehensive Cancer Network–recommended therapy into a single category and indicate that adherence to National Comprehensive Cancer Network guideline care is a significant predictor of improved ovarian cancer–specific survival. This finding represents an important and necessary initial step toward quality improvement by confirming the validity (the degree to which an indicator measures what it is intended to measure) of National Comprehensive Cancer Network ovarian cancer guideline care as a viable quality process measure predicting improved survival outcome.
Structural measures that reflect the physical characteristics and resources of the health care setting and may correlate with both process and outcome measures have been studied to the greatest extent in the field of outcomes research. Of the various structural measures, the number of procedures performed is the easiest to measure and most frequently studied. Previous population-based studies have demonstrated superior clinical outcomes for patients with ovarian cancer when treated by high-volume surgeons and high-volume centers.7–10 Evidence also supports physician specialty or hospital type as the dominant factors driving improved outcomes rather than procedure volume.27–29 Structural indicators associated with National Comprehensive Cancer Network ovarian cancer guideline adherence, and therefore reflective of the overall continuum of care (both surgery and chemotherapy), have not been extensively studied.
Using a structural measure for quality assessment is possible only if the clinical indicator has been shown to increase the likelihood of either a good outcome or a process that has been shown to yield better outcomes. The current data indicate that ovarian cancer provider case volume may satisfy both criteria. Furthermore, high-volume providers were associated with a significant and independent positive effect on patient survival, independent of National Comprehensive Cancer Network guideline care, although the magnitude of the volume–outcomes effect was smaller relative to the appropriateness of treatment. These observations suggest that there are other unmeasured factors contributing to the positive volume–outcomes relationship in ovarian cancer (eg, extent of residual disease, intensity of chemotherapy, supportive care resources).
Strengths of the current study include the large study population size, the proven reliability of the California Cancer Registry, and examination of a contemporary time period during which no major treatment paradigm shifts occurred. There are also several limitations that must be considered when interpreting the data presented. First, this was a retrospective study design using a population-based data set and is subject to the inherent potential for reporting and selection bias that accompanies such methodology. Second, and perhaps most importantly, we were unable to control for potentially important unreported variables that could influence both survival outcome as well as the likelihood of administration of recommended care. Such variables include the presence of medical comorbidities, the extent of initial disease and amount of residual tumor, cumulative chemotherapy dose and dose intensity, and management of recurrent disease. Third, we were unable to perform a detailed analysis of the surgical complexity among patients with advanced disease. Higher surgical complexity predicts improved long-term survival because of the correlation with small-volume residual tumor among patients with advanced-stage disease.30 A fourth potential limitation is that we did not preferentially distinguish the sequence of initial surgery or chemotherapy for patients with advanced-stage disease, because both are acceptable treatment plans according to National Comprehensive Cancer Network guidelines. Previous population-based studies have shown that initial surgery is associated with superior survival compared with neoadjuvant chemotherapy, and that high-volume providers are more likely to undertake initial surgery.10 Finally, we were unable to examine the potential effect of physician specialty, in addition to provider volume, because this information is not captured routinely by the California Cancer Registry.
Despite these limitations, several generalizable conclusions can be drawn from the current data. First, adherence to National Comprehensive Cancer Network guidelines for treatment of ovarian cancer is correlated with disease-specific survival and may be a useful process measure of quality cancer care. Improving adherence to evidence-based processes that improve survival has been cited as a key component of improving the quality of care for ovarian cancer.2 Second, ovarian cancer case volume correlates with a higher likelihood of recommended care and improved survival and may be a useful structural quality measure. Finally, the most sobering finding from the current analysis is that slightly more than one out of three women with ovarian cancer in California received the recommended standard of care. This observation underscores an unmistakable opportunity for quality improvement. Additional research is needed to further define the reasons for deviation from recommended care, adjust for variation attributable to differences in medical comorbidities and performance status, and develop appropriate risk-adjusted measurement models. Until such data are forthcoming, a logical ovarian cancer quality improvement strategy should include selective referral based on a combination of structural indicators and performance-based process and outcome measures, coupled with continuous efforts to improve the overall quality of care by learning from “top performers,” and accompanied by rapid implementation of best practices.21–23
1. Siegel R, Naishadham D, Jemal A. Cancer statistics. CA Cancer J Clin 2012;62:10–29.
2. Reade C, Elit L. Trends in gynecologic cancer care in North America. Obstet Gynecol Clin North Am 2012;39:107–29.
3. Donabedian A. Evaluating the quality of medical care. Milbank Memorial Fund Q 1966;44:166–206.
4. Birkmeyer JD, Dimick JB, Birkmeyer NJO. Measuring the quality of surgical care: structure, process, or outcome? J Am Coll Surg 2004;198:626–32.
5. Morgan RJ, Alvarez RD, Armstrong DK, Burger RA, Chen LM, Copeland L, et al.. NCCN clinical practice guidelines in oncology (NCCN guidelines). Ovarian cancer including fallopian tube cancer prim peritoneal cancer. Version 1.2013. Available at: http://www.nccn.org/professionals/physician_gls/pdf/ovarian.pdf
. Retrieved December 29, 2012.
6. Parikh-Patel A, Allen M, Wright WE. Validation of self-reported cancers in the California Teachers Study. Am J Epidemiol 2003;157:539–45.
7. Goff BA, Matthews BJ, Wynn M, Muntz HG, Lishner DM, Baldwin LM. Ovarian cancer: patterns of surgical care across the United States. Gynecol Oncol 2006;103:383–90.
8. Goff BA, Matthews BJ, Larson EH, Andrilla CH, Wynn M, Lishner DM, et al.. Predictors of comprehensive surgical treatment in patients with ovarian cancer. Cancer 2007;109:2031–42.
9. Bristow RE, Zahurak ML, Diaz-Montes TP, Giuntoli RL, Armstrong DK. Impact of surgeon and hospital ovarian cancer surgical case volume on in-hospital mortality and related short-term outcomes. Gynecol Oncol 2009;115:334–8.
10. Bristow RE, Palis BE, Chi DS, Cliby WA. The National Cancer Database on advanced-stage epithelial ovarian cancer: impact of hospital surgical case volume on overall survival and surgical treatment paradigm. Gynecol Oncol 2010;118:262–7.
11. Fritz APC, Jack A, Parkin DM, Percy C, Shanmugarathan S, Sobin L, et al.. International classification of diseases for oncology. Geneva (Switzerland): World Health Organization; 2000.
12. Morgan RJ, Copeland L, Gershenson D, Locker G, McIntosh D, Ozols R, et al.. Update of the NCCN ovarian cancer practice guidelines. Oncology 1997;11:95–105.
13. Morgan R, Alvarez RD, Armstrong DK, Copeland L, Fiorica J, Fishman DA, et al.. NCCN practice guidelines for ovarian cancer. Version 2000. National Comprehensive Cancer Network.
14. Morgan R, Alvarez RD, Armstrong DK, Copeland L, Fiorica J, Fishman DA, et al.. Ovarian cancer guideline. Version 1.2002. Fort Washington (PA): National Comprehensive Cancer Network. 2002.
15. Morgan R, Alvarez RD, Armstrong DK, Chen LM, Copeland L, Dupont J, et al.. Ovarian cancer. Version 1.2003. Fort Washington (PA): National Comprehensive Cancer Network. 2003.
16. Morgan R, Alvarez RD, Armstrong DK, Chen LM, Copeland L, Fiorica J, et al.. Ovarian cancer. Version 1.2005. Fort Washington (PA): National Comprehensive Cancer Network. 2005.
17. Blumenthal D. Quality of health care, part 1. Quality of care—what is it? N Engl J Med 1996;335;891–4.
18. Mainz J. Defining and classifying quality indicators for quality improvement. Int J Qual Health Care 2003;15:523–30.
19. Earle CC, Schrag D, Neville BA, Yabroff KR, Topor M, Fahey A, et al.. Effect of surgeon specialty on processes of care and outcomes for ovarian cancer patients. J Natl Cancer Inst 2006;98:172–80.
20. Schrag D, Earle C, Xu F, Panageas KS, Yabroff KR, Bristow RE, et al.. Associations between hospital and surgeon procedure volumes and patient outcomes after ovarian cancer resection. J Natl Cancer Inst 2006;98:163–71.
21. Aletti GD, Dowdy SC, Gostout BS, Jones MB, Stanhope RC, Wilson TO, et al.. Quality improvement in the surgical approach to advanced ovarian cancer: the Mayo Clinic experience. J Am Coll Surg 2009;208:614–20.
22. Chi DS, Eisenhauer EL, Zivanovic O, Sonoda Y, Abu-Rustum NR, Levine DA, et al.. Improved progression-free and overall survival in advanced ovarian cancer as a result of a change in surgical paradigm. Gynecol Oncol 2009;114:26–31.
23. Harter P, Muallem ZM, Buhrmann C, Lorenz D, Kaub C, Hils R, et al.. Impact of a structured quality management program on surgical outcome in primary advanced ovarian cancer. Gynecol Oncol 2011;121:615–9.
24. Verleye L, Ottevanger PB, van der Graaff W, Reed NS, Vergote I; Gynaecological Cancer Group (GCG) of European Organisation for Research and Treatment of Cancer (EORTC). EORTC-GCG process quality indicators for ovarian cancer surgery. Eur J Cancer 2009;45:517–26.
25. Harlan LC, Clegg LX, Trimble EL. Trends in surgery and chemotherapy for women diagnosed with ovarian cancer in the United States. J Clin Oncol 2003;21:3488–94.
26. Harlan LC, Greene AL, Clegg LX, Mooney M, Stevens JL, Brown ML. Insurance status and the use for guideline therapy in the treatment of selected cancers. J Clin Oncol 2005;23:9079–88.
27. Tingulstad S, Skjeldestad FE, Hagen B. The effect of centralization of primary surgery on survival in ovarian cancer patients. Obstet Gynecol 2003;102:499–505.
28. Eisenkop SM, Spirtos NM, Montag TW, Nalick RH, Wang HJ. The impact of subspecialty training on the management of advanced ovarian cancer. Gynecol Oncol 1992;47:203–9.
29. du Bois A, Rochon J, Pfisterer J, Hoskins WJ. Variations in institutional infrastructure, physician specialization and experience, and outcome on ovarian cancer: a systematic review. Gynecol Oncol 2009;112:422–36.
30. Aletti GD, Dowdy SC, Podratz KC, Cliby WA. Relationship among surgical complexity, short-term morbidity, and overall survival in primary surgery for advanced ovarian cancer. Am J Obstet Gynecol 2007;197:676.e1–7.