National and international guidelines support the participation of a gynecologic oncologist in the care of women with ovarian cancer.1–7 A meta-analysis of 18 studies found that early involvement of a gynecologic oncologist improved outcomes, concluding: 1) early-stage disease is more likely to have surgical staging, 2) advanced disease is more likely to receive optimal cytoreductive surgery, and 3) advanced disease has an improved median and overall 5-year survival.8 Yet only one third of women with ovarian cancer are referred to a gynecologic oncologist for primary surgery.9,10
To decide who should surgically remove an ovarian tumor, physicians interpret patient history, examination, imaging studies, and laboratory tests. Examination is an unreliable method to detect ovarian tumors.11,12 Pelvic ultrasonography is useful in establishing the risk of tumors based on size and morphology,13,14 but it cannot differentiate benign from malignant consistently, particularly when ovarian size is normal. CA 125 is indicated for monitoring ovarian cancer but not for screening or preoperative assessment. CA 125 has low sensitivity (50%) in early-stage cancers15,16 and low specificity in both premenopausal and postmenopausal women.16 The American College of Obstetricians and Gynecologists (the College) referral criteria4 and the Risk of Malignancy Index17,18 use CA 125, but neither is universally accepted in the United States.
Multivariate index assay is a new multivariate biomarker assay used with physician assessment to determine whether an ovarian tumor warrants referral for surgery. The objective was to compare the effectiveness of physician assessment with a multivariate index assay in identifying high risk ovarian tumors.
MATERIALS AND METHODS
Patients were enrolled from 27 sites throughout the United States, including women's health clinics, obstetrics and gynecology groups, gynecologic oncology practices, community and university hospitals, and health maintenance organizations (see the Appendix, available online at http://links.lww.com/AOG/A240). Institutional review board approval was obtained from each site. Volunteer participants represent a consecutive series of patients who met inclusion criteria from each participating institution. The criteria for inclusion were: female, age 18 years or older, a documented ovarian tumor with planned surgical intervention within 3 months of imaging, agreeable to phlebotomy, and signed informed consent. Before enrollment, all ovarian tumors were confirmed with an imaging study (computed tomography scan, ultrasonography, or magnetic resonance image). The exclusion criteria were age younger than 18, no planned surgical intervention, declined phlebotomy, or had a malignancy diagnosis in the previous 5 years with the exception of a nonmelanoma skin cancer. Menopause was defined as the absence of menses for at least 12 months or age 50 or older when the woman was unsure of her menopausal status. All demographic, clinical, and pathologic information was faxed to Applied Clinical Intelligence for analysis.
Thirty to 50 mL of preoperative venous blood was collected from each patient and placed in BD plastic vacutainer tubes with clot activators and centrifuged after sitting at 18–25°C for a minimum of 1 hour and maximum of 6 hours postphlebotomy. Specimens for each patient were pooled before aliquoting, stored at -65°C to -85°C, and shipped frozen to PrecisionMed International for storage. Biomarker measurements were performed by Quest Diagnostics, Inc., and blinded validation testing was done at Johns Hopkins Medical Institutions and Specialty Laboratories.
The multivariate index assay test incorporates CA 125-II, transferrin, transthyretin (prealbumin), apolipoprotein AI, and beta 2 microglobulin. Many of these individual biomarkers,15,16,19 and the rationale for the development of the multivariate index assay test,20 have been previously described. The multivariate index assay algorithm was derived and validated from two independent serum training sets. The clinically relevant cutoffs were selected to maximize the index over its individual components while maintaining a high level of sensitivity and negative predictive value. The proprietary OvaCalc software combines the values for each assay, and uses the multivariate index assay algorithm to generate an ovarian malignancy risk index score for each. The numeric result ranges from 0.0 and 10.0, with the following clinical report:
- Low probability of malignancy (multivariate index assay less than 5.0)
- High probability of malignancy (multivariate index assay 5.0 or higher)
- Low probability of malignancy (multivariate index assay less than 4.4)
- High probability of malignancy (multivariate index assay 4.4 or higher)
CA 125-II was measured on the Elecsys 2010 (Roche Diagnostics) and the other four markers were measured on the BN II System (Siemens Healthcare Diagnostics).
For each patient, a standard CA 125-II assay was performed. The same assay value was used in the analysis for both CA 125 and the multivariate index assay algorithm. The CA 125 clinical cutoff values were selected in accordance with the published College referral criteria4,21 as more than 200 units/mL for premenopausal women and more than 35 units/mL for postmenopausal women. We also evaluated the modified College criteria proposed by Dearking as more than 67 units/mL for premenopausal women.22
Before surgery, clinicians were required to document the results of the physical examination, family history, imaging, laboratory tests (including CA 125 but not the multivariate index assay), and physician assessment. The majority (80%) of physicians ordered CA 125 as part of their assessment. The enrolling physician formally declared their preoperative assessment by answering the question, “based on all available clinical information, is the physician of the opinion that this is a malignant ovarian tumor? (yes or no).” Physicians were allowed to use any algorithm they desired to determine their answer, but where not required to explain how they arrived at their prediction. A physician assessment was recorded for 516 of 524 patients.
Statistical analysis was stratified based on menopausal status, physician specialty, stage of malignancy, and surgical pathology. Concordances between multivariate index assay results of high or low probability of malignancy and pathologic findings were assessed using McNemar's χ2 test. Clinically relevant criteria such as sensitivity, specificity, positive predictive value, negative predictive value, and likelihood ratios were calculated to estimate the performance of multivariate index assay, CA 125, preoperative assessment alone, and multivariate index assay with preoperative assessment. Ninety-five percent confidence intervals were constructed where appropriate, and those for the area under the receiver operating characteristic curve were calculated using a bootstrap procedure. Statistical analysis was performed with SAS 9.1 (SAS Institute Inc).
From 2007 through 2008, 590 women were enrolled in the study, of which 524 were evaluable for both multivariate index assay and CA 125, and 516 for physician assessment. All patients had an imaging study verifying an ovarian mass. Women were excluded from analysis if surgery was either not performed (27) or delayed more than 3 months (three), pathology report was not available (26), blood specimen was unusable (nine), physician assessment was not available (eight), or imaging study did not confirm an adnexal tumor (one). The clinical characteristics of the study group are summarized in Table 1. More than half of the patients (53%) were enrolled by nongynecologic oncologist. Fewer than 10% of patients had clinical evidence of advanced disease at study enrollment (ascites 9.7%, metastases 4.4%). Final pathology identified 161 malignancies and 363 benign tumors. There were 151 ovarian malignancies (29%), including: 94 epithelial ovarian cancers, 11 nonepithelial ovarian malignancies, 28 ovarian borderline tumors, and 18 malignancies metastatic to the ovary. Of the 10 patients with nonovarian malignancies, one had a synchronous borderline ovarian tumor and an endometrial cancer and nine with documented adnexal tumors on imaging studies had pelvic malignancy but normal ovarian histology (uterine 4, retroperitoneal 2, fallopian tube 1, small bowel 1, other 1).
The performances of physician assessment, multivariate index assay and CA 125 for predicting ovarian malignancy are summarized in Tables 2 and 3. The multivariate index assay improved the sensitivity of physician assessment regardless of gynecologic specialty. The multivariate index assay alone and the multivariate index assay with physician assessment also had the highest sensitivity and negative predictive value, but the lowest specificity and positive predictive value when compared with CA 125 and physician assessment alone. Using McNemar's test, the sensitivity of the multivariate index assay was statistically superior to physician assessment (χ2=20.6, P<.001), CA 125 with cutoff of 200 units/mL (χ2=38.0, P<.001), and CA 125 with cutoff of 67 units/mL (χ2=25.0, P<.001). Receiver operating characteristic curve analysis of the multivariate index assay is shown in Figure 1. The sensitivity of the multivariate index assay test by histologic subtype is: epithelial ovarian cancer 99% (93/94), nonepithelial ovarian cancer 82% (9/11), borderline ovarian tumor 75% (21/28), and metastases to ovary 94% (17/18).
When compared with physician assessment, physician assessment plus the multivariate index assay demonstrated a significant improvement in sensitivity and negative predictive value which was independent of physician specialty (Table 2). Adding the multivariate index assay to physician assessment correctly identified 70% (14/20) of malignancies missed by nongynecologic oncologists, and 95% (19/20) of malignancies missed by gynecologic oncologists. The multivariate index assay had higher sensitivity than either CA 125 or physician assessment (Table 3); however, the addition of the multivariate index assay to physician assessment decreased both specificity and positive predictive value (Tables 2 and 3). In this trial, there were 363 benign ovarian tumors (355 with physician assessment) and 161 malignancies. Of all patients referred for surgery to the gynecologic oncologist, 38% were found to have a malignancy on surgical pathology. There were 261 benign tumors referred to the gynecologic oncologist for surgery, a benign tumor referral rate of 72% (261/363). As documented by physician assessment, the generalist physician predicted that 183 tumors were benign before surgery, and still referred 83 (45%) to the gynecologic oncologist for surgery.
The multivariate index assay test maintained high sensitivity regardless of cancer stage, with 90% sensitivity for stage I cancers, and 100% sensitivity for stages II through IV (Table 4). The sensitivity of the multivariate index assay in early-stage disease (stage I and II) is 94% compared with 61% for CA 125. Moreover, for the cancers missed by physician assessment alone, 70% of the primary ovarian cancers are early-stage (I or II), and 58% had a normal CA 125 value. For early-stage epithelial ovarian cancer, the multivariate index assay sensitivity was 98% compared with 68% for CA 125 (Table 5). Both tests effectively identified late-stage disease (stage III and IV). The sensitivity of the multivariate index assay for all stages of epithelial ovarian cancer and all primary ovarian malignancies was 99% and 97%, respectively.
In premenopausal women with early-stage disease (Tables 4 and 5), the multivariate index assay was more sensitive than CA 125 for all primary ovarian malignancies (82% compared with 29%), and for epithelial ovarian cancers (93% compared with 36%). The multivariate index assay was also more sensitive than CA 125 in postmenopausal women with early-stage disease (Tables 4 and 5). Both tests were effective at detecting late-stage disease, regardless of menopausal status. In this study, 21 of 27 premenopausal patients with ovarian malignancy had positive multivariate index assay tests but indeterminate CA 125 scores (200 units/mL or less), including 9 of the 12 with stage I or II disease. In addition, 17 of 23 postmenopausal women had positive multivariate index assay tests and negative CA 125 scores (35 units/mL or less). Overall, 76% (38/50) of patients with ovarian malignancy had positive multivariate index assay tests despite indeterminate CA 125 results. With the addition of the multivariate index assay, the sensitivity of physician assessment for any ovarian malignancy improved from 60% to 89% (premenopausal) and 81 to 98% (postmenopausal). As expected, in premenopausal women, the negative predictive value increased slightly (90% compared with 94%) and the specificity and positive predictive value decreased (83% compared with 40%; 46% compared with 26%, respectively). Similar trends were noted in postmenopausal women. The multivariate index assay clinical performance for detecting malignancy at varying cutoff levels is displayed for premenopausal and postmenopausal women in Table 6.
The multivariate index assay is a diagnostic test that complements the physician's preoperative assessment; it is not a screening test for ovarian cancer. Although there are many published guidelines for the evaluation of an ovarian tumor,4,5,14,17,18,23–26 none are universally accepted. It is apparent from physician decisions in this trial that there is considerable autonomy in preoperative evaluations, with few following explicit algorithms. The College guidelines permit the physician to choose an imaging modality which is combined with CA 125, patient history, and physical examination. The Risk of Malignancy Index relies exclusively on CA 125, ultrasonography, and menopausal status. Other biomarker algorithms have been proposed,26 but conclusions are premature because they have not been studied in populations in which the test is indicated for use. The dispute with the College criteria is that, although the guidelines are successful in predicting advanced ovarian cancer,21,22 they do not perform well in identifying early-stage disease and particularly in premenopausal women, presumably because CA 125 is less sensitive in these populations.22 Premenopausal women are particularly challenging because they produce the majority of benign cystic tumors and comprise 20% of all ovarian cancers.27,28 Surprisingly, only 10% of women with presumptive early ovarian cancer receive the recommended staging and treatment,29 which is important because their survival is incrementally better.30 The CA 125 value has a considerable influence on the Risk of Malignancy Index and the College criteria. The College guidelines use CA 125 cutoff values for premenopausal (more than 200 units/mL) and postmenopausal (more than 35 units/mL) women, whereas the Risk of Malignancy Index is linked arithmetically to CA 125 because the calculated index score is a product of the actual CA 125 value.18,19 The performance of the Risk of Malignancy Index has not been reported for early-stage disease or premenopausal women. Identifying malignancies for referral in these challenging cohorts is important for comprehensive staging, cytoreduction, appropriate chemotherapy, access to clinical trials, and better outcomes.8–10
To be clinically useful, a preoperative cancer referral test must have high sensitivity. The four additional biomarkers in the multivariate index assay test improve the sensitivity of detecting malignancy when compared with CA 125. Although in clinical practice neither biomarker test is used independently, the multivariate index assay and CA 125 are individually compared to better understand the behavior of each. Considering all patients with pelvic malignancies, the multivariate index assay had a significantly higher sensitivity compared with CA 125. Likewise, the negative predictive value of the multivariate index assay was higher compared with CA 125, regardless of chosen cutoff. These trends were consistent for early- and late-stage cancers, premenopausal and postmenopausal women. When compared with physician assessment alone (which includes the use of CA 125), physician assessment with the multivariate index assay showed similar improvements in sensitivity and negative predictive value. In all, the multivariate index assay identified 76% (38/50) of malignancies missed by CA 125 (more than 200 units/mL), and physician assessment plus the multivariate index assay detected 82% (33/40) malignancies missed by physician assessment alone, and 86% (43/50) of malignancies missed by CA 125.
If used appropriately, the multivariate index assay should not result in additional surgery as it is approved for use in women already scheduled for surgery. It is not known precisely how the multivariate index assay will affect patient referrals. The addition of the multivariate index assay to physician assessment decreased specificity and positive predictive value, a consequence of more false-positive results. This implies that women with nonmalignant tumors may be identified for referral to a gynecologic oncologist. Admittedly, the decision to refer a patient is complex, influenced by many medical and nonmedical concerns. In actual practice, lower specificity may not necessarily result in more benign referrals. Twelve to 40% of women referred to a gynecologic oncologist have a confirmed ovarian malignancy at surgery,14,2,22 meaning more than 60% are referred for benign disease. In this multivariate index assay trial, the specificity of physician assessment alone (nongynecologic oncologists) was 83%, yet the number of nonmalignant tumors referred to the gynecologic oncologist was still high. Surprisingly, 72% of all benign ovarian tumors were referred to a gynecologic oncologist for surgery, including 45% of patients referred despite a negative physician assessment. So the calculated specificity and predictive values of physician assessment do not correlate with observed patient referrals. In patients with a nonmalignant preoperative assessment, the higher negative predictive value of the multivariate index assay may be sufficiently reassuring to discourage referral. Ultimately, this nonrandomized trial was not designed to evaluate the effect of the multivariate index assay on benign tumor referrals.
The strengths of this study include independent biomarker validation, multi-center prospective data collection, and inclusion of all ovarian tumor types for relevance to general gynecologic practice. Also, the majority of patients in the multivariate index assay trial were enrolled at primary care sites where the test is intended for use. A possible study limitation is the lack of a uniform preoperative evaluation for comparison; however, physician assessment was an intentional study design. Preoperative assessment is the most realistic clinical surrogate given that other referral algorithms are not uniformly accepted in the United States. All women enrolled in this trial were scheduled for surgery for an ovarian tumor. This is a cohort with increased cancer prevalence compared with those in whom surgery is not planned. A population with lower cancer prevalence would decrease the test's positive predictive value and increase the negative predictive value. A multivariate index assay is not required if the preoperative evaluation without the multivariate index assay is highly suggestive of malignancy, and the multivariate index assay is not approved for serial surveillance of ovarian tumors without surgery. Hopefully, earlier referral of patients with ovarian cancer will improve survival and reduce the number of required re-operations. The overall value and cost-effectiveness of the multivariate index assay will be determined by future health studies evaluating both economic and outcome measures.
1.NIH consensus conference. Ovarian cancer. Screening, treatment, and follow-up. NIH Consensus Development Panel on Ovarian Cancer. JAMA 1995;273:491–7.
2.Ozols RF. Update of the NCCN ovarian cancer practice guidelines. Oncology 1997;11:95–105.
3.Le T, Giede C. Initial evaluation and referral guidelines for management of pelvic/ovarian masses. Joint SOGC/GOG/SCC Clinical Practice Guideline. Journal Obstet Gynaecol Can 2009;31:668–73.
4.The role of the generalist obstetrician-gynecologist in the early detection of ovarian cancer. ACOG Committee Opinion No. 280. Obstet Gynecol 2002;100:1413–6.
5.Myers ER, Bastian LA, Havrilesky LJ, Kulasingam SL, Terplan MS, Cline KE, et al. Management of adnexal mass. Rockville (MD): U.S. Department of Health and Human Services, 2006.
6.Engelen MJ, Kos HE, Willemse PH, Aalders JG, de Vries EG, Schaapveld M, et al. Surgery by consultant gynecologic oncologists improves survival in patients with ovarian carcinoma. Cancer 2006;106:589–98.
7.Hillner BE, Smith TJ, Desch CE. Hospital and physician volume or specialization and outcomes in cancer treatment: importance in quality of cancer care. J Clin Oncol 2000;18:2327–40.
8.Giede KC, Kieser K, Dodge J, Rosen B. Who should operate on patients with ovarian cancer? An evidence-based review. Gynecol Oncol;99:447–61.
9.Earle CC, Schrag D, Neville BA, Yabroff KR, Topor M, Fahey A, et al. Effect of surgeon specialty on processes of care and outcomes for ovarian cancer patients. J Natl Cancer Inst 2006;98:172–80.
10.Carney ME, Lancaster JM, Ford C, Tsodikov A, Wiggins CL. A population-based study of patterns of care for ovarian cancer: who is seen by a gynecologic oncologist and who is not? Gynecol Oncol 2002;84:36–42.
11.Ueland FR, Depriest PD, Desimone CP, Pavlik EJ, Lele SM, Kryscio RJ, et al. The accuracy of examination under anesthesia and transvaginal sonography in evaluating ovarian size. Gynecol Oncol 2005;99:400–3.
12.Buys S, Partidge E, Greene M, Prorok PC, Reding D, Riley TL, et al. Ovarian cancer screening in the prostate, lung, colorectal and ovarian (PLCO) cancer screening trial: findings from the initial screen of a randomized trial. Am J Obstet Gynecol 2005;193:1630–9.
13.DePriest PD, Shenson D, Fried A, Hunter JE, Andrews SJ, Gallion HH, et al. A morphology index based on sonographic findings in ovarian cancer. Gynecol Oncol 1993;51:7–11.
14.Ueland FR, DePriest PD, Pavlik EJ, Kryscio RJ, van Nagell JR Jr. Preoperative differentiation of malignant from benign ovarian tumors: the efficacy of morphology indexing and Doppler flow sonography. Gynecol Oncol 2003;91:46–50.
15.Bast RC Jr, Klug TL, St John E, Jenison E, Niloff JM, Lazarus H, et al. A radioimmunoassay using a monoclonal antibody to monitor the course of epithelial ovarian cancer. N Engl J Med 1983;309:883–7.
16.Jacobs I, Bast RC Jr. The CA 125 tumour-associated antigen: a review of the literature. Hum Reprod 1989;4:1–12.
17.Jacobs I, Oram D, Fairbanks J, Turner J, Frost C, Grudzinskas J. A risk of malignancy index incorporating CA 125, ultrasound and menopausal status for the accurate preoperative diagnosis of ovarian cancer. Br J Obstet Gynecol 1990;97:922–9.
18.Tingulstad S, Skjeldestad FE, Halvorsen T, Nustad K, Onsrud M. The risk-of-malignancy index to evaluate potential ovarian cancers in local hospitals. Obstet Gynecol 1999;93:448–52.
19.Zhang Z, Bast R, Yu Y, Li J, Sokoll L, Rai AJ, et al. Three biomarkers identified from serum proteomic analysis for the detection of early stage ovarian cancer. Cancer Res 2004;64:5882–90.
20.Zhang Z, Chan D. The road from discovery to clinical diagnostics: lessons learned from the first FDA-cleared in vitro
diagnostic multivariate index assay of proteomic biomarkers. Cancer Epidemiol Biomarkers Prev 2010;19:2995–9.
21.Im SS, Gordon AN, Buttin BM, Leath CA 3rd, Gostout BS, Shah C, et al. Validation of referral guidelines for women with pelvic masses. Obstet Gynecol 2005;105:35–41.
22.Dearking AC, Aletti GD, McGree ME, Weaver AL, Sommerfield MK, Cliby WA. How relevant are ACOG and SGO guidelines for referral of adnexal mass? Obstet Gynecol 2007;110:841–8.
23.Gostout BS, Brewer MA. Guidelines for referral of the patient with an adnexal mass. Clin Obstet Gynecol 2006;49:448–58.
24.Maggina T, Gadducci A, D'Addario V, Pecorelli S, Lissoni A, Stella M, et al. Prospective multicenter study on CA 125 in postmenopausal pelvic masses. Gynecol Oncol 1994;54:117–23.
25.Schutter E, Kenemans P, Sohn C, Kristen P, Crombach G, Westermann R, et al. Diagnostic value of pelvic examination, ultrasound, and serum CA 125 in postmenopausal women with a pelvic mass. An international multicenter study. Cancer 1994;74:1398–406.
26.Moore RG, McMeekin DS, Brown AK, DiSilvestro P, Miller MC, Allard WJ, et al. A novel multiple marker bioassay utilizing HE4 and CA125 for the prediction of ovarian cancer in patients with a pelvic mass. Gynecol Oncol 2009;112:40–6.
27.Jemal A, Siegel R, Ward E, Hao Y, Xu J, Thun M. Cancer statistics, 2009. CA Cancer J Clin 2009;59:225–49.
28.Bristow R, Zahurak M, del Carmen M, Gordon TA, Fox HE, Trimble EL, et al. Ovarian cancer surgery in Maryland: volume-based access to care. Gynecol Oncol 2004;93:353–60.
29.Munoz KA, Harlan LC, Trible EL. Patterns of care for women with ovarian cancer in the United States. J Clin Oncol 1997;15:3408–15.
30.Lee CK, Pires de Miranda M, Ledermann J, Ruiz de Elvira MC, Nelstrop AE, Lambert HE, et al. Outcome of epithelial ovarian cancer in women under 40 years of age treated with platinum-based chemotherapy. Eur J Cancer 1999; 35:727–32.