Obstetrics & Gynecology:
Performance of the American College of Obstetricians and Gynecologists' Ovarian Tumor Referral Guidelines With a Multivariate Index Assay
Miller, Rachel Ware MD; Smith, Alan MS; DeSimone, Christopher P. MD; Seamon, Leigh DO, MPH; Goodrich, Scott MD; Podzielinski, Iwona MD; Sokoll, Lori PhD; van Nagell, John R. Jr MD; Zhang, Zhen PhD; Ueland, Frederick R. MD
From the Department of Obstetrics and Gynecology, Division of Gynecologic Oncology, University of Kentucky Markey Cancer Center, Lexington, Kentucky; Applied Clinical Intelligence, Bala Cynwyd, Pennsylvania; and the Department of Pathology, Center for Biomarker Discovery, Johns Hopkins Medical Institutions, Baltimore, Maryland.
See related article on page 1289.
For a list of OVA1 trial sites and each primary investigator specialty, see the Appendix online at http://links.lww.com/AOG/A243.
Funded by Vermillion Inc., Austin, Texas.
Data from this article were presented at the 41st Annual Meeting on Women's Cancer, the Society of Gynecologic Oncologists, March 14–17, 2010, San Francisco, California, and at the 42nd Annual Meeting on Women's Cancer, the Society of Gynecologic Oncologists, March 6–9, 2011, Orlando, Florida.
Corresponding author: Rachel Ware Miller, MD, Assistant Professor of Gynecologic Oncology, University of Kentucky Markey Cancer Center, 800 Rose Street, Lexington, KY 40536-0298; e-mail: firstname.lastname@example.org.
Financial Disclosure Dr. Frederick Ueland was the principle investigator for the OVA1 trial and became a member of Vermillion's speaker's bureau in January 2011. He also has received honoraria as a speaker for Vermillion. Dr. Alan Smith is an independent statistician hired by Vermillion for data analysis. Dr. Zhen Zhang is an employee of the Johns Hopkins Center for Biomarker Discovery, which has a sponsored research grant from Vermillion. He is entitled to royalty payment through a license agreement between Johns Hopkins University and Vermillion. The other authors did not report any potential conflicts of interest.
OBJECTIVE: The American College of Obstetricians and Gynecologists (the College) published referral guidelines for women with a pelvic mass that incorporate CA 125. A new multivariate index assay assesses the malignant risk of ovarian tumors before surgery. Our objective was to estimate the performance of the College guidelines with this new multivariate index assay.
METHODS: This prospective, multi-institutional trial included 27 primary care and specialty sites throughout the United States. The College guidelines were evaluated in women scheduled for surgery for an ovarian mass. Clinical criteria and blood for biomarkers were collected before surgery. A standard CA 125-II assay was used and the value applied to the multivariate index assay algorithm and the CA 125 analysis. Study results were correlated with surgical pathology.
RESULTS: Of the 590 women enrolled with ovarian mass on pelvic imaging, 516 were evaluable. There were 161 malignancies (45 premenopausal and 116 postmenopausal). The College referral criteria had a modest sensitivity in detecting malignancy. Replacing CA 125 with the multivariate index assay increased the sensitivity (77–94%) and negative predictive value (87–93%) while decreasing specificity (68–35%) and positive predictive value (52–40%). Similar trends were noted for premenopausal women and early-stage disease.
CONCLUSION: Replacing CA 125 with the multivariate index assay improves the sensitivity and negative predictive value of the College referral guidelines while decreasing specificity and positive predictive value. The high sensitivity is maintained in premenopausal women and early-stage disease.
LEVEL OF EVIDENCE: III
The National Institutes of Health released a consensus statement in 1994 declaring “women with ovarian masses who have been identified preoperatively as having a significant risk of ovarian cancer should be given the option of having their surgery performed by a gynecologic oncologist.”1 There have been numerous publications and guidelines recommending that women with ovarian cancer be under the care of a gynecologic oncologist.2–8 Reports indicate that only one third of women with malignant ovarian tumors are referred to gynecologic oncologists for primary surgery.9,10
The American College of Obstetricians and Gynecologists (the College) published referral guidelines incorporating menopausal status, physical examination, family history, imaging, and CA 125.6 These guidelines are useful in predicting advanced-stage ovarian cancer11,12 but “perform poorly in identifying early-stage disease, especially in premenopausal women, primarily due to lack of early markers and signs of ovarian cancer.”12 An effective preoperative test, particularly for younger women and early-stage cancers, can have a favorable effect on women's health as survival is better in these populations.13 This is relevant because only 10% of women with early-stage ovarian cancer receive the recommended staging and treatment.14
The OVA1 test (multivariate index assay) is a new multivariate diagnostic biomarker assay approved by the U.S. Food and Drug Administration for use in conjunction with physician evaluation to determine whether an ovarian tumor warrants referral to a gynecologic oncologist. Our study objective was to estimate the performance of the College referral guidelines and the effect of replacing CA 125 with the multivariate index assay.
MATERIALS AND METHODS
This multi-institutional trial enrolled patients from 27 primary care and specialty sites across the United States (see the Appendix, available online at http://links.lww.com/AOG/A243). The sites included women's health clinics, obstetrics and gynecology groups, community and university hospitals, gynecologic oncology practices, and health maintenance organization groups. Institutional review board approval was obtained from each site. Participants were recruited by medical staff at each participating institution and represent a consecutive series of patients who met inclusion criteria and agreed to participate in the study. Inclusion criteria included: female patients age 18 years or older, a level of understanding sufficient to give informed consent, agreeable to phlebotomy, an ovarian tumor with planned surgical intervention within 3 months of imaging, and signed informed consent. All ovarian tumors were confirmed with an imaging study (ultrasonography, computed tomography scan, magnetic resonance imaging) before enrollment. Patients were excluded from the study if: age younger than 18, surgical intervention was not planned, declined phlebotomy, or had a malignancy diagnosis in the last 10 years (excepting nonmelanoma skin cancer). Menopause was defined as the absence of menses for at least 12 months or age 50 or older in those patients who were unclear about their menopausal status.
Before surgery, 30 to 50 mL of venous blood was collected into BD plastic vacutainer tubes with clot activators and centrifuged after sitting at 18–25°C for a minimum of 1 hour and a maximum of 6 hours postphlebotomy. The serum specimens for each patient were pooled and aliquots stored at −65°C to −85°C. The specimens were shipped frozen for storage to PrecisionMed International. Biomarker measurements were performed at Quest Diagnostics, Inc. and validated at Johns Hopkins Medical Institutions and Specialty Laboratories. Validation results were submitted to the U.S. Food and Drug Administration. All testing sites were blinded to the clinical and pathologic data. Data analysis was performed by Applied Clinical Intelligence.
The multivariate index assay consists of five immunoassays combined into a single numerical result, including: CA 125-II, transthyretin (prealbumin), apolipoprotein A1, beta 2 microglobulin, and transferrin. Many of these individual biomarkers have been previously reported.15–17 The multivariate index assay algorithm cutoffs were derived and validated from two independent serum training sets. The premenopausal and postmenopausal cutoffs were selected to maximize the utility of the composite index over its individual component markers while maintaining a high level of sensitivity and negative predictive value. CA 125-II was measured on the Elecsys 2010 (Roche Diagnostics) and the other four markers were measured on the BN II System (Siemens Healthcare Diagnostics). The OvaCalc software imports, reconciles, and numerically combines the values for each assay and uses the multivariate index assay algorithm to generate an ovarian malignancy risk index score for each individual specimen. The output of the multivariate index assay algorithm is a numeric index between 0.0 and 10.0, with the following clinical report:
Low probability of malignancy (multivariate index assay less than 5.0)
High probability of malignancy (multivariate index assay 5.0 or higher)
Low probability of malignancy (multivariate index assay less than 4.4)
High probability of malignancy (multivariate index assay 4.4 or higher)
A standard CA 125-II assay (Roche Elecsys) was performed for each patient and the value used in the multivariate index assay algorithm and the CA 125 analysis. The CA 125 clinical cutoff values were chosen in accordance with the College referral criteria5,11 as more than 200 units/mL for premenopausal women and more than 35 units/mL for postmenopausal women. Additionally, we evaluated the modified College criteria proposed by Dearking (more than 67 units/mL for premenopausal women).12
The College criteria recommend preoperative consultation with a gynecologic oncologist for one or more of following criteria:
1. Very elevated CA 125 (more than 200 units/mL)
3. Evidence of abdominal or distant metastasis
4. Family history of one or more first-degree relatives with ovarian or breast cancer.
1. Any elevated CA 125 (more than 35 units/mL)
2. Nodular or fixed pelvic mass
4. Evidence of abdominal or distant metastasis
5. Family history of one or more first-degree relatives with ovarian or breast cancer.
The revisions to the College guidelines proposed by Dearking include: 1) eliminating the family history of one or more first-degree relatives with ovarian or breast cancer, and 2) lowering the CA 125 threshold in premenopausal women to 67 units/mL.12
The statistical analysis was stratified based on menopausal status, stage, and pathology diagnosis. Clinically relevant criteria such as sensitivity, specificity, and predictive values were calculated to evaluate the performance of the College and modified College referral criteria. McNemar's χ2 test was used to compare the performance of the College guidelines with and without the multivariate index assay. Test performance was calculated for all pelvic malignancies (including epithelial ovarian cancer, nonepithelial ovarian cancer, borderline ovarian tumors, metastases to the ovary, and other nonovarian pelvic malignancies), with respect to menopausal status. Subanalysis was performed in patients with primary ovarian malignancies (epithelial and nonepithelial ovarian cancers) with respect to menopausal status and stage. Ninety-five percent confidence intervals were constructed, and P values were calculated from t tests and Fisher exact test where appropriate. Statistical analysis was performed with SAS 9.1 (SAS Institute Inc.).
Between 2007 and 2008, the study enrolled 590 women with an ovarian mass verified by an imaging study. Of these, 516 were evaluable. Women were excluded from analysis if surgery was either not performed (n=27) or delayed more than 3 months (n=3) pathology report was not available (n=26), blood specimen was unusable (n=9), physician assessment was not available (n=8), or imaging study did not confirm an adnexal tumor (n=1). The clinical and pathologic characteristics of all evaluable patients are summarized in Table 1. More than half of the patients (52%) were enrolled by physicians who were not specialty trained in gynecologic oncology. There were 161 pelvic malignancies in women with a documented ovarian tumor on preoperative imaging. One hundred and fifty one had ovarian malignancies (29%), nine patients had a pelvic malignancy but normal ovarian histology, and one patient had an ovarian tumor of low malignant potential and a synchronous endometrial cancer. There were 355 patients with benign ovarian conditions.
The performance of the College and modified College referral guidelines in all pelvic malignancies are reported in Table 2. Evaluating all 516 patients, the performance of the Dearking modifications did not differ statistically from the College criteria. When separated by menopausal status, the modified College guidelines were associated with an increase in sensitivity (58–76%) and decrease in specificity (77–70%) for premenopausal women, and increase in specificity (56–71%) for postmenopausal women. On univariate analysis, CA 125, ascites, and radiographic evidence of metastatic disease had the highest odds ratio for predicting ovarian cancer (Table 3).
The multivariate index assay was substituted for CA 125 in the College guidelines and the results are summarized in Tables 4, 5, and 6. Using McNemar's test, the sensitivity of the College guidelines with the multivariate index assay was significantly higher than the College guidelines (χ2 of 21.5 [df=1], P<.001). Compared with the College guidelines, the calculated negative predictive value for the College guidelines with the multivariate index assay also increased, whereas the specificity and positive predictive value decreased (Table 4). The improvement in sensitivity and negative predictive value was most notable for premenopausal women. When a subanalysis for primary ovarian malignancy was performed, the College guidelines with the multivariate index assay were more sensitive but less specific for early-stage disease than the original College criteria for premenopausal (Table 5) and postmenopausal (Table 6) women. When all 161 malignancies are evaluated, the College guidelines with the multivariate index assay identified 79% (15/19) of missed malignancies in premenopausal, and 67% (12/18) of malignancies missed in postmenopausal women compared with the College criteria. Furthermore, for primary ovarian malignancies (epithelial and nonepithelial ovarian malignancies), the College guidelines with the multivariate index assay correctly identified 78% (7/9) of missed early-stage premenopausal malignancies, and all five missed malignancies in postmenopausal women. The College guidelines with the multivariate index assay detected 93% (25/27) of premenopausal and all (76/76) postmenopausal primary ovarian malignancies.
There is agreement on the importance of early involvement of a gynecologic oncologist in the care of women with ovarian cancer.2–10 The challenge is how best to identify tumors at risk for malignancy, particularly in premenopausal women who account for up to 20% of all ovarian cancers.18,19 Examination alone is often unreliable.20,21 Although several algorithms have been proposed,5,6,22–28 they either are used infrequently or are ineffective given that only 30–40% of women with ovarian cancer initially are treated by a gynecologic oncologist.9,10 Sensible referral guidelines are important to help concentrate ovarian cancer care at centers where surgical expertise improves outcomes.29
In this multicenter trial, the College guidelines were evaluated in a diverse group of primary care and specialty centers. The sensitivity of the College referral criteria was lower than previously published.11,12 The predictive values were also lower in this study, which may be a consequence of lower overall cancer prevalence (31% compared with 37%). Considering the Dearking modifications, eliminating family history and lowering the CA 125 threshold for premenopausal women further emphasized the significance of the CA 125 result. In our trial, most patients did not show signs of advanced disease on imaging (8% ascites; 1% metastatic implants). So for all remaining premenopausal women, CA 125 was the only criterion left to determine the risk of malignancy.
The multivariate index assay is approved for use in women scheduled for surgery for an ovarian tumor. This assay combines CA 125 with four additional biomarkers, enhancing its ability to detect malignancy, particularly early-stage cancers. When the multivariate index assay replaces CA 125 in the College guidelines, the new guidelines detect almost 80% of all missed malignancies and more than 90% of missed epithelial ovarian cancers. The high sensitivity in premenopausal women and early-stage cancers is where CA 125 and the College guidelines have underperformed. Identifying these patients for referral is valuable because many are not receiving appropriate surgical staging and treatment.13,14 The College guidelines with the multivariate index assay are also effective at detecting advanced disease, where aggressive cytoreductive surgery and chemotherapy improve overall survival.7–10,29 In addition, the College guidelines with the multivariate index assay permit a simplified algorithm for evaluating a pelvic mass. Because menopausal status is incorporated into the multivariate index assay result and family history appears to be of marginal significance, the referral criteria can be simplified (Box 1). The clinical performance of this simplified algorithm is similar to Table 4, with sensitivity 93%, specificity 40%, positive predictive value 41%, and negative predictive value 93%.
Beyond identifying more malignancies, it is not known precisely how the multivariate index assay will affect the referral of patients. Adding the multivariate index assay to the College criteria resulted in a decrease in specificity, which implies that women with nonmalignant tumors may be referred to gynecologic oncologists. In actual practice, lower specificity does not necessarily translate into more benign tumor referrals. The decision to refer a patient is an individualized integration of medical and nonmedical variables. Historically, 12% to 40% of women referred to a gynecologic oncologist have an ovarian malignancy11,12,23; thus, current practice demonstrates that more than 60–80% of referrals are for benign disease. In this trial, the calculated specificity of the College criteria was 68%, yet the number of nonmalignant tumors referred to the gynecologic oncologist was very high. Of the 355 benign ovarian tumors in the study, 72% were referred to a gynecologic oncologist for surgery, including 45% of patients referred despite the belief by the enrolling physician that the tumor was benign. It is possible that the higher negative predictive value of the College guidelines with the multivariate index assay may add enough reassurance to deter referral in situations where the clinician is uncertain.
One of the strengths of this study is the multicenter design, enrolling a diverse patient population from numerous geographic sites. All study information was collected prospectively and recorded before surgery, including blood, imaging studies, physical examination, and family history. Also, internal biomarker validation was performed at two independent laboratories. A potential limitation of this study is the use of the newer CA 125-II assay rather than the original assay. Today, there are numerous CA 125 and CA 125-II assays which are available and used interchangeably for preoperative evaluations with very similar diagnostic accuracy.30 The cancer prevalence in this study is similar to previous reports.11,12 In a population with lower cancer prevalence, the test performance will have a lower positive predictive value and higher negative predictive value, although sensitivity and specificity will be unaffected.
In conclusion, replacing CA 125 with the multivariate index assay improves the sensitivity and negative predictive value of the College referral guidelines while decreasing specificity and positive predictive value. Using the multivariate index assay in the College guidelines will identify more malignancies before surgery, but further study is needed to determine the effect on patient referral.
1.NIH consensus conference. Ovarian cancer. Screening, treatment, and follow-up. NIH Consensus Development Panel on Ovarian Cancer. JAMA 1995;273:491–7.
2.Morgan RJ Jr, Copeland L, Gershenson D, Locker G, McIntosh D, Ozols R, et al. NCCN Ovarian Cancer Practice Guidelines. The National Comprehensive Cancer Network. Oncology 1996;10:293–310.
3.Hoskins W, Rice L, Rubin S. Ovarian cancer surgical practice guidelines. Oncology 1997;11:896–900, 903–4.
4.Elit L, Plante M, Bessette P, DePetrillo A. Surgical management of an adnexal mass suspicious for malignancy. SOGC Clinical Practice Guidelines 2000;97:1–5.
5.The role of the generalist obstetrician-gynecologist in the early detection of ovarian cancer. ACOG Committee Opinion No. 280. The American College of Obstetricians and Gynecologists. Obstet Gynecol 2002;100:1413–6.
6.Myers E, Bastian L, Havrilesky L, Kulasingam S, Terplan M, Cline K, et al. Management of adnexal mass. Rockville (MD): U.S. Department of Health and Human Services; 2006.
7.Engelen MJ, Kos HE, Willemse PH, Aalders JG, de Vries EG, Schaapveld M, et al. Surgery by consultant gynecologic oncologists improves survival in patients with ovarian carcinoma. Cancer 2006;106:589–98.
8.Giede KC, Kieser K, Dodge J, Rosen B. Who should operate on patients with ovarian cancer? An evidence-based review. Gynecol Oncol 2005;99:447–61.
9.Earle CC, Schrag D, Neville BA, Yabroff KR, Topor M, Fahey A, et al. Effect of surgeon specialty on processes of care and outcomes for ovarian cancer patients. J Natl Cancer Inst 2006;98:172–80.
10.Carney ME, Lancaster JM, Ford C, Tsodikov A, Wiggins CL. A population-based study of patterns of care for ovarian cancer: who is seen by a gynecologic oncologist and who is not?. Gynecol Oncol 2002;84:36–42.
11.Im SS, Gordon AN, Buttin BM, Leath CA 3rd, Gostout BS, Shah C, et al. Validation of referral guidelines for women with pelvic masses. Obstet Gynecol 2005;105:35–41.
12.Dearking AC, Aletti GD, McGree ME, Weaver AL, Sommerfield MK, Cliby WA. How relevant are ACOG and SGO guidelines for referral of adnexal mass?. Obstet Gynecol 2007;110:841–8.
13.Lee CK, Pires de Miranda M, Ledermann J, Ruiz de Elvira M, Nelstrop A, Lambert H, et al. Outcome of epithelial ovarian cancer in women under 40 years of age treated with platinum-based chemotherapy. European J of Cancer 1999;35:727–32.
14.Munoz K, Harlan L, Trible E. Patterns of care for women with ovarian cancer in the United States. J Clin Oncol 1997;15:3408–15.
15.Bast RC Jr, Klug TL, St John E, Jenison E, Niloff JM, Lazarus H, et al. A radioimmunoassay using a monoclonal antibody to monitor the course of epithelial ovarian cancer. N Engl J Med 1983;309:883–7.
16.Jacobs I, Bast RC Jr. The CA 125 tumour-associated antigen: a review of the literature. Hum Reprod 1989;4:1–12.
17.Zhang Z, Bast R, Yu Y, Li J, Sokoll L, Rai AJ, et al. Three biomarkers identified from serum proteomic analysis for the detection of early stage ovarian cancer. Cancer Res 2004;64:5882–90.
18.Jemal A, Siegel R, Ward E, Hao Y, Xu J, Thun M. Cancer statistics, 2009. CA Cancer J Clin 2009;59:225–49.
19.Bristow R, Zahurak M, del Carmen M, Gordon T, Fox H, Trimble E, et al. Ovarian cancer surgery in Maryland: volume-based access to care. Gynecol Oncol 2004;93:353–60.
20.Ueland FR, Depriest PD, Desimone CP, Pavlik EJ, Lele SM, Kryscio RJ, et al. The accuracy of examination under anesthesia and transvaginal sonography in evaluating ovarian size. Gynecol Oncol 2005;99:400–3.
21.Buys S, Partidge E, Greene M, Prorok PC, Reding D, Riley TL, et al. Ovarian cancer screening in the prostate, lung, colorectal and ovarian (PLCO) cancer screening trial: findings from the initial screen of a randomized trial. Am J Obstet Gynecol 2005;193:1630–9.
22.DePriest PD, Shenson D, Fried A, Hunter JE, Andrews SJ, Gallion HH, et al. A morphology index based on sonographic findings in ovarian cancer. Gynecol Oncol 1993;51:7–11.
23.Ueland F, DePriest P, Pavlik E, Kryscio R, van Nagell J Jr. Preoperative differentiation of malignant from benign ovarian tumors: the efficacy of morphology indexing and Doppler flow sonography. Gynecol Oncol 2003;91:46–50.
24.Gostout BS, Brewer MA. Guidelines for referral of the patient with an adnexal mass. Clinical Obstet Gynecol 2006;49:448–58.
25.Tingulstad S, Skjeldestad FE, Halvorsen T, Nustad K, Onsrud M. The risk-of-malignancy index to evaluate potential ovarian cancers in local hospitals. Obstet Gynecol 1999;93:448–52.
26.Maggina T, Gadducci A, D'Addario V, Pecorelli S, Lissoni A, Stella M, et al. Prospective multicenter study on CA 125 in postmenopausal pelvic masses. Gynecol Oncol 1994;54:117–23.
27.Schutter E, Kenemans P, Sohn C, Kristen P, Crombach G, Westermann R, et al. Diagnostic value of pelvic examination, ultrasound, and serum CA 125 in postmenopausal women with a pelvic mass. An international multicenter study. Cancer 1994;74:1398–406.
28.Jacobs I, Oram D, Fairbanks J, Turner J, Frost C, Grudzinskas J. A risk of malignancy index incorporating CA 125, ultrasound and menopausal status for the accurate preoperative diagnosis of ovarian cancer. Br J Obstet Gynecol 1990;97:922–9.
29.Tingulstad S, Skjeldestad F, Hagen B. The effect of centralization of primary surgery on survival in ovarian cancer patients. Obstet Gynecol 2003;102:499–505.
30.Davelaar E, van Kamp G, Verstraeten R, Kenemans P. Comparison of seven immunoassays for the quantification of CA 125 antigen in serum. Clin Chem 1998;44:1417–22.
Figure. No caption available.
This article has been cited 3 time(s).
Bmc CancerA combined blood based gene expression and plasma protein abundance signature for diagnosis of epithelial ovarian cancer - a study of the OVCAD consortiumBmc Cancer
Molecular Diagnosis & TherapyBiomarker Testing for Ovarian Cancer: Clinical Utility of Multiplex AssaysMolecular Diagnosis & Therapy
Bmc CancerElevated levels of circulating microRNA-200 family members correlate with serous epithelial ovarian cancerBmc Cancer
Supplemental Digital Content
© 2011 The American College of Obstetricians and Gynecologists