Dearking, Amy C.1; Aletti, Giovanni D. MD2; McGree, Michaela E.3; Weaver, Amy L.3; Sommerfield, Marla-Kay2; Cliby, William A. MD2
Ovarian cancer is the fifth most common cause of death from cancer in women in the United States, and it is estimated that it will cause 15,280 deaths in the United States in 2007.1 The 5-year survival rate for women diagnosed while the cancer was localized to the ovary (stage I disease) is 93%.1 Unfortunately, 68% of women present with late-stage disease at diagnosis, with a 5-year survival of 30%.1 Therefore, the overall 5-year survival rate for ovarian cancer is 45%.1
At this time there are no accepted effective screening tests to identify women with ovarian cancer, due in part to a relative lack of early signs and symptoms, low prevalence of ovarian cancer in the general population, and poor predictive values of current imaging and serum markers. The cancer marker CA 125, while associated with ovarian cancer, has poor sensitivity and specificity with a high false-positive rate because it is also associated with benign gynecologic conditions and other malignancies.2 Imaging techniques like transvaginal ultrasonography and multi-modal screening methods have also been tested and found to have unacceptably high false-positive rates.3 Additionally, both modalities are relatively insensitive for stage I disease.
Because survival and prognosis in ovarian cancer are closely linked to tumor stage and because there is no effective screening test, there must be a high level of suspicion among providers to identify patients early in the disease process to facilitate referral to appropriate centers for initial management. Different studies have shown that survival is higher in patients whose initial treatment was managed by gynecologic oncologists rather than other providers.4–6 Additionally, second operations for inadequate initial surgical staging or cytoreduction are likely to carry additive morbidity. For these reasons, when a screening test is developed, a greater sensitivity despite a lower specificity is usually preferred. In the case of patients with adnexal masses, factors associated with a low sensitivity (under-referral and inappropriate treatment with additional reoperations) and a low specificity (over-referral and possible “congestion” of patients in tertiary centers, anxiety related to a possible diagnosis of malignancy) seem to tip the balance in favor of a higher sensitivity as well.
To facilitate proper patient triage patterns, the American College of Obstetricians and Gynecologists (ACOG) and the Society of Gynecologic Oncologists (SGO) jointly published guidelines for the referral of women with pelvic masses that are suspicious for ovarian cancer to gynecologic oncologists.7 The published guidelines for preoperative referral to, or consultation with, a gynecologic oncologist are as follows:
“Postmenopausal women who have a pelvic mass that is suspicious for a malignant ovarian neoplasm, as suggested by at least one of the following indicators: elevated CA 125 level; ascites; a nodular or fixed pelvic mass; evidence of abdominal or distant metastasis; a family history of one or more first-degree relatives with ovarian or breast cancer”7
“Premenopausal women who have a pelvic mass that is suspicious for a malignant ovarian neoplasm, as suggested by at least one of the following indicators: very elevated CA 125 level (eg, more than 200 units/mL); ascites; evidence of abdominal or distant metastasis; a family history of one or more first-degree relatives with ovarian or breast cancer”7
The expert opinion referral guidelines were recently tested in a study published by Im et al8 in 2005. The authors evaluated 1,035 patients from seven tertiary centers to estimate the predictive value of the guidelines to distinguish benign from malignant masses. This study found the referral guidelines to have a positive predictive value of 33.8% in premenopausal women and 59.5% in postmenopausal women, with a negative predictive value of greater than 90% in both groups. However, this study was likely influenced by referral center bias: referring physicians likely screened many patients that for a variety of reasons may not have been referred. In addition, at most centers, patients were identified as having a pelvic mass retrospectively, and errors or missed cases based upon the written preoperative listings were another potential source of bias for patients with presumed benign processes. Finally, there was marked variation between centers, with some centers heavily biased toward care of cancer (comprehensive cancer centers).
We proposed to test the predictive value of the referral guidelines in a prospectively enrolled cohort of Mayo Clinic patients evaluated for pelvic mass. Additionally, because Mayo Clinic is the primary provider for the majority of regional patients, we separately performed a subgroup analysis including only regional patients to minimize the influence of referral center bias. Finally, we specifically wanted to examine how the referral guidelines would perform in early-stage disease.
MATERIALS AND METHODS
All consecutive female Mayo Clinic Rochester patients who were undergoing surgery for a pelvic mass were identified from October 2001 through August 2006 for enrollment in the study. After plans for surgery were made, the study was explained by a dedicated study coordinator, and patients were given the option for participation. Written consents were used for all study procedures. Less than 2% of patients declined participation in the study. None of these patients were included in the prior report by Im et al.8 Approval was obtained for all aspects of this study from the Mayo Clinic Rochester Institutional Review Board.
Inclusion criteria for the study was a woman, 35 years of age or older, with an adnexal mass undergoing surgery. Exclusion criteria for the database were 1) women with a previous history of ovarian or primary peritoneal cancer, 2) women with any known active malignancy, and 3) women younger than age 35. In addition, because the referral guidelines require only one or more of the criteria to be met for referral, we excluded 71 patients who were missing data point(s) and would not have been referred by another criterion. For example, if a patient did not have a CA 125 level measured before surgery, she was not excluded if she met one of the other criteria, like evidence of ascites. All data were collected by a dedicated, trained study coordinator. One questionnaire was filled out by the coordinator with the patient preoperatively to obtain preoperative data and family history. After surgery, pathology and operative report findings were entered by the coordinator. Operations were conducted by gynecologic oncologists. If a benign mass was suspected, two minimally invasive surgeons would also perform some of the operations, but if a malignancy was found, a gynecologic oncologist was always available.
The referral guidelines are stratified on the basis of menopausal status. The patients self-reported their menopausal status as postmenopausal, perimenopausal, premenopausal, or unknown. We defined the postmenopausal group as patients who reported being post- or perimenopausal; the premenopausal group comprised patients who reported being premenopausal. In addition, for patients who chose “unknown” for their menopausal status (28 patients), we assigned them based on age into the premenopausal group (age less than 50 years) or the postmenopausal group (age 50 years or more).
The variables analyzed were the criteria specified in the published referral guidelines: preoperative CA 125 level (more than 35 units/mL for postmenopausal and more than 200 units/mL for premenopausal), evidence of ascites on ultrasonography or computed tomography (CT), and evidence of abdominal or distant metastasis on CT, including pleural effusion and family history of a first-degree relative with ovarian or breast cancer. The postmenopausal criteria also include a nodular or fixed pelvic mass. Obviously, we would have had incomplete information of the referring physician’s physical examination findings, so we used evidence of a solid mass on ultrasonography, CT, or other imaging method to meet this criterion. We also believed the referring physician’s examination to be more subjective and, thus, less reliable a variable for subsequent analysis.
To reduce referral center bias due to Mayo Clinic’s role as a tertiary care center, we performed a subgroup analysis of residents from Olmsted County, and the six adjacent counties (Goodhue, Wabasha, Winona, Fillmore, Mower, and Dodge). We defined patients who were residents of these counties as “local” patients.
For statistical analysis, the data were stratified according to the menopausal status to allow independent analysis of both sets of guidelines. The dependent variable or outcome was the detection of primary or metastatic ovarian, fallopian tube, or peritoneal cancer at surgical exploration. The independent variable (referral compared with not referral) was determined based on the referral guideline criteria; the presence of at least one of the criteria was indicative of the need for referral. Sensitivity, specificity, and positive and negative predictive values were calculated to evaluate the performance of the referral guidelines’ criteria in predicting ovarian cancer. Ninety-five percent confidence intervals (CIs) for the performance characteristics were constructed by using a normal approximation or an exact method for a binomial parameter as footnoted in the tables. The χ2 test was used to compare proportions in 2×2 contingency tables. Statistical analysis was performed with SAS 9 (SAS Institute Inc, Cary, NC).
The cohort contained 908 patients, of whom 71 were excluded because they were missing a preoperative CA 125 level or had an unknown family history of cancer and would not otherwise have met referral criteria. The remaining 837 patients were the subjects of this study. Local patients made up 21% of the eligible cohort. Forty-four percent (263/597) of the postmenopausal women with an adnexal mass were diagnosed with ovarian cancer, whereas 20% (48/240) of premenopausal women had ovarian cancer. The majority of cancer diagnoses were primary cancers, and 74% of primary cancers were stage III or IV. Table 1 presents characteristics by menopausal status of the 837 patients included in the study.
We first evaluated the ACOG/SGO referral guidelines within the postmenopausal and premenopausal groups separately to estimate how well the guidelines separated the patients with ovarian cancer from the patients with benign masses for each group. The performance characteristics of the referral guidelines are summarized in Table 2 for each group. For postmenopausal women, the guidelines were 93.2% sensitive in distinguishing ovarian cancer cases from benign cases, with a positive predictive value of 64.6%. In the premenopausal group, the guidelines were 79.2% sensitive, with a positive predictive value of 39.6%. The sensitivity and positive predictive value were both significantly higher in the postmenopausal group than in the premenopausal group (χ2 test, P<.001). The negative predictive value in both groups was greater than 90%.
When looking at the performance of the guidelines based on cancer stage, the referral guidelines for both pre- and postmenopausal groups performed much better for late-stage (stage III or IV) than for early-stage (stage I or II) cancers in both sensitivity and positive predictive value (Table 2). In particular, while only 28 of the 311 patients with ovarian cancer overall would not have been referred to a gynecologic oncologist using the guidelines, the majority (19 [68%]) of these patients had early-stage disease. Table 3 summarizes the cancer origin and stages of the patients not referred by the ACOG/SGO guidelines.
To reduce referral center bias that occurs in tertiary care centers, we evaluated the referral guidelines within a subpopulation of local patients most likely to obtain their routine care at Mayo Clinic. In the postmenopausal group, 109 of the total 597 (18.3%) patients lived in the seven-county area at the time of enrollment and were defined as local patients. The prevalence of ovarian cancer in these local patients was 22.9% (25 of 109), including three women with metastatic cancer. In the 240 premenopausal patients, 66 patients were local (27.5%). Only four of these 66 premenopausal patients had an ovarian cancer (prevalence of 6.1%), and all four cases were primary cancers. For comparison, the prevalence of ovarian cancer in the nonlocal (referral) patients was 48.8% (238 of 488) in the postmenopausal group and 25.3% (44 of 174) in the premenopausal group. Within both the postmenopausal and premenopausal groups, the prevalence of ovarian cancer was significantly higher in the nonlocal patients than in the local patients (P<.001). Because the guidelines are intended to be used by referring physicians, this is extremely important when considering the performance of these tests, and therefore this subset was examined more closely.
Table 4 summarizes the sensitivity, specificity, positive predictive value, and negative predictive values in the local patients. In the local patient subgroup, the ACOG/SGO guidelines were able to identify 22 of 25 ovarian cancers (sensitivity of 88.0%) in the postmenopausal group; three of four ovarian cancers (sensitivity of 75%) in the premenopausal group were also identified by the guidelines. The specificity of the ACOG/SGO guidelines was 67.9% in the local postmenopausal women and 69.4% in the local premenopausal women, resulting in a referral to a gynecologic oncologist of more than 30% of women with benign masses in both groups. The positive predictive values of these guidelines in the local patients were 44.9% in postmenopausal women and only 13.6% in premenopausal women, with negative predictive values of at least 95% in both groups. The three cancers in the postmenopausal group and one cancer in the premenopausal group that were missed by the guidelines were all early-stage ovarian cancers (stage I or II).
We were able to maximize the sensitivity and specificity of the guidelines for referral in premenopausal patients by lowering the cutoff value of the CA 125 marker for referral in premenopausal patients to more than 67 units/mL, which improved the sensitivity of the guidelines from 79.2% to 85.4% and allowed detection of three more patients with ovarian cancer. Lowering the CA 125 level further to more than 35 units/mL, the same level used in postmenopausal patients, further increased the sensitivity to almost 90% but dropped the specificity to less than 50%. Lowering the cutoff level of CA 125 had similar effects on the early-stage patient group as well. Table 5 presents the results of lowering the CA 125 level in referral of premenopausal patients. In addition, we determined that family history of a first-degree relative with breast or ovarian cancer in either group did not contribute significantly to the ability of the guidelines to refer the appropriate patients. Thus, family history could be removed from the referral guidelines criteria without significantly altering the sensitivity of the guidelines, while improving specificity and positive predictive value. Tables 6 and 7 present data for the performance characteristics of these revised referral guideline criteria.
The ACOG/SGO guidelines perform well in predicting advanced-stage ovarian cancer, probably owing to the nature of advanced-stage disease. However, the guidelines perform poorly in identifying early-stage disease, especially in premenopausal women. Unfortunately, although patients with early-stage ovarian cancer have the best prognosis with treatment, they are also the most difficult to identify clinically. The poor performance of the guidelines in early-stage cancers is primarily reflective of both the absence of symptoms in early-stage patients and the lack of meaningful methods of early detection clinically.
The over-referral by the guidelines of more than 30% of women with benign disease in all groups is preferable to having a decreased sensitivity. Although this may increase anxiety to the patient and the number of patients referred to a specialist, it seems a reasonable balance compared with under-referral and inappropriate treatment that results in incomplete information on which to base adjuvant therapy or a reoperation.
One of the strengths of our study was the ability to separately evaluate the guidelines in a regional patient subgroup to try to decrease the impact of referral bias. We observed that the positive predictive values of the guidelines were lower in the subgroup analysis of the local patients for both post- and premenopausal groups, compared with all patients in each group. This is expected because the prevalence of cancer was lower in the local patient group. This is undoubtedly due to the referral bias in the nonlocal patients, ie, identification of patients by outside physicians who were appropriately following the guidelines would have eliminated many low-risk patients from referral to our center. We made the assumption that all local patients within the cohort have had their primary evaluation at the Mayo Clinic, while, in fact, some local patients may have been evaluated elsewhere before being referred to Mayo. This assumption allows us to decrease the referral center bias within our cohort of patients but will not completely eliminate this bias.
To improve the sensitivity of the guidelines in detecting ovarian cancer, especially in the premenopausal group, we looked at the strength of each criterion in contributing to the overall performance of the guidelines. We found that lowering the CA 125 level for referral to more than 67 units/mL in the premenopausal group provided greater sensitivity, with an acceptable decrease in specificity. The new referral level of more than 67 units/mL increased the sensitivity of detecting ovarian cancer in the early-stage premenopausal patients, a group in whom the original guidelines had the worst performance.
We also determined that family history of breast or ovarian cancer did not contribute to the performance of the guidelines, and removing this criterion simplified the guidelines without significantly affecting performance. For example, among the 240 premenopausal patients, 49 women had only family history as their single positive risk factor. Of these 49 patients, just two had cancer; therefore, in removing this criterion the sensitivity is slightly decreased, but the specificity is greatly improved since 47 patients without cancer no longer meet the criteria for referral. Family history of a first-degree relative with breast or ovarian cancer does not contribute to the performance of the guidelines, most probably because of the high prevalence of breast cancer in the general population. Having one relative with a breast cancer likely does not carry any additional risk for ovarian cancer in the vast majority of patients being considered. However, in patients who have a family history suggestive of a syndrome of familial breast and/or ovarian cancer, referral to a specialist for counseling regarding screening and risk management should be done regardless of the other characteristics of the adnexal mass. This rare segment of the population—those at high risk of a familial predisposition to ovarian cancer—should probably be considered separately, thus eliminating family history from the guidelines. Specificity is improved by eliminating women whose only risk factor is a nonrelevant family history.
In addition, we found that presence or absence of a nodular or fixed mass on examination in the postmenopausal group did not contribute to the performance of the guidelines. This is the most subjective criterion in the guidelines because it is dependent on different factors, such as the skill of the clinician, patient’s body habitus, and parity. Obviously, if a fixed or nodular mass is found on examination, the patient should be referred. We recognize that this finding could be clinically important in a small number of cases and, thus, should remain as a criterion, but it is unlikely to contribute much to the referral practices. Similarly, in a prior validation of the ACOG/SGO guidelines by Im et al,8 the authors also found that the sensitivity of guidelines improved by lowering the CA 125 level to more than 50 units/mL in the premenopausal group and leaving out the criteria of family history of breast or ovarian cancer and presence of nodular or fixed pelvic masses.
The ACOG/SGO guidelines are relevant for helping to ensure that women with advanced ovarian cancer are provided with appropriate care by gynecologic oncologists, which increases their survival. However, the guidelines frequently miss early-stage ovarian cancers because the guidelines are biased toward obvious signs and markers of advanced disease. Improvements in the referral of early-stage disease await continued research on identifying factors consistent with early-stage ovarian cancer, highlighting the importance of research into early markers of disease. Until such times, gynecologists should be aware that the current guidelines have poor sensitivity for early-stage and premenopausal ovarian cancer. This should be openly discussed during counseling. When operating on such patients without the intraoperative availability of gynecologic oncologists, procedures should be selected cautiously and with minimally invasive procedures, if possible. Such patients are often more easily managed secondarily by a gynecologic oncologist after a brief surgery (ie, laparoscopy) to make the diagnosis, as opposed to a major operation that is suboptimal and often results in delay of treatment.
In summary, based on our findings, we recommend that the referral guidelines be revised to exclude family history of breast or ovarian cancer and lowering the CA 125 referral level for premenopausal women to more than 67 units/mL. This results in a simplified set of guidelines that will be easier for clinicians to remember. In addition, the sensitivity and specificity of the guidelines are improved, particularly in the premenopausal patients, and result in more appropriate referral of patients to gynecologic oncologists. It should be stated that these revised guidelines are not intended for those patients at high risk for familial breast or ovarian cancer syndromes.