Despite the more than 4 million deliveries in the United States each year,1 there are currently no uniformly accepted measures of obstetric quality.2 A valid obstetric quality measure should have face validity, in that both obstetricians and patients believe that it measures the quality of obstetric care. The measure also should have construct validity demonstrating that hospitals that perform well on the quality measure of interest also perform well on other possible measures of quality. Additionally, the measure should be reproducible across different patient populations and across different time periods.2
The risk-adjusted cesarean delivery rate historically has been a proposed quality measure in obstetric care given its face validity, easy measurability, and construct validity demonstrated in prior work in which a high cesarean delivery rate at individual hospitals was associated with other markers of poor quality of care, such as infections, severe perineal lacerations, and neonatal complications.2–4 However, there are several issues with using the risk-adjusted cesarean delivery rate as a quality measure. First, obstetricians argue that using all cesarean deliveries in the measure is inappropriate because, in some situations, cesarean delivery is the standard of care. This criticism diminishes its face validity. Second, prior studies use data from nearly 10 years ago, when the cesarean delivery rate was significantly lower3–7; these results have not been validated using more recent data or in additional states. Finally, these studies do not compare the association of a hospital's risk-adjusted cesarean delivery rate with other measures, such as the Agency for Healthcare Research and Quality (AHRQ) patient safety indicators, as additional indications of its construct validity. These studies also find that those hospitals with lower-than-expected cesarean delivery rates have higher rates of maternal infection, longer lengths of stay, and neonatal asphyxia than do the hospitals in the expected-rate group.3,4 These results suggest that both higher-than-expected and lower-than-expected rates may be associated with adverse maternal and neonatal outcomes, although more evidence is needed.
With these concerns, we seek to validate the risk-adjusted cesarean delivery rate as a measure of obstetric quality by measuring the correlation, or the statistical relationship, between the risk-adjusted cesarean delivery rate and important maternal and neonatal outcomes in recent data from multiple states. Combined with prior data, this new analysis will help measure the reproducibility of prior risk-adjusted cesarean delivery rate results across different periods of time and across different states.
We evaluated the risk-adjusted cesarean delivery rate as a quality measure in two separate population-based cohorts of women to improve the face validity of the results. The first, general, model includes all women in the data set to evaluate the risk-adjusted cesarean delivery rate as a general quality metric for all delivering women. The second, restricted, model includes only primiparous women with term singleton pregnancies without a history of prior cesarean delivery. This second model is important because of biases inherent in the general model, which is based on all deliveries. These biases include: 1) the role of patient choice in delivery mode after a cesarean delivery, 2) the differential ability of hospitals to perform vaginal birth after cesarean, 3) the increased speed of labor and the lower likelihood of parous women with prior vaginal delivery to have cesarean deliveries, and 4) the higher likelihood of fetal heart rate abnormalities and need for cesarean delivery in preterm neonates. All analyses were performed on each cohort separately.
We collected birth certificates from all deliveries occurring in California and Pennsylvania between January 1, 2004, and June 30, 2005, and linked them to death certificates by each state's department of health using name and date of birth. These linked records then were matched to maternal and newborn hospital discharge records using previously described methods.8 California data were linked by the state department of health using established algorithms, and Pennsylvania data were linked in a similar fashion internally at our center. Using these techniques, we matched more than 98% of all birth certificates in the two states to maternal and newborn hospital records. The institutional review boards of the Children's Hospital of Philadelphia and the departments of health in California and Pennsylvania approved this study.
Birth certificates were excluded if the newborn had a gestational age less than 23 weeks or more than 44 weeks, a birth weight less than 400 g or greater than 8,000 g, or if the birth weight was more than 5 standard deviations from the mean birth weight for the recorded gestational age in the cohort. Cesarean deliveries were identified from an International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) code of 669.7x in the maternal delivery record or a notation of a cesarean delivery in the birth certificate. Cesarean deliveries for previa, herpes, malpresentation, and cord prolapse were excluded because medical standards of care support the delivery of these women via cesarean. Hospitals with fewer than 50 deliveries were combined into two small hospital groups, one for California and one for Pennsylvania, because the small number of deliveries at each individual hospital resulted in less stable assessments of the outcomes at each individual hospital.9
To evaluate the construct validity of the risk-adjusted cesarean delivery rate, we measured the correlation between the risk-adjusted cesarean delivery rate and six outcome measures. A composite maternal outcome included wound infection (ICD-9-CM codes 674.1x, 674.2x, 674.3x), postdelivery hemorrhage (ICD-9-CM codes 641.3x, 641.8x, 641.9x, 660.0x, 660.1x, 660.2x, 660.3x, 667.1x), and blood transfusion (ICD-9-CM codes 99.0 99.00 99.02 99.03 99.04). A composite neonatal outcome included neonatal death rate (defined as any death during the initial birth hospitalization) from death certificate records, neonatal asphyxia (ICD-9-CM codes 768.5, 768.6, 768.9), birth injury (ICD-9-CM codes 767.2, 767.4, 767.5, 767.6, 767.7, 767.8, 767.9), and neonatal seizure (ICD-9-CM codes 779.0, 780.3, 780.39, 780.31). Four patient safety indicators from the AHRQ also were examined: birth trauma (patient safety indicator 17) (ICD-9-CM codes 767.2, 767.4–767.9), injury with instrumented vaginal delivery (patient safety indicator 18), injury with noninstrumented vaginal delivery (patient safety indicator 19), and injury with cesarean delivery (patient safety indicator 20).
Our risk-adjustment model included covariate variables based on their association with one or more study outcomes, the likelihood that a patient with these covariates would undergo a cesarean delivery, biologic plausibility, and previous work.5,10 These variables included maternal comorbid conditions and neonatal congenital anomalies grouped by affected organ system. These maternal and neonatal comorbidities (shown in Table 1) were identified by ICD-9-CM codes. The c-statistics for all patients (general model) and the restricted cohort were 0.866 and 0.616 respectively.
We first calculated the expected risk-adjusted cesarean delivery rate in the following manner. Logistic regression was performed using all explanatory variables (eg, comorbidities, complications, birth weight, year). The model's results were used to calculate the probability that a given patient would undergo a cesarean delivery, known as the expected value. The expected values for each patient in a given hospital were summed to derive the expected rate of cesarean delivery at that hospital. The expected numbers for the composite of maternal outcomes, neonatal outcomes, and each individual AHRQ patient safety indicator also were calculated using similar methods. We then compared the expected rate of each outcome with the observed rate of the outcome at each hospital using the following formula:
After determining hospital-level differences between observed and expected risk-adjusted cesarean delivery rates, we measured the construct validity of the risk-adjusted cesarean delivery rate in two ways. First, we performed a Pearson correlation analysis between the risk-adjusted cesarean delivery rate and each outcome measure using the two models described. This correlation analysis evaluated the statistical relationships between two observed data values (risk-adjusted cesarean delivery rate and one of the six maternal and neonatal outcomes). A positive correlation was signified by a positive coefficient value; a negative correlation was signified by a negative value. Second, we assigned hospitals into statistically lower-than-expected, as expected, and statistically higher-than-expected categories for the risk-adjusted cesarean delivery rate and each of the six outcome measures using the previously described methods of Haberman.11,12 The Haberman method adjusts the standard error, and thus the statistical significance, of each (observed–expected)/n value to account correctly for the fact that the expected event model used the same patients as did the observed values at each hospital. We then examined the relationship between the risk-adjusted cesarean delivery rate and the six outcomes using these rankings. These methods paralleled other methods to identify outlier hospitals in publicly reported data.9 All statistical analyses reported two-tailed P values with a statistical significance level of 5% after adjusting for multiple comparison testing using the methods of Bonferroni. All analyses were performed using SAS 9.2 (SAS Institute Inc., Cary, NC).
A total of 401 hospitals were evaluated. Forty-two hospitals were grouped together into two small hospital groups (24 California hospitals and 18 Pennsylvania hospitals), leaving 361 hospital groups in our study. Initially, 957,438 birth records were identified for this project; 111,787 met the exclusion criteria, leaving 845,651 births in the final cohort in the overall model and 274,371 in the model restricted to primiparous patients with term singleton pregnancies. Table 2 demonstrates the average demographic characteristics for the patients by the general and restricted models.
Table 3 shows the Pearson's correlation coefficients between the corresponding risk-adjusted cesarean delivery rate model and each adverse outcome. For both cohorts, there was a negative correlation between the risk-adjusted cesarean delivery rate and each of the six outcomes, which ranged from a Pearson's correlation coefficient of -0.08 to -0.38. The maternal composite outcome was most negatively correlated with the risk-adjusted cesarean delivery rate in both models. These correlations were all statistically significant except for the correlation between the risk-adjusted cesarean delivery rate and patient safety indicator 19 (injury with noninstrumented vaginal delivery). With respect to individual outcomes, there was a statistically significant correlation between the risk-adjusted cesarean delivery rate and maternal hemorrhage (r=-0.369, P<.001) and wound complications (r=-0.342, P<.001).
We next determined which hospitals had significantly higher-than-expected or lower-than-expected risk-adjusted cesarean delivery rates for each model and each of the six adverse outcomes. We then identified the risk-adjusted cesarean delivery rates for hospitals that had higher-than-expected rates of each of the six additional adverse outcomes. As with the correlation analysis, hospitals with higher-than-expected rates of the six adverse outcomes were more likely to have lower-than-expected risk-adjusted cesarean delivery rates in both the general cohort and the restricted cohort. In the general cohort, 59.8% of the 107 hospitals with lower-than-expected risk-adjusted cesarean delivery rates had a higher-than-expected rate of at least one of the six adverse outcomes, compared with 19.6% of the 102 hospitals with higher-than-expected risk-adjusted cesarean delivery rates and 36.1% of the as expected group. A similar result was seen with the restricted cohort (Table 4). Compared with the expected risk-adjusted cesarean delivery rate group, a statistically similar percentage of hospitals with higher-than-expected risk-adjusted cesarean delivery rates had higher-than-expected rates of the other six outcome measures. Average hospital-level characteristics for hospitals in the lower-than-expected, as-expected, and higher-than-expected risk-adjusted cesarean delivery rate groups are shown in Table 5.
Many clinical fields have accelerated their efforts to improve safety and quality greatly, using techniques such as performance measurement, regionalization and specialization, and communication of best practices. Although the use of the risk-adjusted cesarean delivery rate as an obstetric quality metric has been promising, it has several deficiencies: its lack of acceptance by the provider community, the continued rising rate, and the influence of factors outside of the hospital's control, such as a patient's choice to demand a cesarean delivery. Similar to prior work,3,4 our results in a cohort of more than 360 hospitals with almost 1 million deliveries suggest that hospitals that perform too few cesarean deliveries have higher-than-expected rates of other outcomes, such as maternal and neonatal complications. We speculate that, in some instances, patients benefit from having cesarean deliveries and that hospitals in which medical staff do not act fast enough to perform cesarean deliveries may have a higher rate of adverse outcomes. Additionally, although hospitals with higher-than-expected risk-adjusted cesarean delivery rates do not have higher-than-expected rates of other adverse outcomes, performing more risk-adjusted cesarean deliveries was not associated with improved outcomes. This overuse of medical health care may have significant negative consequences for many women13–15 and results in higher costs to patients and society.
Our findings in regard to the use of the risk-adjusted cesarean delivery rate as a quality measure should be placed into the context of the prior literature. Initial measures of obstetric quality used the raw cesarean delivery rate, which was undesirable given the lack of risk adjustment, and the maternal mortality rate, which is a rare occurrence. The risk-adjusted cesarean delivery rate was promising because it is easily measured and, historically, a high rate had excellent construct validity given its association with poor maternal and neonatal outcomes.3,4 Conversely, a lower-than-expected risk-adjusted cesarean delivery rate also was associated with poorer neonatal outcomes in three separate studies.4,7,16 These findings suggest that the risk-adjusted cesarean delivery rate may be a possible quality measure, either higher or lower than expected.
There are several plausible explanations for our findings that demonstrate an association between worse maternal and neonatal outcomes and AHRQ patient safety indicators when the risk-adjusted cesarean delivery rate is lower than expected but no difference when the risk-adjusted cesarean delivery rate is higher than expected. First, in many clinical settings, outcomes improve with volume. The increasing volume of cesarean delivery1,17 may lead to a decline in complications such as hemorrhage and infection. Second, improvements in sterilization techniques and operating room procedures may result in a lack of correlation between infection and higher-than-expected cesarean delivery rate in our study. Third, changes in practice patterns owing to external, nonmedical forces such as the liability climate and cesarean delivery by maternal request may increase the number of cesarean deliveries for nonmedical reasons.18 In each case, secular changes in the use of cesarean delivery may have led to our results differing from prior work.
Performing too many cesarean deliveries was not associated with adverse maternal and neonatal outcomes, nor was it associated with improved outcomes or protective effects. This lack of a correlation between a higher-than-expected cesarean delivery rate and adverse outcomes should not suggest that a higher-than-expected cesarean rate is desirable. Specifically, it likely reflects an overuse of medical care and the performance of unnecessary procedures. This fact may represent a different aspect of poor quality that is not being measured through these methods. For example, Medicare patients residing in geographic areas with high-intensity practice patterns (ie, those areas with a high quantity of health care services per capita) have been shown to have a higher 5-year mortality rate with conditions such as myocardial infarction and colorectal cancer, reflecting a lower quality of care.19–21 Further, in a cross-sectional analysis of Medicare enrollees, Fisher and colleagues demonstrate that, in areas with greater hospital-bed capacity, there is increased hospital use without detectable mortality benefit.22
Finally, the time period of our study reflects the increasing cesarean delivery rate compared with previously published literature. Using data from previous years to construct our expected rate of cesarean delivery, when the initial studies were performed, would drastically change our results: with this older data as the baseline, almost all hospitals would have higher-than-expected risk-adjusted cesarean delivery rates in 2004–2005. Therefore, the metric of risk-adjusted cesarean delivery rate would not differentiate hospitals of different levels of quality. This result shows the importance of continually testing the construct validity and reproducibility of any obstetric quality measure.
This study has several limitations. First, we identified comorbidities and complications using administrative data including ICD-9-CM codes, not primary chart abstraction. This method allows for the inclusion of a population-based cohort of patients not otherwise obtainable. However, some comorbidities may have been undercoded or overcoded, resulting in a misclassification bias. Second, although the results are adjusted for a large number of observable factors, unobserved changes in the maternal population may be responsible for the observed changes in outcomes. The concurrent observation of lower-than-expected risk-adjusted cesarean delivery rates and higher-than-expected adverse outcomes does not prove causality. However, the observation that the majority of hospitals with lower-than-expected risk-adjusted cesarean delivery rates have higher-than-expected rates for one or more adverse outcomes suggests that an association is present.
In conclusion, in a cohort of 361 hospitals, a lower-than-expected risk-adjusted cesarean delivery rate is associated with a higher-than-expected rate of several adverse maternal and neonatal outcomes. However, with the rapidly rising cesarean delivery rate, performing too many cesarean deliveries is not beneficial and may have significant negative future reproductive consequences to women.13–15 The lack of an association between higher-than-expected risk-adjusted cesarean delivery rates and higher rates of adverse obstetric outcomes suggests that the risk-adjusted cesarean delivery rate may not be a sustainable measure of obstetric quality. These findings underscore the importance of developing novel ways to measure all aspects of obstetric quality to ensure that women are receiving the safest and most effective obstetric care available.
1. Martin JA, Hamilton BE, Sutton PD, Ventura SJ, Menacker F, Kirmeyer S, and Mathews TJ. Births: final data for 2006. Natl Vital Stat Rep. 2009;57(7):1–101.
2. Bailit JL. Measuring the quality of inpatient obstetrical care. Obstet Gynecol Surv 2007;62:207–13.
3. Bailit JL, Love TE, Dawson NV. Quality of obstetric care and risk-adjusted primary cesarean delivery rates. Am J Obstet Gynecol 2006;194:402–7.
4. Gould J, Danielsen B, Korst L, Phibbs R, Chance K, Main E, et al. Cesarean delivery rates and neonatal morbidity in a low-risk population Obstet Gynecol 2004;104:11–9.
5. Bailit JL, Garrett JM. Stability of risk-adjusted primary cesarean delivery rates over time. Am J Obstet Gynecol 2004;190:395–400.
6. Main EK, Moore D, Farrell B, Schimmel LD, Altman RJ, Abrahams C, et al. Is there a useful cesarean birth measure? Assessment of the nulliparous term singleton vertex cesarean birth rate as a tool for obstetric quality improvement. Am J Obstet Gynecol 2006;194:1644–51.
7. Bailit JL, Garrett JM, Miller WC, McMahon MJ, Cefalo RC. Hospital primary cesarean delivery rates and the risk of poor neonatal outcomes. Am J Obstet Gynecol 2002;187:721–7.
8. Herrchen B, Gould J, Nesbitt T. Vital statistics linked birth/infant death and hospital discharge record linkage for epidemiological studies. Comput Biomed Res 1997:290–305.
9. U.S. Department of Health and Human Services. Hospital quality compare. Available at: www.hospitalcompare.hhs.gov
. Retrieved September 15, 2009.
10. Bailit JL, Dooley SL, Peaceman AN. Risk adjustment for interhospital comparison of primary cesarean rates. Obstet Gynecol 1999;93:1025–30.
11. Haberman S. The analysis of frequency data. Chicago (IL): The University of Chicago Press; 1974. p. 138–9.
12. Haberman S. Generalized residuals for log-linear models: generalized residuals for log-linear models. Boston (MA): The Biometric Society; 1976. p. 104–23.
13. Paré E, Quiñones JN, Macones GA. Vaginal birth after caesarean section versus elective repeat caesarean section: assessment of maternal downstream health outcomes. BJOG 2006;113:75–85.
14. Gilliam M. Cesarean delivery on request: reproductive consequences. Semin Perinatol 2006;30:257–60.
15. Gilliam M, Rosenberg D, Davis F. The likelihood of placenta previa with greater number of cesarean deliveries and higher parity. Obstet Gynecol 2002;99:976–80.
16. Li T, Rhoads GG, Smulian J, Demissie K, Wartenberg D, Kruse L. Physician cesarean delivery rates and risk-adjusted perinatal outcomes. Obstet Gynecol 2003;101:1204–12.
17. Centers for Disease Control and Prevention (CDC). Rates of cesarean delivery—United States, 1993. MMWR Morb Mortal Wkly Rep 1995;44:303–7.
18. Yang YT, Mello MM, Subramanian SV, Studdert DM. Relationship between malpractice litigation pressure and rates of cesarean section and vaginal birth after cesarean section. Med Care 2009;47:234–42.
19. Fisher ES. Medical care–is more always better? N Engl J Med 2003;349:1665–7.
20. Sirovich BE, Gottlieb DJ, Welch HG, Fisher ES. Regional variations in health care intensity and physician perceptions of quality of care. Ann Intern Med 2006;144:641–9.
21. Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ, Lucas FL, Pinder EL. The implications of regional variations in Medicare spending. Part 2: health outcomes and satisfaction with care. Ann Intern Med 2003;138:288–98.
22. Fisher ES, Wennberg JE, Stukel TA, Skinner JS, Sharp SM, Freeman JL, et al. Associations among hospital capacity, use, and mortality of US Medicare beneficiaries, controlling for sociodemographic factors. Health Serv Res 2000;34:1351–62.