Share this article on:

The Validity of Discharge Billing Codes Reflecting Severe Maternal Morbidity

Sigakis, Matthew J. G. MD; Leffert, Lisa R. MD; Mirzakhani, Hooman MD, MMSc; Sharawi, Nadir MBBS, FRCA; Rajala, Baskar MBBS, FRCA; Callaghan, William M. MD, MPH; Kuklina, Elena V. MD, PhD; Creanga, Andreea A. MD, PhD; Mhyre, Jill M. MD; Bateman, Brian T. MD, MSc

doi: 10.1213/ANE.0000000000001436
Healthcare Economics, Policy, and Organization: Original Clinical Research Report

BACKGROUND: Discharge diagnoses are used to track national trends and patterns of maternal morbidity. There are few data regarding the validity of the International Classification of Diseases (ICD) codes used for this purpose. The goal of our study was to try to better understand the validity of administrative data being used to monitor and assess trends in morbidity.

METHODS: Hospital stay billing records were queried to identify all delivery admissions at the Massachusetts General Hospital for the time period 2001 to 2011 and the University of Michigan Health System for the time period 2005 to 2011. From this, we identified patients with ICD-9-Clinical Modification (CM) diagnosis and procedure codes indicative of severe maternal morbidity. Each patient was classified with 1 of 18 different medical/obstetric categories (conditions or procedures) based on the ICD-9-CM code that was recorded. Within each category, 20 patients from each institution were selected at random, and the corresponding medical charts were reviewed to determine whether the ICD-9-CM code was assigned correctly. The percentage of correct codes for each of 18 preselected clinical categories was calculated yielding a positive predictive value (PPV) and 99% confidence interval (CI).

RESULTS: The overall number of correctly assigned ICD-9-CM codes, or PPV, was 218 of 255 (86%; CI, 79%–90%) and 154 of 188 (82%; CI, 74%–88%) at Massachusetts General Hospital and University of Michigan Health System, respectively (combined PPV, 372/443 [84%; CI, 79–88%]). Codes within 4 categories (Hysterectomy, Pulmonary edema, Disorders of fluid, electrolyte and acid–base balance, and Sepsis) had a 99% lower confidence limit ≥75%. Codes within 8 additional categories demonstrated a 99% lower confidence limit between 74% and 50% (Acute respiratory distress, Ventilation, Other complications of obstetric surgery, Disorders of coagulation, Cardiomonitoring, Acute renal failure, Thromboembolism, and Shock). Codes within 6 clinical categories demonstrated a 99% lower confidence limit <50% (Puerperal cerebrovascular disorders, Conversion of cardiac rhythm, Acute heart failure [includes arrest and fibrillation], Eclampsia, Neurotrauma, and Severe anesthesia complications).

CONCLUSIONS: ICD-9-CM codes capturing severe maternal morbidity during delivery hospitalization demonstrate a range of PPVs. The PPV was high when objective supportive evidence, such as laboratory values or procedure documentation supported the ICD-9-CM code. The PPV was low when greater judgment, interpretation, and synthesis of the clinical data (signs and symptoms) was required to support a code, such as with the category Severe anesthesia complications. As a result, these codes should be used for administrative research with more caution compared with codes primarily defined by objective data.

From the *Department of Anesthesiology, University of Michigan Medical School, Ann Arbor, Michigan; Division of Obstetric Anesthesia, Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts; Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts; §Division of Reproductive Health, Centers for Disease Control and Prevention, Atlanta, Georgia; Epidemiology & Surveillance Branch, Division for Heart Disease and Stroke Prevention, Centers for Disease Control and Prevention, Atlanta, Georgia; and Department of Anesthesiology, University of Arkansas for Medical Sciences, Little Rock, Arkansas.

Accepted for publication April 29, 2016.

Funding: None.

The authors declare no conflicts of interest.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website.

Reprints will not be available from the authors.

Address correspondence to Matthew J. G. Sigakis, MD, Department of Anesthesiology, University of Michigan Medical School, 1H247 UH, SPC 5048, 1500 East Medical Center Dr, Ann Arbor, MI 48109. Address e-mail to msigakis@med.umich.edu.

Recent analysis of available data within the United States health care system indicates a steady increase in severe maternal morbidity and mortality since the 1990s.1–5 Continued effort is aimed at understanding the causes of these trends by monitoring adverse health events not resulting in death6–8 and developing tools such as prediction models to better identify and care for those with increased health risk.9,10 Clinical researchers, public health epidemiologists, health care economists, and hospital administrators frequently use the International Classification of Diseases (ICD) codes documented in hospital discharge billing data to identify cases with severe maternal complications.

Administrative data have also been used increasingly to help answer questions in clinical epidemiology and comparative effectiveness research; however, results may be affected by the quality of the underlying administrative data used for such purposes.11 At present, little is known about the validity of ICD codes representing severe maternal morbidity. Therefore, studies that assess ICD code validity are essential to substantiate the quality or identify limitations of utilized administrative data. We aimed to define the positive predictive value (PPV) of ICD codes used to identify maternal morbidity in epidemiologic studies in a sample of delivery admissions drawn from 2 large academic medical centers.

Back to Top | Article Outline

METHODS

For this study, we queried billing records to identify all obstetric deliveries at the Massachusetts General Hospital (MGH) from 2001 to 2011 and at the University of Michigan Health System (UMHS) from 2005 to 2011. Time periods reflect the availability of electronic patient billing data based on when each institution’s electronic billing infrastructure was established. This study was approved by the Partners Institutional Review Board, Boston, Massachusetts (2011P001432) and the University of Michigan Institutional Review Board, Ann Arbor, Michigan (IRB00001999). The requirement for written informed consent was waived by the institutional review boards.

The MGH and UMHS are teaching hospitals located in Boston, Massachusetts, and Ann Arbor, Michigan, respectively. The MGH is the largest teaching hospital of Harvard Medical School, with 950 inpatient beds and approximately 48,000 inpatients annually, including approximately 3500 deliveries annually.12,13 The UMHS is one of the largest teaching hospitals in the Midwest, with 993 inpatient beds and approximately 45,000 inpatients annually, including approximately 3800 deliveries annually.14,15

From the population of obstetric deliveries at each institution, we identified deliveries with ICD, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis and procedure codes indicative of severe maternal morbidity. We then grouped these deliveries into 1 of 18 preselected clinical categories of severe maternal morbidity (conditions or procedures) based on the ICD-9-CM code that was recorded.

Table 1

Table 1

Informed by studies that have previously used health care utilization data to describe the patterns of severe maternal morbidity,5–7 the authors focused on 18 clinical categories felt to reflect high acuity and the most severe maternal morbidity and end-organ injury. We reviewed the codes used to define these clinical categories in previously published algorithms5–7 and supplemented the codes with those identified based on our own review of the 2012 ICD-9-CM coding manual. The 18 clinical categories included in our analysis, and the ICD-9-CM codes used to comprise these categories, are shown in Table 1.

Back to Top | Article Outline

Data Source

At the MGH, we used the Partner Enterprise Master Patient Index with records from January 1, 2001, through December 31, 2011. We queried this patient billing database for all obstetric records using ICD-9-CM delivery code V27.x and, concomitantly, ICD-9-CM codes representing maternal morbidity (as shown in Table 1). All records represented events during the delivery hospitalization. We queried the University of Michigan Data Warehouse database using the same methodology for the period of January 1, 2005, through December 31, 2011. These selected records served as our sampling frame.

Back to Top | Article Outline

Sample Selection Methodology

At each institution, within each of the 18 clinical categories, 20 patients were chosen randomly (Rand function, Microsoft Excel, version 14.4.2, Microsoft Corp, Redmond, WA) and the associated medical chart reviewed in detail for supportive clinical information. If the number of patients within the preselected category was ≤20, all patients and their associated records were reviewed. If a patient’s medical chart was missing or unavailable, an alternate patient within the same morbidity category was selected randomly (Rand function) for review, if available. Finally, the results from each institution were combined for aggregate analysis.

Back to Top | Article Outline

Medical Record/Chart Review Methodology

Electronic and paper medical charts were used as a source of clinical information. At each institution, 2 physicians (MGH: M.S. and H.M.; UMHS: J.M. and B.R.) reviewed the billing record and medical chart associated with each code to determine correctness. Evidence within the patient’s medical chart, such as operative procedure note, laboratory data, and physician assessment, was reviewed in detail to substantiate the billing code assigned. Although both physician reviewers were aware of the maternal morbidity ICD-9-CM code before reviewing the record, neither physician was aware of the other’s assessment until the conclusion of data acquisition. If the 2 reviewers disagreed on whether the ICD-9-CM code correctly reflected the patient condition, a third physician (MGH: B.B.; UMHS: N.S.) rendered the definitive opinion as to whether the ICD-9-CM code was correct. For the MGH data set only, length of stay and intensive care unit (ICU) admission data also were collected.

In some cases, a patient’s billing record included >1 ICD-9-CM code of interest for our study sample. For example, a patient might require ventilation (96.7) for respiratory distress (518.5) and undergo invasive monitoring (89.6). In this circumstance, the case was eligible to be counted toward the validation analysis of each clinical category.

We analyzed each delivery record where the associated ICD-9-CM code was deemed incorrect by the above process and identified themes post hoc that summarized possible explanations for incorrect coding (categories A, B, and C).

To assess interrater consistency, we analyzed the rate of disagreement between the primary physician reviewers who examined ICD-9-CM code supportive evidence. We identified categories post hoc that summarized possible explanations for these disagreements (categories I and II).

All data for this study were recorded and analyzed in Microsoft Excel for Mac 2011, version 14.4.2 (Microsoft Corp, Redmond, WA) and were maintained in an encrypted computer with password access restricted to the study team. Paper medical charts were housed in a secure location within the institution’s medical records department. All patient identifiers, including date and year of delivery, were removed from the data set before analysis.

Back to Top | Article Outline

Statistical Analysis

Using the medical chart as the gold standard and comparing it with the identified billing records, we calculated the percentage of correct ICD-9-CM codes in each maternal morbidity category, resulting in a PPV and 99% confidence interval (CI). PPV is defined as the number of true positives (where the ICD-9-CM code from billing record and clinical diagnosis from medical record agree on a positive diagnosis) divided by the sum of all positives identified from parturient billing records, which include true positives and false positives. The Wilson Score interval was used to calculate each CI, and we emphasize the lower confidence limit given the wide intervals in our study.16 Analyses were performed on MGH and UMHS data sets individually and then combined for aggregate results.

For the MGH data set only, we performed a subgroup analysis, restricting codes within our sample frame to those associated with a hospitalization ≥3 days, ≥7 days, and ICU stay, and then calculated the PPV for the codes remaining in each maternal morbidity category. Although imposing a length of stay or ICU restriction decreases the number of cases available for assessment, it may increase the specificity of codes for surveillance efforts; codes for severe events, such as amniotic fluid embolism, cardiac arrest, or stroke, are typically associated with longer hospital stays.7,17,18

Identified coding errors were categorized according to error type for analysis. Cohen’s κ coefficient was calculated to measure interrater agreement between the study teams’ physician chart reviewers.19 Disagreements between physician reviewers were investigated and categorized for analysis.

Back to Top | Article Outline

RESULTS

Sample Population

Figure 1

Figure 1

The query of hospital billing records for ICD-9-CM diagnosis and procedure codes of interest yielded 687 patient deliveries (NMGH = 474; NUMHS = 213; Figure 1) with ICD-9-CM codes reflecting maternal morbidity. As described previously, patients were sorted into 1 of the 18 preselected clinical categories by ICD-9-CM code. After we selected patients from each category, a total of 443 patient billing records and associated medical charts underwent a detailed review (NMGH = 255; NUMHS = 188). Eight patients selected had incomplete medical charts precluding analysis, and alternate patients were selected randomly (NMGH = 7; NUMHS = 1).

Back to Top | Article Outline

Positive Predictive Value

The overall number of correctly assigned ICD-9-CM codes, or PPV, was 218 of 255 (85%; CI, 79%–90%) and 154 of 188 (82%; CI, 74%–88%) at MGH and UMHS, respectively (combined PPV, 372/443 [84%; CI, 79%–88%]). The PPV and 99% CI for the codes reflecting each maternal morbidity category are listed in Table 2. Codes within 6 clinical categories demonstrated a 99% lower confidence limit <50% (Puerperal cerebrovascular disorders, Conversion of cardiac rhythm, Acute heart failure [includes arrest and fibrillation], Eclampsia, Neurotrauma, and Severe anesthesia complications). Codes within 8 additional categories demonstrated a 99% lower confidence limit between 74% and 50% (Acute respiratory distress, Ventilation, Other complications of obstetric surgery, Disorders of coagulation, Cardiomonitoring, Acute renal failure, Thromboembolism, Shock). Codes within 4 categories (Hysterectomy; Pulmonary edema; Disorders of fluid, electrolyte, and acid–base balance; and Sepsis) had a 99% lower confidence limit ≥75%.

Table 2

Table 2

Results of a length of stay restriction on the MGH data set are provided in Table 3. Total PPV (the sum of all categories) is shown with each level of restriction: Total PPVoverall = 86% (CI, 79%–90%), total PPVLOS>3days = 89% (82%–93%), total PPVLOS>7 days = 92% (83%–97%), and total PPVICU 98% (89%–99%).

Table 3

Table 3

Table 4

Table 4

Review of all 71 identified coding inaccuracies yielded 3 categories of errors (Table 4). A detailed description for each error is presented in Supplemental Digital Content 1, Table 1, http://links.lww.com/AA/B451. The most common error identified was lack of clinical or laboratory evidence supporting the condition (category A). Other error categories included history of related condition, but no clinical or laboratory evidence of defect impacting current pregnancy (category B), and confusion between possible or expected side effects and actual complications (category C).

Back to Top | Article Outline

Disagreements Between Physician Reviewers

There were a total of 42 disagreements of 443 (9.5%) charts evaluated by the 2 primary physician reviewers (NMGH = 33/255 [13.3%]; NUMHS = 9/188 [4.8%]). Overall, Cohen’s κ statistic for the 2 primary reviewers was 0.80 (κMGH = 0.72; κUMHS = 0.90; Supplemental Digital Content 2, Table 2, http://links.lww.com/AA/B452). Disagreements occurred because data were either overlooked (category I) or misinterpreted (category II) (Supplemental Digital Content 3, Table 3, http://links.lww.com/AA/B453).

Back to Top | Article Outline

DISCUSSION

Hospital administrative data are an attractive data source because of wide availability and low cost; however, the codes used to record relevant conditions may be subject to error. Our analysis of supporting ICD-9-CM codes illustrates a range of PPVs for severe maternal morbidity categories. Codes demonstrated a high PPV when supporting medical information containing objective data, such as laboratory values, pathology results, or procedure documentation. The PPV was low when greater judgment, interpretation, and synthesis of clinical information (signs and symptoms) was required to support the ICD-9-CM code, such as with the category Severe anesthesia complications. These ICD-9-CM codes should be used for administrative research with more caution compared with codes primarily defined by objective data.

As illustrated in Supplemental Digital Content 1, Table 1, http://links.lww.com/AA/B451, errors in coding identified in this study corresponded to 1 of 3 categories. Lack of clinical or laboratory evidence supporting the condition (category A) was the most common reason for coding errors—either a code was placed erroneously or a charted medical condition misunderstood. Inaccuracies in coding for some disorders including Puerperal cerebrovascular disorders and Thromboembolism occurred when chronic conditions were miscoded as acute conditions and did not reflect a new peripartum morbidity (category B). For other conditions, most notably Severe anesthesia complications, Other complications of obstetric surgery, and Disorders of coagulation, inaccuracies occurred because of confusion between possible or expected side effects and actual complications (category C); examples related to Severe anesthesia complications are discussed below.

Coding for procedures demonstrated a high PPV such as with categories Hysterectomy (PPV, 100%; CI, 86%–100%) and Ventilation (PPV, 92%; CI, 74%–98%). Previous studies in which authors validated coding for obstetric and non-obstetric procedures also demonstrated high PPVs.20 Conditions such as Sepsis or Disorders of fluid, electrolyte and acid-base balance also demonstrate a high PPV, perhaps because of the availability of objective data (medications, vital signs, and laboratory values), which too is consistent with previous validation studies.21

For conditions such as Eclampsia and Disorders of coagulation, chart review occasionally illustrated the presence of related diagnoses, but no evidence for the condition of interest. For example, patients with preeclampsia and no evidence of seizure were miscoded with eclampsia, reflecting misunderstanding of the definition or unintentional error. Our observation of low coding PPV for eclampsia is consistent with Geller et al22 in which the PPV of eclampsia was 41.7%.

The PPV for coding for anesthesia-related complication was also low in our study (PPV, 42%; CI, 15%–74%); however, this finding was based on the review of only 12 cases identified in our query, and therefore, the CI for our estimate is very wide. A previous study validation, in which the authors used a slightly different set of ICD-9-CM codes, reported a high PPV for codes representing obstetric anesthesia complications.23 In our study, the observed inaccuracies in codes for anesthesia complications occurred because of confusion between possible or expected side effects and actual complications (category C). For example, 1 case was assigned pulmonary complications of anesthesia because a patient was intubated and sent to the ICU after developing postpartum hemorrhage in the operating room. Another case was assigned complications of anesthesia because the patient underwent an elective awake fiberoptic intubation for known difficult airway.

ICD-coding inaccuracies have been identified in other anesthesia conditions such as malignant hyperthermia, where only 25% of cases reviewed were true positives,24 and obstructive sleep apnea.25 If claims data are going to be used for the study of these complications, work is needed to improve the quality of documentation around anesthesia-related complications and in the training of coders about these complications. Perhaps explicit identification of whether clinical care is routine or nonroutine for billing purposes would help calibrate coding of anesthetic-related events.

Importantly, codes may be correctly applied to a patient’s hospitalization but not reflect a condition’s severity. When a condition is present, any associated morbidity is on a spectrum, from minor to catastrophic consequences. Some conditions, although clearly present during our chart review, did not pose the same threat to life as others. For example, in Disorders of coagulation, 2 cases of thrombocytopenia existed. These cases were appropriately coded, but no significant bleeding event or platelet transfusion occurred. Thus, even for the ICD-9-CM codes representing maternal morbidity that had a high PPV in this study, the morbidity identified may not be severe.

The requirement for ICU admission appears to be an effective way to verify the severity of any complication identified by ICD-9-CM diagnostic codes as the overall PPV was high at 98% (CI, 89–99%). This finding supports the principle of defining severe maternal morbidity as a combination of both a diagnosed complication and ICU admission. The National Partnership for Maternal Safety and the Joint Commission define severe maternal morbidity as “a patient safety event (not primarily related to the natural course of the patient’s illness or underlying condition) that reaches a patient and results in … either ICU admission or four or more units of erythrocytes transfused.”26–28 Going forward, the ICD Tenth Revision, Clinical Modification codes may better capture these differences in severity. Compared with ICD-9-CM, the ICD Tenth Revision, Clinical Modification contains 5 times as many diagnosis codes and 19 times as many procedure codes to update and enhance coding description and, therefore, data quality.29

Although the primary goal of this study was to assess the PPV of ICD-9-CM codes, we evaluated the level of agreement between primary physician reviewers in this study given the complexity of information often contained in medical records. The extent of disagreement between primary physician reviewers within this study (42/443, 9.5%) is consistent with previously reported ICD coding error rates, ranging from approximately 10% to 20%.30,31 The majority of disagreements between our primary reviewers was related to data misinterpretation (27/42, 64%) and the remaining because of data simply being overlooked (Supplemental Digital Content 3, Table 3, http://links.lww.com/AA/B453). Almost half of the disagreements (20/42, 48%) related to ICD-9-CM codes in maternal morbidity categories with low PPV: Puerperal cerebrovascular disorders, Acute heart failure (includes arrest and fibrillation), Eclampsia, and Severe anesthesia complications.

This study has several limitations. First, for some categories, only a small number of cases were identified and reviewed because such clinical events are rare. As a result, CIs on the PPV estimates for these conditions are wide. Second, variances in regional or institutional coding practices may exist. Despite the high PPV at 2 separate institutions, additional studies should be conducted to ensure the generalizability of the results. For example, information technology architecture, staff resources, and case mix and volume may be different at community or rural health centers, and it is unclear how these factors may impact coding accuracy. Third, although we provide examples of miscoding in the basis of our detailed chart review, our discussion about reasons for incorrect coding are speculative and cannot be fully substantiated without further analysis, such as interviewing coders or evaluation of the coding process. Furthermore, this study assesses the PPV of codes reflecting maternal morbidity but does not evaluate the negative predictive value (NPV)—codes not assigned when a condition truly exists. Although for some studies defining conditions with high PPV (even in the face of limited sensitivity) will be adequate, for others having a high NPV will be important (ie, a low NPV for an ICD code may result in underestimation of maternal morbidity rates). Further research is needed to assess NPV of maternal morbidity ICD codes. In addition, prospective studies are needed to confirm the sensitivity and specificity of maternal morbidity codes. Of note, previous studies validating ICD-9-CM codes for rare pregnancy complications32 and nondelivery-related complications33 had greater specificity than sensitivity. Fourth, although a restriction based on length of stay or ICU admission resulted in a high PPV for most categories, restricting the codes may have also increased the false negatives because there were fewer codes in this restricted data set. Finally, PPV depends on the prevalence of disease, and different prevalences of a condition in other patient populations could yield different results.

Back to Top | Article Outline

CONCLUSIONS

The validity of ICD-9-CM codes capturing severe maternal morbidity during delivery hospitalization depends on the amount of objective clinical data (laboratory values, pathology results, or procedure documentation) supporting the definition of the assigned code. Low PPV was observed in maternal morbidity categories when greater judgment, interpretation, and synthesis of supportive clinical information (signs and symptoms) were required to support the assigned codes, such as with the category Severe anesthesia complications. Therefore, surveillance, research, and quality improvement efforts should focus on codes defined by objective supporting data and, accordingly, maternal morbidity categories demonstrating a high PPV that are represented by these types of codes. Likewise, when a high degree of judgment, interpretation, and synthesis of the clinical scenario is required to substantiate the code, greater caution should be exercised.

Back to Top | Article Outline

DISCLOSURES

Name: Matthew J. G. Sigakis, MD.

Contribution: This author helped design the study, collect the data, analyze the data, and prepare the manuscript.

Name: Lisa R. Leffert, MD.

Contribution: This author helped design the study, analyze the data, and prepare the manuscript.

Name: Hooman Mirzakhani, MD, PhD, MMSc.

Contribution: This author helped design the study, collect the data, analyze the data, and prepare the manuscript.

Name: Nadir Sharawi, MBBS, FRCA.

Contribution: This author helped design the study, collect the data, and prepare the manuscript.

Name: Baskar Rajala, MBBS, FRCA.

Contribution: This author helped design the study, collect the data, and prepare the manuscript.

Name: William M. Callaghan, MD, MPH.

Contribution: This author helped design the study, analyze the data, and prepare the manuscript.

Name: Elena V. Kuklina, MD, PhD.

Contribution: This author helped design the study, analyze the data, and prepare the manuscript.

Name: Andreea A. Creanga, MD, PhD.

Contribution: This author helped design the study, analyze the data, and prepare the manuscript.

Name: Jill M. Mhyre, MD.

Contribution: This author helped design the study, collect the data, analyze the data, and prepare the manuscript.

Name: Brian T. Bateman, MD, MSc.

Contribution: This author helped design the study, collect the data, analyze the data, and prepare the manuscript.

This manuscript was handled by: Franklin Dexter, MD, PhD.

Back to Top | Article Outline

REFERENCES

1. Say L, Pattinson RC, Gülmezoglu AM. WHO systematic review of maternal morbidity and mortality: the prevalence of severe acute maternal morbidity (near miss). Reprod Health. 2004;1:3.
2. Creanga AA, Berg CJ, Ko JY, et al. Maternal mortality and morbidity in the United States: where are we now? J Womens Health (Larchmt). 2014;23:3–9.
3. Hogan MC, Foreman KJ, Naghavi M, et al. Maternal mortality for 181 countries, 1980-2008: a systematic analysis of progress towards Millennium Development Goal 5. Lancet. 2010;375:1609–1623.
4. Wanderer JP, Leffert LR, Mhyre JM, et al. Epidemiology of obstetric-related ICU admissions in Maryland: 1999-2008*. Crit Care Med. 2013;41:1844–1852.
5. Callaghan WM, Creanga AA, Kuklina EV. Severe maternal morbidity among delivery and postpartum hospitalizations in the United States. Obstet Gynecol. 2012;120:1029–1036.
6. Kuklina EV, Meikle SF, Jamieson DJ,, et al. Severe obstetric morbidity in the United States: 1998–2005. Obstet Gynecol. 2009;113:293–299.
7. Mhyre JM, Bateman BT, Leffert LR. Influence of patient comorbidities on the risk of near-miss maternal morbidity or mortality. Anesthesiology. 2011;115:963–972.
8. Ronsmans C. Severe acute maternal morbidity in low-income countries. Best Pract Res Clin Obstet Gynaecol. 2009;23:305–316.
9. Bateman BT, Mhyre JM, Hernandez-Diaz S, et al. Development of a comorbidity index for use in obstetric patients. Obstet Gynecol. 2013;122:957–965.
10. Cecatti JG, Souza JP, Oliveira Neto AF, et al. Pre-validation of the WHO organ dysfunction based criteria for identification of maternal near miss. Reprod Health. 2011;8:22.
11. Neuman MD. The importance of validation studies in perioperative database research. Anesthesiology. 2015;123:243–245.
12. Massachusetts General Hospital News Releases. 2015. Available at: http://www.massgeneral.org/about/pressrelease.aspx?id=1483. Accessed June 15, 2016.
13. Massachusetts General Hospital Obstetrics Program. 2015. Available at: http://www.massgeneral.org/obgyn/services/treatmentprograms.aspx?id=1379. Accessed June 15, 2016.
14. University of Michigan Health System, Facts and Figures. 2015. Available at: http://www.uofmhealth.org/about%20umhs/facts-figures. Accessed June 15, 2016.
15. University of Michigan Health System, Pregnancy and Childbirth. 2015. Available at: http://www.uofmhealth.org/conditions-treatments/pregnancy-childbirth. Accessed June 15, 2016.
16. Brown LD, Cat TT, DasGupta A. Interval estimation for a proportion. Statistical Science. 2001;16:101–133.
17. Callaghan WM, Mackay AP, Berg CJ. Identification of severe maternal morbidity during delivery hospitalizations, United States, 1991-2003. Am J Obstet Gynecol. 2008;199:133.e1–133.e8.
18. Callaghan WM, Grobman WA, Kilpatrick SJ, et al. Facility-based identification of women with severe maternal morbidity: it is time to start. Obstet Gynecol. 2014;123:978–981.
19. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20:37–46.
20. Yasmeen S, Romano PS, Schembri ME, et al. Accuracy of obstetric diagnoses and procedures in hospital discharge data. Am J Obstet Gynecol. 2006;194:992–1001.
21. Quan H, Li B, Saunders LD, et al.; IMECCHI Investigators. Assessing validity of ICD-9-CM and ICD-10 administrative data in recording clinical conditions in a unique dually coded database. Health Serv Res. 2008;43:1424–1441.
22. Geller SE, Ahmed S, Brown ML, et al. International Classification of Diseases-9th revision coding for preeclampsia: how accurate is it? Am J Obstet Gynecol. 2004;190:1629–1633.
23. Romano PS, Yasmeen S, Schembri ME, et al. Coding of perineal lacerations and other complications of obstetric care in hospital discharge data. Obstet Gynecol. 2005;106:717–725.
24. Pinyavat T, Rosenberg H, Lang BH, et al. Accuracy of malignant hyperthermia diagnoses in hospital discharge records. Anesthesiology. 2015;122:55–63.
25. McIsaac DI, Gershon A, Wijeysundera D, et al. Identifying obstructive sleep apnea in administrative data: a study of diagnostic accuracy. Anesthesiology. 2015;123:253–263.
26. Callaghan WM, Grobman WA, Kilpatrick SJ, et al. Facility-based identification of women with severe maternal morbidity: it is time to start. Obstet Gynecol. 2014;123:978–981.
27. Kilpatrick SJ, Berg C, Bernstein P, et al. Standardized severe maternal morbidity review: rationale and process. Obstet Gynecol. 2014;124:361–366.
28. Severe Maternal Morbidity: Clarification of the New Joint Commission Sentinel Event Policy. 2016. Available at: http://www.acog.org/About-ACOG/News-Room/Statements/2015/Severe-Maternal-Morbidity-Clarification-of-the-New-Joint-Commission-Sentinel-Event-Policy. Accessed June 15, 2016.
29. Centers for Disease Control and Prevention, Classification of Diseases – International Classification of Diseases, (ICD-10-CM/PCS) Transition – Background. 2016. Available at: http://www.cdc.gov/nchs/icd/icd10cm_pcs_background.htm. Accessed June 15, 2016.
30. Hsia DC, Krushat WM, Fagan AB, et al. Accuracy of diagnostic coding for Medicare patients under the prospective-payment system. N Engl J Med. 1988;318:352–355.
31. Berthelsen CL. Evaluation of coding data quality of the HCUP National Inpatient Sample. Top Health Inf Manage. 2000;21:10–23.
32. Lain SJ, Hadfield RM, Raynes-Greenow CH, et al. Quality of data in perinatal population health databases: a systematic review. Med Care. 2012;50:e7–e20.
33. Romano PS, Chan BK, Schembri ME, et al. Can administrative data be used to compare postoperative complication rates across hospitals? Med Care. 2002;40:856–867.

Supplemental Digital Content

Back to Top | Article Outline
© 2016 International Anesthesia Research Society