Secondary Logo

Validation of a Formula That Calculates the Estimated Risk of Respiratory Distress Syndrome

Melanson, Stacy E. F. MD, PhD1; Berg, Anders MD, PhD1; Jarolim, Petr MD, PhD1; Tanasijevic, Milenko J. MD, MBA1; McElrath, Thomas F. MD, PhD2

doi: 10.1097/01.AOG.0000245785.70216.8a
Original Research
Free

OBJECTIVE: Several groups, including ours, have developed probabilistic models that incorporate both the surfactant-to-albumin ratio (TDx-FLM II) and gestational age to more accurately predict the risk of neonatal respiratory distress syndrome (RDS) and eliminate the current categorical “immature”/“indeterminate”/“mature” interpretation. We validate our model using a separate data set, with the goal of providing the clinician with a risk score.

METHODS: The medical records of all women who had TDx-FLM II testing performed at Brigham and Women's Hospital between January 1, 2003, and December 31, 2005, were reviewed to gather a population upon which to validate our previous logistic regression model. Receiver operating characteristic curve and Hosmer-Lemeshow analysis was conducted to determine the performance of our model and another model in this new population.

RESULTS: A total of 233 mother-neonate pairs (21 RDS, 212 non-RDS) met criteria for analysis. The receiver operating characteristic analysis illustrated that our previous formula was a strong predictor of the risk of RDS with an area under the curve of 0.902 (95% confidence interval 0.849–0.955). In addition, using the Hosmer-Lemeshow analysis, our formula produced an excellent overall fit (P=.95), whereas another published model was a poor fit to our data (P=.002).

CONCLUSION: Our previously derived logistic regression model formula incorporating TDx-FLM II results and gestational age to predict risk of neonatal respiratory distress syndrome was robust and stable over time in an independent data set. The results suggest that the equation can be implemented clinically to assist physicians and patients and used by other institutions after their own internal validation.

LEVEL OF EVIDENCE: III

Our formula for prediction of neonatal respiratory distress is validated using independent data, and the findings allow for clinical implementation at our institution.

From the 1Division of Clinical Laboratories, Department of Pathology, and the 2Division of Maternal-Fetal Medicine, Department of Obstetrics, Gynecology and Reproductive Biology, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts.

Corresponding author: Thomas F. McElrath, Division of Maternal–Fetal Medicine, Department of Obstetrics, Gynecology and Reproductive Biology, Brigham and Women's Hospital, 75 Francis Street, Boston, MA 02115; e-mail: tmcelrath@partners.org and semelanson@partners.org.

Newborn infants, especially those born before term, are at risk of developing respiratory distress syndrome (RDS) because of insufficient production of surfactant by the neonatal lung. Due to the relatively high prevalence (1% of all live births) and high mortality associated with RDS, obstetricians considering delivery of an infant before term often use biochemical tests to help determine whether or not to proceed with delivery.1–4 These tests measure pulmonary surfactant or phospholipids or both in amniotic fluid.1

One of the most widely used tests for fetal lung maturity is the TDx-FLM II surfactant-to-albumin assay (TDx-FLM II) (Abbott Laboratories, Abbott Park, IL). The TDx-FLM II assay is rapid, easy to perform, and highly reproducible. Several studies have examined the diagnostic performance of the TDx-FLM II method to assess fetal lung maturity.2,3,5,6 Even though the assay provides quantitative results, the manufacturer's guidelines dictate that the results be interpreted as “mature,” “indeterminate,” or “immature.” Although the probability of RDS decreases precipitously as the fetus approaches term, conventional interpretation of the TDx-FLM II applies the same cutoff values to all stages of fetal maturation and gestational age.

Several groups have developed probabilistic models that incorporate both TDx-FLM II results and gestational age to more accurately predict the gestational age–specific risk of neonatal respiratory distress and to aid clinical decision making. The first model was published by Tanasijevic et al.7 This study used the original TDx-FLM assay, which was discontinued in 1995. McElrath et al6 updated this work and published a similar model to estimate the probability of RDS based on the current TDx-FLM II assay. Subsequently, similar models have been published by other groups using different patient data sets.3,8

A comparison of the available models reveals widely variant probabilities of RDS based on similar gestational age and TDx-FLM II values. Using the Karcher model,8 we obtain higher risks of RDS at higher TDx-FLM II values, which does not make sense physiologically. We assumed there was a typographic error in the formula and did not use their formula for comparison in this study. Based on a PubMed literature search using the terms “fetal lung maturity” and “gestational age” with no limitations, none of these formulas have been externally validated using separate data sets drawn from the same population. Therefore, the reproducibility and predictive power of these models remains unexplored. In this study, we analyze all patients over the past 3 years who had TDx-FLM II testing performed at Brigham and Women's Hospital in an attempt to validate our previously derived logistic regression equation. The performance of our model is also compared with another published formula using this more recent data set.

Back to Top | Article Outline

MATERIALS AND METHODS

Approval for this study was received from the institutional Human Research Committee at Brigham and Women's Hospital. The laboratory information system was used to obtain a list of all women who had TDx-FLM II testing performed at Brigham and Women's Hospital between January 1, 2003, and December 31, 2005. Women were excluded if they delivered more than 72 hours after the fetal lung maturity estimation. Only live, singleton, nonanomalous births were included.

The maternal medical records were reviewed to obtain the date of delivery, the results of TDx-FLM II, the gestational age, diabetic status, and any other relevant pregnancy details. Data on maternal race, tobacco use, hypertension, and antenatal steroids were not collected because we illustrated previously that these characteristics did not predict the risk of RDS.6 Gestational age was assigned by the following criteria, listed in order of priority: in vitro fertilization or intrauterine insemination dating, last menstrual period (LMP) confirmed by first-trimester ultrasonography, LMP confirmed by second-trimester ultrasonography, or first-trimester ultrasonography alone. Only cases with reliable dating were included.

Neonatal outcomes were assessed through medical record review. Similarly to a study published previously by our group,6 RDS was defined by the presence of two or more of the following criteria: 1) evidence of respiratory compromise shortly after delivery (tachypnea, retractions, and/or nasal flaring) and persistent oxygen requirement for more than 24 hours, 2) administration of exogenous pulmonary surfactant, and/or 3) radiographic evidence of hyaline membrane disease. Care was taken to exclude cases of newborn pneumonia, apnea of prematurity, and transient tachypnea of the newborn from the affected cases.

Fetal lung maturity testing was performed using the TDx-FLM II surfactant-to-albumin assay. Both vaginal pool and amniocentesis specimens were analyzed jointly because TDx-FLM II results are not affected by fluid type.6 The relative concentrations of surfactant and albumin in amniotic fluid were determined by fluorescence polarization. Quantitative results were reported as milligrams of surfactant per 1 g of albumin. According to the manufacturer, cutoffs of less than 40 mg/g, 40–54 mg/g, and 55 mg/g or more define “immature,” “indeterminate,” and “mature,” respectively.

The logistic regression model our group published previously based upon patient assay data from 1998–2000 was retested with a more recent data set. The previous model is described by the following formula: probability of neonatal RDS=(eβ0+β1×[gestational age in weeks]+β2×[FLM S/A II in mg/g])/(1+eβ0+β1×[gestational age in weeks]+β2×[FLM S/A II in mg/g]) where β0=9.3, β1=−0.26, β2=−0.06). Receiver operating characteristic curve analysis was performed using Analyze-It 1.71 software (Analyze-It, Ltd, Leeds, UK).9 Receiver operating characteristic analysis determines the sensitivity and specificity of the model in detecting RDS when one varies the estimated risk threshold for defining the presence of RDS. The area under the curve (AUC) measures the diagnostic accuracy. Hosmer-Lemeshow analysis was used to measure the overall fit of our model to the newer independent data set.10 A perfect fit demonstrates a P value of 1.00, whereas P<.05 indicates poor fit to the data. The same analysis was used to compare our model with another published logistic regression model, where β0=15.69, β1=−0.3661, β2=−0.1122.3 A χ2 test was performed to compare the prevalence of RDS in the previous studies with our current population.

Back to Top | Article Outline

RESULTS

A total of 569 cases were reviewed. Forty-five percent were excluded because the window was greater than 72 hours between the FLM II result and the delivery, 10% were excluded for multiple gestations, and 4% were excluded for fetal anomalies. A total of 233 mother-neonate pairs (21 RDS, 212 non-RDS) met criteria for analysis. The prevalence of RDS was 9.0%. The gestational ages at delivery ranged from 29.3 weeks to 40.3 weeks, with an average of 35.8 weeks (95% confidence interval [CI] 35.6–36.0), and the surfactant-to-albumin ratio up to 72 hours before delivery ranged from 6 mg/g to 155 mg/g, with an average of 53.9 mg/g (95% CI 50.9–56.9). Using a χ2 analysis, we found no significant difference between the number of diabetics in the non-RDS and RDS groups (P=.065).

Our previous model, which incorporates gestational age and TDx-FLM results, was used to predict the probability of RDS. The mean estimated risk of RDS was 8.9% (95% CI 7.3–10.5). Table 1 presents the 21 cases of RDS and the calculated risk of RDS. All the observed cases of RDS had an estimated risk of 4% or greater, with the majority (66.7%) having a risk above 15%. Case 7 would have been misclassified as “mature” using the manufacturers guidelines for interpretation of TDx-FLM II results (ie, would have been “false negative”). Our model provided a probability of RDS of over 5% in this case.

Table 1

Table 1

The receiver operating characteristic curve for our previous formula is illustrated in Figure 1. Our previous formula has an AUC of 0.902 (95% CI 0.849–0.955). Different risk tolerances provide various levels of sensitivity, specificity, positive predictive value, and negative predictive value (Table 2). Using a 5% risk of RDS as a threshold for defining the presence or absence of RDS, our formula demonstrates a sensitivity of 95.2%, a specificity of 67.9%, a positive predictive value of 22.7%, and a negative predictive value of 99.3%.

Fig

Fig

Table 2

Table 2

Although the assessment of our logistic regression model using positive predictive value and negative predictive value is clinically informative, this analysis does not test whether the model is a good overall fit of the observed data. For a probabilistic model that predicts a binary outcome, the best analytical tool is the Hosmer-Lemeshow goodness-of-fit analysis. Using the Hosmer-Lemeshow analysis, our formula produced an excellent overall fit (P=.95). The strong agreement between observed and expected cases for these groups can also be seen in Figure 2.

Fig

Fig

Using logistic regression the Parvin study3 derived an equation analogous to that in our model to estimate the risk of RDS. This study used comparable inclusion criteria and a cohort (n=509) from nine different clinical sites to derive their model. Their data set produced different coefficients for the equation compared with our derived formula. As shown by the coefficients found in Materials and Methods, in comparison with our model, the Parvin formula would be expected to predict higher risks of RDS overall and give greater weight to FLM values compared with gestational age. In fact, when applied to our new data, the Parvin model produced an estimated risk of 15.7% (95% CI 12.4–19.0) for the entire cohort. This average risk was a relative overestimate when compared with our model's estimate (8.9%) and the actual prevalence observed (9.0%). When the Parvin equation was applied to our new data set and tested in a similar fashion by the Hosmer-Lemeshow analysis, it produced a P value of .002, indicating a relatively poor fit to the new cohort (Fig. 3).

Fig

Fig

The prevalence of RDS in our previous paper, in the Parvin paper, and in the current population was 6.7%, 11.2%, and 9.0%, respectively. A χ2 test shows no statistically significant difference in the prevalence between the Parvin cohort and our current cohort (P=.368) or between our previous cohort and our current cohort (P=.295). There is a significant difference between the prevalence in the Parvin population and our previous population (P=.019).

Back to Top | Article Outline

DISCUSSION

Respiratory distress syndrome is the major cause of death in the newborn infant and plays a significant role in the timing of delivery in noncritical patients.2–4 The risk of RDS must be weighed against the maternal and fetal risks associated with continuing the pregnancy. Respiratory distress syndrome risk decreases with increasing gestational age as the pulmonary system matures and produces surfactant to decrease the risk of alveolar collapse. Although gestational age is a powerful predictor of the risk of RDS, amniotic surfactant concentrations are also a good predictor of pulmonary maturity, and obstetricians frequently order additional biochemical tests to quantify the amount of phospholipids in amniotic fluid. According to recommendations published by the American College of Obstetricians and Gynecologists (ACOG), amniocentesis and fetal lung maturity testing should be performed whenever elective delivery is being considered before 39 weeks of gestational age.11 However, lung immaturity before 33 weeks and lung maturity after 39 weeks can be safely assumed, and amniocentesis should not be performed at these extremes of gestational age because of the risk of complications.

The translation of TDx-FLM II amniotic surfactant concentration assay results as recommended by the manufacturer into “mature,” “immature,” and “indeterminate” categories provides an oversimplified interpretation. Although the manufacturer distributes the predictive values of these assay results within their product insert (as determined by one specific prospective cohort study), categorizing results propagates the misconception that “immature” results are highly predictive that RDS will occur and that “mature” results rule out the possibility of RDS.

To improve upon this method of interpretation, three groups have derived logistic regression models using fetal lung maturity assay results combined with gestational age to assign individualized estimates of RDS risk.3,6,7,12,13 A risk estimate generated by a continuous model provides the patient and clinician a more precise estimate of their risk than a categorical model does and would allow clinicians to make more confident decisions about intentional preterm delivery. Despite the advantages, adopting this method of interpretation in clinical practice has been slow. In addition, the estimated risks differ using the same gestational age and TDx-FLM II result, and external validation of these equations is lacking.

The receiver operating characteristic analysis reveals that our logistic regression model is a powerful predictor of RDS. Traditionally a 5% risk threshold has been used to define the presence or absence of RDS. All but one case of RDS using our model provided a risk greater than 5%. Increasing the risk threshold improves specificity and decreases the number of false positives. However, optimizing sensitivity and negative predictive value will avoid delivery of infants with RDS. Although choosing an acceptable risk cutpoint may simplify the decision process, it contradicts the original purpose of our model, which is to provide a continuum of risk instead of a binary, yes or no, answer. In one case of RDS, the TDx-FLM II result was “mature,” while the risk of RDS was 5.8%, illustrating that the model, by its probabilistic nature, will not misclassify patients.

As illustrated by the Hosmer-Lemeshow analysis, our formula had an excellent overall fit to the new data. This indicates that our equation has remained stable and robust over time at our institution, is applicable to our patient population, and depending upon local prevalence may be generalizable to the populations cared for by other institutions. Respiratory distress syndrome, as stated in Materials and Methods, was precisely defined in both our previous and current patient populations, which played a critical role in the accuracy of the formula. The absolute probability of RDS is also dependent on the prevalence of RDS in the population. In our previous study, the prevalence of RDS was 6.7%, which is not statistically different from the prevalence of RDS (9.0%) in the current study. However, despite a similar prevalence of RDS, the Parvin equation was a poor fit to our data, suggesting that this equation would not be suitable to use clinically with our patients. Although our validation of the formula illustrates its ability to provide an accurate risk estimate, other institutions with enough resources may wish to examine the performance of the formula in their patient population as they would with other laboratory tests. In this case, the definition and prevalence of RDS should be carefully considered.

If the diagnostic accuracy of the logistic model, according to the receiver operating characteristic analysis, is compared with that of FLM alone, similar results are obtained (ie, AUC=0.902 for the model and AUC=0.905 for fetal lung maturity alone). These results are consistent with previous studies examining the accuracy of TDx-FLM II4,12 or evaluating regression models in relation to TDx-FLM II alone.3 Although the receiver operating characteristic analysis suggests that the model is comparable with fetal lung maturity alone, the model offers several distinct advantages. Instead of providing just the test result, the model will provide obstetricians and patients with the probability of RDS. The model also offers consistent sensitivities and specificities across the gestational age range, as opposed to TDx-FLM II alone whose sensitivities and specificities vary according to gestational age. In addition, use of the formula will eliminate the “indeterminate” zone between 40 and 54 mg/g, which clinicians cannot currently interpret with confidence. Although elimination of the “indeterminate” zone has been proposed previously,4 most laboratories are reluctant to deviate from the manufacturer's recommendations. Almost one third of our TDx-FLM II results (28.8%) fell into the “indeterminate” zone. In addition, three cases of RDS (14.3%) occurred in this zone. Implementation of the formula and elimination of the “indeterminate” zone would clarify decision making in a significant portion of patients.

We validated our logistic regression model by using a separate, more recently delivered population of neonates from our hospital. The results suggest that the equation can be used clinically in our patient population and in other hospitals that have examined its performance in their population. Using this formula, clinicians can be provided with a quantitative risk of RDS for patient-specific TDx-FLM II results and gestational age combinations. Due to our findings, we intend to supplement our quantitative results with a risk score that clinicians and patients can use. The equation could be implemented in our laboratory information system and reported with each result, or it could be made available electronically for clinicians to calculate the risk on their own.

Back to Top | Article Outline

REFERENCES

1. Ashwood ER. Standards of laboratory practice: evaluation of fetal lung maturity. Clin Chem 1997;43:211–4.
2. Bildirici I, Moga CN, Gronowski AM, Sadovsky Y. The mean weekly increment of amniotic fluid TDx-FLM II ratio is constant during the latter part of pregnancy. Am J Obstet Gynecol 2005;193:1685–90.
3. Parvin CA, Kaplan LA, Chapman JF, McManamon TG, Gronowski AM. Predicting respiratory distress syndrome using gestational age and fetal lung maturity by fluorescent polarization [published erratum appears in Am J Obstet Gynecol 2005;192:1354]. Am J Obstet Gynecol 2005;192:199–207.
4. Fantz CR, Powell C, Karon B, Parvin CA, Hankins K, Dayal M, et al. Assessment of the diagnostic accuracy of the TDx-FLM II to predict fetal lung maturity. Clin Chem 2002;48:761–5.
5. Kesselman EJ, Figueroa R, Garry D, Maulik D. The usefulness of the TDx/TDxFLx fetal lung maturity II assay in the initial evaluation of fetal lung maturity. Am J Obstet Gynecol 2003;188:1220–2.
6. McElrath TF, Colon I, Hecht J, Tanasijevic MJ, Norwitz ER. Neonatal respiratory distress syndrome as a function of gestational age and an assay for surfactant-to-albumin ratio. Obstet Gynecol 2004;103:463–8.
7. Tanasijevic MJ, Wybenga DR, Richardson D, Greene MF, Lopez R, Winkelman JW. A predictive model for fetal lung maturity employing gestational age and test results. Am J Clin Pathol 1994;102:788–93.
8. Karcher R, Sykes E, Batton D, Uddin Z, Ross G, Hockman E, et al. Gestational age-specific predicted risk of neonatal respiratory distress syndrome using lamellar body count and surfactant-to-albumin ratio in amniotic fluid. Am J Obstet Gynecol 2005;193:1680–4.
9. Shultz EK, Aliferis C, Aronsky D. Clinical evaluation of methods. In: Burtis CA, Ashwood ER, Bruns DE, editors. Tietz textbook of clinical chemistry and molecular diagnostics. 4th ed. St. Louis (MO): Elsevier Saunders; 2006. p. 409–24.
10. Hosmer DW, Lemeshow S. Applied logistic regression. 2nd ed. New York (NY): John Wiley; 2000.
11. American College of Obstetricians and Gynecologists. Assessment of fetal lung maturity. ACOG Educational Bulletin 230. Washington, DC: ACOG; 1996.
12. Kaplan LA, Chapman JF, Bock JL, Santa Maria E, Clejan S, Huddleston DJ, et al. Prediction of respiratory distress syndrome using the Abbott FLM-II amniotic fluid assay. Clin Chim Acta 2002;326:61–8.
13. Stubblefield PG. Using the TDx-FLM assay and gestational age together for more accurate prediction of risk for neonatal respiratory distress syndrome. Am J Obstet Gynecol 2002;187:1429–30.
© 2006 The American College of Obstetricians and Gynecologists