OBJECTIVE: Nuchal translucency medians are used to convert nuchal translucency measurements to multiple of the median values for use in risk assessment. There has been much debate about the use of practitioner–specific medians instead of population-based medians to correct for practitioner bias. This study examined data from the California Prenatal Screening Program to evaluate the effect of individual medians on screening performance.
METHODS: For this retrospective study, we identified first-trimester specimens in the California Prenatal Screening Program database for which screening results were calculated using a practitioner–specific median (n=55,286) and recalculated risk assessment using a population-based median. We looked at positive rates for Down syndrome and Trisomy 18 as well as overall screening results. We also looked at the recalculated risks for cases in which there has been a positive diagnosis of a chromosomal abnormality to determine if cases would have been missed if practitioner–specific medians had not been used.
RESULTS: Nuchal translucency multiples of the median would be lower for the majority of nuchal translucency practitioners if a population-based median were used instead of practitioner–specific medians. Outcomes data from the California Prenatal Screening Program's database indicate that 14 positively diagnosed cases would have been missed by the screening program if population-based medians had been used.
CONCLUSION: The use of practitioner–specific medians corrects for practitioner bias in measuring nuchal translucency. Practitioner–specific medians improve screening performance and help meet detection rate goals.
LEVEL OF EVIDENCE: II
Data From the California Prenatal Screening Program demonstrate that use of practitioner&#x2013;specific medians improves screening performance.
From the Genetic Disease Screening Program, California Department of Public Health, Richmond, the Maternal Fetal Medicine Foundation (Nuchal Translucency Quality Review Program), Center For Fetal Medicine and Women's Ultrasound, David Geffen School of Medicine at the University of California–Los Angeles, Los Angeles, and the Fetal Medicine Foundation, Studio City, California.
Funded as program evaluation by the California Department of Public Health.
Corresponding author: Nerissa Wu, PhD, California Department of Public Health, Genetic Disease Screening Program, 850 Marina Bay Parkway, Room F-175, Richmond, CA 94804; e-mail: Nerissa.Wu@cdph.ca.gov.
Financial Disclosure The authors did not report any potential conflicts of interest.
Prenatal screening detection rates have improved with the incorporation of additional screening analytes. Recently, the addition of nuchal translucency measurements into the screening algorithm has further improved detection of Down syndrome and Trisomy 18.1,2
Nuchal translucency data have a profound effect on the calculated risk.3 Accurate evaluation of nuchal translucency is therefore critical to screening performance and to providing quality patient care. Recent publications have highlighted the issue of quality assurance and nuchal translucency measurements. Studies have shown that there are discrepancies in technique3 and that there is a tendency, particularly among less experienced ultrasonographers, to undermeasure the nuchal translucency.4
The accuracy of nuchal translucency measurement depends on a number of factors including imaging conditions fetal position, machine settings, and correct positioning of the calipers. These factors can only be addressed through proper instruction and supervision. However, among properly trained and credentialed practitioners, consistently higher or lower nuchal translucency measurement is akin to instrument bias in the laboratory and has been addressed by some with the use of practitioner–specific medians to adjust for the differences.2
Nuchal translucency measurements are typically converted to a multiple of the median or Δ nuchal translucency based on a comparison of the expected nuchal translucency measurement at a particular gestational age and the actual measurement. If an nuchal translucency practitioner consistently undermeasures nuchal translucency, the multiple of the median, or Δ nuchal translucency calculated based on comparison with the population-based median will always be relatively low, resulting in a lower estimation of risk (Table 1). The results of a woman's risk assessment, and therefore the accuracy of prenatal screening for an individual woman, may therefore depend on which nuchal translucency practitioner conducts an examination. Although many laboratories use a population-based median, the California Prenatal Screening Program recognizes that there is variability between nuchal translucency practitioners and uses practitioner–specific medians with the goal of providing uniform detection rates to all participants, regardless of which nuchal translucency practitioner conducts the examination. Indeed, the California Prenatal Screening Program was designed to have a nuchal translucency median multiple of the median of 1.0; deviation from this median multiple of the median therefore negatively affects screening performance.
This article assesses the California Prenatal Screening Program's experiences with practitioner–specific medians and compares actual screening results, based on practitioner–specific medians, with risks recalculated using a population-based median.
MATERIALS AND METHODS
The California Prenatal Screening Program has been in existence since 1986. In April 2009, the program expanded to incorporate first-trimester analytes and nuchal translucency examination data to provide first-trimester combined screening for Down syndrome and Trisomy 18. The California Prenatal Screening Program accepts nuchal translucency data only from nuchal translucency practitioners who are actively credentialed by the Nuchal Translucency Quality Review Program or Fetal Medicine Foundation and registered with the California Prenatal Screening Program. Both credentialing agencies require completion of a training class, a passing score on a written examination, and acceptance of submitted scans by a review panel to become credentialed. In addition, the California Prenatal Screening Program works with both credentialing agencies to monitor nuchal translucency data and maintain quality assurance for all accredited practitioners.
Nuchal translucency data are used for risk assessment if the crown–rump length of the fetus(es) is between 45 and 84 mm at the time of the nuchal translucency examination. Converted to gestational age based on the algorithm published by Hadlock et al,5 this corresponds with a gestational age window of 11 weeks 2 days to 14 weeks 2 days. Using a log-linear equation for which the log10(NT) is the dependent variable and crown–rump length is the independent variable to calculate the median nuchal translucency value at a given crown–rump length, the nuchal translucency measurement is converted to a multiple of the median for use in the risk assessment algorithm.
The California Prenatal Screening Program maintains a population-based California nuchal translucency median but also calculates and uses practitioner–specific medians when possible. The California median is calculated annually based on data submitted to the California Prenatal Screening Program by all practitioners who have submitted data from at least 75 examinations in the previous year. Although the Nuchal Translucency Quality Review Program and the Fetal Medicine Foundation both use quadratic medians, the California Prenatal Screening Program, based on data from the Serum, Urine and Ultrasound Screening Study, uses log-linear medians. Figure 1 compares the population-based medians used by California, Fetal Medicine Foundation, and Nuchal Translucency Quality Review Program as well as practitioner–specific medians. Each small point on the figure represents the median equation of an individual practitioner, whereas the larger points represent the population-based median equations used by the Fetal Medicine Foundation, Nuchal Translucency Quality Review Program, and the California Prenatal Screening Program. As shown, the California median falls around the midpoint of the Fetal Medicine Foundation and Nuchal Translucency Quality Review Program medians over the gestational age range. Figure 1 also illustrates the great variability in practitioner–specific medians.
When nuchal translucency practitioners first register in the California Prenatal Screening Program, data they submit are converted to nuchal translucency multiples of the median using the California median. As individual practitioners accumulate data, practitioner–specific medians are calculated and used to determine nuchal translucency multiples of the median for subsequently submitted data. A minimum of 75 examinations per practitioner is required before a practitioner–specific median is calculated. The calculation of a median is conducted by grouping the data into seven bins, each corresponding to approximately 6-mm crown–rump length. The log median nuchal translucency and mean crown–rump length are calculated for each bin, and the coefficients are produced by weighted regression. This smoothing protocol produces a stable estimate when each bin has sufficient data, which leads to the overall volume requirement. Practitioner–specific medians are evaluated each quarter. If the calculated median nuchal translucency at either end of the gestational age range differs from the previously calculated median nuchal translucencies by greater than 10%, the practitioner's median equation is updated, creating a new version. An exception to this protocol is made if an individual practitioner's data indicate an excessively low slope (less than 11% increase in nuchal translucency per week); in these situations, the California Prenatal Screening Program continues to use the California median for risk calculations.
To compare the effect of practitioner–specific medians on screening performance, we used data submitted to the California Prenatal Screening Program between April 2009 and July 2011 that was interpreted using the penultimate practitioner–specific median version for each practitioner (n=71,524), approximately 24% of data submitted within this time period. For example, if a practitioner is currently on the fourth practitioner–specific version, we included data that were interpreted using this individual's third individual median version. For practitioners with only one practitioner–specific median, the penultimate median version was the California median. Data from practitioners who currently are using the California median were not included. Only singleton pregnancies were included. Use of California Prenatal Screening Program data for the purpose of this study was exempt from institutional review board review as per communication with the California Health and Human Services Agency Committee for the Protection of Human Subjects.
We calculated the median nuchal translucency multiple of the median and the standard deviation of log10(NT MoM) based on the interval between the 5th and 95th percentiles for each nuchal translucency practitioner. For the risk results calculated using a practitioner–specific median (n=55,286), we also recalculated the nuchal translucency multiples of the median using the current California median and input the new multiples of the median into the first-trimester risk calculation keeping all other factors (eg, analytical values, maternal age, race) constant.6 In accordance with California Prenatal Screening Program policy, cases were defined as screen-positive for Down syndrome with a risk cutoff of 1 in 100 and screen-positive for Trisomy 18 with a risk cutoff of 1 in 50.6 We then compared the risks calculated with the practitioner–specific medians with the risks calculated using the California median.
For data submitted by practitioners using a practitioner–specific median (n=55,286), the median nuchal translucency multiple of the median was 1.01 and the overall standard deviation of log10(NT MoM) was 0.095 (Table 2). Of the 339 practitioners in this category, nine (3%) had a median multiple of the median less than 0.9, and 37 (11%) had a median multiple of the median greater than 1.1. No practitioners in this group had a standard deviation above the acceptable limit (greater than 0.15).7 Data submitted by 126 practitioners were interpreted using the California median (n=16,238). The practitioners in the California median group had an overall median nuchal translucency multiple of the median of 0.92 and an overall standard deviation of 0.103. Of the 126 practitioners in this group, 59 (47%) had a median multiple of the median less than 0.9, and one practitioner had a standard deviation greater than 0.15. None had a median multiple of the median greater than 1.1. The distribution of median multiple of the median and standard deviation for both groups of nuchal translucency practitioners is shown in Figures 2 and 3. Based on the Z-test, the difference in nuchal translucency multiple of the median between the two groups of nuchal translucency practitioners was statistically significant (P≪.001). The variance ratio test (F-test) demonstrated that the difference in standard deviation between these two groups was statistically significant (P<.01). Within the California median group, there were 11 practitioners who submitted fewer than 75 examinations. If the low-volume practitioners are excluded, there are no practitioners in this group with a standard deviation greater than 0.15; the percentage of practitioners with a median multiple of the median less than 0.9 does not change appreciably (46%).
For the 55,278 pregnancies in our database, which were screened using a practitioner–specific median (eight pregnancies were eliminated because of missing data), we recalculated risks using the California median. Screen-positive results for Down syndrome and Trisomy 18 as well as for overall first-trimester screening are shown in Table 3. The screen-positive rate for Down syndrome using practitioner–specific medians was 3.1%. The positive rate with recalculated nuchal translucency multiples of the median was 2.4%. The screen-positive rate for Trisomy 18 was reduced from 0.22% using practitioner–specific medians to 0.20% using the California median. Overall the screen-positive rate (positive for either disease) fell from 3.3% to 2.6%. Given an annual rate of approximately 160,000 first-trimester specimens, this translates to 1,120 fewer screen-positive cases, or 1,120 fewer women who would be offered first-trimester follow-up services through the state program.
In addition to affecting the screening result (screen-positive compared with screen-negative), the use of the California median would also change the numeric risk result. Women whose screening result would be screen-positive regardless of which nuchal translucency medians were used might think differently about follow-up care if her numeric risk result were 1 in 50 instead of 1 in 100; screen-negative women with a numeric result of 1 in 110 might view the screening results differently if her numeric risk were 1 in 250. This difference is difficult to quantify, but recent studies have shown that the risk number does affect patients' choices regarding follow-up diagnostic procedures.8,9
Overall, the median nuchal translucency multiple of the median was considerably lower when the California median was used. Recalculated using the California median, the overall median nuchal translucency multiple of the median was 0.91, and 148 (44%) of practitioners had a median multiple of the median below 0.90. Thirty-eight (11%) had a median multiple of the median below 0.80. The distribution of median multiple of the median by nuchal translucency practitioner, calculated both by practitioner–specific median and the California median, is shown in Figure 4. The effect on the positive rate by practitioner can be seen in Figure 5.
To further illustrate the clinical implications of using a population-based median, we looked at actual cases in the State Registry for which a diagnosis of a chromosomal abnormality was made and recalculated screening results based on the California median (Fig. 6). These cases included both screen-positive and screen-negative pregnancies.
In our data set of 55,278 pregnancies, 1,815 (3.3%) were screen-positive. Seven hundred seventy-seven screen-positive women obtained diagnostic services based on the first-trimester result. (This does not include women who decided to obtain sequential integrated screening and obtained follow-up services based on the integrated result.) In 138 of the 777 pregnancies (18%), a chromosomal abnormality was confirmed through diagnostic testing. We recalculated risk for these 138 cases using the California median. In 90% of the cases, the screening result did not change, but in 14 cases, risk results were sufficiently altered such that the first-trimester screening result was screen-negative (Table 4). If the California median had been used for the interpretation of these 14 cases, 14 confirmed cases (including seven Down syndrome, one Triploidy, one Triple X, one Turner syndrome, and one Klinefelter syndrome case) would have been missed by our screening program.
For practitioners who typically overmeasure, the individual median results in a lower multiple of the median than the population-based median. To determine if cases were missed as a result of our use of individual medians, we looked at screen-negative cases for which a chromosomal abnormality was diagnosed and recalculated the risk using the California median. From our database of 55,278 first-trimester pregnancies, there were 23 screen-negative cases that appear in the State Registry with a chromosomal abnormality. Among the 23 screen-negative cases, recalculation of risk using the California median did not result in any altered screening results.
The data demonstrate that the use of practitioner–specific medians improves first-trimester screening performance. The comparison between risks calculated using practitioner–specific or a population-based median included only nuchal translucency practitioners with at least two median versions. This excluded new or low-volume practitioners as well as practitioners whose median slope was calculated to be excessively low. The inclusion of only a subset of nuchal translucency practitioners biased the study toward a null finding. Less experienced or low-volume practitioners are more likely to undermeasure. Therefore, the effect of using a population-based median to which data do not necessarily adhere would be expected to be greater in the overall screening population than for this study.
The California Prenatal Screening Program was designed based on a median nuchal translucency multiple of the median of 1.0. Use of the California Median to calculate multiples of the median from undermeasured nuchal translucency values therefore negatively affects screening performance; using practitioner–specific medians helps the California Prenatal Screening Program meet projected detection rates. The use of practitioner–specific medians results in higher detection rates overall and on an individual basis. Recent publications4,10 have presented concerns regarding the use of practitioner–specific medians. It has been suggested that the use of practitioner–specific medians would cover up sloppy performance and eliminate practitioner incentive to improve.10 However, we are not suggesting that the use of practitioner–specific medians precludes rigorous review of practitioner data. Indeed, the calculation and review of practitioner–specific medians is only one of the statistics used by the California Prenatal Screening Program to flag practitioners of concern for the credentialing agencies. The California Prenatal Screening Program also calculates standard deviation and median nuchal translucency multiple of the median for all practitioners, flagging practitioners for review if statistics fall out of range. Both credentialing agencies review scans when practitioners are credentialed and if practitioners are flagged for quality concerns, and Fetal Medicine Foundation requires submittal of scans annually for reaccreditation. Practitioners credentialed by Nuchal Translucency Quality Review Program have the option of sending images annually as well and some have opted to do so. Indeed, the use of the California median would be acceptable if all credentialed practitioners measured within a prescribed variance. However, studies have suggested that uniform adherence to one median line may be difficult to achieve.11,12 Perhaps more stringent and frequent oversight and review of nuchal translucency practitioners than currently exists would bring nuchal translucency practitioners into closer conformity, but the potential negative consequences of increased oversight on the nuchal translucency practitioners and on access to nuchal translucency examinations must be considered. We have presented here an alternative strategy through the use of practitioner–specific medians and have shown evidence suggesting this strategy is in keeping with California Prenatal Screening Program goals of high-quality screening for all women.
It also has been suggested that the use of center-specific medians, or medians based on the data submitted by multiple practitioners from one site, would shift the curve of nuchal translucency data away from the population curve, altering the rate of screen-positive results.4 As long as measures of deviation from the median such as the multiple of the median or Δ nuchal translucency are used in risk calculations, this should not be the case.
We must consider nuchal translucency quality assurance in the context of screening performance. Although data from the Biochemistry, Ultrasound, Nuchal translucency study suggest that nuchal translucency practitioner performance improves with accumulated experience and feedback,3 we are obligated to provide current patients with the best screening performance possible. If performance does improve over time, the quarterly review of data will identify these improvements, and practitioner–specific medians will be adjusted accordingly.
We must also consider the issue of patient equity. Because nuchal translucency practitioners with the least experience are most prone to undermeasure, use of the population-based median will most greatly affect the detection rates for these practitioners. Experienced practitioners are likely to be concentrated in academic centers and urban areas. The California Prenatal Screening Program's goal of equitable access to prenatal screening statewide mandates that we work to provide access as well as equitable quality to all program participants. Use of practitioner–specific medians helps the California Prenatal Screening Program meet these goals.
As with any quality assurance measure, there is a question of implementation. How should the California Prenatal Screening Program deal with practitioners with excessively low slope? The California Prenatal Screening Program currently interprets data submitted by these practitioners using the California median. This is not ideal because detection rates for this group are likely to be lower than California Prenatal Screening Program goals nor is it satisfactory to continue accepting data, which indicates little or no relationship between crown–rump length and nuchal translucency measurements. Ultimately we need to determine why the nuchal translucency practitioner data are showing low slope; is the practitioner over- or undermeasuring on either side of the gestational age range? Is the practitioner measuring with low precision, resulting in almost random measurements? There are a number of technical issues that may lead to inaccurate measurements or bias. Only periodic review of submitted scans or supervision of actual or recorded examinations by qualified examiners can identify these issues.
The use of practitioner–specific medians cannot replace periodic review of technique; the California Prenatal Screening Program will continue to work with the credentialing agencies to monitor data and implement quality control measures. However, when the technique is being practiced in a standardized fashion, use of practitioner–specific medians, in conjunction with active credentialing and quality review, corrects for individual practitioner differences and provides more uniform compliance with screening detection rate goals.
1. Malone FD, Canick JA, Ball RH, Nyberg DA, Comstock CH, Bukowski R, et al.. First-trimester or second-trimester screening, or both, for Down's syndrome. N Engl J Med 2005;353:2001–11.
2. Wald NJ, Rodeck C, Hackshaw AK, Walters J, Chitty L, Mackinson AM. First and second trimester antenatal screening for Down's syndrome: the results of the Serum, Urine and Ultrasound Screening Study (SURUSS). J Med Screen 2003;10:56–104.
3. Snijders RJ, Thom EA, Zachary JM, Platt LD, Greene N, Jackson LG, et al.. First-trimester trisomy screening: nuchal translucency measurement training and quality assurance to correct and unify technique. Ultrasound Obstet Gynecol 2002;19:353–9.
4. Evans MI, Van Decruyes H, Nicolaides KH. Nuchal translucency measurements for first-trimester screening: the ‘price’ of inaccuracy. Fetal Diagn Ther 2007;22:401–4.
5. Hadlock FP, Shah YP, Kanon DJ, Math B, Lindsey JV. Fetal crown-rump length: reevaluation of relation to menstrual age (5–18 weeks) with high-resolution real time US. Radiology 1992;182:501–5.
7. Palomaki GE, Lee JE, Canick JA, McDowell GA, Donnenfeld AE; ACMG Laboratory Quality Assurance Committee. Technical standards and guidelines: prenatal screening for Down syndrome that includes first-trimester biochemistry and/or ultrasound measurements. Genet Med 2009;11:669–81.
8. Nicolaides KH, Chervenak FA, McCullough LB, Avgidou K, Papageorghiou A. Evidence-based obstetric ethics and informed decision-making by pregnant women about invasive diagnosis after first-trimester assessment of risk for trisomy 21. Am J Obstet Gynecol 2005;193:322–6.
9. Jaques AM, Collins VR, Muggli EE, Amor DJ, Francis I, Sheffield LJ, et al.. Uptake of prenatal diagnostic testing and the effectiveness of prenatal screening for Down syndrome. Prenat Diagn 2010;30:522–30.
10. Evans MI, Cuckle HS. Performance adjusted risks: a method to improve the quality of algorithm performance while allowing all to play. Prenat Diag 2011;31:797–801.
11. Canick J. Prenatal screening for trisomy 21: recent advances and guidelines. Clin Chem Lab Med 2011 [Epub ahead of print].
© 2012 The American College of Obstetricians and Gynecologists
12. Sahota DS, Chen M, Leung TY, Chan LW, Fung TY, Ting YH, et al.. Assessment of sonographer nuchal translucency measurement performance—central tendency and dispersion. J Matern Fetal Neonatal Med 2011;24:812–6.