Secondary Logo

Journal Logo

Assessment of Transvaginal Ultrasound Cervical Length Image Quality

Boelig, Rupsa C. MD; Feltovich, Helen MD; Spitz, Jean Lea MPH, RDMS; Toland, Gregory MS; Berghella, Vincenzo MD; Iams, Jay D. MD, on Behalf of the Perinatal Quality Foundation

doi: 10.1097/AOG.0000000000001820
Contents: Original Research
Free

OBJECTIVE: To use data from the Cervical Length Education and Review program to evaluate the quality of transvaginal cervical length ultrasonography by trained imagers (ie, ultrasonographers, radiologists, perinatologists).

METHODS: This is a retrospective observational study of data from the Cervical Length Education and Review program. Candidates underwent an online lecture series, examination, and submitted a batch of images for review. For a candidate’s batch of images to pass, all images must meet at least seven of the nine criteria assessed, the overall batch score needs to be 80% or greater, correct caliper placement must be met for all images, and the same criterion cannot be consistently missed. We also examined a subset of these criteria-appropriate image acquisitions, defined as an image that demonstrated both internal and external os and visualization of the entire endocervical canal. Primary outcome was the overall initial candidate pass rate; secondary outcomes included distribution of criteria missed in images and percentage of images that was inadequately acquired.

RESULTS: Six hundred eighty-seven candidates submitted 3,748 images between June 10, 2012, and August 18, 2016. Eighty-five percent of candidates were ultrasonographers. Of the 687 initial batches submitted, 105 (15%) did not pass. Eight hundred thirty-seven images (22%) of all images failed at least one criterion; the most common image deficiencies were in “anterior width of cervix equals the posterior width” (33%), “failure to visualize” the internal or external os (29%), “cervix occupies 75% of image and bladder area visible” (33%), and incorrect caliper placement (24%). Two hundred fifty-six (7%) of all images failed to meet our criteria for adequate image acquisition.

CONCLUSION: Fifteen percent of trained imagers failed to obtain appropriate cervical length imaging. This highlights the importance of a standardized cervical length training and certification program.

Among trained imagers undergoing structured training for transvaginal cervical length measurement, 15% fail to meet standards, highlighting the importance of standardized training and certification.

Division of Maternal Fetal Medicine, Department of Obstetrics & Gynecology, Sydney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania; the Division of Maternal-Fetal Medicine, Obstetrics & Gynecology, Intermountain Healthcare, Utah Valley Hospital, Provo, Utah; the Perinatal Quality Foundation, Oklahoma City, Oklahoma; and the Ohio Perinatal Quality Collaborative, Columbus, Ohio.

Corresponding author: Rupsa C. Boelig, MD, 833 Chestnut Street, Level 1, Maternal-Fetal Medicine, Philadelphia, PA 19107; email: rupsa.c@gmail.com.

Financial Disclosure The authors did not report any potential conflicts of interest.

Presented at the Society for Maternal-Fetal Medicine's 37th Annual Meeting, January 23–28, 2017, Las Vegas, Nevada.

Each author has indicated that he or she has met the journal's requirements for authorship.

The importance of accurate measurement of transvaginal ultrasound cervical length to identify candidates for intervention to prevent preterm birth has been emphasized by both the American College of Obstetricians and Gynecologists1 and the Society for Maternal-Fetal Medicine (SMFM).2 Improper measurement of cervical length could lead to overtreatment for improperly measured shortened cervices or lack of treatment and prevention of preterm birth in women who in fact have a short cervix. Documentation of a specific training program for or quality control of cervical length measurements in studies is lacking in most studies of transvaginal ultrasound cervical length measurement for preterm birth screening. Furthermore, there are little data on the quality and consistency of cervical length imaging among trained imagers (ie, ultrasonographers, radiologists, perinatologists). The Cervical Length Education and Review program was developed by the Perinatal Quality Foundation, a sister organization to the SMFM, to ensure that persons performing cervical length measurement in pregnancy are properly trained and adhere to standardized criteria for documentation of cervical length (https://clear.perinatalquality.org). The program criteria were developed in conjunction with members from the American College of Obstetricians and Gynecologists, American College of Osteopathic Obstetricians and Gynecologists, American College of Radiology, American Institute of Ultrasound in Medicine, the Society of Diagnostic Medical Sonography, and the SMFM. The purpose of this study is to identify potential deficiencies in consistent cervical length imaging quality by evaluating the pass rate of images submitted to the Cervical Length Education and Review program, assessing the acquisition of key criteria in cervical length imaging, and identifying the most common reasons for deficiencies.

Back to Top | Article Outline

MATERIALS AND METHODS

This is a retrospective observational study of data from the Cervical Length Education and Review program. This program was developed by the Perinatal Quality Foundation in 2012 to assure that persons performing cervical length measurement in pregnancy were properly trained and adhered to standardized criteria for documentation of cervical length. Applicants to the program undergo online lectures and, after completion of the lectures and an examination, submit batches of images of cervical lengths done at 16–32 weeks of gestation for review. Reviewers were selected by the Perinatal Quality Foundation Board and image grading was periodically compared between reviewers to ensure consistency. All images in a batch were evaluated by the same reviewer. A minimum of five images is required in the initial batch. For a candidate's batch of images to pass, all images must meet at least seven of the nine criteria assessed (Box 1), the overall batch score needs to be 80% or greater (ie, if five images are submitted, a minimum score of 36 out of a possible 45 must be achieved), correct caliper placement must be met for all images, and the same criterion cannot be consistently missed on all images. An additional three or five images were requested for failed batches. In addition to examining the pass rate as defined by this program, we also examined appropriate image acquisition, a subset of these criteria, defined as an image that demonstrated both internal and external os and visualization of the entire endocervical canal (Fig. 1). These three criteria were selected as key features to acquiring an appropriate cervical length image.

Fig. 1.

Fig. 1.

Evaluations of images submitted to the Cervical Length Education and Review program by applicants following their online training between June 10, 2012, and May 18, 2016, were reviewed. The geographic location, occupation, and level of training for each candidate were recorded. The percentage of candidates who were required to resubmit an image batch and the pass rate after resubmission were measured. Each image was assessed by the nine criteria (Box 1) to identify the most frequent areas of deficiencies and to determine the percentage of images that was appropriately acquired, as defined previously.

Primary outcome was the overall initial candidate pass rate. Secondary outcomes included distribution of criteria missed and percentage of images that were inadequately acquired. In addition, initial candidate pass rates were compared by year, by occupation, and by region using χ2 analysis from Open Source Epidemiologic Statistics (www.OpenEpi.com). P<.05 was considered significant.

This study was conducted in concert with the Perinatal Quality Foundation. As part of the privacy policy, candidates who participate in the Cervical Length Education and Review program accept that their information may be used in an aggregate, deidentified manner for research. In addition, there were no patient identifiers associated with any images submitted and reviewed. As such, this study was exempt from institutional review board approval.

Back to Top | Article Outline

RESULTS

Six hundred eighty-seven candidates submitted 3,748 images between October 6, 2012, and May 18, 2016. Eighty-five percent of candidates were ultrasonographers, 6% were maternal–fetal medicine subspecialists, 4% were maternal–fetal medicine fellows in training, and 3% obstetrics and gynecology specialists. Candidates were fairly evenly distributed regionally in the United States with 29% from the South, 24% from the West, 23% from the Midwest, 21% from the Northeast, and 3% from Puerto Rico or international (Table 1).

Table 1.

Table 1.

Of the 687 initial batches submitted, 105 (15%) did not pass. Of these, 88 candidates resubmitted images; 85% passed after additional images were reviewed (Fig. 2). The initial pass rate increased from 77% in 2013 to 89% in 2016 (P=.03) (Fig. 3).

Fig. 2.

Fig. 2.

Fig. 3.

Fig. 3.

There was no significant difference in pass rate among ultrasonographers, maternal–fetal medicine subspecialists, maternal–fetal medicine fellows in training, and obstetrics and gynecology specialists (P=.89) (Table 1). Similarly, there was no significant association between region of country and pass rate (0.13) (Table 1).

Of the 3,748 images submitted, 837 (22%) images did not meet at least one criterion. For the individual criterion assessed (Box 1), the percentage of failure among these 837 images is shown in Figure 4; the most common deficiencies were in “anterior width of cervix equals the posterior width” (33%), “failure to visualize” the internal or external os (29%), “cervix occupies 75% of image and bladder area visible” (33%), and incorrect caliper placement (24%). Two hundred fifty-six images, which is 31% of images that failed at least one criterion and 7% of all images submitted, failed to meet our criteria for adequate image acquisition as described in the “Methods” (demonstrated both internal and external os and visualization of the entire endocervical canal).

Fig. 4.

Fig. 4.

Back to Top | Article Outline

DISCUSSION

Our findings are consistent with the Maternal–Fetal Medicine Units Network (MFMU) study in demonstrating that a significant percentage (15%) of candidates fail to submit satisfactory images that are properly measured.3 In contrast to that study, we were able to assess a larger number of candidates who were from a broader background than the MFMU sites and who have accessed a training program with standardized criteria based on a consensus by the Perinatal Quality Foundation and that is available to all.

The top reasons for image failure involved appropriate technique for obtaining an adequate image and in caliper placement. The equal anteroposterior width of the cervix is important because it indicates there is not undue pressure on the cervix from the probe, which could affect cervical length. A significant percentage of failures was also the result of lack of visualization of the internal or external os. The most common deficiencies were criteria that are central to adequate image acquisition, which is particularly concerning. This highlights the need for training in appropriate image acquisition and cervical length measurement.

The fact that the majority (85%) of applicants who failed their first submission subsequently passed with only one additional submission suggests a benefit to a standardized training in cervical length in pregnant women. The Cervical Length Education and Review program provides not only a certification of quality for those who pass, but also a guide for improvement for those who do not and a standardized set of criteria by which to assess quality of cervical length.

This study has a number of strengths. It builds on the previous MFMU study3 with almost twice the number of candidates and a wider geographic and practice distribution, expanding the external validity. Cervical length imaging quality was determined by previously established key criteria and images was examined by expert reviewers with quality control measures to ensure internal validity and reproducibility. Finally, it provides not just an assessment of overall quality of cervical length imaging, but also key areas of deficiency.

This study has limitations. First, there may be selection bias in the population studied because these are imagers who have voluntarily submitted themselves for training in cervical length screening. The external validity is therefore limited and the rate of suboptimal cervical length imaging is likely underestimated. Second, because candidates did not submit images before the training program, we cannot comment specifically on the efficacy of the training program except that the majority of applicants who failed with their first submission passed after their second. Third, although the applicants were fairly evenly distributed across the United States, the overall number is limited and there are no data on the specific practice settings (ie, outpatient office, community hospital, academic medical center). Fourth, we cannot comment on the maintenance of skills because there was no posttraining follow-up after obtaining certifications. Finally, the lack of clinical correlation associated with images submitted limits our ability to comment on the degree of effect suboptimal imaging may have on patient care and outcomes.

The value of cervical length screening has been shown in several randomized studies and guidelines showing a decrease in preterm birth when interventions are applied in women with a short cervical length such as a cerclage in women with prior preterm birth4,5 and vaginal progesterone for women without a prior preterm birth.1,2,6 Recent studies have reported a decreased risk of preterm birth among low-risk women who undergo universal cervical length screening in the second trimester.7,8 Multiple studies have demonstrated the cost-effectiveness of a universal cervical length screening program.9–12 As cervical length becomes increasingly used in the prediction and prevention of preterm birth, and the difference of 1 mm in measurement could mean the difference in treatment recommendations, it is imperative that as health care providers we ensure the quality and consistency of our cervical length imaging. Our study provides evidence to justify the need for standardizing cervical length ultrasound techniques as has been with nuchal translucency measurement. The Nuchal Translucency Quality Review was established to ensure the quality and consistency in nuchal translucency measurements. The data collected from this program have aided in identifying and addressing the problem of undermeasurement in nuchal translucency ultrasonograms.13 Similarly, the Cervical Length Education and Review program provides an avenue for educating imagers and health care providers, addresses a clear deficiency in quality control of cervical length measurement, and establishes protocols for quality review. The Perinatal Quality Foundation has reviewed more than 60,000 nuchal translucency images over the past 11 years. Although it is major work to have central quality review, the Perinatal Quality Foundation has the potential resources to pursue this effort.

There is much room for further research in the area of cervical length measurement quality assessment. Specifically, it is important to identify both the preeducational and posteducational pass rate to assess the efficacy of the training itself and to broaden the utilization of the Cervical Length Education and Review program to have a better understanding of the use and quality of cervical length done across the United States in a variety of practice settings.

Back to Top | Article Outline

REFERENCES

1. Prediction and prevention of preterm birth. Practice Bulletin No. 130. American College of Obstetricians and Gynecologists. Obstet Gynecol 2012;120:964–73.
2. Society for Maternal-Fetal Medicine Publications Committee, with assistance of Vincenzo Berghella. Progesterone and preterm birth prevention: translating clinical trials data into clinical practice. Am J Obstet Gynecol 2012;206:376–86.
3. Iams JD, Grobman WA, Lozitska A, Spong CY, Saade G, Mercer BM, et al. Adherence to criteria for transvaginal ultrasound imaging and measurement of cervical length. Am J Obstet Gynecol 2013;209:365.e1–5.
4. Berghella V, Rafael TJ, Szychowski JM, Rust OA, Owen J. Cerclage for short cervix on ultrasonography in women with singleton gestations and previous preterm birth: a meta-analysis. Obstet Gynecol 2011;117:663–71.
5. Cerclage for the management of cervical insufficiency. Practice Bulletin No. 142. American College of Obstetricians and Gynecologists. Obstet Gynecol 2014;123:372–9.
6. Romero R, Nicolaides K, Conde-Agudelo A, Tabor A, O'Brien JM, Cetingoz E, et al. Vaginal progesterone in women with an asymptomatic sonographic short cervix in the midtrimester decreases preterm delivery and neonatal morbidity: a systematic review and metaanalysis of individual patient data. Am J Obstet Gynecol 2012;206:124.e1–19.
7. Temming LA, Durst JK, Tuuli MG, Stout MJ, Dicke JM, Macones GA, et al. Universal cervical length screening: implementation and outcomes. Am J Obstet Gynecol 2016;214:523.e1–8.
8. Son M, Grobman WA, Ayala NK, Miller ES. A universal mid-trimester transvaginal cervical length screening program and its associated reduced preterm birth rate. Am J Obstet Gynecol 2016;214:365.e1–5.
9. Einerson BD, Grobman WA, Miller ES. Cost-effectiveness of risk-based screening for cervical length to prevent preterm birth. Am J Obstet Gynecol 2016;215:100.e1–7.
10. Conde-Agudelo A, Romero R. Vaginal progesterone to prevent preterm birth in pregnant women with a sonographic short cervix: clinical and public health implications. Am J Obstet Gynecol 2016;214:235–42.
11. Werner EF, Han CS, Pettker CM, Buhimschi CS, Copel JA, Funai EF, et al. Universal cervical-length screening to prevent preterm birth: a cost-effectiveness analysis. Ultrasound Obstet Gynecol 2011;38:32–7.
12. Cahill AG, Odibo AO, Caughey AB, Stamillo DM, Hassan SS, Macones GA, et al. Universal cervical length screening and treatment with vaginal progesterone to prevent preterm birth: a decision and economic analysis. Am J Obstet Gynecol 2010;202:548.e1–8.
13. D'Alton ME, Fuchs KM, Abuhammad A, Benacerraf B, Berkowitz R, Cuckle H, et al. Implementation of a national nuchal translucency education and quality monitoring program. Obstet Gynecol 2014;123:149–54.
Figure

Figure

© 2017 by The American College of Obstetricians and Gynecologists. Published by Wolters Kluwer Health, Inc. All rights reserved.