Morgan, Maria A. PhD*; Driscoll, Deborah A. MD†; Zinberg, Stanley MD*; Schulkin, Jay PhD*; Mennuti, Michael T. MD*†
Guidelines for offering cystic fibrosis carrier screening in the preconception and prenatal care setting were developed by the American College of Medical Genetics (ACMG), American College of Obstetricians and Gynecologists (ACOG), and National Institutes of Health in response to a Consensus Development Panel report indicating that cystic fibrosis carrier screening should be offered to couples planning a pregnancy or seeking prenatal care.1,2 A multidisciplinary process was used to develop clinical and laboratory guidelines. Information was included regarding the clinical course of the disease, the genetics of cystic fibrosis, and the recommended screening process. Information that physicians might find useful in understanding the scientific basis for the recommendations, for counseling patients, and in responding to patients’ questions was included. In addition, patient education materials, consent forms, and sample patient letters were included to facilitate implementation. Factors that patients might find influential in making their decision to have or to not have screening were listed in the patient education brochure. The materials were mailed to all ACOG members in October 2001.
We conducted a survey 2 years after distribution of the guidelines to evaluate implementation. In a previous publication,3 we reported that the majority of obstetrician–gynecologists ask pregnant patients if they have a family history of cystic fibrosis (88.7%), that they provide pregnant patients with information regarding cystic fibrosis (86.6%), and that they offer cystic fibrosis carrier screening to all pregnant patients independent of selection criteria (65.8%) or only in specific situations (32%; eg, she requests it, she has a family history of cystic fibrosis). The proportion of physicians engaging in these practices with nonpregnant patients was substantially lower.
Here we report responses to the same survey, relating self-reported physician familiarity with the guidelines to knowledge, implementation, and opinions regarding cystic fibrosis screening.
MATERIALS AND METHODS
Before distribution, the questionnaire was extensively field-tested on practicing obstetrician–gynecologists, and the final version was reviewed by specialists before being distributed to subjects. The study received institutional review board approval from the University of Pennsylvania's Office of Regulatory Affairs. Questionnaires were mailed in September 2003 to 1,165 American College of Obstetricians and Gynecologists Fellows and Junior Fellows in Practice. A total of 565 of these subjects were members of the Collaborative Ambulatory Research Network. Members of this network are practicing obstetrician-gynecologists who have volunteered to participate in survey studies on a regular basis. The Collaborative Ambulatory Research Network was established to facilitate assessment of clinical practice patterns and aid the development of educational materials. The remaining 600 subjects consisted of a computer-generated random sample of ACOG Fellows and Junior Fellows in Practice who are practicing obstetrics and/or gynecology and had not received a survey from ACOG during the previous 2 years (non-Collaborative Ambulatory Research Network members). All nonrespondents received a second mailing of the questionnaire 5 weeks after the first mailing. A final reminder mailing was sent approximately 6 weeks later. Questionnaires returned by January 9, 2004 were included in the survey. This protocol has typically resulted in a total sample size of more than 450, which is sufficient to detect differences between groups of less than 0.5 standard deviation with power of 80% and significance at the 0.05 level.4
The questionnaire contained a total of 34 questions on 3 double-sided pages, a standard length for our questionnaires. Nine questions addressed demographic details of physicians and their patient population, 9 questions assessed basic knowledge of cystic fibrosis and professional training and experience, and 16 questions inquired about screening practices and opinions regarding cystic fibrosis and screening. The questions having to do with practices and professional concerns regarding cystic fibrosis screening in pregnant and nonpregnant patients were primarily addressed elsewhere.3 The majority of questions were of a multiple-choice format, except for the 4 opinion scale questions, which asked physicians to rate their level of agreement with several statements regarding cystic fibrosis screening.
The data were analyzed using a personal computer-based software package (SPSS 12.0; SPSS Inc., Chicago, IL). Descriptive statistics were computed for the measures used in the analyses, which are reported as mean and standard error of the mean. The t test was used to compare group means of continuous variables. Differences on categorical measures were assessed with the χ2 test. Group differences on ordinal measures were assessed using the Mann-Whitney test or Kruskal-Wallis test. Correlations including an ordinal measure used the Spearman rho coefficient. All analyses were tested for significance using an α level of 0.05.
A total of 753 questionnaires were returned. Data from 22 respondents were judged invalid (physician retired, returned to sender), resulting in a valid response rate of 64% (731 of 1,143 respondents), 419 from Collaborative Ambulatory Research Network members (75% response rate) and 312 from nonmembers (53.4% response rate). There were responding physicians from every state of the United States, as well as the District of Columbia, Puerto Rico, Canada, and overseas military installations. Respondents’ mean age and sex ratio closely matched the characteristics of the larger population to whom the survey was sent.
The remaining analyses will be limited to the 86.5% (632) of respondents who indicated that their primary medical specialty was gynecology or general obstetrics and gynecology, and who include nonpregnant patients of reproductive age in their practice. This subject selection was done to best assess the practices of nonsubspecialist obstetrician and gynecologists. Of these physicians, 13.5% (85) see only gynecologic patients (GynOnly), and 86.6% (547) see both obstetric and gynecologic patients (ObGyn); the term “physicians” is used when referring to all subjects. The mean age of physicians was 47.12 ± 0.39 years, and they had been out of residency a mean of 15.38 ± 0.38 years. Males made up 57.6% of these respondents and were older than females (49.95 ± 0.51 years versus 43.28 ± 0.53 years, P < .001). Collaborative Ambulatory Research Network members made up 57.9% of respondents. Collaborative Ambulatory Research Network females were older than nonmember females (44.44 ± 0.62 years versus 41.69 ± 0.90 years, P < .05). Data are collapsed across Collaborative Ambulatory Research Network member and nonmembers because the two groups did not differ on any nondemographic measure analyzed.
The majority (90.8%) of physicians rated ACOG documents as an important source of information. However, when asked how familiar they were with the ACOG/ACMG cystic fibrosis carrier screening guidelines published in October 2001, only 19.2% of respondents selected “I have read it thoroughly” (see Table 1, column headings). ObGyns were more familiar with the document than GynOnlys (P < .001); more ObGyns than GynOnlys reported having thoroughly read (20.7% versus 9.5%) or skimmed (46.4% versus 32.1%) the document. Gender and years since residency were not associated with document familiarity.
Physicians were asked questions assessing knowledge of cystic fibrosis and ACMG/ACOG screening recommendations (see Table 1). When asked to select the racial/ethnic group(s) that have a cystic fibrosis carrier detection rate of 80% or better if the recommended panel of mutations is tested, 29.7% answered correctly by selecting both Northern European Caucasian and Ashkenazi Jew. ObGyns were more likely than GynOnlys to answer correctly (31.7% versus 15.4%, P = .003).
The majority (82.0%) of physicians knew that the guidelines recommend that cystic fibrosis carrier screening should be offered to both individuals with a family history of cystic fibrosis or reproductive partners of individuals with cystic fibrosis, and to couples in whom one or both partners are Caucasian and are planning a pregnancy or seeking prenatal care. ObGyns were more likely than GynOnlys to answer correctly (83.9% versus 70.2%, P = .002). Of ObGyns, females were more likely than males to answer correctly (90.7% versus 78.9%, P = .001).
Those subjects who were more familiar with the guidelines were significantly more likely to answer questions correctly and to select the correct ACMG/ACOG recommendations than those who had skimmed, not read, or not heard of the guidelines (see Table 1, last 4 columns).
Physicians were also presented with a more difficult question that required either recall of specific information in the guidelines or the ability to calculate the genetic risk for a recessive disease incorporating information about the sensitivity of screening that was provided in the guidelines:
A Caucasian couple has cystic fibrosis carrier screening. The woman is found to be a cystic fibrosis carrier, and her reproductive partner has a negative cystic fibrosis carrier screening test. How would you describe their chance of having a child with cystic fibrosis compared to the Caucasian population that has not had this screening test?
□ Lower □ About the same □ Higher
Only 22.2% correctly answered “higher,” whereas the majority (77.8%) selected “lower” (55.6%) or “about the same” (22.2%). Familiarity with the document was not associated with correct responding to this more complex question (P = .124).
Almost three quarters (72.7%) of the physicians said their practice patterns regarding cystic fibrosis carrier screening and prenatal testing had changed since ACMG/ACOG guidelines were issued. A total of 77.4% of ObGyns had changed their practice patterns, whereas only 41.3% of GynOnlys had. Those more familiar with the guidelines were also more likely to indicate that their screening practices had changed (P < .001) (Table 1).
Increased guideline familiarity of ObGyns was associated with an increased positive response to “ask about cystic fibrosis family history” (P < .001; read guidelines = 96.2%, skimmed = 90.8%, heard of but not read = 80.2%, not heard of = 81.8%), an increased positive response to “provide information about cystic fibrosis” (P < .001; read guidelines = 94.3%, skimmed = 90.7%, heard of but not read = 76.6%, not heard of = 72.1%), and an increased positive response to “offer carrier screening to all pregnant patients” (P < .001; read guidelines = 83.0%, skimmed = 65.1%, heard of but not read = 55.1%, not heard of = 59.1%). Respondents who correctly identified the recommended guidelines were more likely to offer carrier screening to all pregnant patients (rather than not offering screening, or using the recommended selective criteria) than were those who did not correctly identify the recommended guidelines (P = .002; correct = 68.5%, not correct = 50.6%).
Physicians were asked how well they felt their training and experience qualified them to manage general genetic counseling in prenatal care, cystic fibrosis screening in prenatal care, and counseling for a positive prenatal diagnosis of cystic fibrosis. Regarding general genetic counseling, the majority (63.1%) rated themselves as somewhat qualified, and 22.9% said well qualified. For cystic fibrosis screening, 57.2% said somewhat qualified and a quarter (25.5%) said well qualified. Regarding a positive prenatal diagnosis, almost half (48.9%) felt unqualified and 43.2% somewhat qualified. Those completing residency more recently rated themselves more highly qualified in all three situations than did those out of residency longer, as did ObGyns as compared with GynOnlys (all P's <.01). Level of familiarity with the guidelines correlated positively with the 3 qualification ratings (rho = .221, P < .001; rho = .353, P < .001; rho = .290, P < .001, respectively).
Physicians were asked to rate how strongly they agreed with a statement regarding cystic fibrosis carrier screening: “I am confident about my ability to interpret or deal with a positive screening test.” Two thirds (67.3%) of physicians agreed or strongly agreed (4 or 5 on a 5-point scale) that they were confident. As with the qualification ratings above, physicians completing residency more recently were more likely to agree or strongly agree than were those out of residency longer (71.9% versus 62.8%, P = .002); so were ObGyns compared with GynOnlys (30% versus 72.9%, P < .001); and so were physicians who had thoroughly read the guidelines as compared with the other 3 groups (86.3% versus skimmed = 73.0%, heard of = 49.1%, not heard of = 51.9%; P < .001), independent of years since residency.
ObGyns were asked to rate on a 5-point scale their opinion regarding how influential several factors were in determining whether a pregnant patient would accept or reject an offer of cystic fibrosis screening (1 = not influential to 5 = very influential) (see Table 2). Ratings differed by gender, but not by document familiarity. All physicians were asked to rate on a 5-point scale their level of agreement with a set of statements addressing their concerns for the patient regarding cystic fibrosis carrier screening (1 = strongly disagree to 5 = strongly agree) (see Table 3). Document familiarity was highly associated with level of agreement with only 2 of the 6 statements regarding physician concerns about patients. Those more familiar with the guidelines were more likely to disagree that the test is too inaccurate (thoroughly read = 78.6%, skimmed = 68.5, not read = 58.4, not heard of = 50.0%; P < .001) and that there are insufficient educational materials (thoroughly read = 63.3%, skimmed = 58.2%, not read = 40.1%, not heard = 42.3%; P < .001).
Other studies have explored obstetrician–gynecologists’ adherence to ACOG guidelines, their attitudes regarding the importance of ACOG clinical practice guidelines, and perceived barriers to using guidelines.5,6 There is relatively little information regarding how familiarity with new clinical practice guidelines impacts knowledge and implementation in practice. Before October 2001, ACOG recommended that cystic fibrosis screening should only be offered to couples in which one partner had cystic fibrosis or couples who had a family history of cystic fibrosis.7 Thus, the distribution of Preconception and Prenatal Carrier Screening for Cystic Fibrosis: Clinical and Laboratory Guidelines represents an opportunity to study the impact of reading new guidelines on physician knowledge, implementation and opinions regarding cystic fibrosis screening.
We previously reported obstetrician–gynecologists’ practice pattern of offering cystic fibrosis carrier screening to non-pregnant and to prenatal patients after publication of the guidelines.3 The current study reports results from the same survey and is focused on understanding how familiarity with the guidelines impacted physician knowledge and practice implementation. We also explore how familiarity with the guidelines may have influenced opinions on certain issues that were addressed in the guidelines. It is likely that opinions of physicians were also influenced by information from sources other than the guidelines. Further, it is possible that physicians who were more interested in or knowledgeable about the topic of the survey were more likely to respond. In an attempt to control for this, a subset of our subject pool is comprised of Collaborative Ambulatory Research Network members, who respond to several questionnaires a year, covering a wide variety of topics. It is unlikely that cystic fibrosis screening is a topic of greater interest to this control group than to the group of randomly selected ACOG members. Collaborative Ambulatory Research Network members had a 75% response rate and made up 58% of respondents to this questionnaire. They did not differ significantly from nonmembers on nondemographic responses, and the age and sex ratio of respondents as a whole closely matched the characteristics of the larger population to whom the survey was sent, all of which suggests that response bias was minimized. Despite these limitations, we believe that differences in knowledge, practice pattern, and opinions of physicians associated with physician familiarity with the guidelines should be useful in development of future guidelines on genetic screening and other topics.
The low percentage of physicians who read the ACMG/ACOG guidelines, despite the finding that the majority believed that ACOG documents are an important source of information, points to the need to evaluate additional methods of communication of new guidelines. The results demonstrate that basic knowledge regarding cystic fibrosis was associated with having read the guidelines. It is not clear if the greater number of correct answers on basic questions was causally related to having read the guidelines or if individuals who reported reading the guidelines were also more likely to be knowledgeable about cystic fibrosis. In contrast, ability to correctly answer a more difficult question about a common clinical situation was not enabled by reading or skimming the guidelines, even although the information to answer the question was in the guidelines. This suggests that the format of guidelines may not be well suited to in-depth education beyond the prescriptive recommendations. Educational methods, such as case-based teaching, may be better suited to transmitting knowledge that enables problem solving.
Reading the guidelines was significantly associated with implementation, and was also associated with correctly identifying the specific ACMG/ACOG recommendations and selective screening criteria. In direct contrast, familiarity with the guidelines was associated with offering cystic fibrosis screening to all patients, rather than applying the selective criteria recommended in the guidelines. The survey did not include questions designed to understand reasons for compliance or noncompliance with selective screening criteria. It is possible that the low compliance with selective screening criteria may have resulted from the greater ease of offering screening to all patients in a busy obstetric practice or because the difference between “offer” and “make available” was not clear to readers of the guidelines. Further investigation is warranted to understand the cause of the low compliance with selective screening criteria.
We also attempted to elicit physicians’ opinions regarding key issues studied in pilot projects of cystic fibrosis screening and to determine if the responses were influenced by their familiarity with the guidelines.8 Differences in physicians' opinions regarding factors that might influence patient acceptance of screening were not associated with document familiarity or time since completing training. It is generally accepted by clinical geneticists that understanding the range of severity of disease and chronic disease burden are important elements of genetic counseling and influence patient decision making.8,9 Despite descriptions of these aspects of cystic fibrosis in the text of the guidelines and in the patient education materials, ObGyns rated this as likely to be least influential in the patient decision and this was not influenced by familiarity with the guidelines. We believe that this difference between clinical geneticists and ObGyns warrants further exploration.
Physicians strongly agreed with the statement that patients do not understand the limitations of sensitivity of the screening test. This opinion was not influenced by document familiarity, and it is likely that this opinion is strongly influenced by more extensive experience with other screening tests in obstetric practice that have similar sensitivity, such as second-trimester maternal serum screening for Down syndrome. In contrast, those physicians who had read the guidelines were more likely to disagree that the test is too inaccurate. This suggests that familiarity with the guidelines informed physicians of the low risk for having a child with cystic fibrosis after negative screening of one or both partners despite the limitations of sensitivity of the test.
Shorter time interval since completing training was associated with greater physician confidence in their perceived ability to interpret and deal with positive screening results. Reading the guidelines was also associated with greater physician confidence, independent of years since residency. This suggests that physician education through clinical practice guidelines is of value in improving physician confidence with application of new technologies, independently of years from completion of training.
We conclude that reading ACOG practice guidelines is an important source of information for obstetricians and gynecologists with regard to knowledge and implementation in practice. This experience should be useful in formulating and communicating future guidelines, and in development of supplemental educational materials.