Steiner, Anne Z. MD, MPH; Fritz, Marc MD; Sites, Cynthia K. MD; Coutifaris, Christos MD, PhD; Carr, Bruce R. MD; Barnhart, Kurt MD, MSCE
The Accreditation Council for Graduate Medical Education (ACGME) Program Requirements for Graduate Medical Education in Obstetrics and Gynecology states that “A residency program in obstetrics-gynecology must be a structured educational experience. … While such residency programs contain a patient-service component, they must be designed to provide education as a first priority.1” ACGME also requires residency programs to have an educational curriculum containing training in reproductive endocrinology and infertility. The Council on Resident Education in Obstetrics and Gynecology (CREOG) Educational Objectives for Residency Programs provides guidelines as to the knowledge and skills residents should acquire in reproductive endocrinology and infertility before the completion of residency.2 Programs are required to assess individual resident's knowledge, which is commonly done through an annual standardized test.
Although ACGME advises as to what should be learned, there is considerable flexibility allowed as to how these educational objectives are to be met. Programs therefore have the freedom to set the length of subspecialty rotations, number of didactic hours, and clinical responsibilities. In fact, diversity between programs is encouraged.1 Exposure to reproductive endocrinology and infertility, usually in the form of subspecialty rotation(s), differs in structure between programs. However, it is not known whether the structure of the rotation and subsequent resident experience affects accumulation of reproductive endocrinology and infertility knowledge.
The Society for Reproductive Endocrinology and Infertility (SREI) sought to estimate current and past resident knowledge in this area. By examining CREOG In-Training Examination results between 2004 and 2007, they observed that residents were answering fewer answers correctly in reproductive endocrinology and infertility compared with the overall examination. In addition, they perceived a decline in correct answers to specific questions on reproductive endocrinology and infertility, with declines observed in the specific areas of infertility and reproductive endocrinology and no change observed in the areas of menstrual and endocrine disorders and basic reproductive endocrinology (Fig. 1). It was hypothesized that decreased time or quality or both of teaching on the reproductive endocrinology and infertility rotation might lead to insufficient accumulation of knowledge by residents in this area.
The objective of this study was to estimate U.S. residents' experience on rotations in reproductive endocrinology and infertility, estimate resident satisfaction with their rotations in reproductive endocrinology and infertility, and estimate the extent to which rotation qualities are associated with resident satisfaction and reproductive endocrinology and infertility knowledge.
MATERIALS AND METHODS
A survey was created to assess resident educational experience in reproductive and endocrinology and infertility. The anonymous, multiple-choice survey was provided to all examinees during the 2008 CREOG In-Training Examination (N=4,744). The CREOG examination is offered annually as a mechanism for evaluating resident cognitive knowledge of obstetrics and gynecology, a requirement for residency programs in Obstetrics and Gynecology.1 The 25-question, investigator-developed survey (including 13 demographic questions) was administered by CREOG at the annual examination, using the CREOG testing format at the time of the annual examination. Sites were instructed to administer the survey before the examination. The survey answer sheet matched that of the CREOG examination (bubble form), and results were tallied using the same scanning system. A total of 4,670 surveys (98.4% response rate) were returned. This analysis includes all 4,670 surveys. The University of North Carolina Institutional Review Board (IRB) determined this analysis to be exempt from required IRB approval.
The primary outcomes for this study were resident satisfaction with the quality of the reproductive endocrinology and infertility rotation, resident satisfaction with the duration of the rotation, and self-reported knowledge of reproductive endocrinology and infertility. Satisfaction with quality and duration of the rotation was determined from the question “In general, which of the following best describes how satisfied you are with the total duration and quality of your rotations in REI?” Four responses were possible including 1) satisfied with both duration and quality; 2) satisfied with duration, not quality; 3) satisfied with quality, not duration; and 4) satisfied with neither duration nor quality. Responses 1 and 3 were coded as satisfied with quality, and responses 2 and 4 were coded as unsatisfied with quality. Responses 1 and 2 were coded as satisfied with duration, and responses 3 and 4 as unsatisfied with duration. Knowledge of reproductive endocrinology and infertility was determined by the question “Which of the following best describes how you would rate your knowledge base in clinical REI?” Responses were limited to 1) Poor 2) Adequate 3) Good 4) Excellent. Categories were dichotomized for analysis.
The cohort was initially described using all 4,670 surveys. For analyses of resident satisfaction with the quality and duration of the reproductive endocrinology and infertility rotation, 3,709 participants were included after excluding 961 residents who did not answer the question. For analyses of resident knowledge of reproductive endocrinology and infertility, 4,515 residents were included after excluding 155, who did not answer the question.
Information collected in the survey included time on the reproductive endocrinology and infertility rotation per year, number of hours of didactic education per week on reproductive endocrinology and infertility, vacation policy for the rotation, coverage of other duties while on reproductive endocrinology and infertility rotation, and Sunday through Thursday night call while on the rotation. All responses were multiple choice. “Not applicable” responses were coded as missing in the analysis. Covariates collected through the survey included postgraduate year, sex, race and ethnicity, marital status, presence and number of children in the family, CREOG region, number of residents in the program, and type of residency program (university or community based). The χ2 test was used to compare the demographics (proportions) by outcome. All covariates were considered as potential confounders and included in the models.
Deidentified CREOG scores were obtained from the 4,744 residents who took the 2008 CREOG examination. Individual, fourth-year CREOG scores (percent correct) in basic reproductive endocrinology, menstrual and endocrine disorders, infertility, and climacteric sections were statistically compared to their score on the entire examination using Poisson regression with a cluster term3 used to designate that scores were being compared within an individual.
For analysis of the survey results, univariable analyses were conducted to determine the most appropriate method for modeling the categorical exposures. Covariates (time on the rotation, didactic hours) having a linear relationship with the outcomes (quality, duration, and knowledge) were modeled as ordinal variables. Categorical variables without a linear relationship were modeled using indicator variables. Potential collinearity between covariates and exposures was explored before modeling using Spearman rank correlation coefficients. Preliminary analyses revealed potential effect modification by postgraduate year. Multiple interaction terms were created to assess for interaction between postgraduate year and each of the exposures. Subsequently three logistic regression models were created, one for each outcome of interest (quality, duration, and knowledge). All three models contained the exposures, covariates, and potential effect modifiers. Likelihood ratio tests were conducted and effect modifiers that were not statistically significant (P<.05) were removed from the model. The models were then analyzed using binomial regression to generate risk ratios. All statistical analyses were conducted using Stata 11.0.
On the 2008 CREOG examination, postgraduate year 4 residents answered significantly fewer questions correctly on the basic reproductive endocrinology (61% correct), menstrual and endocrine disorders (61%), and climacteric sections (42%) compared with the entire examination (63% correct, P<.001). Scores improved from previous years and were significantly above the entire examination average in infertility (67% in 2008, P<.001).
In total, 4,670 residents completed the anonymous reproductive endocrinology and infertility survey. Duration of rotations varied considerably by postgraduate year and by residents within a postgraduate year (Table 1). Approximately 84% of respondents were permitted or required to take vacation on the reproductive endocrinology and infertility rotation, while 16% were not permitted vacation on the rotation. Excluding those respondents who responded “not applicable,” 80% of residents covered other services while on the rotation. The majority of residents did not have to take night-call while on the rotation. Most residents had some form of didactic lecture on the rotation, most commonly for 1 hour a week (Table 1).
Of the 3,709 residents included in the analysis of the quality and duration of the reproductive endocrinology and infertility rotation, 1,719 (46%) were satisfied with the quality and duration, 738 (20%) were satisfied with the quality but not the duration, 632 (17%) were satisfied with the duration but not the quality, and 620 (17%) were satisfied with neither. Of the 4,515 who were included in analysis of self-assessed reproductive endocrinology and infertility knowledge, 1,784 (40%) described their knowledge as poor, 2,005 (44%) as adequate, 672 (15%) as good, and 54 (1%) as excellent.
Satisfaction with quality, satisfaction with duration, and knowledge were significantly correlated with Spearman correlation coefficients ranging from 0.19 to 0.21. Residents describing themselves as having poor knowledge of reproductive endocrinology and infertility were more likely than residents reporting good knowledge to be in their first year of training (46% compared with 5%, P<0.001), female (79% compared with 69%, P<.001), single (44% compared with 33%, P<.001), and without children (22% compared with 35%, P<.001). Those not satisfied with the quality and duration of their program's reproductive endocrinology and infertility rotation were more likely to be single (42% compared with 34%, P=.003).
Residents reporting less time on the reproductive endocrinology and infertility rotation (required vacation during the rotation, multiple coverage duties during the week on the rotation, frequent night-call), and no didactic hours were more likely to describe their reproductive endocrinology and infertility knowledge as poor (Table 1). Residents with less time on the reproductive endocrinology and infertility rotation and no didactic hours were more likely to be unsatisfied with the duration of the rotation. Residents with conflicting duties on the rotation and having fewer didactic hours were more likely to be unsatisfied with the quality of the rotation.
Required vacation during the reproductive endocrinology and infertility rotation and coverage outside of the rotation more than two times a week were associated with a 40% increase in risk of poor perceived knowledge in reproductive endocrinology and infertility (Table 2). Required vacation, coverage more than three times a week, and call three times a week were also strongly associated with dissatisfaction with duration of the reproductive endocrinology and infertility rotation. Although a required vacation policy had little effect on the perception of reproductive endocrinology and infertility rotation quality, multiple coverage duties elsewhere and frequent call increased the risk of dissatisfaction with the quality of the rotation. Conversely, more didactic hours was associated with reduced dissatisfaction with the quality of a rotation. Three hours of didactics per week was associated with a 61% reduction in risk of poor perceived knowledge.
Time on the reproductive endocrinology and infertility rotation was not associated with the perceived quality of the rotation (P=.29). The effect of time on the rotation on satisfaction with the duration of the rotation and resident knowledge of reproductive endocrinology and infertility differed by postgraduate year (Ptest of interaction =.006 and Ptest of interaction=.06, respectively). In the first 2 years, time appeared to increase the likelihood of satisfaction with the duration of the reproductive endocrinology and infertility rotation and decrease the probability of poor perceived reproductive endocrinology and infertility knowledge while in later years of training, time appeared to have less of an effect (Fig. 2).
This study demonstrates that, subjectively, residents are noting inadequate knowledge in reproductive endocrinology and infertility. Forty percent of residents in obstetrics and gynecology consider themselves to have poor knowledge of reproductive endocrinology and infertility. Although the risk was higher in junior level residents, 40% of senior level residents considered themselves to have only poor or adequate knowledge of reproductive endocrinology and infertility. Low CREOG scores (42%–67% correct answers) in reproductive endocrinology and infertility topics would appear to corroborate resident perceptions. These findings would imply that not all residents are gaining appropriate knowledge in this area during residency. A basic understanding of reproductive endocrine and infertility is vital for the generalist in an outpatient practice. While the generalist spends the majority of their time in the outpatient setting, most of residency is dedicated to inpatient training. Reproductive endocrinology and infertility rotations, in general, provide a unique opportunity for mentored training in an outpatient setting. Menstrual cycle irregularities, climacteric complaints, androgenic disorders, and basic infertility are common issues brought to the general gynecologist for diagnosis and treatment.4,5 While educators may be interested in knowledge accumulation as an endpoint in itself, it is also important to note that resident satisfaction with rotations may ultimately affect choice of future career and subspecialty training.6
A resident's experience on a rotation appears to have a significant effect on their knowledge. These data suggest that compromises made in allocation of a residents' time may have negatively affected education in one of the core Obstetrics and Gynecology education objectives. It is also possible that other aspects of training in Obstetrics and Gynecology have been similarly affected. Competing demands and distractions affect perception of the quality of a rotation and the perception of knowledge acquired. Factors that are associated with a resident's perceived quality of a rotation and subsequent knowledge include limitations on night-call and coverage of off-service duties. Unfortunately a significant percentage of residents (14% of applicable residents) are being pulled more than two times a week from their reproductive endocrinology and infertility rotation and (12% of applicable residents) are taking night-call more than once between Sunday and Thursday during their reproductive endocrinology and infertility rotation.
One could hypothesize that low resident knowledge in reproductive endocrinology and infertility could be due to the introduction of the primary care requirement and the 80-hour work week, which may have disproportionately affected office-based training, such as in reproductive endocrinology and infertility. Recognizing the breadth and limited time of Obstetrics and Gynecology resident training, an increase the duration of rotations, like reproductive endocrinology and infertility, is not always possible. However, the finding that the quality and duration of a rotation appear to be two independent indicators of resident-perceived knowledge allows for potential modifications in training. The duration of the rotation did not ensure quality of the rotation. Quality rotations include limited call and little off-service coverage. These findings suggest that a focused rotation with limited distractions may enhance satisfaction and increase perceived knowledge.
Considering time restrictions in residency, educators may need to reallocate time to address weaknesses in the residents' knowledge at their program. Mid-level providers can be leveraged to alleviate the burden on residents on service rotations. This will allow for more time for education on service rotations and more time to dedicate to educational (nonservice) rotations. A coverage rotation and night float may provide a method to secure dedicated time for educational rotations. Alternatively, vacation maybe considered a “rotation” in the zero sum game of allocation of a resident's rotations (ie, if a resident gets 3 weeks of vacation only 49 weeks, not 52 weeks, are available to distribute among the rotations for that year). Therefore, each rotation will not be “disrupted” by vacation and coverage. Because the effect of rotation duration on reproductive endocrinology and infertility knowledge was most pronounced in the first 2 years of postgraduate training, programs may affect resident knowledge simply by moving the reproductive endocrinology and infertility rotation into the first or second year. Finally, consideration should be given to inducing the tenants of reproductive endocrinology and infertility in other office-based rotations, and the time spend on an reproductive endocrinology and infertility rotation should be more focused on reproductive endocrinology and less on assisted reproductive technologies.
Perceived knowledge was also positively correlated with didactic hours. While these findings may appear intuitive, they provide empiric evidence that didactics are an effective method to expand a resident's knowledge base. Resident training is largely experiential; however, previous research has shown that medical knowledge is promoted most effectively through a mixture of interactive session and didactics.7 Didactic lectures may be used to address CREOG objectives that may be difficult to address experientially (due to the rarity of disease). SREI provides classic references that may be used to support a didactic curriculum.8
The strengths of this study include the high response rate, large number of residents surveyed in an anonymous fashion, and collection of information on covariates and potential confounders. We used multiple measures to assess the quality of resident reproductive endocrinology and infertility education: perceived satisfaction with quality of the rotation, perceived satisfaction with the duration of the rotation, and perceived knowledge of reproductive endocrinology and infertility. A limitation of the study is that because of the anonymous nature of the survey we were not able to link these data to the result of the CREOG examination. Perceived knowledge was used as a surrogate for tested knowledge. Resident physician's perceived knowledge does not always correlate with examination scores,9–11 but standardized examinations may not be the single best method for assessing physician competency.12 While SREI perceived a decline in CREOG scores in reproductive endocrinology and infertility topics, this may be due to differences in the depth or difficulty of the questions or stress on topics no longer taught during residency. This survey did not address knowledge or satisfaction with other rotations or areas of obstetrics and gynecology resident training.
In conclusion, many residents perceive their knowledge of reproductive endocrinology to be lacking. There is potential to improve resident knowledge of reproductive endocrinology and infertility through focused dedicated rotations that limit off-service coverage and call and include didactics. Changes in allocation of residents' responsibility and educational objectives may be needed to improve resident knowledge and satisfaction in this area.
2. Council on Resident Education in Obstetrics and Gynecology. CREOG's educational objective: a core curriculum in obstetrics and gynecology. 9th ed. New York (NY): Professional Publishing Group, Ltd; 2009.
3. StataCorp. 2009. Stata: Release 11. Statistical Software. College Station, TX: StataCorp LP.
4. Wren BG. Dysfunctional uterine bleeding. Aust Fam Physician 1998;27:371–7.
5. Deligeoroglou E, Tsimaris P, Deliveliotou A, Christopoulos P, Creatsas G. Menstrual disorders during adolescence. Pediatr Endocrinol Rev 2006;3(suppl 1):150–9.
6. Lu G, Owen J, Wenstrom K. Obstetrics and gynecology residents' attitudes toward maternal-fetal medicine fellowship training. J Matern Fetal Neonatal Med 2004;16:259–63.
7. Forsetlund L, Bjørndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf F, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. The Cochrane Database of Systematic Reviews 2009, Issue 2. Art. No.: CD003030. DOI: 10.1002/14651858.CD003030.pub2.
8. Society for Reproductive Endocrinology and Infertility. Classic/Essential References in Reproductive Endocrinology & Infertility. Available at www.socrei.org
. Retrieved December 3, 2009.
9. Gordon MJ. A review of the validity and accuracy of self-assessments in health professions training. Acad Med 1991;66:762–9.
10. Hawkins RE, Sumption KF, Gaglione MM, Holmboe ES. The in-training examination in internal medicine: resident perceptions and lack of correlation between resident scores and faculty predictions of resident performance. Am J Med 1999;106:206–10.
11. Parker RW, Alford C, Passmore C. Can family medicine residents predict their performance on the in-training examination? Fam Med 2004;36:705–9.
12. Holmboe ES, Hawkins RE. Methods for evaluating the clinical competence of residents in internal medicine: a review. Ann Intern Med 1998;129:42–8.
© 2011 by The American College of Obstetricians and Gynecologists. Published by Wolters Kluwer Health, Inc. All rights reserved.