Didactic Teaching Conferences for IM Residents

Who Attends, and Is Attendance Related to Medical Certifying Examination Scores?

Abstract

Purpose 

Didactic teaching conferences are a cornerstone of residents' training in internal medicine, yet these programs have received little formal evaluation. This study determines the factors associated with residents' attendance at didactic teaching conferences and the relationship of attendance to residents' scores on medical certification examinations.

Method 

The attendance of 81 residents was recorded at 199 conferences at one university hospital's internal medicine residency program during the 1996–97 academic year. Characteristics of the conferences were recorded, including the date, whether lunch was provided, the daily census on the medicine general wards, daily ambient temperature, and conference type. Residents' scores on the United States Medical Licensing Examination (USMLE) Step 2 and American Board of Internal Medicine (ABIM) certification examination were collected.

Results 

Residents' attendance at conferences was 34% overall or 54% excluding excused absences. Adjusting for the conferences' and residents' characteristics, attendance declined by one third as the year progressed. Providing lunch at noon conferences enhanced residents' attendance (odds ratio [OR] 1.26 overall and OR 1.64 for residents on inpatient rotations). Higher attendance was not associated with improvement in standardized medical knowledge test scores.

Conclusions 

Absenteeism and waning attendance through the year have important implications for structuring didactic internal medicine teaching. Providing lunch improves attendance, but this incentive should be weighed against the potential burdens of the pharmaceutical industry's funding of these lunches. The lack of relationship between attendance and residents' adjusted board scores calls for a better understanding of the value of this high-intensity medical education intervention.

Didactic teaching conferences are a cornerstone of the education of internal medicine residents. Significant effort goes into the scheduling, preparation, and presentation of the conference series in internal medicine residency programs. Underscoring the conferences' importance, the Accreditation Council of Graduate Medical Education (ACGME) recommends that a minimum average of 60% of residents attend didactic teaching conferences.¹ Despite the emphasis placed on these conferences, there has been a paucity of evaluation of attendance, the characteristics associated with residents' participation, or the outcomes of attending these conferences.

METHOD

We performed a prospective, longitudinal observational study of residents' attendance at didactic teaching conferences, the factors associated with residents' attendance at these conferences, and the effect of residents' attendance on the scores of their American Board of Internal Medicine (ABIM) certification examinations.

Data Collection

Using a daily sign-in sheet, we collected attendance data for 81 internal medicine residents for each didactic teaching conference at one university hospital during the 1996–97 academic year. All observed conferences were considered mandatory by the residency program. We coded residents as present, absent, or excused for each conference. Residents were excused if they were physically incapable of attending or if scheduling made it difficult, but not impossible, for them to attend (for instance, they were assigned to intensive or cardiac care units or the bone marrow transplant service).

For each conference, we recorded the date, whether lunch was provided, the daily census on the medicine general wards, daily ambient temperature, and the type of conference. Conferences were categorized into the following groups: grand rounds that met weekly in the morning, medicine and pathology, morbidity and mortality (M&M), journal club, and ethics, all of which met monthly at noon. General-topic noon conferences provided by either faculty or resident speakers met four times per week when they were not pre-empted by another noon conference.

We collected data on the characteristics of the residents: age, sex, race, marital status, and postgraduate training year. Residents' schedules were matched with the conference schedule to determine for each resident each day whether the resident was assigned to an inpatient or outpatient rotation and whether the resident would be considered excused.

Using a de-identified mechanism, we collected the percentages of questions answered correctly on the ABIM certification examination for 67 of the 81 residents (83%). The percentages of correct answers were comparable across cohorts.² To control for baseline medical knowledge and test performance, the three-digit scores for the United States Medical Licensing Examination (USMLE) Step 2 were collected for each resident. These scores were comparable across years of residency.³

To provide for comparison between the two standardized exams, we created a single scale for both the USMLE and ABIM certification examination. Scores were rescaled for each so that the mean resident score was 50 with a standard deviation of 10 using the following formula:

We calculated the difference between the rescaled scores to determine change in performance between the two examinations.

Statistical Analysis

Univariate analyses were performed for all the characteristics of the conferences and the residents attending them. We excluded 22 conferences with missing attendance data from the analyses (11% of the total 199 conferences).

To capture both the conferences' and the residents' characteristics in a single multivariate model, we computed a logistic regression model using generalized estimating equations with repeated measures, which account for within-resident correlations.⁴ Thus, we modeled the likelihood of each resident's attendance at a particular conference. After deleting conferences with incomplete data and excused absences, there were 9,293 unique opportunities for the 81 residents at 177 conferences. The logistic regression model included the conferences' and the residents' independent variables hypothesized to be related to attendance.

Conference-specific variables included time of year, inpatient or outpatient rotation assignment, type of conference, whether lunch was provided, daily ambient temperature, and daily census on the medical wards. Residents' characteristics included age, sex, race, postgraduate year, and prior medical knowledge as assessed by USMLE Step 2 performance. Residents' attitudes regarding perceived value of the conferences, their career goals, and specialty preferences derived from a year-end survey designed specifically for this analysis were also included in the model.

Because conferences are held in the hospital (closer for residents on inpatient assignment), we postulated an interaction between inpatient assignment and whether lunch was provided. We performed an additional logistic model that added an interaction term between lunch and rotation assignment.

To examine the effect of attendance on ABIM certification examination scores, each resident's mean attendance was compared with his or her ABIM score by creating a scatterplot. To control for baseline test performance, each resident's mean attendance was then compared with the difference between his or her ABIM certification examination and USMLE scores.

RESULTS

Of the 81 residents in our study, 58% were male, the mean age was 28 years, 38% were Caucasian, 42% Asian, and 20% other races and ethnicities. Three residents were graduates of foreign medical schools. Forty-seven percent stated a preference for practicing general medicine, 43% for specialty medicine, and 10% were undecided. Nine percent planned a research career, 42% expressed an interest in community clinical practice, and 49% intended to pursue a clinical/teaching career.

The 177 didactic teaching conferences with complete attendance data included 45 grand rounds, ten M&Ms, nine medicine—pathology conferences, eight journal clubs, and ten ethics conferences. The remaining 95 conferences were noon conferences on general topics given by faculty or residents. The mean raw attendance at all conferences was 34%. Adjusting for excused absences, the mean attendance was 54%. M&M conferences, with a mean attendance of 70%, had significantly higher attendance than other conferences, while journal clubs and ethics conferences were more poorly attended, with mean attendance of 44% and 27%, respectively (see Table 1).

Table 1: Percentages of Residents Who Attended Didactic Teaching Conferences, by Type of Lecture, at One Internal Medicine Residency Program, 1996–97

Third-year residents had significantly lower attendance (49%) than either first- or second-year residents (56% and 57%, respectively, p < .05). Residents had significantly higher attendance when they were assigned to inpatient rotations compared with outpatient assignments (64% versus 48%, respectively, p < .001).

Complete data for conferences and residents were available for 7,249 conference opportunities for the 81 residents (78% of the total 9,293 conference opportunities). The multivariate logistic model predicting the odds that a resident will attend a particular conference demonstrated that residents were less likely to attend conferences as the year progressed. Compared with the first third of the year, residents' odds of attending were 0.88 (95% confidence interval [CI] 0.70, 1.10) and 0.70 (95% CI 0.55, 0.90) for the middle and final third of the year, respectively. This finding was consistent across classes. Residents assigned to inpatient rotations more often attended conferences than did residents assigned to outpatient rotations (odds ratio [OR] 1.85, 95% CI 1.48, 2.31). Compared with the general-topic noon conferences given by residents and faculty, residents were more likely to attend M&M conferences and grand rounds (ORs 1.99 and 1.65, respectively) and less likely to attend journal clubs and ethics conferences (ORs 0.58 and 0.33, respectively). Curiously, daily census on the medical wards was not associated with resident attendance (see Table 2).

Table 2: Logistic Model—Independent Predictors of Residents' Attendance at Didactic Teaching Conferences at One Internal Medicine Residency Program, 1996–97^*

In addition, the residents were more likely to attend conferences when lunch was provided. However, the magnitude of this effect was modest (OR 1.26, 95% CI 1.07, 1.48). This effect was more pronounced among residents assigned to inpatient rotations (OR 1.64) but not outpatient rotations. There was also a trend for residents to attend conferences on warmer days (OR 1.14 for every 10°F of increased daily ambient temperature).

The residents' demographics, career intentions, evaluations of the conferences, estimated workloads, and evaluations of incentives were not associated with attendance. However, a resident's class was associated with attendance, with second-year residents more likely to attend conferences than first-year residents (OR 1.40, 95% CI 1.04, 1.87), and third-year residents less likely to attend than first-year residents (OR 0.76, 95% CI 0.53, 1.10). Residents who had higher USMLE Step 2 scores during medical school were more likely to attend conferences (OR 1.17, 95% CI 1.01, 1.36).

ABIM certification examination scores were available for 67 of the 81 residents (83%). USMLE scores were available for all residents. The 14 residents who did not release their ABIM certification examination scores had lower USMLE Step 2 scores than did the residents who did release their ABIM scores (42.0% versus 51.7%, p = .0007). However, they did not have significantly lower mean attendance (47.0% versus 52.1%, p = .17.) Figure 1, top panel, demonstrates that higher attendance was only weakly associated with higher ABIM certification examination scores (Pearson product—moment correlation coefficient r = .15, p = .2). After adjusting for baseline test performance by examining the difference between the ABIM certification examination and USMLE internally standardized scores, we found little appreciable association between percent attendance and change in standardized test score (r = 0.09, p = .5) (see Figure 1, bottom panel). Residents' USMLE and ABIM certification examination scores were highly correlated (r = 0.69, p < .001).

Figure 1: Associations between standardized test scores of medical knowledge and the percentages of 67 residents who attended didactic teaching conferences at one institution, 1996–97. The top panel shows American Board of Internal Medicine (ABIM) certifying examination scores versus percentages of conferences attended by the residents. The bottom panel shows changes in residents' performances on the ABIM certification examination and United States Medical Licensing Examination (USMLE) versus percentages of conferences attended by the residents. (Both ABIM certification examination and USMLE scores have been internally standardized about a mean of 50 with a standard deviation of 10.)

DISCUSSION

Didactic teaching conferences are a core element of internal medicine training. The ACGME recommends that a core conference series be developed and repeated during residents' three-year tenure.¹ Because many residents regularly miss conferences (due to either scheduling conflicts or personal choice), we concur with this recommendation. Residents' attendance declines as the year progresses; therefore, the core conference series should be presented early in the year. In addition, because residents on outpatient rotations were less likely to attend conferences, repetition of this series might be designed to capture different cohorts of residents beginning inpatient rotations.

Our study's findings are consistent with program directors' fears that not providing lunch would be associated with a decline in attendance.⁵ Because the residents in our study were provided meal cards that adequately covered the cost of lunch in the hospital cafeteria, this observed effect is largely due to the convenience of having lunch provided at the conference. The residents' responses to a questionnaire confirmed that receiving lunch was an important facilitator of their attendance at the conferences. However, providing lunch at the conferences may entail significant financial and moral costs. We estimated a cost of approximately $30,000 per year for lunches that would be required to supply a similar-sized program with the same number of conferences. To off-set this expense, 89% of U.S. internal medicine residency programs rely upon external financial sponsorship, usually by pharmaceutical companies.⁵ While pharmaceutical companies' support of residents' education has been considered ethically acceptable,⁶ industry's sponsorship in other contexts has been shown to affect physicians' practice patterns.⁷ Even gifts as small as a free lunch may influence faculty's and residents' prescribing patterns.⁸ Several groups have published guidelines that suggest restricting physicians' interaction with pharmaceutical companies.^9,10 Despite concerns about the potential harmful influences of industry's sponsorship on residents, most program directors believe that the benefits of sponsorship outweigh the costs.⁵ Program directors, faculty, and residents need to mull over our data to decide whether the attendance benefits outweigh the potential adverse effects of corporate sponsorship.

We believe that ours has been the first study to evaluate the relationship of residents' attendance at didactic teaching conferences and standardized test scores. Better attendance at the conferences was weakly correlated with better performance on the ABIM certification examination. This observed association may be biased toward the null hypothesis because residents who did not release their ABIM certification examination scores had slightly lower attendance and significantly lower USMLE scores (and, therefore, likely lower ABIM certification examination scores). However, the association between attendance and USMLE Step 2 performance was weak and, therefore, the magnitude of this bias is likely small.

The lack of association between attendance at the conferences and an objective measure of medical knowledge should stimulate consideration of how we measure the value of medical education interventions. Aside from imparting medical knowledge to residents, didactic teaching conferences expose residents to research opportunities, demonstrate methods to critically evaluate the literature, and provide a venue to exhibit professional role models and ethical decision making. Better ways to measure the value of a didactic conference series should be developed so that these educational programs can be studied and improved.

Our study has several limitations. We evaluated the relationship of residents' participation in only one venue of education. Future studies addressing this issue will need to rigorously evaluate the effects of the full spectrum of educational activities during residents' training. Furthermore, we evaluated only one residency program over one year. In addition, the effects of tardiness were not captured. Providing lunch at conferences is a time-saving convenience for residents that allows them to bypass the crowded cafeteria and arrive at conferences more promptly. Therefore, the full benefits of a lunch incentive may be underestimated if, as in our study, tardiness is ignored.

Our findings demonstrate substantial deficits in residents' attendance at didactic teaching conferences and point to aspects of conference programs that might be manipulated to improve attendance. Perhaps the most important finding is the lack of relationship between conference attendance and improvement in medical knowledge. Given the tremendous resources dedicated to these teaching programs, more research is needed to better understand the value of these programs. Other outcome measures should be developed to evaluate and improve this educational intervention.