Hemmer, Paul A. MD, MPH; Durning, Steven J. MD; Papp, Klara PhD
Clerkship directors (CDs) occupy an important position as managers of academic programs at medical schools. They have responsibilities that cover the breadth of curricular development, evaluation, feedback, and administrative oversight of educational programs that may span multiple training sites.1–4 Although conducted more than 20 years ago, a survey of CDs identified serious challenges to evaluation that likely ring true today, including no warning system for “problem” students, a breakdown in information transfer across clerkships, inadequate guidelines about students remediating clinical clerkships, no follow-up on the effectiveness of remediation, and a lack of integrated information about students over time.5,6
Most clerkships in the U.S. and Canada are discrete, discipline-based experiences lasting 4 to 12 weeks per clerkship, although longitudinal clerkship experiences are gaining in popularity.7 In either case, it is not known how CDs formally communicate with one another and the medical school's administration about issues relevant to medical student education. In addition, there is growing interest in the topic of discussing students who are struggling during clinical clerkships, and how and whether that information is or should be shared.8–11 Frellsen and colleagues10 have noted that most CDs (64%) believed that they should discuss struggling students' performance “outside” of the promotions (or student progress) committee meetings, although they did not characterize when or where these discussions occurred. Thus, it could be inferred that these discussions about struggling students are occurring informally rather than in the presence of the medical school administration.
Therefore, in this study, we sought to clarify whether CDs meet with one another regularly within their institutions, at U.S. and Canadian medical schools; the frequency, content, and usefulness of the meetings; and whether this was the venue in which CDs and medical school administrations discussed the performance of students on clinical clerkships, but not limited to struggling students.
In May 2007, the Clerkship Directors in Internal Medicine (CDIM) conducted an annual, confidential, online survey of its 114 institutional members from U.S. and Canadian medical schools (not all Liaison Committee on Medical Education [LCME]-accredited medical schools have an institutional member in the CDIM). The CDIM has one institutional member for each of its 114 schools. The survey was developed with input from CDIM members, coordinated by the CDIM research committee, and approved for distribution by the CDIM council. The CDIM research committee contacted nonresponders up to three times by e-mail to complete the survey. The survey was reviewed and approved by the institutional review board at the Uniformed Services University of the Health Sciences. The first section of the survey inquired about CDs' demographic characteristics (age, gender, academic rank, primary role, and years in primary role). The section on CDs' meetings within each medical school asked whether or not CDs at a school met with one another on a regular basis. If there were regular meetings, we asked respondents to characterize the frequency (weekly, monthly, quarterly, semiannually, annually, or other). We asked who was present at the meetings, with a pick list of core CDs, preclinical course directors, representatives from the dean's office, and “other” (followed by a free-text response area). We asked what was discussed at the meetings (progress of students on clerkship rotations, students having academic difficulty on clerkships, planning for “at-risk” preclerkship students in the clerkship years, policy decisions from the dean's office to the CDs, policy decisions/recommendations from the CDs to the dean, input to the curriculum committee (curriculum development, preparation for accreditation [LCME] reviews), general announcements, and “other” topics with a space for written comments. We asked whether respondents found the CD meetings to be useful (rated very useful, mostly useful, mostly not useful, waste of my time) and whether they believed the CD meetings helped or harmed students at their institution (with each question rated strongly disagree, disagree, neutral, agree, or strongly agree).
In addition, we asked some questions similar to those on the 2006 CDIM survey10 examining for stability of responses and also expanding responses beyond struggling students (the focus of the 2006 survey) to all students. Specifically, we repeated a question about whether the respondent was prohibited from discussing students who were in academic difficulty. We asked for CDs to rate their opinions about whether there were benefits or drawbacks to students in discussing students' performance (each rated on a scale of strongly disagree, disagree, neutral, agree, or strongly agree), without limiting the response to struggling students. We also allowed free-text responses to what CDs viewed as the benefits and drawbacks to students with regard to discussing their performance, not limited to those in academic difficulty.
We used SPSS, version 12.0 (SPSS Inc., Chicago, Illinois) for statistical analysis, including descriptive statistics, and chi-square analysis or Fisher exact test as appropriate, for categorical variables. For the free-text responses, two authors (P.H. and K.P.) independently reviewed approximately 25% of the responses to create a preliminary coding structure and, through an iterative process, discussed the coding and collapsed or created new codes, until agreement was reached on the final themes. The two authors (P.H. and K.P.) then independently coded the remainder of the free-text responses; we reviewed responses for a subset of cases and found agreement to be acceptable (kappa = 0.492; P = .005).
The survey response rate was 74% (84/114). Respondent's primary role included CD (66, or 79%,), vice chair for education (5, or 6%), fourth-year CD (6, or 7.1%), core GME faculty (3, or 3.6%), ambulatory block director (1, or 1.2%), second-year course director (1, or 1.2%), associate residency program director or program director (1, or 1.2%), fellowship director (1, or 1.2%), and other (7, or 8.3%), with five reporting their role as division chief, one as associate hospital epidemiologist and infectious disease physician, and one as senior associate chair for educational affairs. Several respondents indicated more than one primary role. Respondents had been in their primary role for a mean of 8.1 years (SD 4.8, range 1–20 years). The mean age of respondents was 45.6 (±7.1) years, with 60% men, and an academic rank distribution of assistant professor (29%), associate professor (42%), and professor (25%).
Virtually all respondents (81/84, or 96%) indicated that CDs meet regularly at their institution, with 77% meeting monthly, 15% meeting quarterly, 6% meeting every other month, and 1% meeting weekly. Table 1 summarizes the types of attendees described as being present at these meetings. The meetings were typically attended by “core” CDs as well as a representative from the dean's office. A minority of respondents indicated that preclinical course directors (15%), fourth-year CDs (10%), or students (7%) were present or included in these meetings. Others who were noted by some respondents to attend the meetings included the administrative assistant to the dean (2), registrar (1), curriculum consultant (1), head of scheduling (1), and student counselors (1).
Table 2 summarizes the topics of discussion at these meetings. Most commonly, the content of the meetings was reported to be communication between the dean's office and the CDs, input into the curriculum, or preparation for accreditation visits. Approximately half of the respondents indicated that these meetings were the venue to discuss not only students who are in academic difficulty (“struggling”) but also students' performance in general during clerkships. A few of the respondents (22%) indicated that they used these meetings to help create a plan for the needs of struggling preclinical students as they make the transition to the clerkship years.
As with a prior study,10 13 respondents (16%) noted that they were prohibited from discussing students who were currently having difficulty on any clerkship rotation, but only 4% (3) of all respondents believed that they should be prohibited from having such discussions. The reasons cited by the respondents who were prohibited from discussing students included the administration's fear of biasing future CDs (cited by seven respondents), that such discussions take place in a different venue such as a promotions committee (cited by four), fear of litigation (cited by two), and a general opposition to such discussions (cited by one).
Some respondents (30, or 35%) were not specifically precluded from discussing students with the other CDs yet indicated that they did not discuss students at the CD meetings. Seventeen of these respondents provided reasons (in their free-text comments) for not discussing students: The issue was addressed at a different meeting/venue (7), members (deans or CDs) oppose the discussion (3), or this is simply not done (3). However, two respondents noted that discussions of students do take place at the meeting, and two other respondents noted that individual CDs will discuss students about whom they are concerned, to plan for placing them in an environment or with teachers to help them succeed, but that for these two respondents, their discussions occurred outside of the CD meetings.
The CD meetings were noted to be mostly useful (37%) or very useful (46%), but 16% (13) found the meetings to be mostly not useful. Of those who found the meetings mostly not useful, six (46%) were prohibited from discussing students in academic difficulty, which was statistically significant when comparing the usefulness of the meetings to CDs who were not prohibited from discussing struggling students (P = .002, Fisher exact test).
The overwhelming majority (93.6%) of respondents either agreed or strongly agreed that there were benefits to students if CDs are allowed to discuss students' performance (not limited to struggling students); three respondents (3.8%) were neutral, and two (2.5%) disagreed. Respondents were less uniform with their agreement about whether there are drawbacks to students if CDs are allowed to discuss students' performance, with 27.6% agreeing or strongly agreeing there are drawbacks, 25% neutral, and 47.6% disagreeing or strongly disagreeing.
Table 3 summarizes the themes that emerged from the free-text responses asking for respondents to explain their reasons for agreeing or not agreeing with the questions about benefits or drawbacks to students if CDs discuss their performance. With regard to the benefits, respondents cited several themes. First, such discussions help CDs provide a student a more personalized learning experience, such as scheduling the rotations with the best faculty and residents, proactively planning the experience (rather than rediscovering or ignoring a problem), and creating accurate evaluations. One respondent noted,
If a CD knows that a student struggles with, say, academics, then a proactive, rather than reactive plan can be put in place at the beginning of a clerkship to help forestall any academic difficulties. Same idea with professionalism issues—an ounce of prevention is worth a pound of cure.
Second, CDs would have the opportunity to intervene early, allowing early identification of problems to avert future failure, provide earlier feedback and attention, bring resources to bear, and simply help the student in a more timely fashion. Third, discussions enable communication among the CDs and longitudinal tracking of students' progress. Below are some representative comments.
If the student is having problems or you are having problems dealing with a student, consulting with your peers and possibly people who have encountered that student before the discussion can assist with developing a strategy to best deal with them.
I fear that many problem students slip through when there is minimal communication.
It is helpful to discuss these students together to obtain others' perspectives with the experience of multiple CDs pooled … a more valuable remediation or support scheme could be devised.
The final benefit to students cited was the belief that the interests of society are ultimately best served when CDs can discuss students' performance. One response captured many of these themes:
By and large this is helpful to students by preventing future failure. In the short term, [students] may feel it is damaging if faculty share information about them and if it leads to recommendations for additional work. However, we are a faculty of a medical school, producing physicians who will ultimately be taking care of patients. We have a responsibility to produce graduates who are competent and professional. By [holding discussions] we are more likely to accomplish these goals by intervening early rather than by passing the buck.
With regard to drawbacks to students of CDs discussing their performance, respondents indicated concern about a prejudicial attitude or bias that may develop that could adversely affect a student. One CD noted,
Too much interclerkship talk can lead to unfounded/secondhand preconceptions and unfair treatment.
Even though the concern was cited, it was often couched in terms of how the prejudice or bias could be mitigated, as two respondents' comments illustrate:
There is potential for bias. The direct evaluators of student performance are usually not informed of past performance difficulties to minimize this effect.
The only drawback I see is biasing the CD to grade the student lower. The CD needs to be professional in this regard and specifically avoid this bias.
The second drawback raised a more specific concern that discussing students' performance could lead to a specific action (rather than an attitude) such as lower grades, evaluations, or unfair treatment. These CDs' comments note,
By placing students under more scrutiny than other students, [there] might be some unfairness there.
If the shared information is not used in the best interest of the student, they may be subject to bias in evaluation and grading.
The risk of “labeling” a student and having resulting lower grades is real.
The third drawback was that a few respondents felt such discussions could violate student privacy or confidentiality, particularly if the discussion was shared outside of the CD meeting. An illustrative comment:
It could be an issue if the CD would not maintain confidentiality and the student's performance on other clerkships was known to the people involved in grading them beforehand [emphasis in original].
Finally, one respondent noted that having such discussions could result in an increased academic workload for the student, which the student would view negatively.
There were several respondents whose responses identified a third theme about CDs discussing students' performance that went beyond benefits and drawbacks to students, namely, a theme about the character of the CDs. Several respondents identified that CDs are professionals, in academic leadership positions within their medical schools, and that they need to be trusted to act on behalf of the best interests of society and the students. Several respondents noted that prohibiting discussions of student performance was or could be interpreted as a lack of faith on the part of the administration in their CDs and that this could or did undermine the relationship with the medical school administration. One respondent noted,
CDs must be trusted agents of the university and/or medical school. CDs have an obligation to keep any information learned about the student in confidence or limited to those who need to know. Precluding discussions of students likely places the medical school at risk—incompetent students who graduate and move to residency are the medical school's legacy and [the school] will likely be seen as bearing responsibility for the decision to graduate such students, particularly at risk if information about students was known but not shared.
CDs are key academic managers in medical schools, often with significant experience and academic rank.4 Prior to our study, to our knowledge little was known about whether CDs met at medical schools, the topics of their discussions, and their opinions about the utility of such meetings. Importantly, “forward feeding” of information about medical students, whether across clerkships or across years in medical school, is a topic of significant interest to academic managers, medical education program leadership, and teachers.8,9,11 A prior study by Frellsen and colleagues10 noted that such discussions about struggling students were occurring among CDs outside of academic promotion meetings but could have left the impression that these discussions might be informal or in “back channels.”
We found that CDs and representatives from the dean's office meet quite frequently at their medical schools. The CDs present are usually those whom one would consider “core,” with some including fourth-year CDs and few including students on the committee. For fourth-year directors, this number may be low partly because many third-year directors also serve as fourth-year directors,12 or there may be fewer required clerkships in the fourth year. We do not know the reason why so few respondents noted the participation of students at these meetings. It could be logistical reasons (e.g., students away on clerkships) or a desire for privacy, but students might provide important contributions to such meetings by perhaps giving more real-time feedback and insight about rotations and helping with planning. It might be worthwhile for individual medical schools to consider whether students should participate in these meetings.
The most common topics discussed at the CD meetings reveal that they are a venue for communication between the medical school administration and CDs. This serves a critical role not simply in meeting LCME accreditation standards13 but also in keeping open lines of communication, sharing ideas, brainstorming about educational research ideas, and understanding the views and needs of each group.
Furthermore, CDs clearly find these regular meetings to be valuable in their management of educational programs. Interestingly, CDs who are specifically precluded from discussing students' performance at these meetings are much more likely to find that the meetings are mostly not useful to them compared with those who are not precluded from discussing students who find the meetings mostly not useful (42% versus 11%). CDs would appear to want to engage with each other and the administration in discussion of student-specific issues in order to make the meetings a valuable use of their time.
Less common topics, although cited as present by nearly half of the respondents, included discussion of students' performance in general and discussion of students in academic difficulty on clerkships. According to Frellsen and colleagues,10 51% of the respondents to the 2006 CDIM survey noted that they do share information about struggling students with other CDs outside of academic promotion committee meetings. Thus, our findings support that when such discussions of students are taking place, they are within the framework of a regularly scheduled meeting among CDs and medical school administration officials rather than in “back-channel” discussions. This finding would apply to both students in general and students who are struggling on clinical clerkships. In fact, only two respondents indicated that they do discuss students outside of the CD meetings (where such discussions do not take place at their institutions) but do so in order to plan an educational experience that can address areas of concern and/or deficiency.
It is evident that the respondents to our survey overwhelmingly endorse the notion that there are benefits to students if CDs are allowed to discuss their performance—whether “struggling” students or students in general—with other CDs and the medical school administration. Their reasons for believing this included that it improved the timeliness of identification of, intervention for, and feedback to students during the clerkship. They also believed it would help them proactively (rather than reactively) plan for the student, such as tailoring the educational program in the clerkship, placing the student with teachers who are felt to be in the best position to help the student, and intervening early if problems arise. Most CDs did not tell the teachers working with any such identified student anything specific about that student, but were choosing teachers who are best suited to working with the student.10
Additional benefits cited by the respondents were that such discussions among CDs are beneficial to students in terms of longitudinal tracking of problems and progress, that seeking guidance from colleagues on how to best handle a given situation can result in greater expertise being brought to bear in addressing the needs of the student, and that such discussions are in the best interest of society. With regard to the latter, we believe that CDs are striking a balance between meeting the individual needs of a student, on the one hand, and, on the other, fulfilling the medical school faculty's duty to society to ensure that students are ready to graduate and move to the next level of training.
Even though respondents strongly believed in the benefits to students of such discussion, they identified potential drawbacks. Most commonly, this dealt with the possibility that the CD and/or future teachers would be unfavorably biased (to the positive or negative) about a given student, that they would prejudge a student, or that the student might feel stigmatized, thus potentially threatening the evaluation process. As noted, CDs often do not tell teachers about students, a reflection of their concern about such bias. Some respondents noted that to discuss a student's performance is a violation of her or his privacy, might lead to unfair treatment during the clerkship rotation, or may impact the evaluation process. Once again, the issue of balancing the obligations to society and to the student seems to be the central issue.
A recent article by Hauer et al14 about the remediation of medical trainees and/or physicians outlined a detailed approach to identifying those with deficiencies, creating specific and tailored interventions (remediation) to address and help the specific deficiency (or deficiencies), and then tracking the outcome. For medical students, such an approach would call for close and careful communication among administration officials and academic managers, perhaps as well as teachers of students. Such sharing would greatly improve the likelihood that supervisors know that deficiencies are addressed, that students are helped, that individual faculty and/or learning environments are used most effectively, and that the student improves to meet standards. The meetings among CDs and medical school administrators are already in place and are an ideal forum for such discussion and planning to occur, and could be the venue for ensuring close follow-up of interventions during clinical clerkships. As one respondent noted,
The argument that students with academic problems will have prejudicial treatment assumes that CDs cannot see beyond the problems to the students' potential and can't be useful in helping them learn. If that is the case, then the CDs need more training, not more blindfolding.
As this comment suggests, an unexpected theme emerged about discussing students' performance at CD meetings, namely, that of the integrity and professionalism of the CD. Respondents noted that CDs are advocates for students and that to discuss students' performance with other CDs is done in an effort to reach a more coordinated approach to the student, rather than continuing to deal with issues in isolation. Respondents clearly indicated that CDs are professionals and that to prohibit discussions of students presumes that CDs are not student advocates and that they cannot be trusted by their administration. This is reflected in our finding that those who are prohibited from discussing students' performance often find the CD meetings unhelpful. We see this theme from the comments of respondents to be a core issue that could undermine the trust and relationship between the administration and faculty and, thus, could be potentially harmful to all involved in the educational process.
We did not ask whether specific plans for individual students were discussed, created, or implemented at or as a result of these meetings, but respondents' qualitative comments would suggest that such planning is occurring. However, there is little known about the efficacy of discussing students' performance at CD meetings and of “feeding forward” this information to other academic managers; although there are believed to be benefits and potential drawbacks to students, there is little research to say whether or not this is the case.9,11,14 Given this, a presumption of benefit to discussing students may not be entirely justified, but by the same token, neither should there be a presumption of harm. Until such time that there is clarifying research on this issue, there is ample evidence that problematic behaviors later in medical training or practice can be traced to students' performance in medical school, and to avoid taking action in the best interest of society, as well as the student, would seem to violate our duty as a profession.15–20
It is evident that “forward feeding” of information about medical students' performance among CDs is occurring, usually (but not always) with representatives of the medical school administration, and that there needs to be a consensus on how this information can and/or should be shared. As a result, there are important steps to be taken and questions yet to be answered. Specifically, we need to expand beyond the views of CDs to understand the views of deans and students, and then develop general guidance for academic managers and medical school officials. Such guidance needs to address multiple issues: for instance, should medical students (either in general or the involved student) be part of these discussions? And, how should we ensure that students are evaluated fairly and longitudinally without blindfolding teachers and/or CDs? Such guidance must simultaneously address that medical students must be treated fairly, without being subject to scrutiny that is arbitrary or capricious, and that academic leaders (from deans to individual faculty) must recognize that society is placing enormous trust in the medical education process to graduate medical students in whom we are confident of future success.
There are limitations to our study, in addition to those mentioned previously. It is a cross-sectional survey of internal medicine CDs, although there is stability of responses across a two-year period (from the 200610 to 2007 CDIM surveys), and the questions we asked about the CD meetings were general. The views expressed in the qualitative comments are made by internal medicine physicians, and thus we do not know whether they accurately reflect the views of CDs from other disciplines. Although providing a pick list from which respondents could select as many items as relevant (e.g., the topics discussed at the CD meetings, Table 2) might have restricted responses, we consistently provided respondents an opportunity to offer open-ended responses for items we may have overlooked in the iterative process of survey development; the use of this “Other” category allowed us to fully capture the breadth of responses.
CDs and medical school officials meet frequently, with open lines of communication, to discuss important issues related to the educational programs within their institutions. Students' progress on clinical clerkships was discussed at nearly half of all responding medical schools, reflecting that these discussions are formal rather than informal. Further research is needed on the effectiveness of these discussions and subsequent plans that are developed. In the meantime, the views of deans and students could inform the process of discussing students' performance at these meetings within medical schools.
The authors wish to thank the CDIM Research Committee and the CDIM administrative staff for their help in the survey review process and Internet survey instrument development.
The survey was reviewed and approved by the institutional review board at the Uniformed Services University of the Health Sciences.
The views expressed in this report are those of the authors and do not represent the official views of the Department of Defense, the United States Air Force, or other federal agencies. The data presented in this report are the property of the Clerkship Directors in Internal Medicine and are used with permission.