One morning while I finished up assigning grades for the family medicine and ambulatory care clerkship that I direct, a student stopped by to find out how she had done. She had earned an A, and since it is always fun to give students good news in person, I decided to tell her and risk another dozen students dropping by.
She looked over her faculty comments and gave me a puzzled look. “Where does Dr. ____ work? I don't remember working with that attending.” After discussing where she had worked, it became clear that she had not worked with or even met this attending. Yet this attending had filled out an evaluation of her! Chagrined, I promised to figure out what happened. Envisioning the A slipping out of her grasp, she said half-jokingly, “That's OK, we can leave things as they are.”
The clerkship secretary and I quickly determined what had happened. Because of an error on a list, an evaluation form was sent to the wrong attending. Interestingly, the errant evaluation was mediocre and included no comments—unlike the excellent evaluations the student had received at other sites. Discussions with my codirector confirmed that the attending who completed the errant form is conscientious and thoughtful but works in a high volume outpatient site that receives a new student every two weeks. We suspected that the attending assumed the student must have been there many weeks ago and dutifully completed the form based on the vaguest of impressions.
How can we maximize the accuracy of these crucial components of the educational process? Our evaluation forms include a picture of the student to stimulate recall, but this was not enough to prevent this mishap. Timely completion of the evaluation can enhance the accuracy and detail of the feedback as well as minimize cases of mistaken identity. In this instance, back-up systems to assure the correct distribution of forms also would have helped. More globally, faculty who feel they lack sufficient information to accurately evaluate a given student should seek additional information from other potential evaluators. After this, if they still lack adequate discriminating information, they should discuss the situation with the clerkship director. Finally, when a student is independently evaluated by multiple faculty and receives an evaluation that appears out of line with others, then that evaluation should be suspect, especially if it includes few or no comments.
Our inpatient medicine clerkship uses a summative evaluation system that solves most of these issues. Although this model is not practical for all clerkships, including the outpatient clerkship the student with the errant evaluation rotated through, it is an innovative system worth highlighting. At the end of each rotation, the entire team meets with a medical educator who helps the group reach consensus on each student's performance. All team members must agree on a final grade in each competency category. Input from junior residents has increased and faculty feel these evaluations are more accurate.
The validity and reliability of student evaluations are vital; in some ways, they represent the integrity of the institution. Additionally, if done poorly, these evaluations may have a direct effect on later quality of patient care. Organizations of clerkship directors should develop, implement, and study ways of obtaining accurate student evaluation in a variety of settings. This would benefit both students and patients.
Robert L. Hatch MD, MPH
Richard A. Davidson MD, MPH