Quality In-Training Evaluation ReportsDoes Feedback Drive Faculty Performance?

Dudek, Nancy L. MD, MEd; Marks, Meridith B. MD, MEd; Bandiera, Glen MD, MEd; White, Jonathan MD; Wood, Timothy J. PhD

doi: 10.1097/ACM.0b013e318299394c
Research Reports

Purpose: Clinical faculty often complete in-training evaluation reports (ITERs) poorly. Faculty development (FD) strategies should address this problem. An FD workshop was shown to improve ITER quality, but few physicians attend traditional FD workshops. To reach more faculty, the authors developed an “at-home” FD program offering participants various types of feedback on their ITER quality based on the workshop content. Program impact is evaluated here.

Method: Ninety-eight participants from four medical schools, all clinical supervisors, were recruited in 2009–2010; 37 participants completed the study. These were randomized into five groups: a control group and four other groups with different feedback conditions. ITER quality was assessed by two raters using a validated tool: the completed clinical evaluation report rating (CCERR). Participants were given feedback on their ITER quality based on group assignment. Six months later, participants submitted new ITERs. These ITERs were assessed using the CCERR, and feedback was sent to participants on the basis of their group assignment. This process was repeated two more times, ending in 2012.

Results: CCERR scores from the participants in all feedback groups were collapsed (n=27) and compared with scores from the control group (n = 10). Mean CCERR scores significantly increased over time forthe feedback group but not the controlgroup.

Conclusions: The results suggest that faculty are able to improve ITER quality following a minimal “at-home” FD intervention. This also adds to the growing literature that has found success with improving the quality of trainee assessments following rater training.

Dr. Dudek is associate professor, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada.

Dr. Marks (deceased) was professor, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada.

Dr. Bandiera is associate professor, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada.

Dr. White is associate professor, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Canada.

Dr. Wood is assistant professor, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada.

Funding/Support: This study was funded by a Medical Council of Canada Multi-Year Grant, file number: MCC-2(s)/08.

Other disclosures: None.

Ethical approval: This study was approved by the institutional review boards of the Ottawa Hospital Research Ethics Board, Health Sciences II–University of Toronto Research Ethics Board, the Capital Health Research Ethics Board at Dalhousie University, and the Health Research Ethics Board at the University of Alberta.

Previous presentations: An earlier version of this article was presented in part at the University of Ottawa’s Academy for Innovation in Medical Education Day, March 23, 2012, and the Canadian Conference on Medical Education, April 21, 2013.

Correspondence should be addressed to Dr. Dudek, Ottawa Hospital Rehabilitation Centre, 505 Smyth Rd., Room 1105D, Ottawa, Ontario, Canada, K1H 8M2; telephone: (613) 737-7350, x75596; fax: (613) 739-6974; e-mail: ndudek@toh.on.ca.

© 2013 by the Association of American Medical Colleges