Musick DW, Bockenek WL, Massagli TL, Miknevich MA, Poduri KR, Sliwa JA, Steiner M: Reliability of the physical medicine and rehabilitation resident observation and competency assessment tool: A multi-institution study.
Objectives: To assess the psychometric qualities of a method of resident physician evaluation by faculty.
Design: Multicenter study by seven Physical Medicine and Rehabilitation training programs. Faculty physicians observed residents in brief patient encounters or teaching sessions, rated specific competencies, and provided residents with immediate feedback. The resident observation and competency assessment form included competencies in patient care, professionalism, interpersonal and communication skills, systems-based practice, and practice-based learning and improvement. Residents and faculty rated satisfaction with the process.
Results: Three hundred sixty-two ratings were completed on 88 different residents. Each resident received an average of 3.8 ratings across two academic years. Overall internal consistency reliability was high (0.98); reliability of the individual competencies ranged from 0.74 to 0.76. Item means were correlated with year of training for two skill sets, with higher means for more experienced residents. The majority of participants gave high ratings of satisfaction; correlation between satisfaction ratings of attending and resident physicians was 0.63 (P < 0.01).
Conclusions: The resident observation and competency assessment is a reliable method to assess resident skills in five of six general competencies. Construct validity of the tool is supported by the fact that faculty rated two skill sets higher for senior residents.
From the Departments of Medical Education, Physical Medicine and Rehabilitation, and Medical Humanities (DWM); Brody School of Medicine, East Carolina University, Greenville; Department of Physical Medicine and Rehabilitation (WLB); Carolinas Rehabilitation (WLB), Carolinas Medical Center, Charlotte, North Carolina; Departments of Rehabilitation Medicine and Pediatrics (TLM) University of Washington School of Medicine, Seattle, Washington; Department of Physical Medicine and Rehabilitation (MAM), Mercy Hospital of Pittsburgh, University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania; Department of Physical Medicine and Rehabilitation (KRP), University of Rochester Medical Center, Rochester, New York; Department of Physical Medicine and Rehabilitation (JAS), Northwestern University/Feinberg School of Medicine; Rehabilitation Institute of Chicago (JAS), Chicago, Illinois; Department of Orthopedic Surgery and Rehabilitation (MS), Loyola University/Stritch School of Medicine; Hines Veterans Affairs Medical Center (MS), Chicago, Illinois.
All correspondence and requests for reprints should be addressed to: David W. Musick, PhD, Departments of Medical Education, Physical Medicine and Rehabilitation, and Medical Humanities, Brody School of Medicine, East Carolina University, 600 Moye Blvd., Brody 2N-72D, Greenville, NC. 27834.
Limited startup funding was provided by the American Board of Physical Medicine and Rehabilitation Foundation, Rochester, MN. Presented at the Annual Meeting of The Association of Academic Physiatrists, February 22, 2008, Anaheim, California. Financial disclosure statements have been obtained, and no conflicts of interest have been reported by the authors or by any individuals in control of the content of this article.