Skip Navigation LinksHome > October 2007 - Volume 86 - Issue 10 > Reliability of a 360-Degree Evaluation to Assess Resident Co...
American Journal of Physical Medicine & Rehabilitation:
doi: 10.1097/PHM.0b013e318151ff5a
Education & Administration: Competency

Reliability of a 360-Degree Evaluation to Assess Resident Competence

Massagli, Teresa L. MD; Carline, Jan D. PhD

Collapse Box


Massagli TL, Carline JD: Reliability of a 360-degree evaluation to assess resident competence. Am J Phys Med Rehabil 2007;86:845–852.

Objective: To determine the feasibility and psychometric qualities of a 360-degree evaluation of physical medicine and rehabilitation (PM&R) residents’ competence.

Design: Nurses, allied health staff, and medical students completed a 12-item questionnaire after each PM&R resident rotation from January 2002 to December 2004. The items were derived from five of the six competencies defined by the Accreditation Council for Graduate Medical Education (ACGME).

Results: Nine hundred thirty evaluations of 56 residents were completed. The alpha reliability coefficient for the instrument was 0.89. Ratings did not vary significantly by resident gender. Senior residents had higher ratings than junior residents. A reliability of >0.8 could be achieved by ratings from just five nurses or allied health staff, compared with 23 ratings from medical students. Factor analysis revealed all items clustered on one factor, accounting for 84% of the variance. In a subgroup of residents with low scores, raters were able to differentiate among skills.

Conclusion: Resident assessment tools should be valid, reliable, and feasible. This Web-based 360-degree evaluation tool is a feasible way to obtain reliable ratings from rehabilitation staff about resident behaviors. The assignment of higher ratings for senior residents than junior residents is evidence for the general validity of this 360-degree evaluation tool in the assessment of resident performance. Different rater groups may need distinct instruments based on the exposure of rater groups to various resident activities and behaviors.

© 2007 Lippincott Williams & Wilkins, Inc.


Article Tools


Article Level Metrics

Search for Similar Articles
You may search for similar articles that contain these same keywords or you may modify the keyword list to augment your search.