Assessment of the Reliability and Validity of a Clinical Evaluation Instrument in Physical Therapy Education

Abstract

Introduction. 

Competency-based education is a proven, effective strategy for developing skilled health care practitioners. The purpose of this study was to reassess the validity and reliability of the Emory University Internship Clinical Evaluation (EUICE) instrument by reassessment of the component behaviors of the curricular competencies (provision of patient care, interpersonal communications, the teaching–learning process and administration) developed by the faculty of the Emory University Division of Physical Therapy. Five aims were investigated. The first three assessed content validity by evaluating respondent ratings of the essentialness of the component behaviors to the definition of the related competency, the exhaustiveness of the component behaviors per competency, and the measurability/observability of the component behaviors in the clinical setting. The fourth aim assessed the construct validity of the EUICE by examining whether the component behaviors were reducible to the four competencies in the EUICE. The fifth aim was to determine whether the EUICE has internal consistency.

Methods. 

A nationally representative clinical educator participant pool consisted of Directors of Clinical Education (DCEs)/Academic Coordinators of Clinical Education (ACCEs), along with Center Coordinators of Clinical Education (CCCEs) and clinical instructors (CIs) affiliated with the Emory Doctor of Physical Therapy (DPT) program. A total of 637 respondents (121 DCEs/ACCEs; 186 CCCEs; 330 CIs) completed a survey about the competencies and requisite component behaviors needed to practice as an entry-level physical therapist. The survey contained 50 items representative of the component behaviors in the EUICE. Descriptive statistics, principal component analysis (PCA), and Cronbach's alpha were used to assess content and construct validity and reliability of the component behaviors in the EUICE. Of the 50 survey items, 46 were included in the PCA and reliability analysis.

Results. 

There was agreement among educators on the essentialness, exhaustiveness, and measurability/observability of each component behavior. Principal component analysis yielded four factors corresponding to the four curricular competencies and explaining a total of 38.47% of the variance of the entire set of component behaviors. Each component behavior loaded onto one of the four factors representing the EUICE competencies. The clustering of items revealed by the PCA was consistent with the assignment of component behaviors to the competencies in the EUICE, thereby supporting construct validity. Cronbach's alphas were all above 0.7, suggesting good internal consistencies and that each subscale is reliable.

Discussion and Conclusion. 

In this study, we confirmed content validity, demonstrated construct validity, and established internal consistency of the component behaviors of the competencies in provision of patient care, interpersonal communications, the teaching–learning process and administration. Therefore, results support the use of the EUICE as a valid competency-based instrument by which to evaluate whether a student demonstrates competency in the clinical setting.