Whether physician assistant (PA) students' self-assessment or standardized patient (SP) evaluations of students' medical Spanish proficiency accurately reflect their language proficiency is unclear. This study compares PA student and SP ratings with an expert faculty member's rating to determine whether student or SP ratings can be used to evaluate language proficiency.
Fifty-eight students participated in a single-station Spanish Objective Structured Clinical Examination (OSCE) at the midpoint of a medical Spanish curriculum. Using the Interagency Language Roundtable (ILR)—a 6-point, single-item language proficiency scale previously validated among physicians—PA students and SPs evaluated students' medical Spanish proficiency. Their scores were then compared with the scores derived by an expert faculty rater who had viewed a video of each student–SP encounter. The faculty's score was considered the gold standard. Correlation between scores was calculated using Spearman's rank correlation coefficient.
Mean student scores were highest when rated by SPs (M = 3.8, SD = 0.9), followed by self (M = 3.0, SD = 0.9), and then faculty (M = 2.5, SD = 1.2). Spearman's rank correlation coefficient showed a strong positive correlation between students and the expert faculty rater (rs = 0.67, P < .001) and between SPs and the expert faculty rater (rs = 0.72, P < .001). The correlation was stronger for high- than for low-proficiency students. Students' self-rated scores showed significant improvement from baseline to the OSCE.
PA students participating in a medical Spanish curriculum and SPs show good correlation with an expert faculty rater in assessing Spanish proficiency during an OSCE. Standardized patients demonstrate scoring leniency. The ILR has potential for tracking aggregate student progress and curriculum effectiveness. With training, student self-rating could be used for interval assessment of medical Spanish communication.
Désirée A. Lie, MD, MSEd, is a clinical professor of family medicine at the University of Southern California, Keck School of Medicine, Alhambra, California.
Christopher P. Forest, MSHS, PA-C, is a professor and founding program director of the Master of Science Physician Assistant Program at California State University, Monterey Bay, California. Dr. Forest conducted this research while he was an assistant professor of family medicine at the University of Southern California, Keck School of Medicine, Alhambra, California.
Regina Richter-Lagha, PhD, is a consultant for the University of Southern California, Keck School of Medicine, Alhambra, California.
Correspondence should be addressed to: Désirée A. Lie, MD, MSED, Division of Physician Assistant Studies, Department of Family Medicine, University of Southern California, Primary Care Physician Assistant Program, Keck School of Medicine of USC, 1000 S Fremont Avenue, Unit 7, Building A11, Room 11166, Alhambra, CA 91803. Telephone: (626) 457-4074; Email: firstname.lastname@example.org
Supported by the Health Resources and Services Administration (HRSA) of the US Department of Health and Human Services (HHS), Grant number D57HP23251, LEAP-FIT: Longitudinal Education to Activate Physician Assistant Students for Interprofessional Teamwork (PI Kevin Lohenry, PhD, PA-C).
C. P. Forest and D. A. Lie are co-first authors who have contributed equal roles and effort to the design, execution, and implementation of the study, and writing of the manuscript.
The authors declare no conflict of interest.