Secondary Logo

Journal Logo

Institutional members access full text with Ovid®

Developing End-of-Training Entrustable Professional Activities for Psychiatry: Results and Methodological Lessons

Young, John Q., MD, MPP, PhD; Hasser, Caitlin, MD; Hung, Erick K., MD; Kusz, Martin; O’Sullivan, Patricia S., EdD; Stewart, Colin, MD; Weiss, Andrea, MD; Williams, Nancy, MD

doi: 10.1097/ACM.0000000000002058
Research Reports

Purpose To develop entrustable professional activities (EPAs) for psychiatry and to demonstrate an innovative, validity-enhancing methodology that may be relevant to other specialties.

Method A national task force employed a three-stage process from May 2014 to February 2017 to develop EPAs for psychiatry. In stage 1, the task force used an iterative consensus-driven process to construct proposed EPAs. Each included a title, full description, and relevant competencies. In stage 2, the task force interviewed four nonpsychiatric experts in EPAs and further revised the EPAs. In stage 3, the task force performed a Delphi study of national experts in psychiatric education and assessment. All survey participants completed a brief training program on EPAs. Quantitative and qualitative analysis led to further modifications. Essentialness was measured on a five-point scale. EPAs were included if the content validity index was at least 0.8 and the lower end of the asymmetric confidence interval was not lower than 4.0.

Results Stages 1 and 2 yielded 24 and 14 EPAs, respectively. In stage 3, 31 of the 39 invited experts participated in both rounds of the Delphi study. Round 1 reduced the proposed EPAs to 13. Ten EPAs met the inclusion criteria in Round 2.

Conclusions The final EPAs provide a strong foundation for competency-based assessment in psychiatry. Methodological features such as critique by nonpsychiatry experts, a national Delphi study with frame-of-reference training, and stringent inclusion criteria strengthen the content validity of the findings and may serve as a model for future efforts in other specialties.

J.Q. Young is professor, Department of Psychiatry, Zucker School of Medicine at Hofstra/Northwell, New York, New York.

C. Hasser is assistant professor, Department of Psychiatry, UCSF School of Medicine, San Francisco, California.

E.K. Hung is associate professor, Department of Psychiatry, UCSF School of Medicine, San Francisco, California.

M. Kusz is research assistant, Department of Psychiatry, Hofstra Northwell School of Medicine, New York, New York.

P.S. O’Sullivan is professor, Department of Medicine and Surgery, UCSF School of Medicine, San Francisco, California.

C. Stewart is assistant professor, Department of Psychiatry, Georgetown School of Medicine, Washington, DC.

A. Weiss is associate professor, Department of Psychiatry and Behavioral Sciences, Albert Einstein School of Medicine, New York, New York.

N. Williams is professor, Department of Psychiatry, University of Iowa Carver College of Medicine, Iowa City, Iowa.

Funding/Support: None reported.

Other disclosures: None reported.

Ethical approval: Reported as not applicable.

Supplemental digital content for this article is available at and

Correspondence should be addressed to John Q. Young, 75-59 263rd St., Kaufman 217A, Zucker Hillside Hospital, Glen Oaks, NY 11004; telephone: (718) 470-8005; e-mail:

Written work prepared by employees of the Federal Government as part of their official duties is, under the U.S. Copyright Act, a “work of the United States Government” for which copyright protection under Title 17 of the United States Code is not available. As such, copyright does not extend to the contributions of employees of the Federal Government.

© 2018 by the Association of American Medical Colleges