Secondary Logo

Share this article on:

Transforming Resident Assessment

An Analysis Using Deming’s System of Profound Knowledge

Warm, Eric J., MD; Kinnear, Benjamin, MD; Kelleher, Matthew, MD, MEd; Sall, Dana, MD, MEd; Holmboe, Eric, MD

doi: 10.1097/ACM.0000000000002499
Article: PDF Only

W. Edwards Deming, in his System of Profound Knowledge, asserts that leaders who wish to transform a system should understand four essential elements: appreciation for a system, theory of knowledge, knowledge about variation, and psychology. The Accreditation Council for Graduate Medical Education (ACGME) introduced the milestones program as a part of the Next Accreditation System to create developmental language for the six core competencies and facilitate programmatic assessment within graduate medical education systems. Viewed through Deming’s lens, the ACGME can be seen as the steward of a large system, with everyone who provides assessment data as workers in that system. The authors use Deming’s framework to illustrate the working components of the assessment system of the University of Cincinnati College of Medicine’s internal medicine residency program and draw parallels to the macrocosm of graduate medical education. Successes and failures in transforming resident assessment can be understood and predicted by identifying the system and its aims, turning information into knowledge, developing an understanding of variation, and appreciating the psychology of motivation of participants. The authors offer insights from their experience for educational leaders who wish to apply Deming’s elements to their own assessment systems, with questions to explore, pitfalls to avoid, and practical approaches in doing this type of work.

E.J. Warm is professor of medicine and program director, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio; ORCID: https://orcid.org/0000-0002-6088-2434.

B. Kinnear is assistant professor of medicine and pediatrics, and associate program director, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio.

M. Kelleher is assistant professor of medicine and pediatrics, and associate program director, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio.

D. Sall is assistant professor of medicine, and associate program director, Department of Internal Medicine, University of Cincinnati College of Medicine, Cincinnati, Ohio.

E. Holmboe is senior vice president, Milestones Development and Evaluation, Accreditation Council for Graduate Medical Education, Chicago, Illinois; adjunct professor of medicine, Yale University, New Haven, Connecticut; and adjunct professor, Feinberg School of Medicine at Northwestern University, Chicago, Illinois.

Funding/Support: None reported.

Other disclosures: None reported.

Ethical approval: The use of milestones data was approved by the Institutional Review Board of the University of Cincinnati, Study # 2014-2042.

Previous presentations: These concepts were presented as part of a mini-course at the Accreditation Council for Graduate Medical Education meeting, Orlando, Florida, March 11, 2017.

Correspondence should be addressed to Eric J. Warm, University of Cincinnati College of Medicine, 231 Albert Sabin Way M.L. 0557, Cincinnati, OH 45267-0557; email: warmej@ucmail.uc.edu; Twitter @CincyIM.

© 2018 by the Association of American Medical Colleges