RIME

Comparing Diagnostic Performance and the Utility of Clinical Vignette-Based Assessment Under Testing Conditions Designed to Encourage Either Automatic or Analytic Thought

Ilgen, Jonathan S. MD, MCR; Bowen, Judith L. MD; McIntyre, Lucas A.; Banh, Kenny V. MD; Barnes, David MD; Coates, Wendy C. MD; Druck, Jeffrey MD; Fix, Megan L. MD; Rimple, Diane MD; Yarris, Lalena M. MD, MCR; Eva, Kevin W. PhD

Author Information
Academic Medicine 88(10):p 1545-1551, October 2013. | DOI: 10.1097/ACM.0b013e3182a31c1e

Abstract

Purpose 

Although decades of research have yielded considerable insight into physicians’ clinical reasoning processes, assessing these processes remains challenging; thus, the authors sought to compare diagnostic performance and the utility of clinical vignette-based assessment under testing conditions designed to encourage either automatic or analytic thought.

Method 

This 2011–2012 multicenter randomized study of 393 clinicians (medical students, postgraduate trainees, and faculty) measured diagnostic accuracy on clinical vignettes under two conditions: one encouraged participants to give their first impression (FI), and the other led participants through a directed search (DS) for the correct diagnosis. The authors compared accuracy, feasibility, reliability, and relation to United States Medical Licensing Exam (USMLE) scores under each condition.

Results 

A 2 (instructional condition) × 2 (vignette complexity) × 3 (experience level) analysis of variance revealed no difference in accuracy as a function of instructional condition (F[1,379] = 2.44, P = .12), but demonstrated the expected main effects of vignette complexity (F[1,379] = 965.2, P < .001) and experience (F[2,379] = 39.6, P < .001). Pearson correlations revealed greater associations between assessment scores and USMLE performance in the FI condition than in the DS condition (P < .001). Spearman–Brown calculations consistently indicated that alpha ≥ 0.75 could be achieved more efficiently under the FI condition relative to the DS condition.

Conclusions 

Instructions to trust one’s first impres-sions result in similar performance when compared with instructions to consider clinical information in a systematic fashion, but have greater utility when used for the purposes of assessment.

© 2013 by the Association of American Medical Colleges

You can read the full text of this article if you:

Access through Ovid