Letters to the Editor
To the Editor:
Ilgen and colleagues1 recently compared the effects on diagnostic accuracy of automatic and analytic reasoning, the latter induced by instructions described as similar to those that we have employed in our research. In contrast with our studies (e.g., Mamede et al, 20102) and their own previous findings,1 analytic reasoning did not improve performance. We believe the instructions laid out in our previous work2 and Ilgen and colleagues’ application of them in their study substantially differ.
Reflective reasoning involves, in our view, critically scrutinizing the initial diagnostic impression of a problem. This usually requires physicians to escape from the point of view created by their first hypothesis and look at the case from a different perspective. Our reflection instructions therefore request physicians to first formulate one diagnostic hypothesis and subsequently identify case findings that speak in favor of and against that hypothesis. Only after searching for contradictory evidence are physicians asked to generate alternative diagnoses. These alternative hypotheses triggered by the contradictory evidence, in turn, support the restructuring of initial ideas. In Ilgen and colleagues’ study, however, physicians wrote a summary representation of the problem and formulated possible diagnoses before listing features inconsistent with each diagnosis. Consequently, generation of alternative diagnoses is guided by the initial representation of the problem rather than by contradictory evidence, which may have been less powerful for redirecting reasoning. Whereas we advocate an inductive, bottom-up approach (contradictory findings lead to alternative hypotheses), Ilgen and colleagues adopted a deductive, top-down one (alternative hypotheses possibly lead to contradictory findings).
Further, Ilgen and colleagues’ study was Web based, which allows for large samples but hinders control over what participants actually do. We noted that their participants needed almost twice the amount of time to reach a diagnosis compared with studies using the same or similar cases.2,3 Therefore, one cannot exclude the possibility that their participants engaged in off-task behaviors that may have contributed to experimental error, despite their large N value. Studies of thinking and reasoning usually require tight control over the way materials are processed.
Ilgen and colleagues concluded that their results contradict the assertion that analytical reasoning is likely to reduce diagnostic errors. We raise the possibility that the results are due to conceptual and methodological shortcomings and hope this letter helps to focus our efforts to better understand under which conditions doctors can minimize diagnostic errors.
Sílvia Mamede, MD, PhD
Associate professor, Institute of Medical Education Research Rotterdam, Erasmus MC, and associate professor, Department of Psychology, Erasmus University, Rotterdam, The Netherlands; firstname.lastname@example.org.
Henk G. Schmidt, PhD
Professor, Department of Psychology, Erasmus University, Rotterdam, The Netherlands.
1. Ilgen JS, Bowen JL, McIntyre LA, et al. Comparing diagnostic performance and the utility of clinical vignette-based assessment under testing conditions designed to encourage either automatic or analytical thought. Acad Med. 2013;88:1541–1551
2. Mamede S, van Gog T, van den Berge K, et al. Effect of availability bias and reflective reasoning on diagnostic accuracy among internal medicine residents. JAMA. 2010;304:1198–1203
3. Sherbino JS, Dore KL, Wood TJ, et al. The relationship between response time and diagnostic accuracy. Acad Med. 2012;87:785–791