Objective: Currently, surgical skills assessment relies almost exclusively on subjective measures, which are susceptible to multiple biases. We investigate the use of eye metrics as an objective tool for assessment of surgical skill.
Summary Background Data: Eye tracking has helped elucidate relationships between eye movements, visual attention, and insight, all of which are employed during complex task performance (Kowler and Martins, Science. 1982;215:997–999; Tanenhaus et al, Science. 1995;268:1632–1634; Thomas and Lleras, Psychon Bull Rev. 2007;14:663–668; Thomas and Lleras, Cognition. 2009;111:168–174; Schriver et al, Hum Factors. 2008;50:864–878; Kahneman, Attention and Effort. 1973). Discovery of associations between characteristic eye movements and degree of cognitive effort have also enhanced our appreciation of the learning process.
Methods: Using linear discriminate analysis (LDA) and nonlinear neural network analyses (NNA) to classify surgeons into expert and nonexpert cohorts, we examine the relationship between complex eye and pupillary movements, collectively referred to as eye metrics, and surgical skill level.
Results: Twenty-one surgeons participated in the simulated and live surgical environments. In the simulated surgical setting, LDA and NNA were able to correctly classify surgeons as expert or nonexpert with 91.9% and 92.9% accuracy, respectively. In the live operating room setting, LDA and NNA were able to correctly classify surgeons as expert or nonexpert with 81.0% and 90.7% accuracy, respectively.
Conclusions: We demonstrate, in simulated and live-operating environments, that eye metrics can reliably distinguish nonexpert from expert surgeons. As current medical educators rely on subjective measures of surgical skill, eye metrics may serve as the basis for objective assessment in surgical education and credentialing in the future. Further development of this potential educational tool is warranted to assess its ability to both reliably classify larger groups of surgeons and follow progression of surgical skill during postgraduate training.
Currently, surgical skills assessment relies almost exclusively on subjective measures that are susceptible to multiple biases. We investigate the use of complex eye and pupillary movements, collectively referred to as eye metrics, as objective tools for assessment of surgical skill.
From the *The Smith Institute for Urology, The North Shore, Long Island Jewish Health System, New Hyde Park, NY; †Department of Urology, The University of Texas, Southwestern Medical Center, Dallas, TX; ‡Department of Psychology, San Diego State University, San Diego, CA; and §EyeTracking, Inc., San Diego, CA.
Supported (in part) by an Empire Clinical Research Investigators Program (ECRIPS) grant.
Sandra Marshall is CEO of Eyetracking, Inc, which participated in data analysis.
Eye metrics can reliably distinguish expert from non-expert surgeons and show promise as a unique and objective assessment tool in surgical education and credentialing.
Reprints: Lee Richstone and Michael J. Schwartz, The Smith Institute for Urology, 450 Lakeville Road, New Hyde Park, NY 11042. E-mail: LRichsto@nshs.edu and firstname.lastname@example.org.