Visualization Techniques to Enhance the Explainability and Usability of Deep Learning Models in Glaucoma

Glaucoma, an irreversible cause of blindness worldwide, necessitates accurate and timely detection for efficacious treatment.^1–3 Deep learning (DL) models have recently become powerful tools in glaucoma research, utilizing intricate algorithms to extract valuable data from medical images.⁴ The application of DL models in glaucoma screening, diagnosis, and classification has yielded promising results.⁵ Nevertheless, these models present a key challenge: their opacity, or the “black box” phenomenon, which leads to a lack of clarity about the processes underlying final predictions. This opacity incites substantial concerns about the models’ interpretability and potential embedded biases, possibly affecting the reliability of the outcomes and, subsequently, the quality of patient care. Given these challenges, visualization techniques have emerged as an influential tool to augment the interpretability of these intricate DL models. These techniques yield a visual representation of the model’s decision-making process, empowering users to comprehend how inputs are converted into outputs. By visually mapping these transformations, it is feasible to discern the factors that the model deems most significant in generating predictions, potentially aiding in bias identification.

This issue of the Asia-Pacific Journal of Ophthalmology covers a narrative review by Gu et al⁶ that presents a comprehensive overview of the different types of data and visualization techniques that are used to illustrate predictions made by DL models. The article delves into novel strategies to improve the interpretation of predictions made by artificial intelligence (AI) models. These innovative approaches involve an alignment of the findings with the viewpoints of clinical practitioners. This aspect is critical, as understanding AI model predictions from a practitioner’s perspective ensures that these predictions are applicable and beneficial in a real-world clinical context. By incorporating the practical insights of clinicians, these strategies foster a more effective integration of AI technology in health care practice.

During the process of development, visualization techniques in imaging data fall into 3 primary categories: gradient-based, perturbation-based, and attention-based techniques. Of these, gradient-based techniques, such as saliency maps, gradient-weighted class activation mapping, integrated gradients, and layer-wise relevance propagation, are most frequently employed in ophthalmological studies for detecting and screening diabetic retinopathy and glaucoma.^7,8 Among the frequently used methods for scrutinizing visual cues utilized by deep neural networks, gradient-weighted class activation mapping and its derivatives have undergone successive enhancements, leading to increasingly high-resolution and efficient output maps. These advancements have resulted in improved detection and classification accuracy, thereby strengthening their ability to precisely pinpoint the extent of lesions.

Contrarily, perturbation-based methods modify or eliminate sections of the image using a blank sliding window or random masks to observe their influence on the algorithm.⁹ The most discriminative regions are subsequently identified based on the impact of these diverse regions on the overall accuracy. However, this method tends to be computationally intensive and has garnered scant attention in the field of glaucoma research. Attention-based methodologies produce attention maps by subduing irrelevant features during the training phase. Playout et al implemented this technique on the attribution map for the classification of retinal images.¹⁰ These attention-based strategies have been effective in recognizing relevant regions and minimizing irrelevant or redundant features that could affect the model’s accuracy. Progress in visualization techniques has allowed these methods to detect even subtle variations in the features contributing to the severity or progression of the disease. Such techniques play an indispensable role in interpreting the results of glaucoma diagnosis and assisting clinicians in understanding the pathophysiology of the disease.

Tabular data, such as the data from electronic health records, are also frequently used in developing DL models. Gu et al also discussed the strategies to improve the interpretability of clinical features from tabular data used to train explainable AI models.⁶ Among these, Local Interpretable Model-Agnostic Explanations (LIME)¹¹ provide key feature visualizations for a model’s glaucoma classification, increasing medical professionals’ trust, and the Submodular Pick Local Interpretable Model-Agnostic Explanation (SP-LIME)¹² explicates predictive results and glaucoma risk factors, facilitating clearer decision-making. Shapley's value from Cooperative Game Theory, used to quantify individual contributions, is another form of explainable AI. Implemented in the XGBoost algorithm and SHapley Additive exPlanations (SHAP),¹³ it elucidates the reasoning behind DL output. However, considering the scarcity of explainable AI research within the glaucoma domain, this field necessitates additional exploration.

During the process of clinical deployment, a user-friendly interface could further enhance the explainability of AI models and reduce the cognitive burden of the users. Gu et al further investigated several examples of published dashboards or interfaces with a focus on DL predictions to facilitate end-user engagement, such as the GLANCE interface.¹⁴ The relevant study showed that AI models that elucidate automated decisions can enhance clinicians’ understanding and align their trust in DL-based measurements during clinical decision-making. Specifically, clinicians altered their initial management choice and confidence in their predictions in 31% of the cases after reviewing the DL model’s results with a visual heatmap explanation. In contrast, without the aid of a heatmap, only 11% of the cases resulted in a change of opinion.¹⁴ The development and implementation of standards for various data types will be crucial in accelerating AI techniques in ophthalmology.

To summarize, DL holds significant promise to revolutionize glaucoma diagnosis and management through medical image analysis. However, deciphering the inner mechanisms of DL models remains a prerequisite for clinical adoption and incorporation into existing workflows. This issue underscores key visualization strategies under investigation that tackle critical issues of trust, reliability, and explainability. The ultimate success of DL in glaucoma care hinges on striking a suitable balance between performance and transparency. As research and technology advance, DL-assisted diagnostics and monitoring backed by interpretable DL models may play a pivotal role in individualizing glaucoma management. The insights from the review by Gu et al² can potentially guide the development of future AI tools that are not only powerful in their predictive capacities but also intuitive and user-friendly for the clinicians who will be utilizing them.