Review of Visualization Approaches in Deep Learning Models of Glaucoma

Abstract

Glaucoma is a major cause of irreversible blindness worldwide. As glaucoma often presents without symptoms, early detection and intervention are important in delaying progression. Deep learning (DL) has emerged as a rapidly advancing tool to help achieve these objectives. In this narrative review, data types and visualization approaches for presenting model predictions, including models based on tabular data, functional data, and/or structural data, are summarized, and the importance of data source diversity for improving the utility and generalizability of DL models is explored. Examples of innovative approaches to understanding predictions of artificial intelligence (AI) models and alignment with clinicians are provided. In addition, methods to enhance the interpretability of clinical features from tabular data used to train AI models are investigated. Examples of published DL models that include interfaces to facilitate end-user engagement and minimize cognitive and time burdens are highlighted. The stages of integrating AI models into existing clinical workflows are reviewed, and challenges are discussed. Reviewing these approaches may help inform the generation of user-friendly interfaces that are successfully integrated into clinical information systems. This review details key principles regarding visualization approaches in DL models of glaucoma. The articles reviewed here focused on usability, explainability, and promotion of clinician trust to encourage wider adoption for clinical use. These studies demonstrate important progress in addressing visualization and explainability issues required for successful real-world implementation of DL models in glaucoma.

INTRODUCTION

Early intervention in glaucoma, a major cause of irreversible blindness, may prevent vision loss.¹ A multitude of novel ideas for screening, diagnosing, and detecting changes over time in glaucoma have been proposed to facilitate its early detection and ameliorate its progression. Artificial intelligence (AI) has emerged as a rapidly advancing tool to achieve these objectives.²

Ophthalmology, in particular, has become a prime area for applications of AI and deep learning (DL),³ with many studies using DL models to address glaucoma.^2,4–13 Although these models have demonstrated encouraging results, several challenges remain.⁷ One key challenge is that models often appear as “black boxes” to clinicians or other users. This means that while a model may provide accurate predictions, it may not necessarily provide an explanation or an intuitive understanding to users about how those predictions were made.^14,15 This can be an obstacle that causes a cognitive burden or hesitation in use among clinicians.¹⁶

Recently, visualization approaches have been developed to better understand DL models and enhance their interpretability.¹⁷ These visualization techniques may improve the cognitive processing of data effectively and efficiently.¹⁸ Furthermore, model interpretability allows users to understand the system and, therefore, engenders trust in the model output.¹⁹ User-facing dashboards are another strategy developed to help clinicians quickly review large amounts of data. These are data-driven clinical decision support (CDS) tools that can execute queries across multiple databases and allow the review of many clinically relevant indicators visually in a single report.²⁰ The utility of dashboards comes from their ability to provide a compact overview of important information.²⁰

Although many AI and DL models for glaucoma have been developed, advancing these models to real-world clinical translation requires investigation of visualization approaches and user interface (UI) considerations for model outputs. In this narrative review, we summarize current publications regarding visualization approaches and UI considerations in relation to DL models of glaucoma, with a focus on approaches aimed toward enhancing interpretability, trust, and usability.

METHODS

Eligibility Criteria and Information Sources

Peer-reviewed journal articles published in English between August 1, 2013 and August 1, 2022, with available full text were reviewed. The rationale for this timeframe was to review literature from the most recent 10 years preceding the time the review was initiated, particularly in light of the increasing integration of DL in the medical field since the early 2010s.²¹

An initial search of PubMed, Cochrane, and Google Scholar was conducted in July 2022, and the search was repeated in August 2022 to evaluate any additional articles before collating the observations for this review. Details of the search strategy and study selection are provided in Supplementary Digital Content 1 (https://links.lww.com/APJO/A250).

RESULTS AND DISCUSSION

Using the above search strategy, we identified 18 primary research articles that described DL models for glaucoma. We also included an element of visualization of model results and/or discussion of approaches to enhance the interpretability of the models. These visualization approaches may help inform the design of future UIs for real-world glaucoma management. Table 1 provides a summary of findings from the articles.

TABLE 1 - Summary of Studies Using Visualization Approaches in DL Models of Glaucoma

References	Network	Country of Data Sets/Single or Multicenter	Input Data Type	Input Image	Output	Built Dashboard or Interface	Visualization Method (Designed Way)	Purpose
Li et al22	AG-CNN	China/Multicenter	FP	ONH and it around RNFL	Glaucoma vs nonglaucoma (binary)	No	Heatmap	Detection
Christopher et al23	CNN	US/Multicenter	FP	ONH-focused	Softmax probability of GON	No	Heatmap	Detection
Kucur et al24	CNN	Hungary/Single-center	VF	Voronoi image converted from VF	Values in the maps range from 0 to 1*	No	Voronoi image	Detection
Liu et al25	GD-CNN (based on the ResNet)	China, US/Multicenter	FP	ONH-focused	Glaucoma vs nonglaucoma (binary)	No	Heatmap	Detection
Ran et al26	ResNet	China, Hong Kong/ Multicenter	OCT	OCT (3D volume, 2D enface)	Glaucoma vs nonglaucoma (binary)	No	Heatmap	Detection
Ajitha S et al27	CNN	India/Multicenter	FP	Whole FP image	Glaucoma vs nonglaucoma (binary)	No	Feature maps	Detection
Oh et al28	CNN, Shapley value	Korea/Single-center	VF, RNFL, OCT, IOP, FP	Whole FP image	Glaucoma vs nonglaucoma (binary), estimating global (VFI) and its degree of certainty	Yes	Gauge chart, Radar chart, and SHAP chart	Detection
Dixit et al29	The convolutional LSTM model	US/Single-center	VF	VF represented as an 8×9 grid	Progression or not	No	Heatmap	Progression
Christopher et al15	ResNet	US, Tokyo/Multicenter	FP	ONH-focused	Glaucoma vs nonglaucoma (binary)	No	Heatmap	Detection
Yu et al30	Adam RMSprop with Nesterov momentum	US/Single-center	OCT	OCT (MAC, ONH)	Estimating “global VFI”	No	Heatmap	Progression†
Li et al31	EfficientNet-B0	China/Multicenter	FP	Whole FP image	Glaucoma vs nonglaucoma (binary) + progression	No	Saliency maps	Detection and progression
Huang et al32	ResNet	US, China/Multicenter	OCT, VF, FP	Voronoi image converted from VF	Grading (5 grade)	Yes	Developed interface (FGGDL)	Detection and grading (progression)
Maetschke et al33	CNN	US/Single-center	OCT	OCT (enface, side)	Glaucoma vs nonglaucoma (binary)	No	Class activation maps	Detection
van den Brandt et al17	ResNet50	US/Multicenter	OCT	OCT (B scan, RNFL, and GCIPL thickness maps)	Predicted mean deviation	Yes	Heatmap	Detection
Baxter et al34	Random Forests	US/Multicenter	Clinical EHR data	not applicable	Need for glaucoma surgery as a proxy for progression	No	Gini variables of importance	Progression
Christopher et al35	ResNet50	US/Multicenter	OCT, VF	OCT (RNFL thickness map, RNFL enface image, CSLO image)	GVFD vs non-GVFD	No	Heatmap	Detection
Kamal et al36	SNN, ANFISs	Kaggle Data Set (Google) /Public	FP, clinical records	Whole FP image	Glaucoma vs nonglaucoma (binary) by pixel density analysis	No	SP-LIME	Detection
Chayan et al37	CNN, fully connected neural network	US/Single-center	FP	Whole FP image	Glaucoma vs nonglaucoma (binary)	No	LIME explanation	Detection

AG-CNN indicates attention-based convolutional neural network for glaucoma detection; ANFIS, adaptive neuro-fuzzy inference system; CNN, convolutional neural network; CSLO, confocal scanning laser ophthalmoscopy; DL, deep learning; EHR, electronic health record; FGGDL, fine-grained grading deep learning system; FP, fundus photography; GCIPL, ganglion cell-inner plexiform layer; GD-CNN, glaucoma diagnosis with CNN; GON, glaucomatous optic neuropathy; GVFD, glaucomatous visual field damage; IOP, intraocular pressure; LSTM, long short-term memory; MAC, macula; N/A, not applicable; OCT, optical coherence tomography; ONH, optic nerve head; RNFL, retinal nerve fiber layer; SHAP, SHapley Additive exPlanations; SP-LIME, submodular pick local interpretable model-agnostic explanation; SNN, spike neural network; VF, visual field test; VFI, visual field indices.

^*Values in the maps range from 0 to 1, where 0 indicates the pixel or region has no impact on the CNN decision whereas 1 indicates a region with maximal importance.

^†Technically, they investigated global visual field indices, visual field index, and mean deviation.

Data Types and Visualization Approaches for Presenting Model Predictions

The included articles represent a wide range of data types used for training DL models of glaucoma. Evaluation of imaging data plays a prominent role in the clinical management of glaucoma, so unsurprisingly many of the DL models that have been developed for glaucoma entail some components of automated image analysis. Images are amenable to saliency methods for visualization. Saliency methods are tools that highlight features in an input that are relevant for generating a prediction.^38–40 For example, a “heatmap” (saliency map or activation map) can depict regions of an image that highly influence a model’s decision-making process.^7,17 Nearly half (8/18, 44%) of the review articles in this study introduced a visualization approach that allowed superimposition on a single fundus photograph.^{15,22,23,25,27,31} These results are not surprising given the widespread use of fundus photography in ophthalmology due to its availability, affordability, and ease of use.^41,42 Other tools used by researchers included photography encompassing the optic nerve head with the surrounding fundus (Fig. 1A)^15,22,23 and whole fundus photography to detect the retinal nerve fiber layer (RNFL, Fig. 1B).^25,27,31 Some studies used optical coherence tomography (OCT), which is useful in 3-dimensional visualization, to build heatmaps that evaluate the performance of DL models (Fig. 1C).^{17,26,30,33,35}

FIGURE 1: Heatmaps for several examples of DL models. These heatmaps highlight the areas, which are the most relevant for analysis by DL. In section (A), bright pink regions indicate a large impact on model predictions. The heatmaps are shown overlaid on a representative healthy (left) and GON (right) eye (adapted from Christopher et al²³). B, In the same way, a heatmap is presented from a DL model, which can be superimposed on the input image to highlight the areas of the model important for diagnosis. In this example, note that the heatmap strongly highlights the RNFL defect (adapted from Liu et al²⁵). C, Representative examples of heatmaps in enface view, highlighting retinal regions strongly contributing to the macular and ONH regions estimates of VFI. Upper photos show healthy eyes (VFI = 99), whereas lower photos show glaucomatous eyes (VFI = 33), and primary open angle glaucoma (adapted from Yu et al³⁰). DL indicates deep learning; GON, glaucomatous optic neuropathy; ONH, optic nerve head; RNFL, retinal nerve fiber layer; VFI, visual field index. Adaptations are themselves works protected by copyright. So in order to publish this adaptation, authorization must be obtained both from the owner of the copyright in the original work and from the owner of copyright in the translation or adaptation.

Researchers have also explored DL models that incorporate functional modalities, such as visual field (VF) tests.^28,32 To distinguish healthy and early glaucomatous VF for DL analysis, 2 studies introduced the “Voronoi images” concept, which transformed VF into 2-dimensional images.^24,32 In addition, some studies provided heatmaps, highlighting which VF regions contributed to DL analysis.^24,29 Consideration of both the structural and functional aspects can enhance the discriminatory power of DL models and lead to a more comprehensive evaluation of glaucoma.²⁴ In addition, supplementation of images with clinical data is associated with an improvement in the performance of DL models.^43,44 Dixit et al²⁹ generated a model that integrated both VF and clinical data and used a heatmap to understand the extent, to which various points in the VF contributed to the DL. This model demonstrated a superior ability to detect glaucoma progression compared with a model based exclusively on VF data.

Importance of Data Source Diversity and Data Standardization to Improve the Utility and Generalizability of Deep Learning Models

In addition to the supplementary use of structural and clinical data, multicenter evaluation and inclusion of patients with diverse demographic characteristics are important in improving the utility and generalizability of DL models.^5,10,45 As shown in Table 1, a majority of the studies (11/18, 61%) obtained their data from multicenter sources. Single-center studies often have limited external validity, with decreased model performance when applied to populations with different characteristics than the training data.^46,47 Studies that use multicenter input data tend to have less bias, improved generalizability, and thus enhanced potential for clinical implementation.⁴⁷ Futoma et al⁴⁸ argued, however, that an emphasis on generalizability results in machine learning systems that have poor performance at several sites at the expense of systems that have robust performance at a single site. They suggest shifting the focus from broad applicability to gaining a better understanding of how and why certain machine learning systems work.

A lack of standardization is a significant barrier to the development of robust AI models with generalizability across different patient populations. Although these models have shown powerful diagnostic performance,^49,50 challenges in applying them to clinical practice include (1) a lack of large-image data sets from multiple devices, (2) nonstandardized imaging and/or postprocessing protocols between devices, (3) limited graphics processing unit capabilities, and (4) inconsistency in reporting metrics.⁵¹ Proposed solutions include standardizing images from various imaging platforms and training models with large data sets consisting of annotated real-world data, a wide range of image quality, and different types of imaging data.^45,51,52 The call for data standardization in the ophthalmology community across multiple data modalities has become more prominent in recent years^53–55 and broad-based efforts, such as through DICOM Working Group 9, the American Academy of Ophthalmology, the National Eye Institute, and the Observation Health Data Sciences and Informatics organization, are ongoing to advance standardization and enable improved interoperability and data harmonization across different data sources.

Design Features to Enhance Artificial Intelligence Explainability and Trustworthiness

A key challenge to the implementation of AI models is engendering trust among clinicians. Clinicians will often trust models to a greater extent if the decision-making underlying the model predictions mimics human clinical decision-making processes.^16,28 The examples below illustrate some innovative approaches to understanding predictions of AI models and alignment with clinicians, beyond heatmaps alone.

Li et al²² proposed a human attention-based DL model (convolutional neural network) for glaucoma detection and pathologic area localization. This stemmed from the idea that glaucoma is correctly detected when heatmaps correspond with attention maps used by ophthalmologists in glaucoma detection.²² The model was based on blurred regions of fundus images that ophthalmologists manually cleared to diagnose glaucoma.²² Their model implemented an attention prediction subnet that located the salient areas and extracted unessential features of fundus images that did not play a role in DL recognition (Fig. 2). Similarly, clinician eye-tracking data and their generated heatmaps have contributed to explainable AI models with applications to both ophthalmology and radiology.^56,57 The incorporation of a human-attention mechanism may not only enhance the performance of models, but may also provide insight into the value of attention-based methods for identifying pathologic areas, increasing reliability, and building trustworthiness among users.

FIGURE 2: Attention maps used by ophthalmologists to detect glaucoma and heatmaps used for a human attention-based DL model. Li et al²² reported that glaucoma was detected correctly by an attention-based convolutional neural network when the highlighted areas visualized by the heatmaps were consistent with the pathologic areas used by ophthalmologists (adapted from Li et al²²). DL indicates deep learning. Adaptations are themselves works protected by copyright. So in order to publish this adaptation, authorization must be obtained both from the owner of the copyright in the original work and from the owner of copyright in the translation or adaptation.

In a study conducted by Christopher et al,²³ DL models identifying glaucomatous optic neuropathy were evaluated. They determined that neuroretinal rim areas, specifically the inferior rim and superior rim, were more important for the models’ decisions than other peripheral lesions (Fig. 1A).²³ Given that these regions correspond to those used by clinicians to diagnose glaucomatous optic neuropathy,²³ clinicians may have increased trust in the ability of DL to model their clinical reasoning. As a result, clinicians may have enhanced confidence in their capacity for decision support and adoption in clinical practice.¹⁶

Recently, a new data-efficient image transformer algorithm has been proposed as an alternative approach, generating AI models with greater generalizability than ResNet and superior explainability compared with saliency maps.⁵⁸ Efforts to explain models’ decision-making processes will likely continue to evolve and provide developers and validators with areas to focus on for improving performance.

The integration of DL in clinical practice can facilitate more individualized glaucoma care for patients.⁵⁹ Specifically, DL can help clinicians depend less on VF tests that often are highly variable, and instead, determine the frequency of individualized VF tests based on the predicted VF loss from OCT scans.³⁵

Explainable Artificial Intelligence for Tabular Data

In addition to imaging-based approaches, there are also methods to enhance the interpretability of clinical features from tabular data used to train AI models.⁶⁰ Guidotti et al⁶¹ described the black box issue, which conceals the internal logic of AI models to users; this is both a practical and ethical issue. They detailed issues with model explanation, model inspection, and outcome explanation. An innovative model-agnostic technique, local interpretable model-agnostic explanations (LIME), was introduced to provide medical professionals with a visual representation of key features considered in the model’s classification of glaucoma.¹⁹ Chayan et al³⁷ proposed that providing explainability through LIME would provide medical professionals with comprehensive information to make decisions and thus would build their trust in the DL model. Kamal et al³⁶ proposed another model, submodular pick LIME (SP-LIME), that explained the predictive results and associated risk factors for the determination of glaucoma class. They claimed their model allowed clinicians to better understand the decision-making process and obtain convincing and consistent decisions.³⁶

Another form of explainable AI, the Shapley value, is a formula derived from a novel solution concept, Cooperative Game Theory.⁶² It takes into account all the contributions other players make when interacting with a player and is considered the standard for quantifying an instance’s contribution.⁶³ Oh et al²⁸ built their model and charts using the XGboost algorithm and SHapley Additive exPlanations, which is a variant of Shapley value. These statistical charts provide insight into why the DL output produced a certain result. Decision trees are an additional technique that can help users interpret model explanations.⁶⁴ Tree ensembles, such as random forests, are combinations of decision trees that create excellent predictive execution compared with a single decision tree.⁶³ This approach was used by Baxter et al³⁴ to assess variables of importance driving predictions generated by a random forest model for predicting glaucoma progression using systemic data in electronic health records (EHRs). Given the limited explainable AI studies conducted in the glaucoma field, this area warrants further exploration.

Dashboards/Interfaces Focused on Explaining Deep Learning Measurements

Here, several examples of published dashboards or interfaces with a focus on DL predictions to facilitate end-user engagement are provided. These illustrate various approaches to interface development to enhance the use and adoption of DL models for glaucoma.

Example 1: “Understandable prediction model”

Oh et al²⁸ proposed a DL interface for glaucoma prediction and provided explanations for each individual prediction. Their aim was to build an “understandable prediction model” rather than a “highly accurate model”. Several properties, including a VF (with pattern SD), RNFL OCT (with superior, inferior, and temporal), and intraocular pressure test, were analyzed for glaucoma prediction.²⁸ When users input these properties in the prediction model, they obtain a binary output as “glaucoma” or “healthy”. The interface includes a gauge chart that shows the value of the superior RNFL quadrant in a distribution of training data.²⁸ Gauge and radar charts display the position of input values among the overall distribution of values, while the SHapley Additive exPlanation chart displays the role of each value in the decision. The interface also includes a radar chart, which provides a visualization of multivariate data in the form of a 2-dimensional chart of 3 or more quantitative variables²⁸ (Fig. 3). Their model shows an accuracy of 0.95 and an area under the curve of 0.95, allowing users to obtain the basis for a glaucoma prediction and providing clinical insight to users.²⁸

FIGURE 3: DL interface implemented by Oh et al²⁸. Oh and colleagues implemented their prediction model and 3 explanation charts [gauge (A) and radar and SHAP charts (B)] in their interface for glaucoma prediction (adapted from Oh et al²⁸). DL indicates deep learning; SHAP, SHapley Additive exPlanation. Adaptations are themselves works protected by copyright. So in order to publish this adaptation, authorization must be obtained both from the owner of the copyright in the original work and from the owner of copyright in the translation or adaptation.

Example 2: “Intuitive AI for glaucoma”

Huang et al³² developed an interactive glaucoma grading interface with DL-based recommendations. They emphasized that the relationship between structure and function is a priority for proper patient evaluation and developed a fine-grained grading DL system (FGGDL), which integrates these 2 facets to introduce a more unified patient evaluation.³² The system converts VF data to Voronoi images and derives a grading system from the classification of VF defects and saliency maps. The vision loss in VFs corresponds to maps of the structural damage in the optic nerves of the fundus photographs.³² They reported that their objective model accomplished an accuracy of 0.85 and an area under the curve of 0.90, showing superior results compared with medical students (accuracy = 0.56–0.73, respectively, P < 0.01, at 95% CI) and comparable to ophthalmologists (accuracy = 0.87, P = 0.61, at 95% CI). They then determined that using FGGDL by clinicians improved performance compared with assessment without FGGDL help.³² As a result, the study concluded that the FGGDL could potentially provide productive guidance to the clinical setting, demonstrating a tool that corresponds with users’ intuitive methods for detecting the progression of glaucoma.

Example 3: “Clinical dashboard for visualizing AI predictions”

In a prior study conducted by our group,¹⁷ we proposed a visualization approach that incorporated a VF prediction model into a multifaceted UI to provide CDS in managing glaucoma progression. Interface development emphasized the interpretability of explanations and the reliability of predictions.¹⁷

This interface (GLANCE, a visualization tool designed to help clinicians make DL-based glaucoma progression management decisions efficiently, ie, at a “glance”) is designed to allow clinicians to select patients, examine demographics, and display OCT and derived data (such as thickness maps) all in one view. This is similar to the existing CDS system for OCT-based assessment in glaucoma. New features in this interface include the DL-generated Mean deviation (MD) prediction that informs the clinician of expected VF loss and the visual descriptions corresponding to these predictions.¹⁷ Historical VF MD values (alongside DL-generated predictions) also provide an assessment of model reliability to clinicians. Clinical decision-making was based on predicted MD, as opposed to real VF MD. In most cases (54%), clinicians made a decision on management that was based on the predicted MD.¹⁷ In particular, in 31% of the cases that changed their recommendation, clinicians had changed their first choice of the management and their reliability for the prediction only after going over the data with a visual explanation (heatmap) of the DL model’s results, as compared with only 11% of the cases where they changed their opinion without a heatmap.¹⁷ This provides evidence that models that reinforce explanations of automated decisions can augment clinicians’ knowledge and calibrate their trust in DL-based measurements during clinical decision-making.¹⁷

Decision Support to Minimize Time and Cognitive Burden on Users

Even if an interface based on DL had good performance, detailed explanations, and high clinical trust, users may be hesitant to use it in clinical practice if it imposed a cognitive burden or a substantial time burden. Read-Brown et al⁶⁵ described the concerns regarding the time burden and the negative impact on productivity associated with using EHR systems. Studies demonstrate that the mean time for clinician use of EHR systems per appointment was around 10 minutes, with time in clinical data review only comprising 1 minute.^65,66 In a survey study, all respondents who were ophthalmologists considered time and documentation a burden.⁶⁶ In time-sensitive clinical settings, an easy-to-use interface was the most important CDS characteristic to clinicians.⁶⁷ The “System Usability Scale (SUS)” and the “Post-Study System Usability Questionnaire” are useful tools for assessing the usability of software interfaces.⁶⁸ The Post-Study System Usability Questionnaire is a usability evaluation survey form based on scenarios that were developed by IBM.⁶⁹ It consists of 19 items focused on 5 usability characteristics of a system: rapid completion of the task, ease of learning, high-quality documentation and online information, functional adequacy, and rapid acquisition of productivity.⁷⁰ These instruments are a reliable way to evaluate user satisfaction with DL models and to assess the cognitive and time burden a model imposes.⁷⁰ Chen et al⁷¹ recently applied one of these tools, the SUS score, to calculate a commonly used and validated scoring system that ranges from 0 to 100 for evaluating the user-friendliness of the GLANCE interface. They found that while their interface was revealed to show mediocre usability (SUS score in the 43rd percentile, mean ± SD SUS score = 66.1±16.0), earlier work has shown that clinicians commonly have unfavorable perceptions of EHR usability when estimated using SUS scores (mean scores < 10th percentile). This highlights the challenges of developing usable CDS instruments in the EHR and the demand for continuous work in this area.⁷¹

Integration into clinical workflows is also essential to ensure that AI models are seamlessly incorporated into existing clinical information systems (such as existing EHR and picture archiving and communications systems), rather than residing in external systems that take additional time and effort for clinicians to access. This requires the adoption and implementation of data standards to enable clinical operability. For example, Fast Health Care Interoperability Resources comprises a data exchange standard that can be used stand-alone or in combination with other existing standards.⁵³ The need for innovation in AI from the EHR data perspective has led to the development of a “common data model” for big data storage and analytics.⁵³ One data model, the Observation Medical Outcomes Partnership Common Data Model, provides a standard for merging and unifying data from different EHR systems.^72,73 In the future, developing and implementing standards for various types of data will be essential for accelerating AI techniques in ophthalmology.⁵⁵

CONCLUSIONS

In summary, we have discussed key principles regarding visualization approaches in DL models of glaucoma: (1) using both imaging and clinical data to develop DL models of glaucoma; (2) promoting model interpretability and explainability to help engender clinician trust, particularly when important features for prediction align with traditional clinical decision-making processes; and (3) designing interfaces that minimize cognitive burden and can be successfully integrated with existing clinical information systems.

To contextualize this with a framework developed by the broader biomedical informatics community, the American Medical Informatics Association has recommended that the safe and effective use of AI in medicine includes the following stages: (1) achieving technical performance (stage 1); (2) evaluating usability and integration into clinical workflows (stage 2); and (3) assessing health impact (stage 3).⁷⁴ Although ophthalmology has seen a wealth of AI models published in recent years, most of these studies fall into the stage 1 category, as they are focused on technical performance and optimizing the performance metrics of the models themselves. However, to advance the field further to clinical translation, there is a substantial need to expand into stages 2 and 3 and place a greater emphasis on implementation science. AI-enabled diabetic retinopathy screening is one area where ophthalmology has advanced into clinical practice^75,76 and is demonstrating impact (stage 3) from the perspective of enhancing screening rates. AI implementation in glaucoma, however, has not yet reached that stage. The articles reviewed here represent cutting-edge innovation in stage 2 studies focused on usability. These studies demonstrate that important progress has been made in advancing toward real-world clinical implementation of AI models in glaucoma, but that much more work is required before these models can be implemented into clinical practice.