Deep Learning-Based Optical Coherence Tomography and Optical Coherence Tomography Angiography Image Analysis: An Updated Summary

Abstract

Deep learning (DL) is a subset of artificial intelligence based on deep neural networks. It has made remarkable breakthroughs in medical imaging, particularly for image classification and pattern recognition. In ophthalmology, there are rising interests in applying DL methods to analyze optical coherence tomography (OCT) and optical coherence tomography angiography (OCTA) images. Studies showed that OCT and OCTA image evaluation by DL algorithms achieved good performance for disease detection, prognosis prediction, and image quality control, suggesting that the incorporation of DL technology could potentially enhance the accuracy of disease evaluation and the efficiency of clinical workflow. However, substantial issues, such as small training sample size, data preprocessing standardization, model robustness, results explanation, and performance cross-validation, are yet to be tackled before deploying these DL models in real-time clinics. This review summarized recent studies on DL-based image analysis models for OCT and OCTA images and discussed the potential challenges of clinical deployment and future research directions.

Over the past two decades, optical coherence tomography (OCT) has become one of the most important imaging modalities in ophthalmology. It is a non-invasive technology that can generate in vivo structural images by detecting interference signals between the reflected signals from the reference mirror and the backscattering signals from biological tissues.¹ OCT visualizes structures of the eye with cross-sectional and three-dimensional (3D) volumetric scans objectively and quantitatively. The advances in OCT technology (ie, from time-domain OCT to spectral/Fourier-domain OCT and swept-source OCT) have largely improved the image quality, axial resolution, acquisition speed, and structural details. With retinal layered segmentation by image processing methods and cross-sectional morphology assessment, it offers quantitative thickness measurement and more detailed pathology evaluation, which provides a more accurate evaluation for retinal features compared with 2D fundus photographs.^2,3 Currently, OCT is widely used for retinal disease detection (eg, diabetic macular edema and age-related macular degeneration) and prognosis monitoring (eg, response to anti-vascular endothelial growth factor treatment).⁴ It has also been applied to assess glaucoma-related structural damage (eg, reduction of retinal nerve fiber layer thickness) in eye clinics soon after its demonstration,⁵ and progress were made in using it for glaucoma diagnosis and management.⁶

In addition to structural imaging, OCT has been explored and extended for “dynamic” imaging and en face imaging to map the retinal capillary network and choriocapillaris without the use of exogenous intravenous dye injection, based on the principle of mapping red blood cell movement over time by comparing sequential OCT sagittal-scans at a given cross-section, namely OCT angiography (OCTA). OCTA not only offers high-quality images of retinal and choroidal vessels at capillary-level but also allows quantification of vascular metrics that correlate with clinical disease severity and progression.⁷ Recently, quantitative OCTA metrics and features (eg, foveal avascular zone area, capillary network density, non-perfusion area) have been defined and explored for assessing retinal diseases such as age-related macular degeneration (AMD), diabetic retinopathy (DR), and retinal vein occlusion (RVO),^8,9 and glaucoma.¹⁰

Artificial intelligence (AI) has been explosively applied to a wide variety of medical fields in the last decade. Machine learning (ML) is a subset of AI that teaches a computer system to perform a task or predict an outcome without explicitly programmed.^11,12 In the development of traditional ML models, meaningful features are manually extracted by highly-trained domain expert. It is challenging to design and generalize such features, especially in medical images with variances in the anatomy and pathologies. On the contrary, deep learning (DL), under the term of ML, specifically uses multiple levels of classification with automatically extracted data features. Besides, with neural networks, DL models can handle data growth better than traditional ML models.^13,14 Convolutional neural network (CNN),¹⁵ one kind of neural networks, was proposed by Krizhevsky et al¹⁵ and gathered much attention for its high performance in image recognition tasks compared to conventional methods. Since then, many models for image recognition tasks have been developed based on DL techniques. Contrary to natural images that are RGB images, OCT/OCTA includes 2D grayscale images and 3D volumes. Besides, OCT/OCTA images often contain quantitative information that is useful to improve neural networks performance. The exact intensity of a pixel, the scale of abnormalities, and their locations in a scan can all be essential cues. Thus, when applying DL in OCT/OCTA images, changing the network architectures developed for natural images and taking differences into account can significantly improve the results and push for clinically viable products.¹⁶ In ophthalmology, an imaging-driven specialty, DL was particularly well-suited and promising for disease evaluation.^17–20 With the promising results from DL-based image analysis of 2D fundus photography,^21–23 efforts quickly expanded toward 3D OCT and OCTA image analysis, given its widespread adoption and integration into the routine management of retinal diseases and glaucoma. Advances in DL have significantly improved the state-of-the-art OCT and OCTA image analysis, including disease detection and prognosis prediction, retinal layers and avascular area segmentation, and image quality assessment. DL-based models can be trained with different kinds of input from conventional OCT, such as 2D en face images, 2D B-scans, and 3D volumetric scans, and OCTA images.^9,24

This review summarized recent studies on DL-based OCT and OCTA image analysis models and discussed the potential challenges of DL models’ clinical deployment and future research directions.

Deep Learning–Based Single Common Eye Disease Detection on OCT/OCTA

AMD is one of the leading causes of visual impairment in elderly patients. Drusen, pseudodrusen, neovascularization, fibrovascular scarring, and intraretinal or subretinal macular edema are typical retinal changes in AMD patients, which are detectable in OCT images.²⁵ Many studies have shown the promise of DL in AMD detection from OCT images. Treder et al²⁶ found that with a DL model developed by a pre-trained Inception-v3 network, it is possible to detect AMD in cross-sectional OCT B-scans with a sensitivity of 100%, a specificity of 92.0%, and an accuracy of 96.0%. As accurate differentiation of AMD subtypes is essential for treatment, Motozawa et al²⁷ proposed two DL-based models for differentiating not only AMD and normal, but also wet and dry AMD from OCT 2D B-scans, respectively. The proposed DL models achieved area under the receiver operating characteristics (AUROC) values of 0.995 and 0.991, with sensitivities of 100% and 98.4%, specificities of 98.8% and 91.1%, accuracies of 99.0% and 93.3%, respectively. In the computer science domain, advanced preprocessing methods were effective in improving DL model performance for OCT image analysis. Rong et al²⁸ proposed a surrogate-assisted CNN using different methods, including image denoising, thresholding, and morphological dilation for mask extraction, to generate surrogate images for CNN training. It achieved AUROC values of 0.978 and 0.986 for AMD detection in the internal validation and external testing datasets, respectively.

DR is a leading cause of blindness in the working-age population, and of an estimated 285 million people worldwide with diabetes mellitus (DM), more than one-third have signs of DR.²⁹ DL is at the forefront in DR assessment with OCT or OCTA images. Despite the application of traditional CNN models, advanced techniques were also used to refine the models. For example, Li et al³⁰ proposed a novel DL model called OCTD_Net based on modified DenseNet and ReLayNet. It classified early-stage DR (ie, mild DR), DM without DR, and normal subjects from OCT 2D B-scans, which achieved a sensitivity, specificity, and accuracy of 90.0%, 95.0%, and 92.0%, respectively. Heatmaps generated from their model demonstrated that patients with early-stage DR showed different features around the myoid and ellipsoid zones, inner nuclear layers, and photoreceptor outer segments. Le et al³¹ used a pre-trained visual geometry group (VGG-16) network to differentiate non-proliferative DR, DM without DR, and normal eyes from OCTA images. The overall AUROC, sensitivity, specificity, and accuracy were 0.965, 83.8%, 90.8%, and 87.3%, respectively. Heisler et al³² used ensemble learning techniques, which created and combined multiple models for performance improvement, to differentiate referable DR from non-referable DR on en face OCT and OCTA images. Though ensemble learning methods improved diagnostic performance, they also greatly increased the computational cost and training time.

RVO is the second most common retinal vascular disease after DR.³³ Studies showed that the foveal avascular zone (FAZ) and visual acuity are reportedly inversely correlated in RVO.³⁴ Thus, it is important to detect nonperfusion area (NPA) caused by RVO accurately. Nagasato et al³⁵ found that a DL model could distinguish RVO from normal based on NPA measurement from OCTA images with an AUROC value of 0.986, a sensitivity of 93.7%, and a specificity of 97.3%. More importantly, it took less average time for evaluation than ophthalmologists (176.9 s vs 700.6 s, P value > 0.01). As shown from their heatmaps, the DL model focused on the FAZ area in normal OCTA images and the FAZ area and NPA in RVO OCTA images.

Studies have demonstrated good performance by using DL to interpret OCT optic disc scans for discriminating glaucomatous eyes from normal eyes. For example, Thompson AC et al³⁶ developed a segmentation-free DL algorithm based on OCT 2D circular B-scans, and they found it achieved better performance for detecting glaucomatous structural changes comparing with conventional RNFL thickness parameters (AUROC: DL model vs RNFL thickness = 0.960 vs 0.870). Ran et al³⁷ developed and validated a 3D DL model with volumetric data and achieved comparable performance to 2 specialists with more than 10 years’ experience in glaucoma. The heatmaps generated by class activation map (CAM) showed that the regions with the most discriminative power for the 3D DL model to detect glaucomatous optic neuropathy (GON) were similar to what ophthalmologists usually observe in clinics. The DL-based method was also proven applicable to detect referable glaucoma from OCT volumetric macula scans. It achieved high diagnostic performance in both internal validation (AUROC = 0.880) and 2 external testing (AUROC = 0.780 and 0.950, respectively).³⁸

Deep Learning–Based Multiple Retinal Disease Detection on OCT/OCTA

The above DL algorithms were all proved to be feasible for detecting single eye disease or abnormality. However, DL models capable of detecting specific eye diseases as well as offering referral suggestions would be more applicable and valuable in clinics. Kermany et al³⁹ proved that DL-based OCT B-scan analysis could provide accurate referral suggestions (urgent vs. non-urgent referral) by differentiating images with choroidal neovascularization (CNV) and diabetic macular edema (DME) from images with drusen and normal. The model was developed with VGG-16 pre-trained on the ImageNet dataset and achieved an AUROC value of 0.999, a sensitivity of 97.8%, a specificity of 97.4%, and an accuracy of 96.6%. They also found that a “limited model” trained with only 1000 cross-sectional OCT images was not much inferior to the model trained with a larger dataset including 108,312 cross-sectional images (AUROC 0.988, sensitivity 96.6%, specificity 94.0%, and accuracy 93.4%). More importantly, they released the OCT dataset with ground truth labels for other researchers’ further investigation.^40–43 Li et al⁴⁰ fine-tuned the network with VGG-16 and achieved slightly better performance with an AUROC value of 1, a sensitivity of 97.8%, a specificity of 99.4%, and an accuracy of 98.6%. Another study proposed a multi-task CNN to discriminate five retinal status (ie, CNV, DME, AMD, drusen, and normal), which showed an overall sensitivity of 97.1%, a specificity of 99.3%, and an accuracy of 97.1%.⁴¹ Tsuji et al⁴² proposed a capsule network, which achieved a higher mean accuracy with 99.6% on the released dataset. Zhang et al⁴³ used a holistically-nested edge detection structure based on the VGG-16, which not only provided disease detection results but also extracted detection targets and enhanced data quality.

Another landmark study by De Fauw et al⁴⁴ proved the clinical applicability of DL technology on interpreting images with higher volume and complexity. It successfully addressed the challenges of detecting retinal diseases on 3D OCT volumetric data. The DL architectures first produced tissue segmentation and then classified the tissue-segmentation maps into different diseases with referral suggestions (ie, urgent, semi-urgent, routine, and observation only). This study showed that the classification given by the DL model was based on device-independent tissue segmentation maps and its accuracy was free from the influence of device types. The referrable suggestions performance was comparable to experts. It was a novel framework to analyse OCT scans for multiple retinal diseases and provide referral suggestions, which could potentially reduce the burden of retinal disease triage.

It is noteworthy that the aforementioned studies were based on supervised DL methods, which requires substantial efforts to label large amount of data manually. New DL techniques, such as weakly supervised learning, semi-supervised learning, and unsupervised learning⁴⁵ have great potential to deal with this issue. For example, Wang et al⁴⁶ developed a weakly supervised learning-based model called uncertainty-driven deep multiple instance learning (UD-MIL) to discriminate DME and AMD patients from normal subjects on OCT images, which achieved robust performance in both B-scan levels and volumetric-scan levels.

Deep Learning–Based Rare Retinal Disease Detection on OCT/OCTA

Despite the detection of the above prevalent retinal diseases, the advances in DL technologies for small datasets, such as few-shot learning (FSL),⁴⁷ also largely improved the feasibility of DL application to rare retinal disease detection. Yoo et al⁴⁸ proposed the FSL model using a technique called generative adversarial network (GAN),⁴⁹ which achieved an accuracy of 93.9% for nine kinds of retinal diseases classification. They proved that GAN was not only useful in improving the accuracy of rare retinal disease detection (ie, central serous chorioretinopathy, macular telangiectasia, macular hole, Stargardt disease, and retinitis pigmentosa) from OCT images, but also maintaining the diagnostic performance of prevalent retinal diseases detection (ie, DME, CNV, and drusen).

Deep Learning–Based Retinal Diseases Prognosis Prediction on OCT/OCTA

The introduction of anti-vascular endothelial growth factor (VEGF) has revolutionized the treatment of individuals with DME, CNV, and RVO. Although intravitreal anti-VEGF agents are the most common first line of therapy for these patients, not every patient responds to them. Besides, frequent injections of anti-VEGF agents are costly and burdensome.^50,51 Thus, precise treatment response prediction could potentially avoid unnecessary anti-VEGF treatment and reduce clinical burden. A pilot study⁵² found that a refined VGG-based DL algorithm could predict responsive patients from non-responsive ones with an AUROC of 0.866, and average sensitivity, specificity, and precision of 80.1%, 85.0%, and 85.5%, respectively. Although only OCT images from DME patients were used, this study proved the feasibility of DL applications on disease prognosis prediction.

The level of treatment requirement is also essential for making the clinical decision, such as whether and when to conduct anti-VEGF treatment. A predictive DL algorithm was developed on a longitudinal OCT dataset collected from 350 neovascular AMD (nAMD) patients with 2 years’ follow-up.⁵³ There were three kinds of predictive tasks: 1) treatment requirement score (RQS), 2) multi-class classification task including low, intermediate, and high requirement, 3) binary classification tasks (ie, low requirement vs. the remaining patients, high requirement vs. the remaining patients). The agreement between predicted RQS and the ground truth was moderate, but the model performed well for identifying low or high requirement patients. Although the predictive task was still in need of improvement, this study showed that DL was promising in disease prognosis prediction. A larger dataset with a longer follow-up time was warranted for further model refinement.

Deep Learning–Based Segmentation on OCT/OCTA

Macular fluids, such as the presence of intraretinal cystoid fluid (IRC) and subretinal fluid (SRF), can be used as qualitative OCT biomarkers to assist with treatment decision and prognosis evaluation. Schlegl et al⁵⁴ proposed a DL algorithm to conduct automated detection, segmentation, and quantification of macular fluids across exudative macular diseases (ie, AMD, DME, and RVO) from two commonly used OCT devices (ie, Cirrus spectral-domain OCT and Spectralis SDOCT). It achieved the mean AUROC values of 0.94 and 0.92 for IRC and SRF detection, respectively. Quantification of fluid was also highly concordant with manual expert assessment (ie, Pearson's correlation coefficient of 0.90 for IRC and 0.96 for SRF). This DL method was further proven to be applicable to quantify the fluid volumes after anti-VEGF therapy in another clinical trial,⁵⁵ demonstrating that the DL-based approach could offer a precise measurement of nAMD activity and allow identification of response patterns to improve therapeutic management of nAMD and avoid discrepancies between clinicians/investigators potentially.

Pekala et al⁵⁶ also proposed a DL model using DenseNet together with post-processing based on Gaussian processes for retinal layers segmentation on OCT B-scans from subjects with mild non-proliferative DR. The DL model performed on par with humans and other ML methods.

In recent years, studies found that the analysis of the choroid in the eye was crucial for understanding a range of ocular diseases and physiological processes. However, only a very limited number of commercial OCT instruments provide methods for automatic segmentation of choroidal tissue. DL methods could segment the Bruch's membrane and choroid layer on OCT B-scans and calculate the choroidal thickness map calculation with a lower error rate and higher dice similarity coefficient.⁵⁷ Kugelman et al⁵⁸ developed several supervised deep learning models, such as Cifar CNN, Complex CNN, RNN, and U-net, to detect the location of the choroidal boundaries of interest, ie, inner limiting membrane (ILM), retinal pigment epithelium (RPE), and choroid-scleral interface (CSI). Their results suggested that all methods exhibit similar accuracy and good performance on the retinal layer (ILM and RPE) whereas performance on the CSI showed more variability between methods. All these studies demonstrated the potential of DL-based technologies for clinical and research tasks involving OCT choroidal segmentation.

Retinal blood vessels are not only valuable for the evaluation of retinal diseases but also for systemic diseases including hypertension, stroke, and cardiovascular diseases, potentially. A DL algorithm developed by U-net achieved an accuracy of 86.8% for artery and vein segmentation with OCT and OCTA en face images as the input.⁵⁹ DL-based methods could also conduct microvascular segmentation on the SCP and DCP OCTA en face images with high accuracy and dice similarity coefficient.⁶⁰ DL-based segmentation method was also applied on measurement and segmentation of FAZ area that was proven to outperform the automated measurements of FAZ by the built-in software.^61,62

Deep Learning–Based Visual Function Prediction on OCT/OCTA

The above studies mainly focused on the detection or prediction of structural changes. Accurate assessments of visual functions, such as visual acuity (VA) and retinal sensitivity, were also important for clinical trial endpoints or treatment biomarkers evaluation. It could potentially be predicted from OCT measurements.⁶³ Studies showed VA was correlated with central retinal thickness derived from OCT,⁶⁴ whereas retinal sensitivity was significantly correlated with outer retinal thickness.⁶⁵ Advancements in DL can provide promising tools for visual function predictions from structural changes detected on OCT images. Kawczynki et al⁶⁶ proposed a DL-based regression model to predict the best-corrected visual acuity (BCVA) from OCT images of treatment-naïve nAMD patients. They used volumetric OCT scans as the input to predict the BCVA and classify poorer BCVA from better BCVA. The mean R² of all visits and all eyes was 0.79. The AUROC values of binary classification of BCVA of < 69 letters, < 59 letter, and ≤ 38 letters were 0.92, 0.95, 0.96 at baseline, and 0.87, 0.89, 0.89 at month 12, respectively. It demonstrated that DL-based OCT image analysis could help predict concurrent and future BCVA in patients with nAMD, which would potentially assist with visual function evaluation during treatment. Kihara et al⁶⁷ utilized a modified VGG-16 network to predict retinal sensitivity from OCT images of macular telangiectasia type 2 patients. They mapped the microperimetry sensitivities on OCT B-scans and generated en face sensitivity prediction maps. The correlation showed a high degree of agreement between predicted and real retinal sensitivities (Pearson correlation coefficient = 0.78).

Deep Learning–Based Image Quality Assessment of OCT/OCTA

Image quality control beforehand is essential for further accurate disease evaluation on OCT and OCTA images. Ran et al⁶⁸ developed a DL-based image quality assessment model to differentiate ungradable OCT optic disc volumetric scans from gradable ones. It showed good performance in both internal validation and 2 external testing datasets, which could potentially be incorporated into DL-based OCT analysis model for glaucoma detection. Kauer et al⁶⁹ proposed an automatic quality assessment network based on CNN to evaluate the quality of OCTA scans and the model achieved an accuracy of 99.5% to classify the image as good, bad, upper, and lower quality. Lauermann et al⁷⁰ developed a multilayer deep CNN for automated assessment of superficial capillary plexus OCTA image quality (sufficient vs. insufficient). The sensitivity, specificity, and accuracy of the DL model were higher than 90.0%. Other studies utilized DL to reconstruct⁷¹ or denoise⁷² OCTA images, which largely enhanced the image quality and would potentially increase the number of sufficient images, decrease the image interpretation time, and further benefit retinal disease detection.

In addition to general image quality assessment, the detection of a specific artifact is also valuable. Guo et al⁷³ had developed a CNN-based DL model to distinguish the capillary NPA from signal reduction artifacts in 6 × 6 mm² OCTA. The capillary NPA is a key quantifiable biomarker in the evaluation of DR using OCTA. The proposed DL model achieved good performance for NPA detection across a wide range of DR severity and scan quality. It can potentially address the problem of signal reduction artifacts in the detection and quantification of capillary dropout in the retina using OCTA.

DISCUSSION

In summary, DL-based models were able to assist diseases classification, prognosis monitoring, functional changes prediction, structural segmentation, and image quality control on OCT and OCTA images. The most commonly used networks were VGG-16, Inception, ResNet, RNN, DenseNet for classification, prediction, and image quality control tasks, whereas U-Net was used for segmentation task. Other techniques, such as transfer learning and few-shot learning were also used in several studies to deal with relatively small training sample size. Major studies were summarized in eTables 1–5 (https://links.lww.com/APJO/A86). Most of the studies were still in the stage of “proof-of-concept” and substantial issues are yet to be dealt with before deploying these DL models in real-time clinics.

First, the training sample size of the existing DL models for OCT or OCTA image analysis was relatively smaller than those for retinal fundus photographs. It potentially reduces the data diversities, results in model overfitting, and decreases the generalizability in unseen OCT/OCTA datasets. Different data augmentation methods, such as rotation, flipping, shifting, and cropping, are useful to enlarge the training sample size. Besides, new explorations in the computer science domain, such as transfer learning, low-shot learning,⁷⁴ or few-shot learning,⁴⁸ also show promises in dealing with this issue. Although current DL-based OCT and OCTA models can be further upgraded for rare diseases, researchers should be cautious when using advanced DL technologies such as GAN in medicine as the pseudo images could contain unobservable or unexplainable features, or vice versa, losing some features in the true images.⁷⁵

Second, OCT images are 3D volumetric data, containing a lot of noises and variances. It requires high-performance preprocessing methods, a series of operations on the initial images, to improve the image quality and produce standardized training input from different devices. Denoising, contrast enhancement, image normalization, segmentation were commonly used methods.⁷⁶ However, the preprocessing operations may increase the computation time and memories for dealing with 3D images, more explorations are still warranted to tackle to the dilemma. Wang et al's⁴⁶ study potentially reduced great efforts on sophisticated preprocessing as the weakly supervised learning model did not require preprocessing and was robust to handle speckle noise.

Third, most of the DL models for OCT/OCTA image analysis were usually trained and validated on an internal dataset without proper external testing. However, there are less available open-source datasets of OCT or OCTA than that of retinal fundus photographs,⁷⁷ and the data sharing among institutions is quite challenging due to issues of patient privacy and large storage requirement. “Model-to-Data” approach, by packaging an easily share DL model and bringing it to the data, was proven a potential solution to the limitation of data exchange among different institutions. It could allow for central construction of a robust DL model, followed by distribution to multiple clinical sites where distributed training can take place.⁷⁸ Federated learning (FL), a learning paradigm training algorithms collaboratively without exchanging the data itself, has the potential to address the problem of data governance and privacy. The DL model training occurs locally at each participating institution and only model characteristics (eg, parameters and gradients) are transferred. It enables gaining insights collaboratively, for example, in the form of a consensus model, without moving patient data beyond the firewalls of the institutions in which they reside.⁷⁹

Fourth, different OCT and OCTA devices produce different types of images from various imaging protocols. Most of the existing DL models were trained and validated on one kind of image from one particular OCT or OCTA device, which may have limited capability to generalize on other devices or other types of images. Romo-Bucheli et al⁸⁰ proposed unsupervised unpaired image translation models based on cycle consistency losses to deal with image variability across 2 kinds of OCT devices. These models improved the generalizability of segmentation models to other unseen OCT images from a different device. Future development to unify all types of OCT scans into one framework in a device-agnostic manner is needed, in addition to consistent upgrade and refinement of DL models.

Fifth, current DL models are relying on AI to detect things overlooked by human eyes, but the “black box” nature reduces their interpretability and reliability. Occlusion tests, activation maps, and salient maps were often used to generate heatmaps and to increase the transparency of DL models. Investigating the “black box” has now become known as explainable AI (XAI), which provides tools to reveal AI decisions. Maloca et al⁸¹ proposed a CNN for OCT segmentation task in monkey eyes, which was enhanced with a post hoc XAI technique called Traceable Relevance Explainability (T-REX). Post hoc XAI can highlight and visualize regions of the input data that lead to relevant prediction decisions after the neural network training process. In the future, more efforts are warranted to provide a real-time illustration of DL-based OCT and OCTA analysis results in clinics.

Sixth, although many studies claimed the non-inferior performance of proposed DL models to experienced ophthalmologists or retina specialists, the reports were not fully met with the guidelines^82–84 and not comparable among different studies. Thus, standardized randomized control trials (RCT) and result reporting are warranted to provide a more comprehensive insight into the clinical impact of these DL models. For RCT, instead of just comparing the performance between DL-based methods and humans, or emphasizing its superiority, the comparison should be made between the performance of decision-makers with and without the assistance of AI tools.⁸⁵

Seventh, current DL-based OCT/OCTA image analysis models are mainly aiming for one particular task. In clinics, it is ideal to have a full set and well-integrated DL system for image quality control, disease classification, triage, and prognosis prediction. Nevertheless, extra engineering is required for developing a user-friendly interface and incorporating different models. Besides, prospective validation in real-world settings with consistent upgrade and refinement is necessary to make sure the integrated DL-based OCT/OCTA image analysis system's performance is on par with human expertise.

Eighth, although some DL models performed well in internal validation, there are still present pertinent challenges for real-world application among different settings (ie, domain shift) due to the diversities in devices and imaging protocols, variances in ocular physiological anatomy, imbalance in the data distribution. Currently, domain adaptation techniques, such as synergistic fusion⁸⁶ and batch normalization,⁸⁷ are proposed to improve DL model robustness for medical image analysis.

Last but not least, this review mainly focuses on DL-based posterior OCT and OCTA analysis. However, there are other kinds of OCT in ophthalmology, such as anterior segment OCT,²⁴ and intra-operative OCT,⁸⁸ which are also with large potentials for the application of DL technologies. Besides, studies proved the feasibility of DL-based methods for systemic disease evaluations from fundus photographs.^89,90 DL-based OCT image analysis for systemic disease evaluation is yet to be explored. It would surely be a promising research area as some systemic diseases showed structural changes measured by OCT and OCTA.^91,92

CONCLUSIONS

DL-based OCT and OCTA image analysis models are showing their advantages in eye disease detection, feature segmentation, prognosis prediction, and image quality control. In the future, the explainability and transparency of DL models are paramount in final clinical deployment. Consistent upgrade and refinement of these models are necessary to make sure the models’ performance is on par with human expertise.

Literature Search

We searched databases of PubMed, Medline, Web of Science, Google Scholar, and Scopus for studies published in English up to January 31, 2021, using these keywords: “retinal diseases”, “macular diseases”, “glaucoma”, “optical coherence tomography/OCT”, “optical coherence tomography angiography/OCTA”, “image analysis”, “artificial intelligence/AI”, “deep learning/DL”, “deep neural network”, “convolutional neural network/CNN”, “deep learning techniques”, and “deep learning update”. The final reference list was generated based on relevance to the topics covered in this review.