The pace of population aging is increasing around the globe, and patients suffering from eye diseases are expected to increase at the same rate. Thus, early diagnosis and appropriate treatment of ophthalmic diseases are of great importance to prevent avoidable visual loss and improve quality of life. However, conventional ophthalmic diagnostic methods profoundly depend on physicians' experience and professional knowledge, which may lead to a high rate of misdiagnosis and waste a large amount of medical data. Therefore, deep integration of artificial intelligence (AI), machine learning (ML), and deep learning (DL) into ophthalmology has the potential to revolutionize the existing disease diagnosis system and create a significant clinical effect in ophthalmic health care service.1
The name of ML was first coined by Arthur Samuel in 1959, when he defined it as an application of AI that provides systems the ability to automatically learn and improve from experience without being explicitly programmed.2 On the other hand, AI is a technique that enables computer to mimic human behavior. It is divided into artificial narrow intelligence, artificial general intelligence, and artificial super intelligence. Of note, AI contains ML, DL, conventional machine learning (CML), natural language processing, computer vision, robotics, reasoning, general intelligence, expert system, automated learning, and scheduling.1,3
As mentioned, ML is a subfield of AI technology that systematically implements algorithms to synthesize the underlying interrelation between data and information.4 The scientific discipline, ML, is focusing on how computers learn from data; it is also an artificial computer intelligence system that allows computers to learn automatically without programming and without human intervention or assistance. In fact, with the advent of the internet, ML has become an important component of the information technology revolution affecting our daily lives in recent years. There are large numbers of successful applications of ML, such as medical practice, speech recognition, handwriting recognition, and machine translation. However, DL and CML are subfields of ML methods. Deep learning learns underlying features in data using neural networks. It usually focuses on data representation rather than task-specific algorithms; it can be supervised, semi-supervised, or unsupervised learning, making use of deep neural networks which are inspired by the structure and function of the human brain.5–7
Taking into consideration the current population growth trends and the limited availability of retina specialists and trained human graders, manual segmentation is time consuming, costly, prone to human errors and bias, and disadvantageous in a clinical ophthalmic health care service. Therefore, the ophthalmic health care system, in particular, needs an automatic, rapid, cost-effective, yet highly sensitive and specific method to detect diabetic retinopathy (DR), age-related macular degeneration (AMD), glaucoma, and other ophthalmic disorders. Unfortunately, many patients lose their vision unnecessarily due to late diagnosis.8–10
Ophthalmology, in comparison with other medical specialties, lends itself well to the implementation of AI-, ML-, and DL-assisted automated screening and diagnosis because of the wide use of ophthalmic images which provide an abundance of data to a computer algorithm. In the near future, for detection and treatment of DR, AMD, glaucoma and other ophthalmic disorders, the unmanned automated applications of AI, ML, and DL will be utilized as a potential alternative to ophthalmologists, retina specialists, and trained human graders.
In this study, we reviewed the potential promising clinical applications of AI, ML, and DL in diagnosing DR, AMD, glaucoma and other ophthalmic disorders, and introduced the current AI, ML, and DL technologies with the ophthalmic imaging modalities. We believe that this review may provide both ophthalmologists and computer scientists a significant and detailed summary on AI, ML, and DL applications in ophthalmology and facilitate such potential promising clinical applications in the ophthalmology health care system.
We searched for reviews and original research studies in PubMed, Cochrane Library, EMBASE, Science Direct, Web of Science, and Google Scholar databases, using the terms “artificial intelligence”, “machine learning”, “deep learning”, “diabetic retinopathy”, “age-related macular degeneration”, “glaucoma”, and “ophthalmic disorders”. Only studies reported in English in recent years were included. Those articles on the potential clinical automated applications of AI, ML, and DL technologies in ophthalmic health care settings, with a focus on diseases with high prevalence and incidence, such as DR, AMD, glaucoma, and some ophthalmic disorders were reviewed.
Application of the Novel Methods in the Ophthalmic Disorders
A variety of articles concerning AI, ML, and DL automated applications in diagnosing ophthalmic diseases have been published recently. The majority of these studies are related to DR, AMD and glaucoma, which are the 3 most prevalent causes of irreversible blindness worldwide. In many recent studies, AI, ML, and DL techniques have been shown to be an effective diagnostic tool for detecting and identifying various eye diseases in ophthalmic health care services. These applications can make a great contribution to providing support to patients in remote areas where there are no experts, medical devices, and adequate infrastructure. These studies have indicated that AI, ML, and DL applications with high accuracy are capable of detecting and diagnosing multiple retina disorders with promising results in automated image analysis.11
AI-ML-DL Algorithms in Diabetic Retinopathy
Diabetes mellitus (DM) has an increasing prevalence and incidence, affecting more than 415 million people worldwide, with approximately 1 of 11 adults being affected. By 2040, it is anticipated that approximately 600 million people will have DM. Diabetes is the most leading cause of adulthood blindness among the working-age population.1,12–15 Because of progressions in the treatment of DM, the surveillance of patients has improved and thus the frequency of DR and diabetic macular edema (DME) has increased. Diabetic retinopathy and DME cause blurring of central vision due to the developing retinal microvascular complications and fluid leakage from abnormal blood vessels in the retina of diabetic individuals. As a prevalent microvascular complication of DM, DR affects one-third of diabetic patients, leading to irreversible blindness. Undiagnosed and untreated DME is the major leading cause of severe visual impairment and blindness in working-age population.1,16,17 Therefore, there is an urgent need for large-scale screening of DR to detect potentially threatening changes at an early stage that would benefit management and treatment. It is known that early intervention is the most cost-effective choice.18 Therefore, in order to prevent vision loss in time due to sight-threatening retinopathy, it is essential to have early detection of DR and DME through regular and close surveillance by clinical examination or grading of retinal photographs. Annual screening of the retina with fundus digital photography, fundus luorescein angiography, and optical coherence tomography (OCT) is recommended but this presents a huge challenge and a problematic issue in many countries. Given the increased global prevalence of DM and DR, the limited availability of ophthalmologists, retina specialists and trained human graders, the delivery of optimal diabetic screening will be a problematic procedure for current global health care management. Analyzing such images is also time consuming, costly, and prone to human error. Therefore, these challenges may only be resolved through unmanned automated applications of novel methods of analysis without the need of engaging human professionals. It is inevitable that DR and DME will be detected by the automated retinal image analysis systems in the near future.19–22 Screening of DR is crucial and hence there is a need for a universal strategy for preventable blindness coupled with timely diagnosis and treatment. Unfortunately, DR screening programs are not being utilized to its full benefit because of organizational issues, lack of human graders, and financial problems.14
Given the current population growth trends and the high prevalence of DR and DME in the community, the applications of automated screening and diagnosis are inevitable in ophthalmic health care settings. In order to improve the management of DR patients and to alleviate the social burden, automatic retinal screening techniques for the diagnosis of DR have been searched. Multiple AI, ML, and DL techniques have been applied to automatically diagnose and grade DR, and the most effective automated applications are based on studies over the past 3 years. Recent studies regarding DR revealed that AI, ML, and DL demonstrated high accuracy, sensitivity, and specificity for the detection of DR.14,22–25
The results of some studies on DR based on AI, ML, and DL modalities are given in detail below.
Ting et al14 assessed the performance of deep learning system (DLS), a ML technology with high potential for screening and detecting referable DR, vision-threatening DR, AMD, and possible glaucoma using 494,661 retinal images. They found that the respective areas under the receiver operating characteristic curve (AUC), sensitivities, and specificities of DLS were 0.936, 90.5%, and 91.6% for referable DR; 0.958, 100%, and 91.1% for vision-threatening DR; 0.931, 93.2%, and 88.7% for AMD; and 0.942, 96.4%, and 87.2% for possible glaucoma. In their multiethnic cohorts of patients with DM, DLS had high sensitivity and specificity for recognizing DR and related eye diseases, but additional studies were required to assess the applicability and validity of the DLS in the ophthalmic health care settings.
Tufail et al22 assessed whether the automated DR image assessment systems (ARIAS) can be safely introduced into DR screening pathways to replace human graders. They detected the sensitivity point estimates for any DR, referable DR, and proliferative DR of the 2 ARIAS were respectively: 94.7%, 93.8%, and 99.6% for EyeArt; and 73.0%, 85.0%, and 97.9% for Retmarker. In view of the acceptable sensitivities for referable DR, they concluded that the ARIAS had a high potential for clinically effective and rapid detection of DR, and that it could be safely introduced into DR scanning programs to replace human graders and help delivery of DR screening in remote health care settings.
Abrámoff et al23 evaluated the sensitivity and specificity of the Iowa Detection Program for detecting referable DR by using automated analysis of retinal images. In this study, the AUC was 0.937, with sensitivity of 96.8% and specificity of 59.4%. However, the sensitivity/specificity of the 3 masked independent retina specialists were 0.80/0.98, 0.71/1.00, and 0.91/0.95, and the average intergrader or interobserver difference (k) was 0.822. They claimed that the Iowa Detection Program had high sensitivity and specificity for automated detection of referable DR. They also claimed that it could be introduced safely into DR screening program, potentially improving access to ophthalmic health care screening and reducing visual loss through early diagnosis and treatment of DR.
Deep learning algorithm (DLA) is expected to be a routine application in ophthalmic health care practice in the immediate future. However, further studies are needed to elucidate the applicability and validity of this algorithm in the clinical ophthalmic health care setting and to elucidate whether use of this algorithm could result in improved care and results comparable to current ophthalmologic appraisal.24 Gulshan et al24 developed a DLA for automated screening and detection of DR and DME in 128,175 retinal images by using a deep convolutional neural network (DCNN). In this study, for detecting referable DR, the algorithm of the AUC was 0.991 for EyePACS-1 data set and 0.990 for Messidor-2 data set. Using the first operating cut point with high specificity, the sensitivity and specificity were 90.3% and 98.1%, respectively for EyePACS-1 data set; 87.0% and 98.5%, respectively for Messidor-2 data set. Using the second operating cut point with high sensitivity, the sensitivity and specificity were 97.5% and 93.4%, respectively for EyePACS-1 data set, and 96.1% and 93.9%, respectively for Messidor-2 data set. They claimed that ML and DL algorithms had high sensitivity and specificity for detecting referable DR.
The detection of DR on the basis of color fundus photographs has been performed for years. The vast majority of studies of automated applications of AI, ML, and DL have focused mainly on the analysis of fundus photographs. If the validity, accuracy, reliability, sensitivity, and specificity of an AI-based DLA are reasonable, this application will be cost-effective in the health care settings. This technology will inevitably offer potential to increase the efficiency, sustainability, and accessibility of DR screening programs worldwide. Li et al25 developed an AI-based DLA for the detection of referable DR on the basis of color fundus photographs. In the internal validation data set, the AUC, sensitivity, and specificity of the DLA for vision-threatening referable DR were 0.989, 97.0%, and 91.4%, respectively. Testing against the independent multiethnic data set, the respective values were 0.955, 92.5%, and 98.5%. Among the false-positive cases, however, 85.6% were due to a misclassification of mild or moderate DR. Unobserved intraretinal microvascular abnormalities accounted for 77.3% of all false-negative cases. They claimed that AI-based DLA could be used with a good accuracy and reliability in the detection of vision-threatening referable DR in retinal images and that DLA technology also had a potential to increase the efficiency and accessibility of DR screening programs worldwide.
Automated application systems can help doctors understand DR predictions better and increase the applicability of intelligent diagnostic models in real-world clinical practice. From the above studies, the accuracy, validity, sensitivity, and specificity of the automated analysis of retinal images for detection of DR were very high and the diagnostic performance of AI, ML and DL was clinically acceptable and highly reproducible for validation data set. However, further studies are needed to elucidate the applicability and validity of these algorithms in clinical ophthalmic healthcare, and to clarify whether the use of this algorithm will lead to better care and outcomes compared with the current ophthalmological evaluation.
AI-ML-DL Algorithms in AMD
As a significant cause of visual loss, AMD is a chronic macular disease characterized by drusen, retinal pigment changes, choroidal neovascularization, hemorrhage, exudation, and sometimes geographic atrophy which is an irreversible serious condition.26 It affects elderly population, resulting in visual impairment, depression, reduction in quality of life, and mortality.27,28 In fact, AMD and DR are the leading causes of blindness in adults older than 50 years in the US.29 Basically, macular degeneration is induced by drusen inside or outside the retinal pigment epithelium and generally leading to visual deterioration in AMD. There are 2 types of drusen, including hard drusen or soft drusen. Hard drusen can be found in all age groups and may progress to soft drusen. However, soft drusen is mostly found among the elderly population and may develop choroidal neovascularization leading to visual impairment. Therefore, the quantitative measurement of drusen is crucial in order to prevent macular degeneration. However, traditional manual drusen measurements with current visual examination take a lot of time, require considerable effort and with a less reliable outcome.30–32 Therefore, there are predictions based on drusen with AI, ML, and DL algorithms for making individualized predictions in AMD. These algorithms can predict about drusen underneath the retina in AMD. Algorithms of AI, ML, and DL provide automated detection of drusen, fluid, and geographic atrophy concerning AMD lesions to improve AMD diagnosis and treatment by using fundus images and spectral-domain OCT (SD-OCT).33–37 The automatic drusen detection with AI, ML and DL is likely to help ophthalmologists to improve the early and rapid diagnostic performance on fundus images.30 The accuracy of AI-, ML-, and DL-based automated assessment of AMD is usually higher than 80%, which is consistent with manual evaluation by professionals, with an agreement reaching 90%.33,36–38
Intravitreal injection of anti-vascular endothelial growth factor (anti-VEGF) drugs is very important in the current management of neovascular AMD (nAMD) and close follow-up observation is paramount. The use of AI, ML, and DL to predict anti-VEGF injection requirements for patients with nAMD and proliferative DR can alleviate the economic burden of patients and facilitate resource management.39
Given the social population aging and the severity of this disease, it is necessary to perform AMD screening regularly. Automatic diagnosis of AMD may apparently reduce the workload of clinicians and hence increase productivity.40 Most of AI, ML, and DL techniques have been applied to automatically diagnose and grade AMD and the most effective automated applications are based on recent studies, which revealed that AI, ML, and DL demonstrated high accuracy, sensitivity, and specificity for the detection of AMD.41–45 The results of some studies on AMD based on AI, ML, and DL modalities are given in detail below.
Burlina et al41 developed an automated grading for detecting AMD from color fundus images by using DL methods and AI, namely, DCNN. They detected that the DCNN gave an accuracy of 88.4% to 91.6%, AUC of 0.94 to 0.96, and kappa coefficient of 0.764 to 0.829. They claimed that a DL-based automated assessment of AMD was consistent with manual professional evaluation and that automated algorithms could play a critical role in the present management of AMD, address costs of screening or monitoring, access to healthcare, and the appraisal of novel treatments.
Schlegl et al42 developed a fully DL-automated method to detect and quantify macular fluid caused by AMD, DME, and retinal vein occlusion (RVO) in conventional OCT images. They demonstrated that an automated diagnostic method based on DL achieved optimal accuracy for the detection and quantification of intraretinal cystoid fluid for AMD, DME, and RVO with a mean accuracy (AUC) of 0.94. However, the detection and measurement of subretinal fluid were also highly accurate with an AUC of 0.92, with superior performance in nAMD and RVO compared with DME. High correlation was verified between automated and manual fluid localization and quantification and the mean Pearson correlation coeficient was 0.90 for intraretinal cystoid fluid and 0.96 for subretinal fluid. They indicated that the detection and quantification of macular fluid based on DL could produce similar results to human performance levels. In addition, they claimed that DL-automated analysis of retinal OCT images ensured a promising horizon in improving accuracy and reliability of retinal diagnosis for clinical studies, practices, and care in the ophthalmic settings.
Burlina et al43 used the DL for severity characterization and estimation of 5-year risk of AMD patients by using 67,401 color fundus images. The weighted k scores were 0.77 for the 4-step and 0.74 for the 9-step AMD severity scales. The overall mean estimation error for the 5-year risk ranged from 3.5% to 5.3%. Based on these findings, they suggested that DL-based automated assessment of AMD grading had performance comparable to that of human performance levels. Consequently, they claimed that DL had a potential to assist clinicians in providing care, clinical research of disease progression, and public screening around the world.
Grassmann et al44 developed a DLA for prediction of the severity scale of AMD based on color fundus photography. They pointed out that the algorithm detected 84.2% of all fundus images with signs of early or late AMD and 94.3% of healthy fundus images were categorized accurately. Their DLA showed a weighted κ outperforming human graders in the AMD study and was appropriate to categorize AMD fundus images in other data sets of individuals older than 55 years.
Peng et al45 assessed the severity and risk of progression of late AMD with DeepSeeNet, which is a DL model for automated classification of AMD severity from color fundus photographs. In this study, the performance of DeepSeeNet was compared with that of retina specialists. They demonstrated that DeepSeeNet achieved high AUCs in the detection of large drusen (0.94), pigmentary abnormalities (0.93) and late AMD (0.97), and that DeepSeeNet (accuracy, 0.671; kappa, 0.558) outperformed retina specialists (accuracy, 0.599; kappa, 0.467). DeepSeeNet also performed better than retina specialists in the detection of large drusen (accuracy, 0.742 vs 0.696; kappa, 0.601 vs 0.517) and pigmentary abnormalities (accuracy, 0.890 vs 0.813; kappa, 0.723 vs 0.535). However, it showed lower performance in the detection of late AMD (accuracy, 0.967 vs 0.973; kappa, 0.663 vs 0.754). In this study, DeepSeeNet had a high reliability and accuracy in the automated AMD risk categories. As a result, they claimed that DL systems had a potential to assist and enhance clinical decision of early AMD detection and risk prediction of the late AMD development.
In light of these studies, AI, ML, and DL have shown high accuracy, sensitivity, and specificity in the detection of AMD. These automated applications have provided similar results with trained human graders. Therefore, these automated applications will be a routine practice in the diagnosis and treatment of AMD in the near future.
AI-ML-DL Algorithms in Glaucoma
Glaucoma, the second most common cause of blindness worldwide, is characterized by progressive neurodegenerative of retinal ganglion cells and irreversible loss of axons from the optic nerve. Early diagnosis and treatment of glaucoma is hugely important for preventing avoidable blindness. It is very important that optic nerve head (ONH) and retinal nerve fiber layer (RNFL) around the optic disc are evaluated for early diagnosis of glaucoma. It is possible to evaluate glaucomatous structural changes quantitatively with OCT. However, extremely limited retinal imaging devices, retina specialists, general ophthalmologists, ophthalmic graders, eye clinics or hospitals pose a great problem in developed and developing countries. In addition, patients with glaucoma suffer from availability, accessibility, affordability, and sustainability of ophthalmic health care services problems especially in poor countries. It is therefore very important for automatic detection of glaucoma via AI, ML, and DL.46,47
The visual field (VF) defect is the main parameter of visual function during glaucoma progression. In order to construct the AI-, ML-, and DL-based glaucomatous diagnostic models, VFs, fundus images, and OCT scans have been used. Although a standard automated VF test is important in the diagnosis and management of glaucoma, it consumes much time and resources. Moreover, such a manual procedure performed by patients is subjective and has been challenging in epidemiological studies. In contrast, AI, ML and DL methods have shown excellent performance in the classification of glaucoma and healthy eyes in a short time. Ophthalmologists can refer to these automated results and make better decisions in clinical practice.1,48
It is considered that AI, ML, and DL have a potential revolution for the screening, diagnosis, and classification of early detection of glaucoma. They also have the potential to recognize the development, progression, and treatment of glaucoma by identifying and assessing new risk factors. However, there is no clearly defined gold standard of these algorithms for determining the presence and severity of glaucoma. Therefore, in future studies, more robust disease definitions should be used to develop and optimize current methodologies and data inputs for AI, ML and DL analyses, and improve information acquisition methods from learned results.49 Multiple AI, ML, and DL techniques have been applied to automatically diagnose and grade glaucoma and the most effective automated applications are based on studies in the past few years. Recent studies on glaucoma revealed that AI, ML, and DL demonstrated high accuracy, sensitivity, and specificity for its detection.10,50–53 The results of some studies on glaucoma based on AI, ML, and DL modalities are given in detail below.
Devalla et al10 developed a DLS to digitally stain OCT images of the ONH and automatically measure its structural parameters. Their DLA was able to digitally stain the RNFL + prelamina, the retinal pigment epithelium, all other retinal layers, the choroid, and the peripapillary sclera and lamina cribrosa. For all tissues, the mean of the dice coefficient, sensitivity, specificity, intersection over union, and accuracy were 0.84 ± 0.03, 0.92 ± 0.03, 0.99 ± 0.00, 0.89 ± 0.03, and 0.94 ± 0.02, respectively. They demonstrated that DLA could simultaneously stain the neural and connective tissues of the ONH. Furthermore, they indicated that digital staining also performed well on OCT images of both glaucoma and healthy individuals and that could offer very high reliability and accuracy for glaucoma management.
Li et al50 evaluated the performance and efficacy of a DLA for the detection and automated classification of referable glaucomatous optic neuropathy based on 48,116 color fundus photographs. They detected that DLS achieved referable glaucomatous optic neuropathy with an AUC of 0.986, sensitivity of 95.6% and specificity of 92.0%. However, coexistence of eye disorders especially pathologic or high myopia was the most common cause resulting in false-negative results. Besides, physiologic cupping of optic disc was the most common reason for false-positive results in their study.
By using standard automated perimetry with a DL modality, Asaoka et al51 differentiated the VFs of patients with preperimetric open-angle glaucoma from healthy eyes. They obtained a significantly larger AUC of 92.6% by using the deep feedforward neural network classifier compared with all other ML methods: 79.0% with random forests (RF), 77.6% with gradient boosting, 71.2% with support vector machine (SVM), and 66.7% with neural network. Using a deep feedforward neural network classifier, they indicated that preperimetric glaucoma VFs could be differentiated from healthy VFs with a good reliability and accuracy.
Kim et al52 aimed to develop ML models that have strong prediction power and interpretability for diagnosis of glaucoma based on RNFL thickness and VF. They indicated the accuracy, sensitivity, specificity, and the AUC were 0.98, 0.983, 0.975, and 0.979, respectively. The developed prediction models showed high accuracy, sensitivity, specificity, and AUC in classifying among glaucoma and healthy eyes. They claimed that ML would be used for predicting glaucoma and hence clinicians could be able to make better decisions.
Asaoka et al53 constructed a DL model to diagnose early glaucoma from SD-OCT images for the input features of the 8 × 8 grid macular RNFL thickness and ganglion cell complex layer thickness. The AUC, RF, and SVM were used for diagnostic accuracy. They demonstrated that the AUC with the DL (DCNN) model was 93.7%. The AUC significantly reduced by between 76.6% and 78.8% without preliminary training. Significantly smaller AUCs were obtained with RF and SVM (82.0% and 67.4%, respectively). Consequently, they detected that a DL model for glaucoma offered a substantive increase in diagnostic performance by using SD-OCT.
From the above studies, it is shown that the automated applications of AI, ML and DL are highly effective and have the potential to support the impending challenge of DR, AMD, and glaucoma screenings in developed as well as developing countries. It is certain that the advent of these novel automated applications is incredibly impressive and the AI, ML and DL algorithms can revolutionize ophthalmology health care system in the near future. In addition, the emergence of AI, ML and DL in ophthalmic health care settings may aid in prevention of DR, AMD, and glaucoma-associated irreversible blindness. Further studies are needed to determine the actual accuracy, sensitivity, specificity, and validity of these automated applications for diagnosis and detection of DR, AMD, and glaucoma.
In many areas of specialties, accurate and rapid evaluation of clinical images is not only for diagnosis, but also for treatment. However, repeatability, validity, accuracy, reliability, sensitivity, and specificity of clinical images are very important in clinical health care practices. For this reason, the development of vision algorithms with computers is crucial to help in the analysis of biomedical images. Evaluation of retinal images is usually performed by retina specialists in many ophthalmological settings. In fact, that is not an objective evaluation, and it is immensely time consuming with variable interpretation, repeatability, and interobserver agreement variation.54 Over the past decade, DL is a promising class of ML models that has become a popular subject in science setting. Deep learning has been used successfully for signal processing, pattern recognition, and statistical analysis. In addition, image processing and segmentation have been eased with DCNN. Undoubtedly, these results will have clinical implications and will positively reflect medical imaging procedures.7,54,55 Application of AI technology depends mainly on ML, which is represented by mathematical algorithms and models generated by many input experiences. In fact, AI can efficiently conduct ophthalmological image processing, mainly based on the fundus photographs. It is also likely to achieve a promising accuracy comparable to clinical experts.40,56 For example, it has been demonstrated that automatic ML algorithm for the detection and quantification of reticular pseudodrusen using multimodal information performed within the same range as the human graders.36 Consequently, in recent years, a variety of studies have highlighted that AI, ML, and DL algorithms were successfully used for automated retinal images applications.
Computer algorithms can make a more objective evaluation of retinal disorders than before. In addition to DR,14,22–25 AMD41–45 and glaucoma,10,50–53 AI, ML and DL have also been used to diagnose other retinal diseases, including central retinal vein occlusion (CRVO),57 rhegmatogenous retinal detachment (RRD),58 retinopathy of prematurity (ROP),59 and reticular pseudodrusen.36 Apart from retina, AI-based systems have been improved to better identify or appraise other ophthalmic disorders, including pediatric cataract,60 keratoconus (KC),61 corneal ectasia,62 oculoplastic reconstruction,63 evaluation of corneal power after myopic corneal refractive surgery,64 making surgical plans in horizontal strabismus,65 and determining pigment epithelium detachment in polypoidal choroidal vasculopathy.66 The results of some studies on CRVO, RRD, ROP, KC, and cataract based on AI, ML, and DL modalities are given in detail below.
As a vascular disease of the retina, CRVO leads to substantial visual morbidity and vision loss in the aging population.67 It shows up with dilated tortuous retinal veins, retinal hemorrhages, cotton-wool spots, macular edema, and optic edema.68 The DL technology has been used to show the presence of CRVO with Optos images. Such DL model has been shown to have higher sensitivity, specificity, and AUC values for detecting CRVO in Optos fundus photographs.57
This technology may have an important potential clinical benefit to reach area with large populations but without retina specialists.57 Therefore, early diagnosis and intervention of CRVO patients living in areas with inadequate ophthalmic care is very crucial for visual recovery. As with other ophthalmic diseases, automatic diagnosis in CRVO will also be potentially beneficial for both patients and ophthalmologists.57
The RRD is a severe condition which can lead to visual loss. Early diagnosis and treatment of RRD is therefore crucial. Such disease is basically treatable, if managed appropriately and in a timely manner. If left untreated and proliferative changes develop, RRD can turn into an uncontrollable state called proliferative vitreoretinopathy. The Optos, the ultra-widefield scanning laser ophthalmoscope (Optos 200Tx; Optos PLC, Dunfermline, UK), can provide non-invasive, nonmydriatic, widefield fundus images, and has been used for diagnosis or follow-up of multiple fundus disorders and treatment assessment.58,69–71 However, due to rising social security costs and lack of retina specialists, the establishment of vitreoretinal centers providing modern ophthalmological procedures is not truly feasible.58 Ohsugi et al58 found that the sensitivity of RRD was 97.6%, specificity was 96.5%, and AUC was 0.988 in the DL model, compared with respective values of 97.5%, 89.3%, and 0.976 in the SVM model. In this study, the sensitivity and specificity of detecting RRD on the Optos fundus photographs were high with DL technology.
As a vasoproliferative disease affecting premature infants, ROP is a leading cause of childhood blindness worldwide and may be successfully treated with appropriate and timely diagnosis.40,59 Clinical studies have shown that ROP requires close observation and timely treatment to prevent blindness. However, the rigorous and arduous repeated screening and follow-up of the ROP consume much resource, including manpower. Therefore, the application of AI in ROP screening may increase the effectiveness of ROP care.40,72,73 Campbell et al59 demonstrated that the diagnostic accuracy of the imaging and informatics in ROP (i-ROP) computer-based system was 95%, whereas the mean accuracy of 11 expert physicians was 87%. Therefore, based on this study, it is possible to perform a computer-based image analysis system of the ROP comparable to retina specialists.
Keratoconus is a bilateral and non-inflammatory eye disease characterized by progressive thinning, protrusion, and scarring of the cornea. However, the deterioration of cornea may be progressive, asymmetric, and thus resulting in distorted and decreased vision.74,75 Although the underlying cause of the disorder is still unknown, it mostly becomes clinically manifest at puberty in both sexes and may be related to various factors such as atopic disease, eye rubbing, contact lens use, connective tissue disease, tapetoretinal degeneration, inheritance, and Down syndrome.74 Model of ML has already been used in KC and other corneal disorders detection. In addition, artificial neural networks and discriminant analysis are ML techniques that have already been used to describe the topographical models of KC.75,76 For instance, Carvalho76 developed an artificial neural network with a sensitivity of 78.75%, specificity of 97.81%, and precision of 94%.
Cataract characterized by clouding of the lens is one of the most prevalent diseases, causing bilateral blindness across the world.40,77 Ultraviolet and infrared rays or electromagnetic waves, smoking, diabetes, alcohol consumption, steroid medications, hormonal replacement therapy, malnutrition, synthetic chemical and pharmaceutical toxins, poor living conditions, hypoparathyroidism, galactosemia, eye surgery, inflammation and injury of the eye can lead to cataracts.78 Early diagnosis and treatment can regain vision and improve patients' quality of life. By applying ML algorithms such as RF and SVM, the diagnosis and grading of cataract has been made by utilizing fundus images, ultrasound images, and visible wavelength eye images.79,80 The risk estimation model for posterior capsular opacification after phacoemulsification surgery is also predicted by algorithms.81 Using DL models, senile cataract can be diagnosed.82 An impressive study showed that pediatric cataract is one of the primary causes of childhood blindness if it is not detected early and treated timely.83 It is not surprising that the application of ML for anterior segment diseases will become a frequent modality of ophthalmic settings.60,82,84,85 Given the prevalence of cataract in the community, its automatic recognition will offer a rapid, cost-effective, and more reliable practice.
There is no doubt that AI, ML, and DL methods have demonstrated significant advances in medicine.49 There are many goals of AI, including ML, DL, CML, natural language processing, computer vision, robotics, reasoning, general intelligence, expert system, automated learning and scheduling. As population aging is a growing trend around the world, more patients will suffer from eye diseases. Early diagnosis and timely treatment of eye diseases are known to be of great importance to prevent visual loss and improve quality of life. Due to the lack of ophthalmologists, retina specialists, graders and optimal eye devices, conventional diagnostic methods of the eye are inadequate. In addition, the conventional diagnostic methods depend on the experience and professional knowledge of physicians, leading to a high rate of misdiagnosis and wasting of large amount of medical data. Therefore, deep integration and adaptation of AI, ML, and DL technologies into ophthalmology has the potential to revolutionize current disease diagnose pattern and create a significant clinical impact.1
Future of the Automated Applications (AI, ML, and DL) in Ophthalmic Clinical Trials
In recent years, AI, ML and DL techniques have been shown in various scientific studies as an effective diagnostic tool to identify various diseases in health care services. The accuracy of the models is incredibly promising, and AI, ML and DL applications can provide support to patients in remote areas by sharing expert knowledge and limited resources. For creating more reliable AI, ML and DL systems, OCT, OCT angiography, VF, and fundus images need to be integrated together. Most of the current studies regarding intelligent diagnosis of eye diseases focus on dual classification problems, whereas many patients suffer from multiple categorical retinal diseases in the clinical setting. It is therefore necessary to have a model for detecting and distinguishing DR, AMD, glaucoma, and other retinal disorders simultaneously. To detect and diagnose different retinal diseases with high accuracy, it is necessary to build further intelligent systems in clinical practice.1
It is certain that the advent of AI, ML, and DL applications is incredibly impressive. Although these technologies are not yet mature enough to be implemented in the clinical setting, they offer a unique revolutionary breakthrough in the health care applications. For this reason, further studies are needed to elucidate the algorithms based on AI, ML, and DL to evaluate their sensitivity, specificity, and validity for detecting DR, DME, AMD, glaucoma, and other ophthalmic disorders. In addition, further researches are essential to identify the applicability of these algorithms in the clinical setting and to identify whether the use of these algorithms could lead to improved health care and to identify their outcomes compared with current ophthalmologic assessment. In the near future, AI-, ML-, and DL-assisted automated screening and diagnosis may help minimize doctors' burden and maximize their role at the ophthalmology clinics.40 The platforms of AI, ML, and DL can provide patients with more medical opportunities and reduce barriers to access to an eye care clinic without an ophthalmologist.40 In addition, new technologies based on AI, ML, and DL can reduce social inequalities.86 It seems that the health care industry will be reshaped by AI, ML and DL completely in the near future. However, as AI, ML, and DL gradually move from the virtual into the real world, artificial neural networks are vulnerable to cyber threat, hacks, and deception.
The intelligent systems will be adopted in some specific clinical ophthalmic studies in the near future. Although ethics, regulatory, and legal issues are challenging, AI, ML and DL will revolutionize the diagnosis and treatment of diseases and will have a significant clinical impact in the health system in the near future.
Automated retinal imaging technologies may potentially reduce the barriers to access to health care system and health screening, thus it may help reduce avoidable blindness across the world. If these technologies are widely embraced by health care authorities and ophthalmologists, it will have an immense favorable impact on medical community and ophthalmology society. On the other hand, it is very important to evaluate the repeatability, validity, accuracy, reliability, sensitivity, specificity, and correct disease staging of retinal scanning algorithms based on AI, ML, and DL in the clinical health care practice. As AI becomes more sophisticated, there may be many ethical challenges ahead, including transparency, bias, human values, data protection and intellectual property, social dislocation, cyber security, decision making, liability, legal and regulatory issue. In spite of these issues, AI, ML, and DL will contribute significantly to make a breakthrough diagnostic and treatment pattern and create a substantial clinical impact in the near future. We believe that this review may provide detailed, important, interesting, and diverse information to both ophthalmologists and computer scientists about the AI, ML and DL applications in the ophthalmology health care platforms and help facilitate promising clinical practices in the future.
1. Lu W, Tong Y, Yu Y, et al. Applications of artificial intelligence in ophthalmology: general overview. J Ophthalmol
2. Samuel AL. Levy DNL. Some studies in machine learning
using the game of checkers. I. Computer Games I.
. New York: Springer; 1988. 335–365.
3. Meskö B, Hetenyi G, Györffy Z. Will artificial intelligence solve the human resource crisis in healthcare? BMC Health Serv Res
4. Awad M, Khanna R. Efficient learning machines: theories, concepts, and applications for engineers and system designers. 2015; Berkeley, CA: Apress, 1–8.
5. Bengio Y, Courville A, Vincent P. Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell
6. Schmidhuber J. Deep learning in neural networks: an overview. Neural Netw
7. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature
8. Lang A, Carass A, Hauser M, et al. Retinal layer segmentation of macular OCT images using boundary classification. Biomed Opt Express
9. Gaiarsa VM, Cardenas D, Furuie SS. Approaches to segment stent area from intravascular optical coherence tomography. Res Biomed Eng
10. Devalla SK, Chin KS, Mari JM, et al. A deep learning approach to digitally stain optical coherence tomography images of the optic nerve head. Invest Ophthalmol Vis Sci
11. Rahimy E. Deep learning applications in ophthalmology. Curr Opin Ophthalmol
12. Gargeya R, Leng T. Automated identification of diabetic retinopathy using deep learning. Ophthalmology
13. Congdon NG, Friedman DS, Lietman T. Important causes of visual impairment in the world today. JAMA
14. Ting DSW, Cheung CY, Lim G, et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA
15. Balyen L, Unlu K, Deniz Balyen LS. Outcomes of intravitreal triamcinolone acetonide injection in patients with diabetic macular edema. Van Med J
16. Yau JW, Rogers SL, Kawasaki R, et al. Global prevalence and major risk factors of diabetic retinopathy. Diabetes Care
17. Sabanayagam C, Banu R, Chee ML, et al. Incidence and progression of diabetic retinopathy: a systematic review. Lancet Diabetes Endocrinol
18. Bek T, Erlandsen M. Visual prognosis after panretinal photocoagulation for proliferative diabetic retinopathy. Acta Ophthalmol Scand
19. Aiello LP, Gardner TW, King GL, et al. Diabetic retinopathy. Diabetes Care
20. Mohamed Q, Gillies MC, Wong TY. Management of diabetic retinopathy: a systematic review. JAMA
21. Shaw JE, Sicree RA, Zimmet PZ. Global estimates of the prevalence of diabetes for 2010 and 2030. Diabetes Res Clin Pract
22. Tufail A, Rudisill C, Egan C, et al. Automated diabetic retinopathy image assessment software: diagnostic accuracy and cost-effectiveness compared with human graders. Ophthalmology
23. Abramoff MD, Folk JC, Han DP, et al. Automated analysis of retinal images for detection of referable diabetic retinopathy. JaAMA Ophthalmol
24. Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA
25. Li Z, Keel S, Liu C, et al. An automated grading system for detection of vision-threatening referable diabetic retinopathy on the basis of color fundus photographs. Diabetes Care
26. Ferris FL 3rd, Wilkinson CP, Bird A, et al. Beckman Initiative for Macular Research Classification CommitteeClinical classification of age-related macular degeneration. Ophthalmology
27. Brody BL, Gamst AC, Williams RA, et al. Depression, visual acuity, comorbidity, and disability associated with age-related macular degeneration. Ophthalmology
2001; 108:1893–1900. discussion 1900–1901.
28. Tseng VL, Yu F, Lum F, et al. Risk of fractures following cataract surgery in medicare beneficiaries. JaAMA
29. Jiang S, Franco YL, Zhou Y, et al. Nanotechnology in retinal drug delivery. Int J Ophthalmol
30. Schlanitz FG, Baumann B, Kundi M, et al. Drusen volume development over time and its relevance to the course of age-related macular degeneration. Br J Ophthalmol
31. Algvere PV, Kvanta A, Seregard S. Drusen maculopathy: a risk factor for visual deterioration. Acta Ophthalmol
32. Kim YJ, Kim KG. Automated segmentation methods of drusen to diagnose age-related macular degeneration screening in retinal images. Comput Math Methods Med
33. Chen Q, Leng T, Zheng L, et al. Automated drusen segmentation and quantification in SD-OCT images. Med Image Anal
34. Bogunovic H, Montuoro A, Baratsits M, et al. Machine learning
of the progression of intermediate age-related macular degeneration based on OCT imaging. Invest Opthalmol Vis Sci
35. Chakravarthy U, Goldenberg D, Young G, et al. Automated identification of lesion activity in neovascular age-related macular degeneration. Ophthalmology
36. van Grinsven MJ, Buitendijk GH, Brussee C, et al. Automatic identification of reticular pseudodrusen using multimodal retinal image analysis. Invest Ophthalmol Vis Sci
37. Feeny AK, Tadarati M, Freund DE, et al. Automated segmentation of geographic atrophy of the retinal epithelium via random forests in AREDS color fundus images. Comput Biol Med
38. Lahmiri S, Boukadoum M. Automated detection of circinate exudates in retina digital images using empirical mode decomposition and the entropy and uniformity of the intrinsic mode functions. Biomed Tech (Berl)
39. Wong TY, Liew G, Mitchell P. Clinical update: new treatments for age-related macular degeneration. Lancet
40. Du XL, Li WB, Hu BJ. Application of artificial intelligence in ophthalmology. Int J Ophthalmol
41. Burlina PM, Joshi N, Pekala M, et al. Automated grading of age-related macular degeneration from color fundus images using deep convolutional neural networks. JAMA Ophthalmol
42. Schlegl T, Waldstein SM, Bogunovic H, et al. Fully automated detection and quantification of macular fluid in OCT using deep learning. Ophthalmology
43. Burlina PM, Joshi N, Pacheco KD, et al. Use of deep learning for detailed severity characterization and estimation of 5-year risk among patients with age-related macular degeneration. JaAMA Ophthalmol
44. Grassmann F, Mengelkamp J, Brandl C, et al. A deep learning algorithm for prediction of age-related eye disease study severity scale for age-related macular degeneration from color fundus photography. Ophthalmology
45. Peng Y, Dharssi S, Chen Q, et al. DeepSeeNet: A deep learning model for automated classification of patient-based age-related macular degeneration severity from color fundus photographs. Ophthalmology
46. Shahsuvaryan ML. Glaucomatous optic neuropathy management: the role of neuroprotective agents. Med Hypothesis Discov Innov Ophthalmol
47. Shibata N, Tanito M, Mitsuhashi K, et al. Development of a deep residual learning algorithm to screen for glaucoma from fundus photography. Sci Rep
48. Oh E, Yoo TK, Hong S. Artificial neural network approach for differentiating open-angle glaucoma from glaucoma suspect without a visual field test. Invest Ophthalmol Vis Sci
49. Zheng C, Johnson TV, Garg A, et al. Artificial intelligence in glaucoma. Curr Opin Ophthalmol
50. Li Z, He Y, Keel S, et al. Efficacy of a deep learning system for detecting glaucomatous optic neuropathy based on color fundus photographs. Ophthalmology
51. Asaoka R, Murata H, Iwase A, et al. Detecting preperimetric glaucoma with standard automated perimetry using a deep learning classifier. Ophthalmology
52. Kim SJ, Cho KJ, Oh S. Development of machine learning
models for diagnosis of glaucoma. PLoS One
53. Asaoka R, Murata H, Hirasawa K, et al. Using deep learning and transform learning to accurately diagnose early-onset glaucoma from macular optical coherence tomography images. Am J Ophthalmol
54. Lee CS, Tyring AJ, Deruyter NP, et al. Deep-learning based, automated segmentation of macular edema in optical coherence tomography. Biomed Opt Express
55. Mitchell JB. Machine learning
methods in chemoinformatics. Wiley Interdiscip Rev ComputMol Sci
56. Lee A, Taylor P, Kalpathy-Cramer J, et al. Machine learning
has arrived. Ophthalmology
57. Nagasato D, Tabuchi H, Ohsugi H, et al. Deep neural network-based method for detecting central retinal vein occlusion using ultrawide-field fundus ophthalmoscopy. J Ophthalmol
58. Ohsugi H, Tabuchi H, Enno H, et al. Accuracy of deep learning, a machine-learning technology, using ultra-wide-field fundus ophthalmoscopy for detecting rhegmatogenous retinal detachment. Sci Rep
59. Campbell JP, Ataer-Cansizoglu E, Bolon-Canedo V, et al. Imaging and informatics in ROP (i-ROP) research consortium.expert diagnosis of plus disease in retinopathy of prematurity from computer-based image analysis. JAMA Ophthalmol
60. Liu X, Jiang J, Zhang K, et al. Localization and diagnosis framework for pediatric cataracts based on slit-lamp images using deep features of a convolutional neural network. PLoS One
61. Ruiz Hidalgo I, Rozema JJ, Saad A, et al. Validation of an objective keratoconus detection system implemented in a Scheimpflug tomographer and comparison with other methods. Cornea
62. Ambròsio R Jr, Lopes BT, Faria-Correia F, et al. Integration of Scheimpflug-based corneal tomography and biomechanical assessments for enhancing ectasia detection. J Refract Surg
63. Tan E, Lin F, Sheck L, et al. A practical decision-tree model to predict complexity of reconstructive surgery after periocular basal cell carcinoma excision. J Eur Acad Dermatol Venereo!
64. Koprowski R, Lanza M, Irregolare C. Corneal power evaluation after myopic corneal refractive surgery using artificial neural networks. Biomed Eng Online
65. Almeida JD, Silva AC, Teixeira JA, et al. Surgical planning for horizontal strabismus using Support Vector Regression. Comput Biol Med
66. Xu Y, Yan K, Kim J, et al. Dual-stage deep learning framework for pigment epithelium detachment segmentation in polypoidal choroidal vasculopathy. Biomed Opt Express
67. Yau JW, Lee P, Wong TY, et al. Retinal vein occlusion: an approach to diagnosis, systemic risk factors and management. Intern Med J
68. Hayreh SS, Zimmerman MB. Fundus changes in central retinal vein occlusion. Retina
69. Nagiel A, Lalane RA, Sadda SR, et al. Ultra-widefield fundus imaging: a review of clinical applications and future trends. Retina
70. Miki D, Hida T, Hotta K, et al. Comparison of scleral buckling and vitrectomy for retinal detachment resulting from lap tears in superior quadrants. Jpn J Ophthalmol
71. Heussen N, Feltgen N, Walter P, et al. SPR Study GroupScleral buckling versus primary vitrectomy in rhegmatogenous retinal detachment study (SPR Study): predictive factors for functional outcome. Study report no. 6. Graefes Arch Clin Exp Ophthalmol
72. Gilbert C, Fielder A, Gordillo L, et al. International NO-ROP GroupCharacteristics of infants with severe retinopathy of prematurity in countries with low, moderate, and high levels of development: implications for screening programs. Pediatrics
73. Hardy RJ, Good WV, Dobson V, et al. The Early Treatment for Retinopathy of Prematurity Clinical Trial: presentation by subgroups versus analysis within subgroups. Br J Ophthalmol
74. Krachmer JH, Feder RS, Belin MW. Keratoconus and related noninflammatory corneal thinning disorders. Surv Ophthalmol
75. Souza MB, Medeiros FW, Souza DB, et al. Evaluation of machine learning
classifiers in keratoconus detection from orbscan IFI examinations. Clinics (Sao Paulo)
76. Carvalho LA. Preliminary results of neural networks and Zernike polynomials for classification of videokeratography maps. Optom Vis Sci
77. Waltz KL, Featherstone K, Tsai L, et al. Clinical outcomes of TECNIS toric intraocular lens implantation after cataract removal in patients with corneal astigmatism. Ophthalmology
78. Gupta VB, Rajagopala M, Ravishankar B. Etiopathogenesis of cataract: an appraisal. Indian J Ophthalmol
79. Yang JJ, Li J, Shen R, et al. Exploiting ensemble learning for automatic cataract detection and grading. Comput Methods Programs Biomed
80. SV, MK, RG. Computer-aided diagnosis of anterior segment eye abnormalities using visible wavelength image analysis based machine learning
. J Med Syst
81. Mohammadi SF, Sabbaghi M, Z-Mehrjardi H, et al. Using artificial intelligence to predict the risk for posterior capsule opacification after phacoemulsification. J Cataract Refract Surg
82. Gao X, Lin S, Wong TY. Automatic feature learning to grade nuclear cataracts based on deep learning. IEEE Trans Biomed Eng
83. Lin D, Chen J, Lin Z, et al. 10-Year overview of the hospital-based prevalence and treatment of congenital cataracts: the CCPMOH experience. PLoS One
84. Gillner M, Eppig T, Langenbucher A. Automatic intraocular lens segmentation and detection in optical coherence tomography images. Z Med Phys
85. Öhnell H, Heijl A, Brenner L, et al. Structural and functional progression in the early manifest glaucoma trial. Ophthalmology
86. Heidary F, Gharebaghi R. Ideas to assist the underprivileged dispossessed individuals. Med Hypothesis Discov Innov Ophthalmol