Artificial intelligence (AI), has emerged as a major frontier in computer science research. Healthcare affordability, quality, and accessibility can be amplified using this technology. AI in simple words means to accomplish a task mainly by a computer or a robot, with minimal involvement of human beings. In other words, AI is simulation of human intelligence by a software/machine. It is essentially the ability of a computerized system to show cognitive abilities. Just like learning in humans, the AI systems need to be exposed to a database which allow them to first “learn” simple targets with regards to a dedicated finding or disease. However, AI is much more than simply a humongous database. Following initial steps of learning, the system or machine is then taught to “improve”, i.e., evolve upon its initial learning to become more accurate and efficient. This learning is further compounded by use of complex mathematical equations for the system to understand nonlinear relationships between different variables through an information flow referred to as “neural networks.” In essence, this form of “higher training” allows AI to judge and weigh possibilities of different outcomes, much like an ideal physician! Many of these technological advances are in part due to software that have now been made available by resources related to information and technology. An example of commercially available machine learning software is Scikit-learn which require computer languages like Python as a platform for working. Scikit learn is a machine learning library for use in python programming language and it has been used for diabetic retinopathy detection. Several commercial software currently integrates artificial Intelligence and machine Learning for fundus image processing and evaluation. Fundus evaluation software are being made as student projects in several engineering colleges for the past few years. Open Indirect Ophthalmoscope by LVPEI and MIT has an inbuilt option for DR detection by Machine Learning. Also, Kavya Kopparapu's Eyagnosis app, which along with a 3D printed smartphone fundus camera, were developed in 2016 and tested in several leading eye hospitals.
AI-assisted medical screening and diagnosis based on images are currently evolving. Application of this technology in ophthalmology is currently focused mainly on the diseases with a high incidence, such as diabetic retinopathy (DR), age-related macular degeneration (ARMD), glaucoma, retinopathy of prematurity (ROP), age-related or congenital cataract, and retinal vein occlusion (RVO).
DR is an eye disease known to cause moderate to severe visual loss and is the leading cause of blindness in working-age people suffering with long standing diabetes. The health burden is accentuated by the huge per capita cost. This has further increased since the introduction of anti VEGF agents. Very often the disease does not show overt symptoms until it reaches an advanced stage; however, if detected early on, vision impairment can be averted by early intervention which is also the most cost-effective option. In view of the alarming increase in the number of people with diabetes and dearth of trained retinal specialists and ophthalmologists, a computer-based analysis of the fundus images by an automated approach would lessen the burden of the health systems in screening for DR and offer a near ideal system for its management. Therefore, screening will be valuable at any stage of the disease and will also be helpful in avoiding blindness among 90% patients.
We searched all the English language studies relative to ophthalmology published on PubMed and springer database. The articles published in last 10 years that we deemed relevant were summarised. The keywords used for the PubMed search were artificial intelligence, ophthalmology, deep learning, machine learning, diabetic retinopathy.
Regardless of the type of diabetes, all individuals diagnosed with DM need regular and repetitive annual retinal screening for timely detection and apt treatment of diabetic retinopathy (DR). Conventionally, retinopathy screening is done by fundus examination by ophthalmologists or with the help of color fundus photography using conventional fundus cameras (mydriatic or non-mydriatic) by trained eye technicians or optometrists. The primary issue is the grading of the retinal images by ophthalmologists (retinal specialists) or trained persons, whose numbers are very scarce compared to the load of patients requiring screening. Second, some of these patients are based in rural areas and can’t visit an eye care provider. Thirdly, as such follow ups are required for years together, the attitude, and/or behavioral aspects negatively impact the patients practice despite knowledge of consequences. These issues can be solved with provision of an automated imaging system within easy reach of the patient. Hence, there has been an increasing interest in the development of automated analysis software using computer machine learning/artificial intelligence (AI) for analysis of retinal images in people with diabetes thus solving at least some part of the problem.
The Solution: Principle Behind Artificial Intelligence
It is basically a process of teaching a machine to recognize specific patterns. Historically, it has been used for various technical tasks including accurate classification of high-resolution images. The techniques of AI devices largely classified into following major categories -the machine learning techniques, the natural language processing methods, speech, vision, expert system, robotics etc. So far, the machine learning techniques are more utilized in ophthalmology.
Machine learning process mainly include two parts, training set followed by validation set. This process occurs by providing large number of training data i.e., thousands of retinal images of varying grades of DR to the machine/system as the training set. Most of the data are labeled as per features in advance by the authoritative professionals. After being exposed to numerous annotated retinal images the machine learns to grade DR by itself by building a model of complex relationships between input data and generalizing a performance standard. In addition, some other data are used to verify the established algorithm i.e., validation set [Flow Chart 1].
Two main deep learning models have been described: the convolutional neural network (CNN) and the massive-training artificial neural network (MTANN).
Diabetic Retinopathy and Artificial Intelligence
At least yearly fundus screening is prudent in patients of uncontrolled diabetes for early diagnosis and treatment to reduce the burden of disease in the community. However only half of such patients undergo screening. At present, screening necessitates referral to an ophthalmologist, and patients may not visit the specialist because of lack of an eye specialist in their community, logistical barriers, or cost of the visit. One way of addressing such problems is by obtaining color fundus images and sending those to eye specialist or optometrists for reading. Although these programs increase screening rates, still there are logistical barriers, cost issues, and time delays. Such limitations created interest in assessment of images using fully automated AI-based grading systems. In real time the system would decide whether a patient requires referral and could potentially be much cheaper than having ophthalmologists conduct screening. In April 2018, the US Food and Drug Administration (FDA) approved an AI algorithm, developed by IDx, used with Topcon Fundus camera (Topcon Medical) for DR identification.
The enthusiasm in the field of artificial intelligence has led to several studies using retinal images to test the performance of AI grading systems for detecting DR. Few of the studies are worth mentioning. [Table 1] Historically the Wisconsin Fundus Photograph Reading Centre (FPRC) has been the gold standard for trials that require grading of the severity of DR, including the Diabetes Control and Complications Trial (DCCT), Diabetic Retinopathy Clinical Research Network (DRCR.net) studies. Wong et al. proposed a model to classify the DR stages based on microaneurysms and hemorrhages. Imani et al. shaped a different technique in which, by using morphological component analysis, they spotted the exudation and blood vessel. The vessel map is obtained by using adaptive thresholding. Yazid et al. published identification of hard exudation and optic disc based using inverse surface thresholding. The main objective of this paper was to detect both hard and soft exudates. However, since the color of the exudates is similar to that of optic disc, they are usually detected together. Basically, fuzzy c-means clustering, edge detection and Otsu thresholding were utilized to separate edge pixels of the exudates from the background. Akyol et al. by using key point detection, texture analysis, and visual dictionary techniques detected the optic disc of fundus images. Studies have reported sensitivity of automatic DR screening ranging from 75% to 94.7% and the specificity, accuracy is comparable. A study using smartphone-based fundus photography system to evaluate the usefulness of an automated AI-based interpretation of screening at a physician clinic has also shown positive results. The study reported a “high sensitivity for detection of DR above 95% using the EyeArt software when used on retinal images taken with Fundus on Phone (FOP)”. A recent study done by EyeNuk with retinal images taken with traditional desktop fundus cameras showed that EyeArt's sensitivity for DR screening was 91.7% and 91.5% specificity. Another very recent major study on validation of deep learning (AI) by Ting et al. done in Singapore with multiple retinal images taken with conventional fundus cameras showed a high sensitivity and specificity for identifying DR and other eye diseases like ARMD. An automated tele-retinal DR screening program, IRIS (intelligent retinal imaging system), compared non-mydriatic retinal images with a standard data set images from Early Treatment Diabetic Retinopathy Study (ETDRS), and proposed recommendations for referral. Any patient with severe NPDR or more advanced disease was considered suitable for the referral. This screening program reported good sensitivity and a low false-negative rate.
Use of AI to evaluate retinal images is alluring as it fits in with the present trend of tele-ophthalmology and telemedicine. Selected patients who have sight-threatening DR would need to meet the eye specialist. Urgent referral of these patients is critical, since DR affects people during their prime productive years of life.
IDx- DR is the first FDA approved AI algorithm for the detection of DR in the offices of non-ophthalmic healthcare practitioners. The device is paired with a non-mydriatic retinal camera (TRC-NW400, Topcon) and the captured images are sent to a cloud-based server. The server then utilizes IDx-DR software and a “deep-learning” algorithm to detect retinal findings consistent with DR based on autonomous comparison with a large dataset of representative fundus images. The software provides one of the two results: (1) If more than mild DR detected, refer to an eyecare professional (ECP); (2) If the results are negative for more than mild DR, rescreen in 12 months.
The FDA approval of IDx-DR device was based on a study on 900-subjects in a primary-care setting (10 primary care sites) with automated image analysis. Two 45-degree digital images per eye (one centered on the macula, one cantered on the optic nerve) were obtained and analyzed. These images were compared with the stereo, widefield fundus imaging interpreted by the Wisconsin Fundus Photograph Reading Centre (FPRC). After procurement of retinal images, the artificial intelligence system is able to make a diagnosis in just 20 seconds.
Based on the analysis a new entity called more than minimal DR (mtmDR) was defined. It in nothing but the presence of ETDRS level 35 or higher (microaneurysms plus hard exudates, cotton wool spots, and/or mild retinal hemorrhages) and/or DME in at least one eye. 96 percent of acquired images were of sufficient quality for algorithmic assessment, which was really high numbers in primary care settings. Sensitivity and specificity of the technology was 87.4% and 89.5% respectively for detecting more than mild DR. It's worth mentioning that 100% of subjects with ETDRS levels of 43 or higher DR were correctly identified by the algorithm. As the device delivers a screening decision without necessitating an eye specialist, it can also be used by non-ophthalmic healthcare professionals.
Downsides of AI
In view of below 90% sensitivity and specificity of the device, 1 in 10 patients theoretically may have a false-positive and false negative result. So, it is not absolutely fail-safe. Thus, it is crucial to educate patients and doctors that the present generation devices are not 100% reliable. A false negative result may provide a pseudo sense of security about the retinopathy status. For the present, a comprehensive dilated eye examination remains the gold standard of screening and cannot be replaced with this device till appropriately proved otherwise. Diabetes has numerous ocular manifestations other than DR, which includes glaucoma, age-related macular degeneration (ARMD), cataract, dry eye. A comprehensive examination is obligatory for proper diagnosis and management in these patients.
Diabetic macular oedema is the leading cause of vision loss in patients with diabetes. Stereoscopic macular examination coupled with optical coherence tomography remains the gold standard for diagnosing this condition. Though all subjects with ETDRS level 43 or higher DR were detected via IDx-DR, but many cases of subtle DME were missed because of non-addressal. Legal accountability in cases of misdiagnosis with artificial intelligence is another subject that is yet to be fixed.
Uses of Artificial Intelligence in Other Ophthalmic Conditions
Age-related macular degeneration
ARMD is a chronic and irreversible macular disease and one of the leading causes of central vision loss in people aged over 50 years. With the demand of a regular screening in such a condition, an automatic AMD diagnosis tool may clearly reduce the work load of clinicians. Few studies used fundus image as the input, where other existing researchers have used OCT as a tool for deep learning of ARMD. Bogunovic et al. employ an algorithm to predict anti-VEGF treatment needs from OCT scans taken during treatment initiation.
Retinal vein occlusion
A research group utilized CNN combined with patch and image-based vote methods to recognize the fundus image of branch retinal vein occlusion (BRVO) automatically. They reported a high accuracy over 97%.
Retinopathy of prematurity
ROP is a leading cause of treatable childhood blindness when diagnosed timely. This disease demands repeated follow up and screening, which is very tedious and demanding. So, application of AI in ROP screening may expand the proficiency in ROP care. Promising results have been obtained from various studies. Most of them are presently based on two-level sorting (plus or not plus disease).
Anterior segment diseases
The disease mainly includes cataract and glaucoma which are very prevalent conditions in community health care. Automated grading of nuclear cataracts using slit lamp has been reported. The detection of glaucoma mainly depends on the intraocular pressure (IOP), thickness of retinal nerve fiber (RNFL), optic nerve, and visual field examination. Unlike diabetic retinopathy, glaucoma is not an imaging disease. Thus, the hurdle would be to incorporate other results tests such as OCT images, IOP, disc photos, and longitudinal visual field data into AI systems. Hwang et al. have combined corneal data from slit-scan tomography and spectral-domain OCT in a method for screening corneas for very early signs of keratoconus using artificial intelligence.
Legal Aspects of AI
AI has tremendous potential to reshape health care and it is more rapidly to do so. But the legal issues involved with the development and implementation of AI algorithms are considerable. Regulation, legal causes of action such as medical malpractice and product liability, intellectual property, and patient privacy all have real implications for the way AI is developed and deployed. When it comes to AI and machine learning, there are currently more legal questions than answers. How can AI systems ensure consent? How will questions of liability be addressed? How does AI fit into existing ethical frameworks in India? How can the security and accuracy of AI solutions be ensured—particularly in the health sector as individual lives can be at stake and highly sensitive data is being handled? These are few questions that are still to be answered.
Use of AI in medical diagnostics, especially in ophthalmology heralds a new era. If proven to be sensitive and specific enough this technology can totally change the way we look at screening programs and community-based ophthalmology programs. Most of the present systems use conventional of 30–50° fundus images. Perhaps applications based on wide field imaging and OCT angiography based vascular analysis might yield even more consistent results. However, the high cost of wide field imaging and OCT angiography may be a limiting factor for this at present. A lot of work is also being done on identifying serum biomarkers for early detection and monitoring of diseases like diabetic retinopathy. Thus, a comprehensive analysis of ocular imaging, systemic parameter profile and other serum biomarkers using AI might provide better insights, perhaps even better conclusions than what human intelligence is capable of deriving.
The AI DR tool can assist the clinician with fundus image analysis, which in turn helps to quickly inform the next steps in the patient's treatment. Also, doctors can attend to more patients that need attention without mydriasis. Emerging healthcare technologies emphasize on reducing visits to eye specialists, curtailing the overall cost of treatment and optimizing the number of patients seen by each doctor. AI can help the health care professional in achieving the goal. Though it assists in health care sector but should not substitute a clinician at its current level. Novel developments in the sector of artificial intelligence are opening up new promises for running DR detection and grading algorithms.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
1. Hamet P, Tremblay J. Artificial intelligence in medicine Metabolism. 2017;69S:S36–40
2. Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, et al Artificial intelligence in healthcare: Past, present and future Stroke Vasc Neurol. 2017;2(2):230–43
3. Doi K. Computer-aided diagnosis in medical imaging historical review, current status and future potential Comput Med Imaging Graph. 2007;31:198–211
4. Doi K. Diagnostic imaging over the last 50 years research and development in medical imaging science and technology Phys Med Biol. 2006;51:R5–27
5. Kocur I, Resnikoff S. Visual impairment and blindness in Europe and their prevention Br J Ophthalmol. 2002;86:716–22
6. Abràmoff MD, Niemeijer M, Suttorp-Schulten MS, Viergever MA, Russell SR, van Ginneken B. Evaluation of a system for automatic detection of diabetic retinopathy from color fundus photographs in a large population of patients with diabetes Diabetes Care. 2008;31:193–8
7. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs JAMA. 2016;316:2402–10
8. Walton OB, Garoon RB, Weng CY, Gross J, Young AK, Camero KA, et al Evaluation of automated teleretinal screening
program for diabetic retinopathy JAMA Ophthalmol. 2016;134:204–9
9. Vashist P, Singh S, Gupta N, Saxena R. Role of early screening
for diabetic retinopathy in patients with diabetes mellitus: An overview Indian J Community Med. 2011;36:247–52
10. Fong DS, Aiello LP, Ferris FL, Klein R. Diabetic retinopathy Diabetes Care. 2004;27:2540–3
11. Namperumalswamy P, Nirmalan PK, Ramaswamy KM. Developing a screening
program to detect sight threatening retinopathy in south India Diabetes Care. 2003;26:1831–5
12. Ryan ME, Rajalakshmi R, Prathiba V, Anjana RM, Ranjani H, VenkatNarayan KM, et al Comparison among methods of retinopathy assessment (CAMRA) study: Smartphone, nonmydriatic, and mydriatic photography Ophthalmology. 2015;122:2038–43
13. Bhaskaranand M, Cuadros J, Ramachandra C, Bhat S, Nittala MG, Sadda SR, et al Automated diabetic retinopathy screening
and monitoring using retinal fundus image analysis J Diabetes Sci Technol. 2016;10:254–61
14. Darcy AM, Louie AK, Roberts LW. Machine learning and the profession of medicine JAMA. 2016;315:551–2
15. Murff HJ, FitzHenry F, Matheny ME, Gentry N, Kotter KL, Crimin K, et al Automated identification of postoperative complications within an electronic medical record using natural language processing JAMA. 2011;306:848–55
16. Lee A, Taylor P, Kalpathy-Cramer J, Tufail A. Machine learning has arrived Ophthalmology. 2017;124:1726–8
17. Suzuki K. Overview of deep learning in medical imaging Radiol Phys Technol. 2017;10:257–73
18. Murchison AP, Hark L, Pizzi LT, Dai Y, Mayro EL, Storey PP, et al Non-adherence to eye care in people with diabetes BMJ Open Diabetes Res Care. 2017;5:e000333
19. Rosenberg JB, Tsui I. Screening
for diabetic retinopathy N Engl J Med. 2017;376:1587–8
20. Taylor CR, Merin LM, Salunga AM, Hepworth JT, Crutcher TD, O’Day DM, et al Improving diabetic retinopathy screening
ratios using telemedicine-based digital retinal imaging technology: The Vine Hill study Diabetes Care. 2007;30:574–578
21. . US Food and Drug Administration. FDA permits marketing of artificial intelligence-based device to detect certain diabetes-related eye problems 2018Last accessed on 2018 Aug 12 Available from: https://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm604357.htm
. Published April 11
22. Wong LY, Acharya R, Venkatesh YV, Chee C, Min LC. Identification of different stages of diabetic retinopathy using retinal optical images Inf Sci. 2008;178:106–21
23. Imani E, Pourreza HR, Banaee T. Fully automated diabetic retinopathy screening
using morphological component analysis Comput Med Imaging Graph. 2015;43:78–88
24. Yazid H, Arof H, Isa HM. Automated identification of exudates and optic disc based on inverse surface thresholding J Med Syst. 2012;36:1997–2004
25. Akyol K, Şen B, Bayır Ş. Automatic detection of optic disc in retinal image by using keypoint detection, texture analysis, and visual dictionary techniques Comput Math Methods Med 2016. 2016 6814791
26. Niemeijer M, Abràmoff MD, van Ginneken B. Fast detection of the optic disc and fovea in color fundus photographs Med Image Anal. 2009;13:859–70
27. Rajalakshmi R, Subashini R, Anjana RM, Mohan V. Automated diabetic retinopathy detection in smartphone-based fundus photography using artificial intelligence Eye. 2018;32:1138
28. Bhaskaranand M, Ramachandra C, Bhat S, Cuadros J, Nittala MG, Sadda S, et al Automated Diabetic Retinopathy Screening
and Monitoring Using Retinal Fundus Image Analysis J Diabetes Sci Technol. 2016;10:254–61
29. Ting DSW, Cheung CY, Lim G, Tan GSW, Quang ND, Gan A, et al Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes JAMA. 2017;318:2211–23
30. Rajalakshmi R, Amutha A, Ranjani H, Ali MK, Unnikrishnan R, Anjana RM, et al Prevalence and risk factors for diabetic retinopathy in Asian Indians with young on set type 1 and type 2 diabetes J Diabetes Complications. 2014;28:291–7
31. Abramoff M. Artificial intelligence for automated detection of diabetic retinopathy in primary care. Paper presented at: Macular Society; February 22, 2018; Beverly Hills, CALast accessed on 2019 Mar 26 Available from: http://webeye.ophth.uiowa.edu/abramoff/MDAMacSocAbst20180222.Pdf[Internet]
32. Pros and Cons of Using an AI-Based Diagnosis for Diabetic Retinopathy: Page 4 of 5 N.d. Optometry Times.Last accessed on 2018 Oct 29 Available from: http://www.optometrytimes.com/article/pros-and-cons-using-ai-based-diagnosis-diabetic-retinopathy
33. Ferris FL, Wilkinson CP, Bird A, Chakravarthy U, Chew E, Csaky K, et al Clinical classification of age-related macular degeneration Ophthalmology. 2013;120:844–51
34. Chou CF, Cotch MF, Vitale S, Zhang X, Klein R, Friedman DS, et al Age-related eye diseases and visual impairment among U.S. adults Am J Prev Med. 2013;45:29–35
35. Mookiah MR, Acharya UR, Fujita H, Koh JE, Tan JH, Noronha K, et al Local configuration pattern features for age-related macular degeneration characterization and classification Comput Biol Med. 2015;63:208–18
36. Burlina P, Pacheco KD, Joshi N, Freund DE, Bressler NM. Comparing humans and deep learning performance for grading AMD: A study in using universal deep features and transfer learning for automated AMD analysis Comput Biol Med. 2017;82:80–6
37. Bogunovic H, Waldstein SM, Schlegl T, Lang G, Sadeqhipour A, Liu X, et al Prediction of Anti-VEGF treatment requirements in neovascular amd using a machine learning approach Invest Ophthalmol Vis Sci. 2017;58:3240–8
38. Zhao RQ, Chen ZH, Chi ZR. Convolutional neural networks for branch retinal vein Conf Proc IEEE. 2015 Available from: https://ieeexplore.ieee.org/document/7279547
39. Gilbert C, Fielder A, Gordillo L, Quinn G, Semiqlia R, Visintin P, et al International NO-ROP Group Characteristics of infants with severe ROP in countries with low, moderate, and high levels of development implications for screening
programs Pediatrics. 2005;115:e518–25
40. Gilbert C, Foster A. Childhood blindness in the context of VISION 2020-the right to sight Bull World Health Organ. 2001;79:227
41. Ataer-Cansizoglu E, Kalpathy-Cramer J, You S, Keck K, Erdogmus D, Chiang MF. Analysis of underlying causes of inter-expert disagreement in retinopathy of prematurity diagnosis. Application of machine learning principles Methods Inf Med. 2015;54:93–102
42. Ataer-Cansizoglu E, Bolon-Canedo V, Campbell JP, Bozkurt A, Erdogmus D, Kalpathy-Cramer J, et al i-ROP Research Consortium Computer-based image analysis for plus disease diagnosis in retinopathy of prematurity performance of the “i-ROP” system and image features associated with expert diagnosis Transl Vis Sci Technol. 2015;4:5
43. Bolón-Canedo V, Ataer-Cansizoglu E, Erdogmus D, Kalpathy-Cramer J, Fontenla-Romero O, Alonso-Betanzos A, et al Dealing with inter-expert variability in retinopathy of prematurity: A machine learning approach Comput Methods Programs Biomed. 2015;122:1–15
44. Gao X, Lin S, Wong TY. Automatic feature learning to grade nuclear cataracts based on deep learning IEEE Trans Biomed Eng. 2015;62:2693–2701
45. Musch DC, Gillespie BW, Niziol LM, Lichter PR, Varma R. Intraocular pressure control and long-term visual field loss in the collaborative initial glaucoma treatment study Ophthalmology. 2011;118:1766–73
46. Oddone F, Lucenteforte E, Michelessi M, Rizzo S, Donati S, Parravan M, et al Macular versus retinal nerve fiber layer parameters for diagnosing manifest glaucoma a systematic review of diagnostic accuracy studies Ophthalmology. 2016;123:939–49