Teleophthalmology is widely recognized as a way to extend care to individuals over a larger region by leveraging limited physician availability and resources. This potential is especially important in low- and middle-income countries, where the imbalance between the prevalence of blindness and the number of practicing ophthalmologists is greatest.1 The XXX Institute of Ophthalmology (XXX) has established a satellite network of 16 community eye centers (CECs) that serve small to medium-sized cities and towns throughout the country. To reach patients in the most remote and isolated regions, ophthalmic technicians (OTs) from these CECs routinely organize and conduct screening camps in smaller nearby villages. They often see hundreds of patients in a day, screening primarily for operable cataracts.
Thanks to these efforts, the prevalence of cataract blindness in Nepal has decreased by more than half over the past several decades.2 However, less progress has been made in treating other causes of blindness, such as glaucoma and retinopathy. These diseases account for an increasing proportion of low vision and blindness in the country, but before this study there was little understanding of what types of pathology, aside from cataracts, were presenting to screening camps. Identifying and managing posterior segment pathology is generally more difficult than for cataracts. Highly trained healthcare workers must operate expensive imaging equipment and interpret the results. In addition, patient access to the equipment is challenging when so much of the population lives in remote rural areas. The Paxos Scope (Verana Health, formerly Digisight Technologies, San Francisco, CA) is an inexpensive, portable, smartphone-based ophthalmic imaging adapter that enables any iPod or iPhone camera to produce high-quality3 magnified anterior and posterior segment photographs and videos (Fig. 1). Despite its long-heralded potential, there are few reports of actual clinical results using smartphone-based teleophthalmology in a developing country. The purpose of this study was to investigate the impact of this technology on the detection and management of all types of ocular pathology seen at remote screening camps, where technicians currently utilize only a direct ophthalmoscope, retinoscope, and handheld portable slit lamp to screen hundreds of patients in a day.
This was a single-institution study comparing diagnoses and clinical decisions made for patients seen at XXX remote screening camps both before and after teleophthalmology consultation. Institutional Review Board approval was obtained from XXX. Best-corrected visual acuity (BCVA) ≤6/18 was chosen as a threshold for study enrollment because it is the World Health Organization criterion for visual impairment, and also because it was a previously established threshold for utility of the technology being used. A prior XXX Institute CEC study demonstrated that eyes with 6/18 BCVA were more likely than eyes with 6/12 BCVA to have pathology that was “missed” by technicians but detected by ophthalmologists reviewing patient imaging (20% vs 9%).4 Based on available data at XXX, it was anticipated that only about 10% of patients presenting to screening camps would meet the criterion of BCVA ≤6/18 in one or both eyes. Average screening camps see 100–150 patients, so 15 screening camps were scheduled to target a study size of 150 patients.
Fifteen screening camps were conducted in the districts of Solukhumbu, Dolakha, and Ramecchap between January and April 2018. Over the 15 camps, 163 patients with 324 eyes were enrolled. One patient was monocular due to past trauma, and another had a phthisical eye; hence, the total number of eyes was 324 instead of 326. Because of the hectic nature of the screening camps, there were some instances of protocol nonadherence resulting in patients who were enrolled and examined, but for whom either disposition or diagnosis was not recorded. In some cases, a diagnosis was given but no disposition, and in others a disposition was given but no definitive diagnosis. In all, 140 patients out of 163 enrolled had complete dispositions, and 283 eyes out of 324 eyes examined in the study had diagnoses available for analysis. Several patients had mature cataracts or other conditions (e.g. nystagmus) that prohibited clear fundus photography. Ophthalmologists did not mark these photos as “ungradable” but rather commented on the pathology that they did see (in anterior segment photos/videos and in limited views of the posterior segment), as the objective was to study this technology's impact on detection and management of pathology, not image quality, which has been assessed previously.5
Camps were conducted in their usual manner, with patients first going through registration and visual acuity testing. Patients were then examined by an OT using a pen light, direct ophthalmoscope, retinoscope, and handheld portable slit lamp. Dilating drops were used if the technician felt that a dilated exam with the direct ophthalmoscope was warranted. However, there was no requirement that technicians dilate patients who were enrolled, only that they perform a screening exam as they normally would. This generally included only undilated direct ophthalmoscopy. The BCVA and diagnosis for each eye were then recorded along with a disposition (Table 1) for the patient. Eligible patients were enrolled in the study (Tables 2 and 3). Extreme photophobia and/or inability to sit still for the photography exam were ultimately the only two reasons for excluding patients, outside of age or inability to provide consent. Informed consent was obtained from all patients.
Once enrolled, patients were dilated if they had not been already. An attempt was made to obtain a contact phone number for every patient. Identifying information, medical data, and photos were recorded in a HIPAA-compliant smartphone app. Anterior segment and central fundus photos or videos were obtained for both eyes in all enrolled patients. Photos were taken either by OTs or by a visiting medical student trained in using the device. These data were uploaded using WiFi to a secure, HIPAA-compliant cloud-based server. For this study, sixth-generation iPods and eighth-generation iPhones were used interchangeably to capture photos and videos.
Ophthalmologists at XXX Institute, who were blinded to the technician diagnoses and plans, reviewed the cloud-based data and images. These data included age, sex, visual acuity, and at least one anterior segment photo and one posterior pole photo for every eye included in this study. Like the examining technicians, the masked XXX ophthalmologists recorded a diagnosis for each eye and a disposition for each patient, and entered these data into the patient's chart in the cloud-based app. Physician diagnoses and dispositions were then compared with those of the technician.
The goal was to determine the feasibility of using mobile device teleophthalmology in this setting and to examine its impact on clinical decision-making. The proportion of discordant diagnoses/dispositions between technicians performing in-person exams (before intervention) and ophthalmologists reviewing data remotely (after intervention) is the simplest way to demonstrate this impact. Cohen kappa was also calculated using SPSS Statistics as a method of rating level of agreement between the technicians and the ophthalmologists.7 No conclusions about sensitivity or specificity could be made because there was no gold standard, although previous studies have validated the sensitivity and specificity of the technology used in this study for certain pathologies.3,6,7
Overall, technicians and ophthalmologists agreed on the ocular diagnosis 57.6% (163/283 eyes) of the time, disagreed entirely 23% (65/283 eyes) of the time, and agreed on one part of the diagnosis but not another (eg, cataract vs cataract plus diabetic retinopathy) 19.4% of the time (55/283 eyes). Analyzed by diagnosis, agreement was highest for the diagnosis of cataract (k = 0.732, 95% CI 0.65–0.81) and lowest for the diagnosis of posterior segment pathology, such as retinopathy and glaucoma suspect (k = 0.057, 95% CI −0.03–0.14). Although there was relatively good statistical agreement for the finding of “No Abnormality Detected” (k = 0.642, 95% CI 0.53–0.75), this obscures the significant fact that technicians diagnosed No Abnormality Detected in far more patients than did ophthalmologists (n = 40 for ophthalmologists, n = 65 for technicians) (Table 4, Fig. 2).
Technicians and ophthalmologists agreed on disposition only 41.4% (58/140 patients) of the time, disagreed 40.7% (57/140 patients) of the time, and partially agreed 17.9% (25/140 patients) of the time. Agreement on disposition was the highest for cataract extraction at a surgical camp (k = 0.623, 95% CI 0.49–0.75). However, agreement on the need for referral to XXX for workup of noncataract pathology was significantly lower (k = 0.12, 95% CI 0.00–0.24). In general, technicians were far more likely to recommend observation (no referral), whereas physicians were much more likely to recommend referral to XXX (Table 5, Fig. 3).
How much were differences in referral decisions due to misdiagnosis versus inappropriate recommendations for correct diagnoses? The overall rate of agreement on disposition was minimal (k = 0.225, 95% CI 0.14–0.31). In cases where technicians and ophthalmologists made the same diagnosis, the rate of agreement on referral need improved substantially (k = 0.5292, 95% CI 0.37–0.69). When the technicians and ophthalmologists made different diagnoses, there was no agreement on need for referral (k = −0.005, 95% CI −0.08–0.07), as expected. However, analyzing only cases where technicians and ophthalmologists both diagnosed retinal or glaucomatous pathology, agreement on the appropriate decision to refer to XXX remained only minimal (k = 0.379, 95% CI 0.16–0.59) (Table 6).
Community screening camps in the XXX healthcare system evolved out of the need to address the overwhelming burden of cataract, which continues to be the leading cause of preventable blindness in Nepal. Our results suggest that this rural screening system is sufficiently equipped to identify cataract patients requiring surgery. However, screening camps provide the opportunity to identify other treatable ocular disorders. In our study, the ophthalmologists recommended referral for further evaluation (beyond cataract surgery) for nearly half (67/140) of the study subjects with BCVA≤6/18 for noncataract diagnoses, whereas technicians recommended further evaluation for only about a tenth (18/140) of patients. We suspect that the discrepancy is due to numerous factors, including minimal training in and exposure to posterior segment pathology for technicians, inadequate examination tools available to technicians, the fact that technicians have only a few minutes to see each patient, and the general emphasis of these camps being on identifying candidates for cataract surgery.
Our study not only quantifies the pathology being encountered in rural village settings, but also provides a model through which diagnostic accuracy and referral decisions at screening camps might be improved without requiring substantially more equipment, human resources, time, or training. Although it seems logical that physician oversight should improve diagnostic yield, the findings from the present study now can quantify and qualify this in terms of specific types of pathology and aspects of the ocular exam and potentially justify the use of low-cost, mobile device-based photography technologies in rural screening camp settings. Smartphone-based ophthalmic camera systems are portable and pocket sized. 1 staff member assigned to photography was able to image at least 30 patients per day in the study without delaying screening camp operations. This staff member did not have any formal medical background, and previous studies have shown that individuals with varying levels of ophthalmologic training can quickly learn to take high-quality photos with the device.7,8 Anyone with the skills to take a cell phone photo has the requisite skill set to take high-quality fundus photos. This allows physician extenders to operate in a telemedicine model to reach remote, resource-limited settings around the world.
This study has several limitations. First, screening camps are a hectic clinical setting. As a result, 23 enrolled patients (out of 163) were missing a referral decision and 41 eyes (out of 324) were missing a diagnosis from either the technician or ophthalmologist, hence the apparent discrepancy in 283 eyes from 140 patients. Additionally, due to the fast pace and need to see so many patients at each camp, it was not feasible to require that OTs dilate every patient before direct ophthalmoscopy, which might have improved OT diagnostic accuracy. However, standardizing the technician exam in this way would have significantly altered the screening process and would have limited the applicability of the results of this study to the actual screening system.
Second, although the technology used in this study has been shown to have a sensitivity of 91% and specificity of 99% for detecting moderate to severe diabetic retinopathy,3 it is only an imaging tool and diagnostic accuracy depends on the ophthalmologist and the quality of the images generated. For this reason, and because there was no gold standard in the study, no definite conclusions can be drawn as to whether technicians or ophthalmologists were correct in their diagnoses and referral recommendations. However, in the eyes included in the analysis, there were no instances of images being graded as unreadable by the ophthalmologist. Thus, in the present study, the assumption is that the ophthalmologist's interpretations of the images were the correct ones. Still, to be accurate, kappa was used for statistical analysis because it obviates the need to establish a correct diagnosis.
Third, though the diagnostic accuracy of this technology has been previously validated,3 the imaging devices (eg, smartphones/iPods) used in this study were different from those used in the initial validation study. However, this is presumed to be of minimal significance, as imaging capabilities and resolution are only improved in the more modern technology used here.
Finally, some aspects of the imaging arm of the study, including how many images to take of each eye and whether to take a still photo or video, were not controlled, but rather left to the judgment of the technician or student acquiring the imaging, so long as at least one anterior image and one posterior image was obtained for each eye. Photographers often felt more adept at one method or another, and some pathology requires video rather than photography (eg, nystagmus). Therefore, it was felt to be most pragmatic to allow for some degree of flexibility to obtain the highest quality images.
To evaluate the impact of smartphone-based teleophthalmology, future investigations should assess the outcomes resulting from referring screened patients to an ophthalmologist at XXX. This includes monitoring and studying patient follow-up compliance and barriers, and the correlation of images obtained in the field with the live examination by an ophthalmologist at XXX.
This study demonstrates that mobile device-based ophthalmic cameras can be used to facilitate teleophthalmology in remote village settings in countries with large rural populations, such as Nepal. This device and system overcome three obstacles that typically limit the delivery of care to remote, low-income areas, namely cost, access, and the need for technical training. Smartphone adapters like the one used in this study are far less expensive than other portable fundus cameras. The pocket-sized device is also very durable, because the only electronic component is a battery-powered light source. Finally, it can be used by anyone with basic familiarity with cell phone photography, regardless of prior medical knowledge or training. Portable fundus photography is particularly important for being able to diagnose treatable posterior segment pathology in patients without access to ophthalmologists in urban clinics.
The authors gratefully acknowledge the collaborative support of the staff at the XXX Institute of Ophthalmology and the Himalayan Cataract Project, as well as generous financial support from the American Society of Cataract and Refractive Surgeons (ASCRS) Foundation. Paxos devices were donated by Digisight Technologies (now Verana Health). Authors from Stanford acknowledge departmental core grants to the Byers Eye Institute from the National Eye Institute (NEI P30 EY026877) and Research to Prevent Blindness.
1. Bastawrous A, Hennig BD. The global inverse care law: a distorted map of blindness. Br J Ophthalmol
2. Nepal Netra Jyoti Sangh. The Epidemiology of Blindness in Nepal. Kathmandu, Nepal:Netra Jyoti Sangh; 2012.
3. Toy BC, Myung DJ, He L, et al. Smartphone-based dilated fundus photography and near visual acuity testing as inexpensive screening tools to detect referral warranted diabetic eye disease. Retina
4. Hong K, Collon SM, Mercado C, et al. Clinical utility of a mobile device teleophthalmology
system at community eye centers in rural Nepal. J Mobile Technol Med
. 2017;6:34-42, Dec. 2017.
5. Ludwig CA, Murthy SI, Pappuru RR, et al. A novel smartphone ophthalmic imaging adapter: user feasibility studies in Hyderabad, India. Indian J Ophthalmol
6. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb)
7. Newman TB, Browner WS, Cummings SR. Hulley SB, Newman TB, Grady D, et al. Designing studies of medical tests. Designing Clinical Research
Lippincott Williams & Wilkins, 4th ed.Philadelphia, PA: 2013.
8. Ludwig CA, Newsom MR, Jais A, et al. Training time and quality of smartphone-based anterior segment screening in rural India. Clin Ophthalmol