Secondary Logo

Journal Logo

Original Articles

Evaluation of a remote telemedicine platform using a novel handheld fundus camera: Physician and patient perceptions from real-world experience

Yeh, Tsai-Chua,b; Lo, Kang-Junga; Hwang, De-Kuanga,b; Lin, Tai-Chia,b; Chou, Yu-Baia,b,*

Author Information
Journal of the Chinese Medical Association: July 2022 - Volume 85 - Issue 7 - p 793-798
doi: 10.1097/JCMA.0000000000000755
  • Open

Abstract

1. INTRODUCTION

A recent estimate from the World Health Organization indicates that approximately 1300 million individuals worldwide suffer from visual impairment, and above 80% of these are considered treatable or preventable.1 However, a considerable percentage of them are under socioeconomic deprivation or in low-resource settings. Without proper intervention, most are trapped in this vicious cycle and experience social disability, loss of productivity, and poor quality of life. With advances in technology and imaging tools, teleophthalmology has emerged as a powerful solution for vision health disparities. It not only overcomes the geographical barriers for eye examinations but also bridges the rural-urban gap of approaching ophthalmologists.

Prior studies have demonstrated the role of teleophthalmology in diagnosing retinal diseases such as DR and retinopathy of prematurity.2–5 However, most of these screening programs use portable cameras that require pupil dilation to capture detailed images or trained physicians to perform the examination. Previous studies have indicated that nonmydriatic retinal imaging tools have relatively low sensitivity and specificity.6,7 Furthermore, real-world experience demonstrating the effectiveness of telemedicine screening using a nonmydriatic fundus camera in an island population is scarce.

This study aimed to establish and evaluate a telemedicine screening system on a remote island where access to eye care is extremely limited or unavailable. We also compared the diagnostic accuracy, including referrals of ophthalmoscopy, with that of telemedicine, and evaluated the perceived value and satisfaction of patients. Furthermore, we analyzed the anticipated barriers and hurdles to the future adoption of teleophthalmology based on our real-world experience. To our knowledge, this is the first study to evaluate the real-world effectiveness of a newly developed nonmydriatic fundus camera in a rural island setting.

2. METHODS

2.1. Study population

This prospective study was conducted on an isolated, remote island (Orchid Island), 62 km east of the southern tip of mainland Taiwan. A total of 176 eyes of 90 participants distributed across four different villages were randomly enrolled. Inclusion criteria were individuals older than 18 years who were willing to participate in a complete ocular and fundus examination and signed a written consent form. This study followed the tenets of the Declaration of Helsinki and was approved by the institutional review board of the Taipei Veterans General Hospital.

2.2. Ophthalmic examination and image acquisition

After standardized history-taking, participants underwent comprehensive ophthalmic examination, including presenting distance visual acuity (with correction if available) on conventional Snellen charts, slit-lamp examination, measurement of intraocular pressure with the Tono-Pen, and indirect ophthalmoscopy examination. Indirect ophthalmic examination was performed and assessed by an experienced retinal specialist. Before pupil dilatation, fundus examination findings were obtained using a nonmydriatic handheld fundus camera built by Medimaging Integrated Solution Inc. (MiiS, Horus Digital Fundus Camera-DEC 200). The Horus handheld fundus camera measured 201 × 90 × 203 mm in size and weighed 450 g. It was equipped with a 3.5-inch full-color liquid crystal display monitor and a high-definition complementary metal-oxide-semiconductor sensor. The front optical lens was positioned at a working distance of 24 mm, owing to a 60° field of view. There was also an eye-up that could be attached to the optical component (Fig. 1). The image resolution was approximately 2560 × 1920 pixels. An inbuilt auto-focus system was used to correct the variability in the axial length and refractive error. Fundus illumination was provided by a white light-emitting diode. Retinal images were captured by a trained photographer.

F1
Fig. 1:
Illustration of the Horus handheld fundus camera. A, The control unit includes several components: A 3.5-inch LCD screen, a working panel with manual focus adjustment, brightness adjustment, and an OK button to capture a photograph, and power indicator. B, An optical lens is attached to the control unit. LCD = liquid crystal display.

2.3. Clinical diagnosis and referral accuracy

The experienced retinal specialist who performed clinical indirect ophthalmoscopic examinations determined a final diagnosis, referral plan, and recommended follow-up for each participant based on the American Academy of Ophthalmology Practice Pattern Guidelines.8 A written referral letter was given to the patient for further management at a partnered hospital with ophthalmology facilities closer to the island.

2.4. Telemedical image reading and quality assessment

The deidentified fundus images were uploaded to a secure web-based database. Two board-certified retinal specialists (TC/DK) with >10 years of clinical experience independently conducted a remote fundus image review. No additional information, such as age, history, or visual acuity, was provided to ensure that the graders focused only on fundus features. Images were graded according to the same criteria as those used for the on-site indirect ophthalmoscopy examinations. Abnormal fundus findings, including enlarged cup/disc ratio, dry age-related macular degeneration (AMD), wet AMD, diabetic retinopathy (DR), and lattice degeneration, were recorded. DR severity was based on the International Clinical Classification for Diabetic Retinopathy disease severity scale. All fundus photographs acquired by the handheld camera were graded for image quality based on a modified FOTO-ED scale9 (Table 1) regarding the possibility of assessing the details and emergent findings of the fundus. Additionally, diagnostic certainty was rated on a three-step scale (Table 1), reflecting how well the diagnosis was established. If additional examinations, including the best corrected visual acuity (BCVA), intraocular pressure (IOP), or slit-lamp examination, are necessary to confirm the diagnosis, the reviewer was asked to select a minimum of them. Referral recommendations for advanced examinations (including visual-field examination, optical coherence tomography, and fluorescein angiography) or further management were made based on all parameters, and graded into urgent (within 1 month) or nonurgent (within 6 months).

Table 1 - A modified FOTO-ED scale for general image quality and a diagnostic certainty scale
General image quality
(Modified FOTO-ED Scale)
Diagnostic certainty
Grade Description Grade Description
1 Ideal image quality: Subtle findings can be discernable 1 Very certain
2 Acceptable image quality: Obvious emergent findings can be discernable 2 Somewhat certain, considered reliable but prefer further information to confirm
3 Inadequate image quality: Discerning all emergency findings is not possible 3 Very uncertain, require further information to diagnose or manage the patient

2.5. Patient satisfaction

Surveys including the experience of current visits using handheld fundus camera examinations and willingness to participate in future teleophthalmology visits were administered to all participants.

2.6. Statistical analysis and rationale for reference standards

Cohen’s kappa test was used to assess agreement between the reviewers. Stata (StataCorp, College Station, TX) software version 15 was used to perform descriptive, correlation, Chi-square, and Cohen’s kappa test analyses. Statistical significance was set at p < 0.05.

We compared telemedicine photograph diagnoses with the standard indirect ophthalmoscopy for each patient. When analyzing sensitivity, specificity, positive predictive value, and negative predictive value using indirect ophthalmoscopy examinations as the reference for referral-warranted retinopathy, any image deemed undecidable by the reviewer was considered a false diagnosis.

3. RESULTS

3.1. Demographics and patient satisfaction

A total of 176 eyes from 90 participants met our eligibility criteria. Table 2 presents the demographic characteristics of the enrolled participants. There were 35 men (38.9%) and 55 women (61.1%) with a mean age of 59.24 ± 13.45 years. Approximately 30.0% of participants had never undergone any eye examination in the past; another 33.3% of participants had not received any eye examination for >1 year, the other 36.7% had a general eye examination within 1 year. Approximately 27.8% of patients were diagnosed with hypertension and 5.6% with diabetes. The mean distance uncorrected visual acuity (decimal) was 0.83 ± 0.48, and the mean IOP (mmHg) was 16.72 ± 3.54. Based on simplified activities of daily living scale, 13.3% of participants had a low capacity for daily living, 33.3% had a moderate capacity, and 53.3% had a high capacity.

Table 2 - Demographic and main clinical data of study participants (total n = 90)
n %
Age (y) (mean ± SD) 59.24 ± 13.45
Sex
 Male 35 38.9
 Female 55 61.1
Distance uncorrected visual acuity (mean± SD) 0.83 ± 0.48
Intraocular pressure, (mean± SD) 16.72 ± 3.54
Diabetes mellitus 5 5.6
Systemic hypertension 25 27.8
Smoking status
 Never 67 74.4
 Active 16 17.8
 Ex-smoker 7 7.8
Activities of daily living scale
 Low 12 13.3
 Moderate 30 33.3
 High 48 53.3
Previous eye examination
 Never 27 30.0
 ≤1 y 33 36.7
 >1 y 30 33.3

3.2. Accuracy in diagnosis

Table 3 describes the diagnostic accuracy of individual telemedicine graders compared with indirect ophthalmoscopy reports. The level of agreement compared with indirect ophthalmoscope was 89.8% (TC) and 88.6% (DK), respectively. Cohen’s kappa was 0.80 ± 0.06 (TC) and 0.78 ± 0.06 (DK), indicating good agreement between indirect ophthalmoscopic and photographic assessments. Using indirect ophthalmoscopy examinations as the reference to detect referral-warranted retinopathy, photograph reading by two reviewers (TC and DK) showed sensitivity of 78% (95% CI, 57%-91%) and 78% (57%-91%) and specificity of 99% (95%-99%) and 98% (93%-99%), with positive predictive values of 95% (75%-99%) and 88% (67%-97%) and negative predictive values of 95% (90%-98%) and 95% (90%-98%), respectively.

Table 3 - Comparison of accuracy in diagnosis and referral recommendation between telemedicine and ophthalmoscopy
Diagnosis Referral
Ophthalmoscopy agreement, % (95% CI) κ Ophthalmoscopy agreement, % (95% CI) κ
Reviewer 1 (TC) 89.8 (85.3-84.3) 0.802 95.3 (91.9-98.7) 0.829
Reviewer 2 (DK) 88.6 (83.9-93.4) 0.777 94.3 (90.6-97.9) 0.790

3.3. Referral accuracy

As compared to the referral decisions based on indirect ophthalmoscopic examination, the level of agreement was 95.3% (TC) and 94.3% (DK). The Cohen’s kappa was 0.83 ± 0.08 (TC) and 0.79 ± 0.08 (DK) (Table 3). The intergrader referral agreement based on fundus images was 92.1% (kappa = 0.81 ± 0.06) among two different retinal specialists. In our study, based on the clinical indirect ophthalmoscopic assessment, two patients were referred for dry AMD, one for wet AMD, two for lattice degeneration, two for an enlarged cup-disc ratio, and one for retinitis pigmentosa.

3.4. Evaluation of fundus photograph quality and certainty in diagnosis

All captured fundus color photographs were judged for quality to evaluate subtle or obvious emergent fundus findings. Fig. 2 shows fundus color photographs captured using a handheld fundus camera. Approximately 97.7% (TC) and 95.5% (DK) of the images were graded as acceptable or ideal quality, respectively. The intergrader agreement was 97.6% with a squared Cohen’s kappa of 0.73 ± 0.07 (Table 4).

Table 4 - Agreement in fundus photograph quality and comfortability scale in diagnosis
Photograph quality reviewer 2 (DK) Diagnostic certainty reviewer 2 (DK)
Reviewer 1 (TC) Grade 1 Grade 2 Grade 3 Grade 1 Grade 2 Grade 3
Grade 1 154 9 1 97 50 4
Grade 2 1 4 3 5 7 2
Grade 3 0 0 4 0 0 11
Agreement = 97.6% Agreement = 89.6%
Weighted κ = 0.733 Weighted κ = 0.499

F2
Fig. 2:
Examples of Horus handheld fundus camera-captured photographs. A, Fundus photograph demonstrates normal retina and optic discs. B, Fundus photograph demonstrates the presence of a few drusen, which is consistent with dry AMD. C, Fundus photograph demonstrates focal retinal hemorrhage, which is consistent with early wet AMD. D, Fundus photograph demonstrates waxy-pallor of the optic disc, severe arteriolar attenuation, and bone spicule pigmentation in the mid-peripheral retina, consistent with retinitis pigmentosa. AMD = age-related macular degeneration.

As for certainty in diagnosis, 85.8% (TC) and 58.0% (DK) of the diagnoses were considered certain; 8.0% (TC) and 32.4% (DK) of the diagnoses were considered reliable; however, the reviewer would prefer further information to confirm; 6.3% (TC) and 9.7% (DK) of the diagnoses were considered made based on insufficient information, and the reviewer requested for more details to confirm the diagnosis (Table 4). According to the reviewer’s survey, the most common additional examination requested to aid in the diagnosis was BCVA, followed by IOP.

4. DISCUSSION

Our study using a handheld fundus camera demonstrated high accuracy in detecting referable retinopathy compared to the gold standard indirect ophthalmoscopy. Key findings in this real-world telemedicine study were as follows: (1) The level of intergrader agreement was excellent when fundus images were used to detect referral-warranted retinopathy. (2) Over 95% of the images were graded as of acceptable or ideal quality. (3) High patient satisfaction was achieved using this teleophthalmology approach.

Although some other handheld fundus cameras have demonstrated their validity and utility in telemedicine platforms,10–13 these camera systems may not be suitable for use in rural settings, where resources, technicians, and infrastructure are limited. The nonmydriatic handheld fundus camera used in this study not only provides substantial image quality, but also increases portability, low implementation cost, excellent data storage capacity, and stable wireless communication of digital images. Our results demonstrate that this camera is appropriate for patient screening and referral, which are the two main purposes of implementing a telemedicine platform in a rural area.

We found high intergrader agreement in diagnosis. Although it was a relatively subjective perception experienced by the reviewer, the fact that each reviewer rated <10% of diagnosis uncertainty further supports the reliability of these image readings. When reviewing discrepancies, the common cause for decreased accuracy was subtle retinal lesions lying outside the view of the handheld fundus camera. Furthermore, there was good intergrader agreement for referral. Disagreements occur most often in cases of dry AMD, which lack clinical consensus on referral and may require more complex imaging modalities. When evaluating the discrepancies between two graders performing image-based diagnosis, we found that one reader follows a more stringent standard in either image quality or referral criteria than the other. Although this variability among experts might create challenges for the implementation of image-based clinical referral systems, a clear definition of a reference standard for referral can help triage patients under remote or virtual care.

The majority of fundus images in this study were graded as at least of acceptable quality, which is comparable to that of the traditional tabletop fundus camera. Furthermore, there was a low rate of ungradable images in the present study. In previous studies, portable or smartphone-based fundus cameras often remained difficult to use for photographers without ophthalmology training.14 However, this handheld fundus camera built an automatic fixation so that the photographer was able to capture good-quality images with minimal training. Additionally, an eyecup attached to the optics helps achieve physiological mydriasis and minimizes reflections from the cornea to decrease artifacts. These features enable the wider adoption of teleophthalmology among primary care physicians, even in remote areas with limited healthcare resources.

Despite some geographical limitations in telemedicine, patient satisfaction was overwhelmingly positive. Approximately 97.4% of patients in the study are satisfied with this telemedicine screening program and would like to participate in future televisits. This number is higher than anticipated, given that this remote island with aboriginal tribes is traditionally considered a relatively conservative population for new technology. Despite the similarity in the prevalence of ocular disease, previous study reported that people in rural areas are twice as likely to have never seen an eye care provider.15 In our study, high perceived value and satisfaction level also reflects lack of eye care resources on the island.

The strengths of this study include a comprehensive evaluation of image quality and agreement with diagnosis and referral compared with the gold standard indirect ophthalmoscopy. Although other studies have validated nonmydriatic fundus cameras against conventional approaches, the present study contributes new features with real-world experience from a rural island population. Additionally, the study population encompassed a variety of retinal manifestations that were not limited to a single disease, such as DR, mimicking real-world conditions to a larger extent. Prior studies have concluded that one of the major barriers to teleophthalmology adoption is the high rate of ungradable images.16 However, with advancements in retinal camera technology, our study displays a low percentage of ungradable images to enable accurate triage.

In this study, the use of a handheld fundus camera in teleophthalmology screening was robust with high patient satisfaction. However, our study has some limitations. Since this was a community-based screening study, the number of abnormal fundus findings was limited. Nonetheless, the results still demonstrate the great potential of telemedicine for triage in a non-ophthalmic setting for further referral and management. A previous study showed that cost was an important influencer of the patient-perceived value of teleophthalmology.17 However, patient satisfaction based on willingness to pay was not rigorously investigated in our study. Despite these limitations, the current study provided qualitative and quantitative data to assess the utility of telemedicine in an island population.

In conclusion, this telemedical eye screening study using a novel nonmydriatic fundus camera is comparably effective in detecting referable retinopathy compared with the gold standard indirect ophthalmoscopy. An easily accessible platform to facilitate teleophthalmology implementation plays an essential role in responding to the pandemic. Further development of sustainable infrastructure is of great concern for teleophthalmology to play a permanent role.

REFERENCES

1. Anon. >WHO | Global data on visual impairment. WHO. Available at http://www.who.int/blindness/publications/globaldata/en/. Accessed June 16, 2020.
2. Biten H, Redd TK, Moleta C, Campbell JP, Ostmo S, Jonas K, et al.; Imaging & Informatics in Retinopathy of Prematurity (ROP) Research Consortium. Diagnostic accuracy of ophthalmoscopy vs telemedicine in examinations for retinopathy of prematurity. JAMA Ophthalmol. 2018;136:498–504.
3. Rajalakshmi R, Arulmalar S, Usha M, Prathiba V, Kareemuddin KS, Anjana RM, et al. Validation of smartphone based retinal photography for diabetic retinopathy screening. PLoS One. 2015;10:e0138285.
4. Yogesan K, Constable IJ, Barry CJ, Eikelboom RH, McAllister IL, Tay-Kearney ML. Telemedicine screening of diabetic retinopathy using a hand-held fundus camera. Telemed J. 2000;6:219–23.
5. Polack S, Yorston D, López-Ramos A, Lepe-Orta S, Baia RM, Alves L, et al. Rapid assessment of avoidable blindness and diabetic retinopathy in Chiapas, Mexico. Ophthalmology. 2012;119:1033–40.
6. Scanlon PH, Malhotra R, Thomas G, Foy C, Kirkpatrick JN, Lewis-Barned N, et al. The effectiveness of screening for diabetic retinopathy by digital imaging photography and technician ophthalmoscopy. Diabet Med. 2003;20:467–74.
7. Murgatroyd H, Ellingford A, Cox A, Binnie M, Ellis JD, MacEwen CJ, et al. Effect of mydriasis and different field strategies on digital image screening of diabetic eye disease. Br J Ophthalmol. 2004;88:920–4.
8. Feder RS, Olsen TW, Prum BE Jr, Summers CG, Olson RJ, Williams RD, et al. Comprehensive adult medical eye evaluation preferred practice pattern(®) guidelines. Ophthalmology. 2016;123:P209–36.
9. Lamirel C, Bruce BB, Wright DW, Delaney KP, Newman NJ, Biousse V. Quality of nonmydriatic digital fundus photography obtained by nurse practitioners in the emergency department: the FOTO-ED study. Ophthalmology. 2012;119:617–24.
10. Jin K, Lu H, Su Z, Cheng C, Ye J, Qian D. Telemedicine screening of retinal diseases with a handheld portable non-mydriatic fundus camera. BMC Ophthalmol. 2017;17:89.
11. Bats FD, Nitenberg CV, Fantino B, Denis P, Kodjikian L. Age-related macular degeneration screening using a nonmydriatic digital color fundus camera and telemedicine. Ophthalmologica. 2014; 231:172–6.
12. Bursztyn L, Woodward MA, Cornblath WT, Grabe HM, Trobe JD, Niziol L, et al. Accuracy and reliability of a handheld, nonmydriatic fundus camera for the remote detection of optic disc edema. Telemed J E Health. 2018;24:344–50.
13. Kubin AM, Wirkkala J, Keskitalo A, Ohtonen P, Hautala N. Handheld fundus camera performance, image quality and outcomes of diabetic retinopathy grading in a pilot screening study. Acta Ophthalmol. 2021;99:e1415–20.
14. Ting DS, Tay-Kearney ML, Kanagasingam Y. Light and portable novel device for diabetic retinopathy screening. Clin Exp Ophthalmol. 2012;40:e40–6.
15. Keeffe JE, Weih LM, McCarty CA, Taylor HR. Utilisation of eye care services by urban and rural Australians. Br J Ophthalmol. 2002;86:24–7.
16. Mansberger SL, Gleitsmann K, Gardiner S, Sheppler C, Demirel S, Wooten K, et al. Comparing the effectiveness of telemedicine and traditional surveillance in providing diabetic retinopathy screening examinations: a randomized controlled trial. Telemed J E Health. 2013;19:942–8.
17. Ramchandran RS, Yilmaz S, Greaux E, Dozier A. Patient perceived value of teleophthalmology in an urban, low income US population with diabetes. PLoS One. 2020;15:e0225300.
Keywords:

Ocular screening; Physician perceptions; Patient perceptions; Remote care; Teleophthalmology; Telemedicine

Copyright © 2022, the Chinese Medical Association.