Validity and Acceptance of Color Vision Testing on Smartphones : Journal of Neuro-Ophthalmology

Secondary Logo

Journal Logo

Original Contribution

Validity and Acceptance of Color Vision Testing on Smartphones

Ozgur, Omar K. MD; Emborgo, Trisha S. BA; Vieyra, Mark B. BA; Huselid, Rebecca F. PhD; Banik, Rudrani MD

Author Information
Journal of Neuro-Ophthalmology 38(1):p 13-16, March 2018. | DOI: 10.1097/WNO.0000000000000637

Abstract

Color vision testing (CVT) is an important part of the standard ophthalmic examination. It is necessary for evaluating and monitoring progression of ophthalmic disease. Ishihara color plates (ICP) were first published in 1906 and are the most commonly used color vision test worldwide (1).

Several properties may affect the validity of CVT including the quality of the printed or displayed image, the viewing distance of the image, background illumination, and an individual's visual acuity. With the advent of new technologies, attempts have been made to streamline the process of CVT.

iPhone- and Android-based smartphones currently are in use by most physicians in the United States (2). In addition to being communication devices, smartphones provide applications that may help with the care and management of patients. As hardware and software evolve, smartphone-based testing modalities may aid ophthalmologists in performing more efficient ophthalmic examinations (2). Smartphone applications such as the Eye Handbook (EHB) (http://www.eyehandbook.com/) (3) provides CVT and other ophthalmic tools for iPhone- and Android-based smartphones.

However, there is minimal evidence to support the validity of these nonstandardized tools. Campbell et al (4) found a strong correlation between testing color vision with ICP and a CVT application on an iPad. Shah et al (5) presented data on 100 normal subjects tested with iPhone EHB and with standard ICP. They found no significant difference between the tests, although 89 of those tested had no color vision deficits on either test.

Our study sought to compare the results of CVT on smartphones using EHB and ICP in patients with a range of ocular pathology and healthy controls.

METHODS

An institutional review board–approved prospective study was performed on individuals between the ages of 18–85 years with Snellen near visual acuity of 20/100 or better at 14 inches, the minimum resolution required for accurate testing with ICP (6). Subjects were recruited from the comprehensive and subspecialty eye clinics at a tertiary eye hospital. Patients 50 years and younger were considered in the younger age range, whereas those older than 50 years were considered in the older age range. All individuals underwent a complete ophthalmic examination including dilated funduscopic examination. The study group included patients with ocular pathology and was grouped by disorders of the retina, optic nerve (nonglaucomatous optic neuropathy and glaucoma were analyzed as separate categories), cornea and/or ocular surface disease lens, orbit, or known congenital color deficiency. If a subject had more than 1 ocular pathology, that person was assigned to the category believed to most compromise visual function. The control group included those with no known pathology by history and examination. Patients with a history of previous ocular surgery were excluded from the study.

CVT was performed with both EHB and ICP under a standardized background illuminance, as measured with Sper Scientific Light Meter FC—840021 (Sper Scientific Direct, Scottsdale, AZ) before turning on the phone and performing any study testing.

Color vision was assessed using the Ishihara color vision test (The Series of Plates Designed as a Test for Colour-Deficiency: Concise Edition Book, Kanehara & Co, Lt, Tokyo, Japan). In the 14-plate edition, a total of 11 plates were used: plates 1–8, 10, 12, and 13. An Apple iPhone 4 with retina display and maximal brightness of 500 cd/m2 (7) and a Samsung Infuse 4G with maximal brightness of 300 cd/m2 (personal communication with Samsung Technical Support, October 2014) were used to conduct the EHB color vision tests using full brightness screen illumination.

The study was randomized by order of testing and phone model. Each eye was tested independently. The eye with pathology was considered the study eye. If both eyes were affected, the eye with the worse vision was considered the study eye. Each subject was presented with eleven color plates; each modality scored by number of correct plates of eleven test plates. Subjects, who failed more than 2 plates, were considered color vision deficient (CVD) (8).

Statistical analyses were performed using IBM SPSS version 22 (SPSS, Inc, Armonk, NY) software. The difference between the 2 methods was plotted against the mean of the 2 scores using the Bland–Altman plot with limits of agreement (LOA) at the 95th percentile of differences in score (9). The difference between ages of the control and study groups was calculated and tested using an independent samples t test with 95% confidence interval (CI). Pearson χ2 tests were used to analyze the subject's method preference.

RESULTS

Of 206 subjects enrolled, 10 did not complete all required study testing, and 7 withdrew from the study after enrollment. Of the 189 remaining patients, 113 with ocular pathology were categorized in the study group and 76 with no ocular pathology were assigned to the control group.

Mean background illuminance for CVT was 727.7 cd/m2. Of 113 study subjects, 41 (36%) had a primary diagnosis of nonglaucomatous optic neuropathy, 22 (20%) had glaucoma or suspicion of glaucoma, 26 (23%) had cornea and/or ocular surface pathology, 11 (10%) had retinal pathology, 10 (9%) had cataracts, and 3 (3%) had thyroid eye disease. Thirty-five percent of study patients (39/113) had more than 1 diagnosis.

Study subjects had a mean age of 50.8 ± 17.5 years (range: 18–82 years) and 64% were female. Control individuals had a mean age of 38.3 ± 14.8 years (range: 18–74 years) and 63% were female. Using an independent samples t test, we found a significant difference in age for the control and study groups; t (187) = −5.123, P = 0.001, 95% CI −17.31 to −7.68, although not for sex (P = 0.343).

The Bland–Altman plot showed agreement between correct number of plates in EHB and ICP for the study subjects (bias, −0.24; LOA −1.92 to 1.42) (See Supplemental Digital Content, Figure E1, https://links.lww.com/WNO/A296). Agreement also was observed between the correct number of plates in EHB and ICP for the controls (bias, −0.01; LOA, −0.61 to 0.59) and CVD (bias, −0.50; LOA, −4.64 to 3.65) patients (See Supplemental Digital Content, Figures E2 and E3, https://links.lww.com/WNO/A297, https://links.lww.com/WNO/A298). The Bland–Altman LOA data are found in Supplemental Digital Content 1, (see Table E1, https://links.lww.com/WNO/A295).

Of the 113 study subjects, 12 (11%) were characterized as CVD, missing more than 2 of eleven plates on either the EHB and/or ICP. Study group individuals found to be CVD had the following diagnoses: optic neuropathy (10/12; 83%) and cornea/other ocular surfaces (2/12; 17%). Eleven of 12 CVD subjects were found to be CVD on both EHB and ICP testing. One CVD patient with a primary diagnosis of cornea/other ocular surfaces, combined with a diagnosis of cataracts, was found to be color deficient with 11 correct plates on the EHB, but 8 correct plates on the ICP. Two CVD subjects had an underlying diagnosis of congenital color deficiency. Both of these recognized 3 correct plates on the EHB; on ICP testing, 1 identified 3 correct plates and the other only 1 correct plate. The number of plate errors and discordances for study subjects without CVD and control subjects is shown in Supplemental Digital Content 2, (see Table E2, https://links.lww.com/WNO/A299). Using ICP as the “gold standard” for CVT, the sensitivity of EHB was 0.92 (95% CI 0.76–1.07) and the specificity of EHB was 1.00 (95% CI 1.00–1.00).

After CVT with both ICP and EHB, patients were asked which modality they preferred, as well as for additional comments. The overall testing preference was 59% for the EHB were as follows: “no comment” (43%), “clearer” (24%), “brighter” (18%), “sharper” (5%), “easier” (4%), and “too bright” (2%). Of this group, 3% preferred the large plate size of the ICP. Of study subjects in the older group, 34 (57%) preferred the EHB, 7 (12%) preferred the ICP, and 19 (32%) had no preference. In the younger group, 31 (59%) preferred the EHB, 7 (13%) preferred the ICP, and 15 (28%) had no preference.

The controls comments regarding EHB were as follows: “no comment” (41%), “clearer” (24%), “easier” (11%), “sharper” (8%), “brighter” (7%), and “too bright” (3%). Of the controls, 1% preferred the larger plate size of the ICP. Of control subjects, the older 12 (63%) preferred the EHB, 1 (5%) preferred the ICP, and 6 (32%) had no preference. In the younger group, 35 (61%) preferred the EHB, 7 (12%) preferred the ICP, and 15 (26%) had no preference.

The mean age for individuals who preferred the ICP and EHB was 43.4 ± 19.5 years and 45.7 ± 19.5 years and 45.7 ± 16.5 years, respectively. Using Pearson χ2 test, there was no significant difference between the control and study groups vs EHB and ICP (P = 0.63). The mean age for patients with no preference was 46.9 ± 18.8 years.

DISCUSSION

Previous studies have reported that approximately 60% of normal subjects identify all color plates correctly on ICP testing (8). In our study, 70 (92%) and 71 (93%) of control patients were able to identify all plates correctly on ICP and EHB, respectively. In our study group with ocular pathology, 78 (69%) identified 11 color vision plates correctly with ICP and 96 (85%) identified all 11 color vision plates correctly with EHB. These results suggest that CVT with the EHB may be easier to perform, thus the potential of missing color vision deficiencies.

We showed that there were no discrepancies in CVT between ICP and EHB in control subjects. Our results are in agreement with the study by Shah et al (5) in which there was no difference in ICP and EHB CVT testing in control patients. We found that individuals with ocular pathologies also had an agreement between ICP and EHB results on CVT. The Bland–Altman plot visually demonstrated that there were no positive nor negative trends for the differences between the ICP and EHB CVT testing. For the CVD group, our study had only 1 discordance in which the patient read all plates correctly on the EHB and missed 3 plates on the ICP; the CVD subject was not considered CVD with the EHB but considered CVD on the ICP. It is important to note that the error made by the CVD patient on EHB was either a misreading or an atypical error.

There are numerous factors which may potentially lead to discordant results between ICP and EHB CVT. Screen size, resolution, brightness, and color saturation of each smartphone model likely impacts the accuracy of EHB CVT. Smartphones also have differences in lighting uniformity on screen, which may lead to variability in colors seen on EHB. Lighting also may play a role; ICP testing is performed with front-light, whereas EHB on smartphones uses back light. Similarly, de Fez et al (10) suggested that mobile device applications should specify colorimetric characterizations, visualization conditions, illumination conditions, screen brightness, and/or distance for specific devices to ensure more reliable measurements.

Limitations of our study include the fact that the study and control groups were not age-matched; notably, the mean age of the study group was older than that of the control group. There was also a female preponderance in both the study and control groups. The pathologies studied were diverse, with small sample sizes for each category, including only 12 CVD individuals, which made it impossible to draw meaningful conclusions regarding expected EHB CVT results based on specific diagnoses.

Although CVT on smartphones may be a convenient bedside tool, there are disadvantages to such testing. One is the rapidly changing smartphone industry. New smartphone models are developed every 1 to 2 years and upgrades in screen displays may require frequent validity testing of applications such as EHB. In addition, applications which allow for digital CVT such as EHB are freely accessible to download. Individuals eager to score well on CVT may download the application and memorize the correct responses. Random order of administration of the plates or the institution of a passcode for eye care providers to allow a download of the application may help address these issues.

Despite the limited sensitivity of EHB in CVD individuals, most subjects preferred the EHB to the ICP because of the EHB's perceived clearer, easier, brighter, and sharper image. Patients who preferred the ICP cited the harsh lighting on the EHB and larger plate sizes on the ICP as reasons. Further studies evaluating CVD individuals are needed. CVT on smartphones using EHB may be useful as a screening test; however, the authors still recommended ICP testing in patients with suspected ocular pathology.

STATEMENT OF AUTHORSHIP

Category 1: a. Conception and design: O. K. Ozgur and R. Banik; b. Acquisition of data: O. K. Ozgur, T. S. Emborgo, M. B. Vieyra, and R. Banik; c. Analysis and interpretation of data: O. K. Ozgur, T. S. Emborgo, M. B. Vieyra, R. F. Huselid, and R. Banik. Category 2: a. Drafting the manuscript: O. K. Ozgur, T. S. Emborgo, R. F. Huselid, and R. Banik; b. Revising it for intellectual content: O. K. Ozgur, T. S Emborgo, R. F. Huselid, and R. Banik. Category 3: a. Final approval of the completed manuscript: O. K. Ozgur, T. S. Emborgo, M. B. Vieyra, R. F. Huselid, and R. Banik.

REFERENCES

1. Dain SJ. Clinical colour vision tests. Clin Exp Optom. 2004;87:276–293.
2. Chhablani J, Kaja S, Shah VA. Smartphones in ophthalmology. Indian J Ophthalmol. 2012;60:127–131.
3. Eye Handbook. Available at: http://www.eyehandbook.com. Accessed February 16, 2016.
4. Campbell TG, Lehn A, Blum S, Airey C, Brown H. iPad colour vision apps for dyschromatopsia screening. J Clin Neurosci. 2016;20:92–94.
5. Shah AA, Pasadhika S, Kim J, Wang M. Pseudoisochromatic color vision testing on an iPhone. Invest Ophthalmol Vis Sci. 2012;53:6399.
6. McCulley TJ, Golnik KC, Lam BL, Feuer WJ. The effect of decreased visual acuity on clinical color vision testing. Am J Ophthalmol. 2006;141:194–196.
7. Apple, iPhone 4—Technical Specifications. Available at: https://support.apple.com/kb/sp587?locale=en_US. Accessed February 16, 2016.
8. Birch J. Efficiency of the Ishihara test for identifying red-green colour deficiency. Ophthalmic Phsyiol Opt. 1997;17:403–408.
9. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;327:307–310.
10. de Fez D, Luque MJ, Garcia-Domene MC, Camps V, Pińero D. Colorimetric characterization of mobile devices for vision applications. Optom Vis Sci. 2016;93:85–93.

Supplemental Digital Content

Copyright © 2018 by North American Neuro-Ophthalmology Society