Secondary Logo

Evaluation of the Waggoner Computerized Color Vision Test

Ng, Jason S.*; Self, Eriko; Vanston, John E.; Nguyen, Andrew L.; Crognale, Michael A.

doi: 10.1097/OPX.0000000000000551
ORIGINAL ARTICLES
Free

Purpose Clinical color vision evaluation has been based primarily on the same set of tests for the past several decades. Recently, computer-based color vision tests have been devised, and these have several advantages but are still not widely used. In this study, we evaluated the Waggoner Computerized Color Vision Test (CCVT), which was developed for widespread use with common computer systems.

Methods A sample of subjects with (n = 59) and without (n = 361) color vision deficiency (CVD) were tested on the CCVT, the anomaloscope, the Richmond HRR (Hardy-Rand-Rittler) (4th edition), and the Ishihara test. The CCVT was administered in two ways: (1) on a computer monitor using its default settings and (2) on one standardized to a correlated color temperature (CCT) of 6500 K. Twenty-four subjects with CVD performed the CCVT both ways. Sensitivity, specificity, and correct classification rates were determined.

Results The screening performance of the CCVT was good (95% sensitivity, 100% specificity). The CCVT classified subjects as deutan or protan in agreement with anomaloscopy 89% of the time. It generally classified subjects as having a more severe defect compared with other tests. Results from 18 of the 24 subjects with CVD tested under both default and calibrated CCT conditions were the same, whereas the results from 6 subjects had better agreement with other test results when the CCT was set.

Conclusions The Waggoner CCVT is an adequate color vision screening test with several advantages and appears to provide a fairly accurate diagnosis of deficiency type. Used in conjunction with other color vision tests, it may be a useful addition to a color vision test battery.

*OD, PhD, FAAO

PhD

BS

Southern California College of Optometry, Marshall B. Ketchum University, Fullerton, California (JSN, ALN); Department of Psychology (ES) and Department of Mathematics (ALN), California State University, Fullerton, California; and Department of Psychology, University of Nevada, Reno, Nevada (JEV, MAC).

Jason S. Ng, Southern California College of Optometry, Marshall B. Ketchum University, 2575 Yorba Linda Blvd, Fullerton, CA 92831, e-mail: jng@ketchum.edu

Over the past 50 years, clinicians have used a standard set of color vision tests including pseudoisochromatic plate (PIP) tests such as the Ishihara or HRR (Hardy-Rand-Rittler), the Farnsworth D15, and the anomaloscope. These common tests are sometimes supplemented with additional tests such as the Farnsworth-Munsell 100 hue test, desaturated versions of the D15, and the Medmont C100 test.1,2 The PIP tests are used to screen (or detect) for color vision deficiencies (CVDs), whereas the D15 and other tests are used to provide a diagnostic type (e.g., protan or deutan) and severity grading (e.g., mild, moderate, or severe). The anomaloscope is considered the standard for the most accurate screening and determining diagnostic type, whereas severity categorization does not have a single standard. In recent years, versions of common tests and new tests have been developed that are designed to be presented on computer screens.3–10 Computerized testing is appealing for a number of reasons (e.g., the ability to display colors impossible to print, convenient randomization of stimuli, and automatic scoring). However, producing an accurate digital color vision test for commercial/public use is challenging because the product must be able to reproduce specific chromaticities while using different computer and display hardware. Strictly speaking, such a product is not possible without specifying the hardware and software settings. Yet, it is possible that some degree of variability in chromaticities may be tolerable for color vision testing and an affordable software-based color vision test might perform adequately in real-world conditions.

The Waggoner Computerized Color Vision Test (CCVT) is a new PIP color vision test with a screening section designed to detect protan, deutan, and tritan CVDs. Additionally, it has diagnostic sections that allow the determination of type and severity of the CVD identified by the screening section. The primary purpose of this investigation was to evaluate the performance of the CCVT against standard clinical PIP tests.

Back to Top | Article Outline

METHODS

A prospective, observational, multicenter trial was conducted. One author (ES) contributed most of the data from subjects with normal color vision (CVN), and the other authors contributed most of the data from subjects with CVD.

The CCVT was administered to 59 CVD subjects and 361 CVN subjects (Table 1). The subjects were also tested with the 24-plate Ishihara test, the HRR test, and an anomaloscope (Table 2). The CCVT was conducted either before or after all of the other tests. For a subset of subjects (24 CVD and 7 CVN), the CCVT was given twice, once using the default setting of the computer monitor and another time after the computer screen had been set to a correlated color temperature (CCT) of 6500 K (i.e., standard illuminant D65 used in color matching/specification and computer graphics).11 For the subset sample for which the CCVT was administered twice, it was given first and last to minimize learning/practice effects, with the default and set CCT (“SetCCT”) conditions being alternated between first and last for alternate subjects.

TABLE 1

TABLE 1

TABLE 2

TABLE 2

Back to Top | Article Outline

Subjects

Subjects, mostly undergraduate students, were recruited by one author (ES) from general advertisement in their academic department. The other authors, at their respective institutions, recruited subjects sequentially presenting for clinical color vision assessments and from an existing participant bank. The mean (±SD) age for all subjects was 22.3 (±8.4) years. Normal color vision subjects (21.2 ± 6.8 years) were, on average, younger than CVD subjects (29.0 ± 13.1 years). The determination of CVN or CVD status for each subject was made by anomaloscopy (i.e., CVN, midpoint and range within 2 SDs of the normative data for each site). Mean (±SD) visual acuity for all subjects was 0.03 (±0.1) logMAR (logarithm of the minimum angle of resolution). Subjects were not given comprehensive eye examinations as part of the study but were assumed free from overt ocular pathology based on case history and measured visual acuity. All subjects provided consent. The procedures complied with the Declaration of Helsinki and were approved by our organizations’ institutional review boards.

Back to Top | Article Outline

Materials/Procedures

Methods and materials are presented in Table 2. All tests, except for the anomaloscope, were presented binocularly. Color tests were illuminated with a Verilux F15T8 fluorescent tube (CCT, 6280; CRI,12 94.5), and the illuminance was about 500 lux as measured at the testing surfaces. Subjects were considered to have failed the HRR screening test if any symbols on any plates were missed, and they failed the Ishihara test if three or more screening plates were missed (i.e., any answer that did not match the key provided with the test).

The anomaloscope match midpoint was used as the standard for determining whether a subject was color normal or color deficient.13 The anomaloscope matching range was used as a standard to differentiate between anomalous trichromacy, including extreme anomalous trichromacy, and dichromacy. Subjects with dichromacy will accept the entire matching range on the anomaloscope (typically 0 to 73), although misdiagnosis of dichromacy is possible in some instances.14 Classification of subjects by anomaloscopy in accord with Pokorny15 is shown in Table 3.

TABLE 3

TABLE 3

The CCVT was performed online via the creator’s Web site (www.testingcolorvision.com). Test instructions were followed for the CCVT, including having no glare on the monitor and a testing distance of 24 to 30 in. Monitors used in the study used native refresh rates and pixel resolutions. Monitors were set to a CCT of 6500 K for the SetCCT condition and set to factory default settings for the default condition, which was measured to have a CCT of about 5500 K. The CCVT uses numbers as targets and consists of four sections (Fig. 1): (1) a screening test (30 plates) that starts with a single demonstration plate analogous to the demonstration plate in the Ishihara test (the screening test includes vanishing and transformation plate designs for all three types of CVDs, i.e., protan, deutan, and tritan), (2) a diagnostic test using 32 protan plates, (3) a diagnostic test using 32 deutan plates, and (4) a diagnostic test using 12 tritan plates. The diagnostic tests use only the vanishing plate design. Each plate was displayed for 2 seconds, after which a screen appears with nine possible number choices (including “nothing”). The subjects selected among the nine possible choices and were instructed to guess as needed.

FIGURE 1

FIGURE 1

A passing score on the CCVT screening test was 26 or more plates correct. All the subjects, including those who passed the screening test, took all three subsequent diagnostic tests. Each diagnostic test had predetermined cutoff scores for pass/fail and severity determination from mild to moderate to severe. It is possible for the test to return a diagnosis of a tritan deficiency in addition to a red-green deficiency. A subject was determined to be protan if he or she scored lower on the protan test compared with the deutan test. Similarly, a subject was classified as deutan if he or she scored lower on the deutan test compared with the protan test. If the deutan and protan test scores were equal, as is possible with the HRR diagnostic plates, the diagnostic type was considered unclassifiable. If a subject identified neither or both (and the numerals within a plate were reported to have equal visibility) diagnostic plates of the Ishihara test or the two diagnostic plates gave opposing diagnoses, the diagnostic type was considered unclassifiable by the Ishihara test.

Back to Top | Article Outline

Data Analysis

All statistical analyses were performed in Stata 12.1 (StataCorp, College Station, TX) or R.16 For all tests, statistical significance was chosen to be p ≤ 0.05. Overall sensitivity and specificity was calculated (along with their binomial confidence intervals) for the screening section of each of the three PIP tests: CCVT, Ishihara, and HRR. Additionally, correct classification proportions were calculated for the tests with reference to the CCVT for both diagnostic type and severity.

Finally, comparisons were made between the results of the default condition and the SetCCT condition. For subjects who were tested under both conditions, only the first CCVT test result was used in this analysis. The Fisher exact test was used to analyze the pass rates for the two independent samples.

Back to Top | Article Outline

RESULTS

Screening Ability

The screening performance of the test is shown in Table 4 along with the results from the Ishihara and HRR tests. The CCVT achieved high sensitivity and specificity that were not statistically significantly different compared with the Ishihara and HRR tests.

TABLE 4

TABLE 4

Back to Top | Article Outline

Type Diagnostic Ability

Classification of diagnostic type by the tests relative to the CCVT is shown in Table 5. For subjects with CVD, the CCVT had a lower unclassifiable rate compared with the Ishihara and HRR tests. The CCVT classified subjects more similarly to the anomaloscope than the HRR or Ishihara tests.

TABLE 5

TABLE 5

Back to Top | Article Outline

Severity Diagnostic Ability

The HRR test was found to classify (i.e., mild, moderate, or severe) 29 of 54 (54%; 95% confidence interval [CI], 0.40 to 0.67) subjects the same as the CCVT. Direct comparison of the raw data for extent by the CCVT and HRR is shown in Table 6. Of the 25 instances of disagreement, 23 resulted in the CCVT assigning subjects to a more severe category as compared with the HRR. Fig. 2 shows CCVT and HRR classifications compared with the anomaloscope matching range. Although 11 of the 13 subjects with dichromacy were determined to be severe by the CCVT, discrimination of subjects with small matching ranges was poor as they could be found in any severity category. In general, the CCVT appears to diagnose subjects as having a more severe deficiency than other tests. When the CCVT gives a mild or moderate severity, it is generally mild based on the anomaloscope, whereas when the CCVT gives a grading of severe, a subject may have any range of severity based on the anomaloscope. Comparatively, a grading of medium on the HRR test has some more utility as compared with a grading of medium on the CCVT.

TABLE 6

TABLE 6

FIGURE 2

FIGURE 2

Back to Top | Article Outline

Default versus SetCCT Monitor Settings

Using the CCVT as a screening test only (i.e., analysis of the proportion passing the test), the default (78% passed; 95% CI, 72 to 83%) versus SetCCT (87% passed; 95% CI, 82 to 91%) conditions were significantly different (p = 0.017). The same analysis examining only data from subjects with CVD (default condition, 6.1% passed; 95% CI, 1.3 to 16.9%; SetCCT condition, 5.9% passed; 95% CI, 0.7 to 19.7%) did not show a significant difference (p > 0.05), but this was an underpowered analysis.

Back to Top | Article Outline

Subset of Subjects Performing CCVT under Both Monitor Settings

Although there was not enough statistical power to analyze the paired data, we note that the 7 CVN subjects and the 24 CVD subjects who repeated the CCVT had screening test results that were the same for both test presentations. Of the 23 CVD subjects who failed, 6 had different diagnostic results on the two runs of the CCVT (Table 7).

TABLE 7

TABLE 7

One subject with tritan deficiency was enrolled in the study, but we excluded these data from the aforementioned analyses. The subject’s aggregate results on all clinical tests, including a measured Moreland matching range (using the HMC II with 4 degrees attachment), supported the diagnosis. The subject passed the screening portion of the CCVT (which included tritan plates). The tritan diagnostic section of the CCVT failed the subject, but this section would not normally be presented unless the screening test was failed.

Back to Top | Article Outline

DISCUSSION

We have evaluated the CCVT in a population of color normal and color vision–deficient subjects. In general, the screening performance of the CCVT was very similar to the other PIP tests, whereas the diagnostic performance was not.

The sensitivity and specificity of the CCVT were high and not statistically significantly different from the Ishihara and HRR tests. The point estimate of sensitivity for the HRR in this study was slightly lower than that found by Cole et al.,17 who used a more stringent fail criterion.

Diagnostically, the CCVT was relatively good at categorizing subjects as either having a deutan or protan deficiency (90% correctly classified compared with the anomaloscope). Cole et al.17 reported a point estimate of 86% correctly classified by the HRR and thus the CCVT appears to perform similarly to the HRR in this respect and better than the Ishihara test.2

Severity categorization by the CCVT was not as useful as the HRR test. The CCVT generally appears to categorize subjects with mild CVD appropriately, whereas the HRR has a bit more discrimination among the severity categories relative to the anomaloscope range. Some of the subjects with dichromacy included in the study were labeled as “moderate” severity by the CCVT and the HRR. Cole et al.17 also found this occurrence with the HRR test.

In comparing results between the default and SetCCT monitor conditions, the specificities were essentially the same, but the sensitivities could not be rigorously assessed because of the lack of statistical power. However, descriptive statistics showed that 18 of 24 (75%) subjects had the same diagnosis (type and extent) with either monitor condition. Of the six subjects with disparate results (Table 7), the SetCCT condition appeared to have somewhat better agreement with the anomaloscope than the default condition.

Given the widespread use of software applications and their varied platforms (e.g., tablet and smartphone) for presentation, it is now commonplace to find vision testing software. Although it is quite easy to present such software on multiple platforms, for color vision testing, it may not be trivial. In this study, we used relatively common desktop computers and monitors. However, our results may not extend to other hardware/software configurations. Pigment-based tests once printed should have the same spectral reflectance profiles and, when used with a standard illuminator, should always have the same spectral profile distribution presented to the eye (note, however, that many printed pigments can potentially fade with time and light exposure—although the extent is unknown). Software applications have their own inherent advantages, but those involving color stimuli are susceptible to variable testing conditions with changes in hardware (computers, displays, and display settings), software (display drivers and color management within applications), and viewing environment. Thus, generalizability of any computerized color vision software is suspect. Nonetheless, the screening performance of the CCVT found in this study is quite good.

Although the CCVT has definite advantages over many traditional clinical color vision tests, its use would still benefit from oversight. The CCVT improves upon most traditional tests in that it can be self-administered, has strict control over presentation time per plate, can automatically randomize order of plate presentation, and has automated scoring. Conventional cautions are still needed in its use, however. It would still be possible to cheat on the test perhaps by viewing it with a software application on a smartphone, using a colored filter,18,19 or simply having another individual supply the answers during the test. Thus, the CCVT does not eliminate oversight by an examiner, although with the CCVT test, the examiner less likely needs to be a color vision specialist or even a health care provider. Additionally, a single examiner could perhaps administer multiple tests simultaneously—a difficult task with most traditional tests.

When used carefully and appropriately, the CCVT may be sufficient as a screening color vision test. Additionally, it may be used to provide a diagnosis of type that is as accurate as the Richmond HRR. However, the CCVT has a tendency to assign subjects as more severe than does the HRR. Like other PIP tests, a result from the CCVT would likely require confirmation using additional color vision tests conducted by an experienced clinician, especially with regard to the severity of the CVD.

Jason S. Ng

Southern California College of Optometry

Marshall B. Ketchum University

2575 Yorba Linda Blvd

Fullerton, CA 92831

e-mail: jng@ketchum.edu

Back to Top | Article Outline

ACKNOWLEDGMENTS

The authors thank Christy Guenther, OD, Brian Shih, BS, and Sophia Liem, BS, for their contributions to data entry and data management. Additionally, appreciation is given to the creators of the test for supplying the test for our research use. The Waggoner CCVT is now marketed also as ColorDx. The software has been ported to other device platforms and changed in some ways, which were not evaluated in this study.

None of the authors has any conflicts of interest to disclose.

Received July 16, 2014; accepted February 5, 2015.

Back to Top | Article Outline

REFERENCES

1. Dain SJ. Clinical colour vision tests. Clin Exp Optom 2004; 87: 276–93.
2. Cole BL. Assessment of inherited colour vision defects in clinical practice. Clin Exp Optom 2007; 90: 157–75.
3. Rabin J, Gooch J, Ivan D. Rapid quantification of color vision: the cone contrast test. Invest Ophthalmol Vis Sci 2011; 52: 816–20.
4. Regan BC, Reffin JP, Mollon JD. Luminance noise and the rapid determination of discrimination ellipses in colour deficiency. Vision Res 1994; 34: 1279–99.
5. Goulart PR, Bandeira ML, Tsubota D, Oiwa NN, Costa MF, Ventura DF. A computer-controlled color vision test for children based on the Cambridge Colour Test. Vis Neurosci 2008; 25: 445–50.
6. Mancuso K, Neitz M, Neitz J. An adaptation of the Cambridge Colour Test for use with animals. Vis Neurosci 2006; 23: 695–701.
7. Barbur JL, Harlow AJ, Plant GT. Insights into the different exploits of colour in the visual cortex. Proc Biol Sci 1994; 258: 327–34.
8. Miyahara E, Pokorny J, Smith VC, Szewczyk E, McCartin J, Caldwell K, Klerer A. Computerized color-vision test based upon postreceptoral channel sensitivities. Vis Neurosci 2004; 21: 465–9.
9. Regan BC, Freudenthaler N, Kolle R, Mollon JD, Paulus W. Colour discrimination thresholds in Parkinson’s disease: results obtained with a rapid computer-controlled colour vision test. Vision Res 1998; 38: 3427–31.
10. Rodriguez-Carmona M, O’Neill-Biba M, Barbur JL. Assessing the severity of color vision loss with implications for aviation and other occupational environments. Aviat Space Environ Med 2012; 83: 19–29.
11. Berger A, Strocka D. Quantitative assessment of artificial light sources for the best fit to standard illuminant d65. Appl Opt 1973; 12: 338–48.
12. International Lighting Vocabulary Standard CIE S 017/E:2011. Vienna, Austria: Commission Internationale de I’Eclairage; 2011.
13. Committee on Vision, National Research Council. Procedures for Testing Color Vision: Report of Working Group 41. Washington, DC: The National Academies Press; 1981. Available at: http://www.ncbi.nlm.nih.gov/books/NBK217818/pdf/TOC.pdf. Accessed January 30, 2015.
14. Alpern M, Wake T. Cone pigments in human deutan colour vision defects. J Physiol 1977; 266: 595–612.
15. Pokorny J (Ed). Congenital and Acquired Color Vision Defects, New York, NY: Grune & Stratton; 1979.
16. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2013. Available at http://web.mit.edu/r_v3.0.1/fullrefman.pdf. Accessed January 30, 2015.
17. Cole BL, Lian KY, Lakkis C. The new Richmond HRR pseudoisochromatic test for colour vision is better than the Ishihara test. Clin Exp Optom 2006; 89: 73–80.
18. Swarbrick HA, Nguyen P, Nguyen T, Pham P. The ChromaGen contact lens system: colour vision test results and subjective responses. Ophthalmic Physiol Opt 2001; 21: 182–96.
19. Hovis JK. Long wavelength pass filters designed for the management of color vision deficiencies. Optom Vis Sci 1997; 74: 222–30.
Keywords:

color vision; color vision testing; color vision deficiency

© 2015 American Academy of Optometry