Since its introduction in 1989,1 the Pelli-Robson Chart has been used in over 700 published studies including in large-scale studies.2 It is a letter chart with optotypes of constant size and varying levels of contrast. It is easy to administer and has documented repeatability,3–8 especially when a by-letter scoring method is used.4 It may be considered the gold standard for clinical contrast sensitivity measurement and has spawned similar tests including the Mars Letter Contrast Sensitivity Test.8–12
Over the last 15 years, several commercially available computer-based contrast sensitivity tests have become readily available, potentially offering some advantages over printed tests. These advantages include allowing the user to specify the letter size and contrast steps, while providing automatic scoring and potentially less susceptibility to letter fading and fingerprints.13 These tests may require some technical expertise and equipment to properly calibrate the luminance and contrast of the monitor, and may not be very portable. One example that is adaptable, freely available, and widely used is the Freiburg Visual Acuity and Contrast Test (http://www.michaelbach.de/fract/index.html).14
More recently, the introduction of tablet devices with high-quality displays may offer further opportunity for portable, rapid, and convenient testing in the clinical setting or the field. Unlike computer systems where screen technology and format varies, tablet displays are fixed and predictable. In particular, an iPad has a Retina display with a resolution of 264 pixels per inch and 8-bit color (http://www.apple.com/ipad/specs/). An iPad-based letter contrast sensitivity test was developed by Ridgevue Vision (http://ridgevue.com). The test was created using iBooks Author and is commercially available through the iTunes Store. Each page displays two letters, with each page having 0.1 log units lower contrast than the previous page.
In the present study, we assess the agreement among the Ridgevue Letter Contrast Sensitivity Test, the Pelli-Robson Test, and the Freiburg Test and the repeatability of the three tests in normally sighted subjects and those with low vision.
Study procedures were approved by the Indiana University Human Subjects Office and Institutional Review Board. Informed consent was obtained from all participants before testing and after an explanation of the procedures was given.
Contrast Sensitivity Tests
The iPad Contrast Sensitivity test consists of 46 letters created in iBooks Author and arranged in 23 pages. Each page consists of two letters of equal contrast. The contrast of the letters on each subsequent page decreases by 0.1 log units from −0.10 (80% contrast) to −2.30 (0.5% contrast). The lower contrast levels are achieved using a technique known as bit stealing wherein the luminance of the red, green, and blue pixels is adjusted independently.15 The letters are the same 10 British letters used on the Bailey-Lovie Visual Acuity Chart, all having an aspect ratio of 5:4. Each letter subtends 2.8 × 2.2 degrees at the test distance of 1 m (equivalent to 20/672). The subject was seated with their head against a headrest while a 1-m-length string attached to each chart was held by the examiner at the subject’s eye to assure a 1-m testing distance.
The Pelli-Robson test measures approximately 59 × 84 cm and is printed on rigid cardboard. It consists of 48 letters arranged in eight rows of six letters each. Each line consists of two triplets of letters. Each triplet contains letters of equal contrast, and the contrast of each triplet decreases by a factor of 0.15 log units. Stated contrast varies from 100% (0.00 log units) to 0.56% (–2.25 log units). Each letter subtends 2.8 × 2.2 degrees at the test distance of 1 m (equivalent to 20/672).
The computer-based Freiburg Visual Acuity and Contrast Test (FrACT version 3.7.1)14 uses a Landolt C target of variable contrast (0.1% to 99.9%) presented at eight possible orientations. The subject used a standard numeric keypad to select which of the eight orientations the opening of the C is located. The Best PEST procedure16 was used with 30 trials (every sixth trial at 3× threshold).
All testing was monocular and performed at 1 m with habitual correction and an appropriate near add. All subjects over 40 years of age were provided with a +0.75 D near add. Each normally sighted subject used his or her right eye, but each low-vision subject was allowed to use his or her better eye. To be classified as low vision, the best-corrected visual acuity had to be 20/30 or poorer in the better-seeing eye.
All testing was performed in the same room. An iPad 3 was placed on a stable desktop and the room lights turned off. Within the iPad settings, the auto brightness was turned off and brightness adjusted to the middle of the scale. The screen luminance was around 150 cd/m2. The Pelli-Robson Chart was illuminated by multiple ceiling mounted fluorescent lamps. Chart luminance ranged from 120 to 160 cd/m2. For the Freiburg test, the room lights were off, and the background screen luminance of an Apple Powerbook Pro (2.66 GHz Intel Core 2 Duo, OS 10.6.8) ranged from 120 to 160 cd/m2.
Contrast sensitivity was measured with the iPad test, the Pelli-Robson tests, and the Freiburg test. After a short break, contrast sensitivity was measured again using alternate forms of all tests. The chart forms used (two Pelli-Robson and two iPad) and the order of testing were varied systematically. One third of the subjects were tested first with the Pelli-Robson test, one third first with the iPad test, and one third first with the Freiburg test. For all subjects, the order of testing was reversed for the second administration.
For both letter contrast sensitivity tests, subjects were instructed to read all letters on each contrast level, beginning with the highest contrast letters. Sideways head movements were allowed. Subjects were allowed up to 30 seconds per letter if needed and forced to guess until the stopping rule was reached. Any of the 26 letters in the alphabet were accepted as a possible response, not just the ten-letter set used on the charts.
Testing on the iPad test ended when the subject missed both letters at a given contrast level. Contrast sensitivity was scored as the last level when at least one letter was identified correctly minus 0.05 for each letter incorrect on that and previous levels. This also represents 0.05 multiplied by the total number of letters correct. Testing on the Pelli-Robson test ended when the subject missed two of three letters in a triplet. Contrast sensitivity was scored as 0.05 multiplied by the number of letters correct minus 0.15 because the first three letters are 100% contrast.4 For the Freiburg test, there were 30 200-millisecond presentations displayed on a gamma corrected monitor, with each presentation separated by approximately 1 second. The subject was required to respond to each presentation and scoring was performed and reported automatically.14
For both letter tests, all subject responses were recorded on standard score sheets. When an incorrect response was made, the incorrect letter was recorded on the sheet to complete scoring calculations later. For the Pelli-Robson chart, responses of “O” for “C” and “C” for “O” were accepted as correct when scoring and when applying the stopping rule.17 This represents a small but accepted deviation from the original published instructions.1
All data were analyzed as the log of Weber contrast sensitivity. Repeatability and agreement were assessed by determining the 95% limits of agreement (LoA), which correspond to ±1.96 SD of the differences between administrations or tests. The difference between the scores for each administration or test was calculated for each subject. The distribution of these differences was described by calculating the mean, SD, and the 95% LoA. The breadth of these LoA indicates the repeatability of the test. The narrower the LoA, the more repeatable the test.18 The repeatability of each test was compared by calculating the ratio of the test-retest variances (F-ratio) between pairs of tests.
A repeated measures multivariate analysis of variance (MANOVA) was used to compare the values from the three tests utilizing a Wilks Lambda criteria. For this analysis, the subject group (low vision, normal vision) represented the between-subjects investigation and the method (iPad, Pelli-Robson, Freiburg) and trial (one, two) were within-subjects investigations. Finally, pairwise comparisons with adjusted significance levels based on the Bonferroni inequality were performed for factors that reached significance in the MANOVA. In this instance of multiple testing, the Bonferroni correction sets the maximum family-wise error rate equal to the sum of the error rates for the individual comparisons.
Twenty normally sighted subjects (age: 21–38 years) and 20 low-vision subjects (age: 18–89 years) were recruited. Median visual acuity in the low vision group was 20/60 and the range was 20/30 to 20/200. Six subjects had visual acuity worse than 20/100.
On average across both the normal- and low-vision groups, contrast sensitivity scores on the second trial were higher than the first by 0.02 units (F = 5.35, p = 0.02). Although this trial effect was not dependent on the vision group, it did vary as a function of testing method (F = 3.72, p = 0.03). Specifically, the second administration of the iPad test was significantly (mean +0.08 units) higher than the first (t = 3.71, p = 0.002). Mean (±SD) contrast sensitivity for all three tests and for each administration in the normally sighted subjects and low-vision subjects are shown in Tables 1 and 2, respectively. The multivariate test of differences between the low- and normal-vision groups was statistically significant (p < 0.001). Specifically, on average the normal-vision group (Table 1) had better contrast sensitivity (0.20 units) compared to the low-vision group (Table 2) with all testing methods, as expected.
The repeatability of the iPad, Pelli-Robson, and Freiburg tests in the normally sighted (open symbols) and low-vision (filled symbols) subjects is shown in Fig. 1, where the test-retest difference is plotted as a function of the mean of the two administrations. The shaded area indicates the 95% LoA for all subjects, while the dashed and dotted lines show the 95% LoA for the low vision and normally sighted subjects, respectively. The repeatability in the normally sighted subjects (Table 1) was quite similar (iPad = ±0.19, Pelli-Robson = ±0.19, Freiburg = ±0.15; F < 1.7, p > 0.13). The repeatability was poorer in the low-vision subjects, although it was similar among the three tests (iPad = ±0.24, Pelli-Robson = ±0.23, Freiburg = ±0.21; F < 1.3, p > 0.22).
Fig. 2 displays the agreement between the iPad test and the Freiburg test (Fig. 2A), the agreement between the Pelli-Robson test and the Freiburg test (Fig. 2B), and the agreement between the iPad and the Pelli-Robson test (Fig. 2C). In all cases, the difference between tests is significantly associated with the mean contrast sensitivity. Specifically, as the mean contrast sensitivity increases iPad contrast sensitivity begins to exceed that of the Freiburg test (r = +0.48, p = 0.002, Fig. 2A), Freiburg contrast sensitivity exceeds that of the Pelli-Robson test (r = –0.57, p < 0.001, Fig. 2B) and iPad contrast sensitivity exceeds that of the Pelli-Robson test (r = +0.72, p < 0.001, Fig. 2C). The 95% LoA are thus plotted symmetrically about the best-fit regression line as advocated by Bland and Altman.19
Evaluating the normal- and low-vision groups independently, the iPad test showed excellent agreement with the Freiburg test (Fig. 2A) in normal subjects (mean difference = +0.02, 95% LoA = −0.15 to +0.15, t = −1.24, p = 0.66), but lower scores than the Freiburg test in the low-vision subjects (mean difference = −0.09 ± 0.13, 95% LoA = −0.36 to +0.17, t = 5.81, p < 0.001). The Pelli-Robson gave lower contrast sensitivity scores than the Freiburg test (Fig. 2B) in both the normally sighted (t = 18.88, p < 0.001) and low-vision subjects (t = 15.29, p < 0.001). Note that all the data points fall below zero, indicating consistently lower scores with the Pelli-Robson test. Specifically, in the normally sighted subjects the mean difference was −0.30 (t = 23.1, p < 0.001, 95% LoA = −0.42 to −0.19) and similarly in the low-vision subjects the mean difference was −0.25 (t = 9.41, p < 0.001, 95% LoA = −0.48 to −0.02). As expected from the comparisons of the other tests, the Pelli-Robson also gave consistently lower contrast sensitivity scores than the iPad (Fig. 2C) in both normally sighted (t = −20.12, p < 0.001) and low-vision subjects (t = −9.48, p < 0.001).
All three tests of contrast sensitivity show similar test-retest repeatability. The repeatability of a letter contrast sensitivity test can be predicted based the number of letters per change contrast.4,9,20,21 In essence, contrast sensitivity tests on which each letter represents a smaller change in contrast should have better repeatability. On both the iPad and Pelli-Robson tests, each letter corresponds to 0.05 log units and thus it is no surprise that the two tests have similar repeatability.9 The Freiburg test uses an adaptive thresholding technique and while the test comprised 30 presentations, most will have been close to the subject’s threshold leading to better repeatability than the other tests. While the method of decreasing limits is used on the iPad test, and is also used on visual acuity charts, improvements in repeatability could be anticipated if an app were developed that incorporated a staircase procedure or an adaptive thresholding technique. Consistent with previous studies, the low-vision subjects gave less repeatable contrast sensitivity scores than individuals with normal vision.6,8 Given the 95% limits of agreement for the iPad test shown in Fig. 1, a change of 0.25 log units, or five letters, could be regarded as clinically meaningful.
Consistent with previous studies, the between-test LoA (Fig. 2) were always wider than the within-test (repeatability) LoA (Fig. 1), regardless of test or subject group. While the 95% LoA between tests are given in the Results section, the differences clearly vary with contrast sensitivity. Thus, Fig. 2 gives a better indication of the extent to which scores between tests may differ. A surprising finding of the present study was the lower contrast sensitivity scores for the Pelli-Robson test. The mean value of 1.65 in normally sighted subjects is around 0.3 lower than the mean for both the iPad and the Freiburg tests. The discrepancy in scores is smaller for the low-vision subjects, but still statistically significant (p = 0.001). Previous studies of the Pelli-Robson test have reported mean scores in normally sighted subjects between 1.70 and 1.88.3–8 Thus, our mean value of 1.65 is slightly poorer than this range whereas our scores for the other two tests are slightly better than this range. The range of normal scores is also very small (Table 1). Nonetheless, the repeatability of Pelli-Robson test scores in the present study was comparable to those of previous studies.3–8
The iPad contrast sensitivity test had comparable repeatability to the other tests. In the normally sighted subjects, however, the scores for the second administration were slightly but significantly higher than the first administration. This effect was not observed in the low-vision subjects but contributes to the asymmetric 95% LoA observed in Fig. 1. This small, potential learning effect unique to the iPad test may represent an improved familiarity of the viewer with the iPad or their improved concentration on the smaller screen. Agreement between the iPad and Freiburg tests was excellent in normally sighted subjects, but the iPad test gives slightly lower scores than the Freiburg test in low-vision subjects.
The 9.7-inch (diagonal) Retina display of the iPad has 2048 × 1536 resolution, which translates to 264 pixels per inch (http://www.apple.com/ipad/specs/). The contrast sensitivity test was calibrated for the luminance properties of the display on the model used and will not accurately display contrast on iPad models that do not have the Retina display. It is unclear how the properties of the display might change over time or how the images might display on subsequent models. These issues require further investigation and are germane to any computer-based measures. Given the results presented here, there are several physical features of the iPad test that may make it desirable for use in a clinical or research setting. Specifically, it is much easier to transport and store than large-format charts like the Pelli-Robson test, as well as typical computers/laptops. Additionally, the test is self-illuminated avoiding the need to install and calibrate appropriate lighting. In summary, our results suggest that the iPad contrast sensitivity test is appropriate for use in normally sighted as well as low-vision populations. The flexibility and repeatability of the Freiburg test makes it an attractive for contrast sensitivity when a desktop computer is available.
Mark A. Bullimore
356 Ridgeview Lane
Boulder, CO 80302
MAB is the sole owner of Ridgevue Vision. Ridgevue Vision sells the iPad Contrast Sensitivity Test through the iTunes Store. None of the other authors have any conflicts of interest.
Received May 28, 2013; accepted November 4, 2013
1. Pelli D, Robson J, Wilkins A. The design of a new letter chart for measuring contrast sensitivity
. Clin Vis Sci 1988; 2: 187–99.
2. Klein BE, Moss SE, Klein R, Lee KE, Cruickshanks KJ. Associations of visual function with physical outcomes and limitations 5 years later in an older population: the Beaver Dam eye study. Ophthalmology 2003; 110: 644–50.
3. Elliott DB, Bullimore MA. Assessing the reliability, discriminative ability, and validity of disability glare tests. Invest Ophthalmol Vis Sci 1993; 34: 108–19.
4. Elliott DB, Bullimore MA, Bailey IL. Improving the reliability of the Pelli-Robson contrast sensitivity
test. Clin Vis Sci 1991; 6: 471–5.
5. Elliott DB, Sanderson K, Conkey A. The reliability of the Pelli-Robson contrast sensitivity
chart. Ophthalmic Physiol Opt 1990; 10: 21–4.
6. Haymes SA, Chen J. Reliability and validity of the Melbourne Edge Test and High/Low Contrast Visual Acuity chart. Optom Vis Sci 2004; 81: 308–16.
7. Lovie-Kitchin JE, Brown B. Repeatability
and intercorrelations standard vision tests as a function of age. Optom Vis Sci 2000; 77: 412–20.
8. Dougherty BE, Flom RE, Bullimore MA. An evaluation of the Mars Letter Contrast Sensitivity
Test. Optom Vis Sci 2005; 82: 970–5.
9. Arditi A. Improving the design of the letter contrast sensitivity
test. Invest Ophthalmol Vis Sci 2005; 46: 2225–9.
10. Haymes SA, Roberts KF, Cruess AF, Nicolela MT, LeBlanc RP, Chauhan BC, Artes PH. Evaluation of the new Lighthouse Letter Contrast Sensitivity
Test. Invest Ophthalmol Vis Sci 2005; 46: 4605.
11. Rabin J, Wicks J. Measuring resolution in the contrast domain: the small letter contrast test. Optom Vis Sci 1996; 73: 398–403.
12. Rabin J. Small letter contrast sensitivity
: an alternative measure of visual resolution for aviation candidates. Aviat Space Environ Med 1995; 66: 56–8.
13. Ehrmann K, Fedtke C, Radić A. Assessment of computer generated vision charts. Cont Lens Anterior Eye 2009; 32: 133–40.
14. Bach M. The Freiburg Visual Acuity test—automatic measurement of visual acuity. Optom Vis Sci 1996; 73: 49–53.
15. Tyler CW. Colour bit-stealing to enhance the luminance resolution of digital displays on a single pixel basis. Spat Vis 1997; 10: 369–77.
16. Pentland A. Maximum likelihood estimation: the best PEST. Percept Psychophys 1980; 28: 377–9.
17. Elliott DB, Whitaker D, Bonette L. Differences in the legibility of letters at contrast threshold using the Pelli-Robson chart. Ophthalmic Physiol Opt 1990; 10: 323–6.
18. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1: 307–10.
19. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res 1999; 8: 135–60.
20. Raasch TW, Bailey IL, Bullimore MA. Repeatability
of visual acuity measurement. Optom Vis Sci 1998; 75: 342–8.
21. Bailey IL, Bullimore MA, Raasch TW, Taylor HR. Clinical grading and the effects of scaling. Invest Ophthalmol Vis Sci 1991; 32: 422–32.
Keywords:© 2014 American Academy of Optometry
contrast sensitivity; low vision; visual assessment; iPad; repeatability; Freiburg test; Pelli-Robson test