Automatic objective refractors, which estimate the refractive error without requiring either operator or patient judgment, have been available since 1969. These instruments are easy to operate, are quicker than other techniques of objective refraction such as retinoscopy, and are better appreciated by the patients.1
For these reasons, autorefractors are enjoying an increased popularity in ophthalmologic and optometric practice for objectively assessing the refractive error of patients and, in some situations, for completely dispensing with retinoscopy.
Although autorefraction produces a fast, repeatable measurement of refractive error, its validity is as important as its efficiency. Fortunately, however, most clinicians do not prescribe spectacles based exclusively on results from autorefraction. Assessment of agreement allows comparison with a “gold standard,” such as subjective refraction.2–4
In addition to use in clinical practice, autorefractors are also widely used in optometric and ophthalmic research (e.g., to examine refractive error development, accommodative responses, and comparison of preto-postoperative condition).5–7
For research purposes and clinical practice, the ideal autorefractor should provide valid and repeatable measurement of refractive error and be fast and easy to use and absolutely objective. Previous studies have found most models are valid, accurate, and reliable when compared with the subjective refraction. Nevertheless, for some instruments, pseudomyopia caused by accommodation and inadequate autofogging mechanisms have been reported.5,8–11
The aim of this study was to estimate the agreement between an autorefractor (Nidek ARK 700A, Gamagori, Japan) and retinoscopy with the subjective refraction. It was carried out on noncycloplegic eyes because, with the exception of Ireland and the U.K., in Europe it is forbidden for optometrists to use cycloplegic agents. In addition, most refractions in the U.S. and in the U.K. are also done without cycloplegia.
MATERIALS AND METHODS
One hundred ninety-two eyes from an equal number of health science students took part in this study. Sixty-four of these were male, and one hundred twenty-eight were female. The age range was 18 to 34 years, with 21.6 years as mean value with an SD of 2.66 years.
Measurements of refractive errors were performed after complete explanation of procedures to obtain informed consent from each patient. The research followed the rules of the Declaration of Helsinki, which were reviewed and approved by the Scientific Council of the School of Sciences at the University of Minho (Portugal).
The eye examinations took place at the Laboratory of Clinical Optometry, School of Sciences of the University of Minho, and the first author performed all the measurements using the same equipment and method as described below in all the examinations.
Autorefraction was carried out with the ARK 700A. The display was masked for the first author, and the second author made the readings. Three readings were taken for each eye, and the final autorefractor prescription was calculated from the average result. A hardcopy of the results was then printed.
The retinoscopy was performed in each eye over the phoropter lenses, attempting to refine the retinoscopy to be within a ±0.25 D range over the real power for the spherical and the cylindrical components and the axis to ±5° maximum error. The spherical power, cylindrical power, and cylinder axis display on the phoropter were covered so the examiner could not see them, as was proposed by Rosenfield and Chiu,12 for the spherical power display. Under these conditions, the first author acted as retinoscopist and refractionist in each case, whereas the second author recorded the results. The phoropter was reset to zero before each use by the second author.
Subjective refraction was performed monocularly. The traditional endpoint of maximum plus was used [i.e., the best visual acuity (VA) with the maximum plus], followed by cross-cylinder to locate the axis within 5° and its power within 0.25 D. The data obtained from both eyes were initially analyzed, and no significant differences were found between the left and the right eye. For this reason, only right eye measurements were submitted for analysis.
Previous studies used the coefficient of correlation to represent the agreement between methods. This statistical procedure has been considered inappropriate when comparing results from different instruments. Bland and Altman13 described a method of measuring test agreement by calculating the mean difference between measurements. Moreover, this statistical method can be used in comparisons among different tests. Since then, plots of differences against means were recommended as the best method to compare measurements obtained with different instruments when the actual measurement is unknown.14–16 In this study, the data were analyzed using the statistical package SPSS v. 11.5 (Chicago, IL). The bias was assessed statistically as the mean of the differences compared with zero. The hypothesis of zero bias was examined by a paired t-test. The 95% limits of agreement (mean of the difference ± 1.96 × SD of the differences) were also calculated. This type of analysis makes it easier to assess the level of agreement between techniques, spot outliers, and determine any underlying trend.
Assessing the variance in the astigmatism poses a problem in the conventional clinical notation (e.g., −2.25 DC × 10°). Therefore, the sphere, cylinder, and axis component were also converted into a vector representation:8,17
1. A spherical lens of power M (equal to the mean spherical equivalent = sphere + cylinder/2).
2. Jackson cross-cylinder at axis 0° with power J0 [equal to − cylinder/2*cos (2*axis)]
3. Jackson cross-cylinder at axis 45° with power J45 [equal to − cylinder/2*sin (2*axis)]
The refractive error of the sample, as represented by subjective refraction, ranged from −9.00 to +2.25 D mean spherical equivalent (mean ± SD, −0.29 ± 1.39 D). The maximum amount of astigmatism was −2.50 D.
Table 1 displays the mean and SD of M, J0, and J45 components of the orthogonal functions obtained with the autorefractor, retinoscopy, and subjective refraction. These values show that the autorefractor reads more negative or less positive when compared with the other methods for M component. Conversely, retinoscopy shows a mean value that is closer to that obtained by subjective refraction.
Table 2 presents the mean difference, level of statistical significance, and the limits of agreement within each pair of instruments to be compared at the 95% confidence interval. The comparison between the autorefractor and the values obtained by subjective refraction shows that for the component M, the autorefractor yields more negative values (−0.44 ± 0.54 D; p = 0.000); for the cylindrical vector J0, the autorefractor yields more positive values than the subjective ones (0.05 ± 0.13 D; p = 0.000); and for the J45 vector, the autorefractor results are the most negative (−0.02 ± 0.09 D; p = 0.019). The differences found for each component M, J0, and J45 are statistically significant. By comparing retinoscopy with the subjective examination, there are no statistically significant differences found for the M component (−0.02 ± 0.33 D; p = 0.304). For the J0 and J45 components, the differences are statistically significant (0.07 ± 0.10 D, p = 0.000; −0.01 ± 0.08 D, p = 0.008).
To graphically analyze the agreement between measurements obtained with different instruments, plots of differences as a function of the mean for each pair of techniques are displayed in Fig. 1. This analysis allows detection of any trend in difference variability as a function of the mean value to be measured.
Table 3 presents the percentage agreement between the data obtained for the autorefractor and retinoscopy with subjective refraction. We consider sphere and cylinder within ±0.25 D, cylinder axis within ±10°, and also the three conditions previously imposed sphere, cylinder, and axis (Sph + Cyl + Ax) simultaneously. The results display high agreement of subjective refraction with autorefractor and retinoscopy when cylinder power is considered. Agreement between retinoscopy and subjective refraction is also clinically acceptable when the spherical component of the refraction is considered.
In this study, three different techniques of refraction were compared without the use of a cycloplegic agent. In Portugal, as well as in other European countries, with the exception of Ireland and U.K., the use of diagnostic drugs in optometric practice is forbidden. Therefore, these results should be particularly useful for most European optometrists and also for those in the U.S. and in the U.K. when cycloplegic is not warranted.
Discrepancies have been reported between the results for refraction obtained with objective autorefraction, retinoscopy, and subjective methods in relation to the amount of astigmatism and to the degree of ametropia. One possible source of disagreement between methods is the presence of higher-order aberrations in the human eye, which could influence retinoscopy and autorefraction. The larger pupil size when performing autorefraction or retinoscopy compared with the pupil size when using subjective techniques could be responsible for such disagreement. The larger the amount of higher-order aberrations present, the greater the amount of disagreement.18
However, the usefulness of comparing autorefractors and retinoscopy against subjective refraction has been questioned. Because noncycloplegic subjective refraction is generally accepted by optometrists for adult prescribing, it provides the standard reference for autorefraction and retinoscopy measurements, and this methodology has been widely accepted by previous authors.1,11,19–21
Although there have been several studies published about the comparison between different refraction techniques, there are only a few that do not use a cycloplegic agent and use simultaneously the orthogonal functions proposed by Thibos17 as a form of representing the refractive error. At the same time, the statistical analysis of results is done using the Bland and Altman method.13 Obviously, a direct comparison of the results obtained in this study with those of previous studies is difficult to achieve.
The results obtained for the value of the spherical equivalent (M) show that the autorefractor values are more negative in the myopia and less positive in the hypermetropia than retinoscopy and subjective refraction. Similar results were also found by Bullimore et al. and Zadnik et al. for the autorefractor and subjective refraction values.2,8 Other studies, considering different models of autorefractor, show the same tendency of the autorefractor to underestimate the value of the refractive error in relation to the other two methods.6,9,22–25
Nayak et al.26 showed that without cycloplegic there was a greater tendency of the autorefractors to measure more negative or less positive values than subjective refraction.
It is interesting to verify that the retinoscopy and subjective refraction confidence interval (limits of agreement, ±0.65) is one-half that of the autorefractor and subjective refraction (limits of agreement, ±1.05).
This means that retinoscopy could be half a dioptre more precise than autorefraction in the estimation of an objective start point for noncycloplegic refraction. The differences found in this process are statistically significant. Similar results were found by Bullimore et al.8 and Kinge et al.9
Regarding astigmatic components, autorefractor gives more positive values than subjective refraction for the J0 vector, whereas the J45 component is more negative for autorefractor, as occurred with the spherical equivalent component (M). Pair comparisons of astigmatic components were found to be statistically significant.
The confidence interval for cylindrical components, J0 and J45, follows the same tendency observed for the components M [i.e., the confidence interval is narrower when comparing retinoscopy against subjective refraction (limits of agreement, J0 = ±0.19; J45 = ±0.15) than when autorefractor and subjective refraction are considered (limits of agreement, J0 = ±0.25; J45 = ±0.18)]. Autorefractor and retinoscopy values correlated in a similar way with subjective refraction. These values were similar to those found by Bullimore et al. for J0 (±0.31) and J45 (±0.22) components.8
Elliot et al.27 used matrix representations of dioptric power, which makes direct comparisons with our results difficult. Their 95% limits of agreement were converted by Bullimore et al.8 by applying Thibos’ recommendations, dividing torsional power by √2 to obtain the J45 component. After conversion, the results obtained are similar to ours, with limits of agreement of ±0.17.
Goss and Grosvenor28 have performed an exhaustive review about the reliability and repeatability of different methods of refraction. They concluded that the reliability of subjective clinical refraction, in terms of repeated refractions by the same examiner and comparisons of the results of two or more examiners, could be expressed by stating that there is 95% agreement within ±0.50 D and 80% agreement within ±0.25 D. This is applicable for spherical equivalent power, sphere power, and cylinder power. In the same way, they concluded that the reliability of objective and subjective autorefractor results is similar to the reliability of conventional clinical refraction. Considering Goss and Grosvenor’s review and keeping in mind that many patients are sensitive to changes as small as 0.25 D, this was the value adopted for any change to be classified as clinically significant.
To make interpretation of results easier for the clinician, we have reconverted the mean M, J0, and J45 values to the conventional spherocylindrical form.
Autorefractor = −0.63 −0.22 × 168
Subjective refraction = −0.23 −0.12 × 165
Retinoscopy = −0.19 −0.26 × 170
To clinically evaluate the values recorded in Table 3 for the sphere power, differences within a clinical range of ±0.25 D were found in 85 eyes (44.3%) for the sphere power obtained by autorefraction and subjective refraction. When sphere power obtained with retinoscopy and subjective refraction were compared, 143 eyes (74.5%) displayed bias within the clinical range.
Regarding cylinder power, the same clinical acceptable range was assumed for bias between methods. In this case, differences within the range of ±0.25 D were found in 172 eyes (89.6%) when autorefraction and subjective refraction were compared and in 186 eyes (96.9%) when retinoscopy was compared with subjective refraction.
Finally, we have established a clinically acceptable range for cylinder axis bias between methods of ±10°. Results reveal that this condition is verified in 106 eyes (55.2%) when we compare autorefraction with subjective refraction, whereas 126 eyes (65.6%) satisfied the condition when retinoscopy and subjective refraction cylinder values are compared.
Nevertheless, we are interested in knowing how many cases satisfy simultaneously the three clinical criteria previously imposed. In these conditions, refraction obtained by different methods can be considered fully comparable. Only 42 eyes (21.9%) satisfy this stringent criterion when we compare autorefraction with subjective refraction and 96 eyes (50.0%) when we compare retinoscopy with subjective refraction values.
The results presented above reveal that both comparisons, autorefraction against subjective refraction and retinoscopy against subjective refraction, display similar agreement with the clinical range in terms of cylinder power and axis. Conversely, higher agreement was found between retinoscopy and subjective refraction for the sphere power component. These findings agree with other previously published studies that observed a closer agreement between autorefraction and other refraction methods regarding the cylinder component, whereas poorer agreement was reported for the sphere component.28,29
In summary, autorefraction may be used as a starting point for the subjective refractive examination but never as its substitute. Present results confirm that when carried out by an experienced clinician, retinoscopy is more accurate than automatic refraction, giving a better starting point to noncycloplegic refraction.
We thank José Manuel González-Méijome for his contributions.
Departamento de Física
Universidade do Minho
Campus de Gualtar
1. Wood MG, Mazow ML, Prager TC. Accuracy of the Nidek ARK-900 objective refractor in comparison with retinoscopy in children ages 3 to 18 years. Am J Ophthalmol 1998;126:100–8.
2. Zadnik K, Mutti DO, Adams AJ. The repeatability of measurement of the ocular components. Invest Ophthalmol Vis Sci 1992;33:2325–33.
3. Walline JJ, Kinney KA, Zadnik K, Mutti DO. Repeatability and validity of astigmatism measurements. J Refract Surg 1999;15:23–31.
4. Strang NC, Gray LS, Winn B, Pugh JR. Clinical evaluation of patient tolerance to autorefractor prescriptions. Clin Exp Optom 1998;81:112–8.
5. Mallen EA, Wolffsohn JS, Gilmartin B, Tsujimura S. Clinical evaluation of the Shin-Nippon SRW-5000 autorefractor in adults. Ophthalmic Physiol Opt 2001;21:101–7.
6. Salchow DJ, Zirm ME, Stieldorf C, Parisi A. Comparison of objective and subjective refraction before and after laser in situ keratomileusis. J Cataract Refract Surg 1999;25:827–35.
7. McBrien NA, Adams DW. A longitudinal investigation of adult-onset and adult-progression of myopia in an occupational group. Refractive and biometric findings. Invest Ophthalmol Vis Sci 1997;38:321–33.
8. Bullimore MA, Fusaro RE, Adams CW. The repeatability of automated and clinician refraction. Optom Vis Sci 1998;75:617–22.
9. Kinge B, Midelfart A, Jacobsen G. Clinical evaluation of the Allergan Humphrey 500 autorefractor and the Nidek AR-1000 autorefractor. Br J Ophthalmol 1996;80:35–9.
10. McCaghrey GE, Matthews FE. Clinical evaluation of a range of autorefractors. Ophthalmic Physiol Opt 1993;13:129–37.
11. Davies LN, Mallen EA, Wolffsohn JS, Gilmartin B. Clinical evaluation of the Shin-Nippon NVision-K 5001/Grand Seiko WR-5100K autorefractor. Optom Vis Sci 2003;80:320–4.
12. Rosenfield M, Chiu NN. Repeatability of subjective and objective refraction. Optom Vis Sci 1995;72:577–9.
13. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1:307–10.
14. Bland JM, Altman DG. Comparing methods of measurement: why plotting difference against standard method is misleading. Lancet 1995;346:1085–7.
15. Zadnik K, Mutti DO, Bullimore MA. Use of statistics for comparing two measurement methods. Optom Vis Sci 1994;71:539–41.
16. Shakespeare TP, Gebski VJ, Veness MJ, Simes J. Improving interpretation of clinical studies by use of confidence levels, clinical significance curves, and risk-benefit contours. Lancet 2001;357:1349–53.
17. Thibos LN, Wheeler W, Horner D. Power vectors: an application of Fourier analysis to the description and statistical analysis of refractive error. Optom Vis Sci 1997;74:367–75.
18. Guirao A, Williams DR. A method to predict refractive errors from wave aberration data. Optom Vis Sci 2003;80:36–42.
19. Elliott M, Simpson T, Richter D, Fonn D. Repeatability and accuracy of automated refraction: a comparison of the Nikon NRK-8000, the Nidek AR-1000, and subjective refraction. Optom Vis Sci 1997;74:434–8.
20. Wood IC, Papas E, Burghardt D, Hardwick G. A clinical evaluation of the Nidek Autorefractor. Ophthalmic Physiol Opt 1984;4:169–78.
21. Allen PM, Radhakrishnan H, O’Leary DJ. Repeatability and validity of the PowerRefractor and the Nidek AR600-A in an adult population with healthy eyes. Optom Vis Sci 2003;80:245–51.
22. Harvey EM, Miller JM, Wagner LK, Dobson V. Reproducibility and accuracy of measurements with a hand held autorefractor in children. Br J Ophthalmol 1997;81:941–8.
23. Isenberg SJ, Del Signore M, Madani-Becker G. Use of the HARK autorefractor in children. Am J Ophthalmol 2001;131:438–41.
24. Cordonnier M, Dramaix M. Screening for refractive errors in children: accuracy of the hand held refractor Retinomax to screen for astigmatism. Br J Ophthalmol 1999;83:157–61.
25. Cordonnier M, Kallay O. Non-cycloplegic screening for refractive errors in children with the hand-held autorefractor Retinomax: final results and comparison with non-cycloplegic photoscreening. Strabismus 2001;9:59–70.
26. Nayak BK, Ghose S, Singh JP. A comparison of cycloplegic and manifest refractions on the NR-1000F (an objective Auto Refractometer). Br J Ophthalmol 1987;71:73–5.
27. Elliott M, Simpson T, Richter D, Fonn D. Repeatability and comparability of automated keratometry: the Nikon NRK- 8000, the Nidek KM-800 and the Bausch and Lomb keratometer. Ophthalmic Physiol Opt 1998;18:285–93.
28. Goss DA, Grosvenor T. Reliability of refraction—a literature review. J Am Optom Assoc 1996;67:619–30.
29. McBrien NA, Millodot M. Clinical evaluation of the Canon Autoref R-1. Am J Optom Physiol Opt 1985;62:786–92.