A major development in laser refractive surgery is the emergence of sophisticated devices for measuring higher-order optical aberrations of the eye. Studies have demonstrated that the correction of higher-order aberrations with an adaptive optics system can lead to supernormal visual performance in normal eyes.1,2 The success of the technique encourages implementation of higher-order correction in everyday vision through customized laser refractive surgery, contact lenses, or intraocular lenses.
Techniques for measuring ocular aberrations include Hartmann-Shack (Shack-Hartmann) wavefront sensing,1,3 Tscherning aberrometry sensing,4,5 Tracey ray-tracing aberrometry,6–8 spatially resolved refractometry,9 and optical path difference (OPD) scans.10,11 An obvious requirement of each of these devices is accuracy of the measurement of spherical and cylindrical errors as well as of the higher-order aberrations of the eye. We are unaware of clinical studies that report the accuracy of the WaveScan device. In a previous study,8 we evaluated the accuracy and reproducibility of an earlier prototype of Tracey ray-tracing aberrometry in pseudophakic eyes. The current study was designed to evaluate the accuracy and repeatability of the WaveScan WavePrint system and the Tracey wavefront aberrometer in measuring the refractive errors in phakic eyes.
Patients and Methods
The WaveScan WavePrint system (Visx, Inc.) is a diagnostic instrument designed to measure and display refractive errors and wavefront aberrations of the eye using a Hartmann-Shack wavefront sensor. A small spot of laser light is projected onto the retina and reflects back through the pupil. Reflected light exiting the eye is imaged by a lenslet array, and the array of spot images is captured by a video sensor. The location of each spot gathered from the video sensor is then compared to the theoretical ideal locations by computer, and the eye's refractive errors and wavefront aberrations are computed.
The latest version of the Tracey device (Tracey Technologies Inc.) has software and hardware improvements. The main change is the addition of an optometer; to eliminate accommodation, a fogged target with 2.0 diopters (D) of hyperopic shift appears just before the laser beam enters the eye. Tracey measures 64 points with 1 measurement at each point, and the dynamic range of spherical equivalent has been extended to −15 to +15 D.
Inclusion criteria for entering the study included normal subjects and patients who had a best spectacle-corrected visual acuity (BSCVA) of 20/25 or better. Manifest refraction (MR) was carefully performed using a standard phoropter at a vertex distance of 12.0 mm, and all patients were carefully fogged. All measurements with the WaveScan and Tracey devices were performed by 1 observer (L.W.) following the steps in the operator's instructions.
Both devices were on loan from the manufacturers. The WaveScan device was available throughout the study period, but the Tracey device was available for 2 weeks only, which restricted the number of patients who could be evaluated.
Sixty-nine eyes of 42 patients (mean age 43 years ± 11 [SD]) with a mean manifest refraction spherical equivalent (MR SE) of −1.32 ± 2.32 D (range −8.38 to 3.63 D) were prospectively examined using the WaveScan device.
An additional 48 eyes of 26 patients (mean age 42 ± 11 years) with a mean MR SE of −1.28 ± 2.92 D (range −10.63 to 3.50 D) were prospectively examined using the WaveScan and Tracey devices. Data from these 48 eyes were used to analyze the accuracy and reproducibility of the Tracey system. The WaveScan system did not obtain measurements in 16 of the 48 eyes. Therefore, to compare the WaveScan and Tracey devices, only the 32 eyes in which both devices were able to obtain measurements were evaluated.
Spherical equivalent, sphere, and cylinder from the MR and WaveScan measurements at a vertex distance of 12.0 mm were recorded, and data displayed by the Tracey device were converted to a vertex distance of 12.0 mm from the corneal plane. The MR values were used as the standard for comparison with the WaveScan and Tracey values. Various statistical analyses were performed. Correlation coefficients were evaluated using the Pearson product moment correlation coefficient. Bland-Altman plots12 were made showing the differences between 2 measurements against their mean to check whether the error of measurement was independent of the magnitude of the mean score. Limits of agreement were estimated by the mean difference (d) and standard deviation of the differences (SD). If the differences are normally distributed (Gaussian), 95% of the differences will lie between the limits of d − 2 SD and d + 2 SD. This is referred to as 95% limits of agreement (95% LA).12
Analysis of aggregate astigmatism was performed using the vector analysis method described by Holladay and coauthors.13 To assess the reproducibility of each device, 2 SDs for the differences among the sets of 3 consecutive measurements for both WaveScan and Tracey were calculated. Assuming a Gaussian distribution, 2 SDs indicate the 95% range of repeated measurements for each device.
Intraclass correlation coefficients (ICCs) (1-way random effects model) (SPSS for Windows, version 10) were used to assess the repeatability of the 3 WaveScan and 3 Tracey measurements. The ICC is the parametric analog of the chance-corrected measurement of agreement.14,15 The ICC values can be arbitrarily interpreted as poor (<0.20), fair (0.21 to 0.40), moderate (0.41 to 0.60), substantial (0.61 to 0.80), or almost perfect (0.81 to 1.00).15,16
A paired Student t test was used to test the absolute differences between the MR values and values from the WaveScan and Tracey devices and the McNemar test, to compare the frequencies within 0.5 D of deviation from the standard. A probability of less than 5% (P<.05) was considered statistically significant.
Accuracy and Repeatability of the Wavescan System
A demographic and refractive summary of the patients in the WaveScan evaluation are shown in Table 1.
Pearson correlation coefficients for SE, sphere, and cylinder between values obtained from MR and WaveScan were 0.984, 0.979, and 0.838, respectively (all P<.001). The mean differences between MR and WaveScan (MR − WaveScan) were −0.26 D, −0.12 D, and −0.28 D, and the magnitude of the error of measurements was independent of the magnitude of the mean values (MR and WaveScan) (Figures 1 and 2). Values of 95% LA were −1.09 to 0.57 D for SE, −1.14 to 0.89 D for sphere, and −0.95 to 0.40 D for cylinder (Table 2). The percentages of eyes with WaveScan values within ±0.5 D of the MR SE, sphere, and cylinder were 65%, 70%, and 83%, respectively, and within ±1.0 D, 96%, 94%, and 97%, respectively.
In normal eyes, the mean differences between MR and WaveScan in SE, sphere, and cylinder were −0.35 D, −0.23 D, and −0.25 D, respectively, compared to −0.20 D, −0.05 D, and −0.30 D, respectively, in eyes that had had refractive surgery (Table 2).
Vector analysis showed that the mean difference between MR and WaveScan was −0.47 + 0.07 × 9. The arithmetic mean of the vectorial difference in astigmatic magnitude was 0.42 ± 0.26 D. The centroid or mean of the aggregate astigmatism measured by MR and WaveScan was +0.12 ± 0.38 D × 94° and 0.19 ± 0.53 D × 96°, respectively. The vector differences in astigmatism are shown in Figure 3; the mean difference was +0.07 ± 0.34 D × 9°.
Of the 69 eyes, 35 had 3 consecutive WaveScan measurements. The 2 SDs were 0.26 D for SE, 0.29 D for sphere, and 0.16 D for cylinder and the ICC values, 0.993, 0.992, and 0.902, respectively.
Vector analysis showed that the centroids of the aggregate astigmatism were +0.09 ± 0.39 D × 109° for WaveScan measurement 1, +0.13 ± 0.37 D × 111° for WaveScan measurement 2, and +0.13 ± 0.42 D × 109° for WaveScan measurement 3. Graphically, the centroids and the ellipses around the centroids for these 3 data sets were almost superimposable (Figure 4).
Accuracy and Repeatability of the Tracey Device
A demographic and refractive summary of patients in the Tracey evaluation are shown in Table 1.
Pearson correlation coefficients for SE, sphere, and cylinder between values obtained from MR and Tracey were 0.980, 0.976, and 0.498, respectively (all P<.001). The mean differences between MR and Tracey (MR − Tracey) were −0.21 D, −0.01 D, and −0.40 D, and the magnitude of the error of measurements was independent of the magnitude of the mean values (MR and Tracey) (Figures 5 and 6). Values of 95% LA were −1.37 to 0.95 D for SE, −1.27 to 1.26 D for sphere, and −1.16 to 0.35 D for cylinder (Table 3). The percentages of eyes with Tracey values within ±0.5 D of MR SE, sphere, and cylinder were 60%, 58%, and 60%, respectively, and within ±1.0 D, 94%, 94%, and 96%, respectively.
In normal eyes, the mean differences between MR and Tracey in SE, sphere, and cylinder were −0.43 D, −0.31 D, and −0.22 D, respectively, compared to −0.02 D, 0.25 D, and −0.55 D, respectively, in eyes that had had laser in situ keratomileusis (LASIK) (Table 3).
Vector analysis showed that the mean difference between MR and Tracey was −0.53 ±0.27 D × 12°. The arithmetic mean of the vectorial difference in astigmatic magnitude was 0.64 ± 0.38 D. The centroid of the aggregate astigmatism values measured by MR and Tracey were +0.10 ± 0.34 D × 84° and 0.36 ± 0.55 D × 98°, respectively. The vector differences in astigmatism are shown in Figure 7; the mean difference was +0.27 ± 0.49 D × 12°.
All 48 eyes had 3 consecutive Tracey measurements. The 2 SDs were 0.31 D for SE, 0.36 D for sphere, and 0.33 D for cylinder and the ICCs, 0.994, 0.992, and 0.764, respectively.
Vector analysis showed that the centroids of the aggregate astigmatism were +0.36 D ± 0.55 D × 98° for Tracey measurement 1, +0.37 ± 0.61 D × 99° for Tracey measurement 2, and +0.38 ± 0.60 D × 104° for Tracey measurement 3. The centroids and the ellipses around the centroids almost overlapped (Figure 8).
Comparison of the Wavescan and Tracey Devices
Three (14%) of the 22 normal eyes and 13 (50%) of the 26 post-LASIK eyes could not be measured by the WaveScan device. Table 4 shows the differences between the methods. The absolute deviations from the MR values were comparable with the 2 devices (P>.05). Cylindrical values from Tracey had a larger deviation from the manifest cylinder than the values from WaveScan (P=.03) (Table 4), although the mean difference, 0.14 D, was not clinically significant.
The WaveScan system had a higher percentage of eyes with cylindrical values within ±0.5 D of the MR values than the Tracey device (88% versus 63%) in the entire group (P=.04) (Table 5). In 1 eye, the difference in cylinder magnitude between the MR and WaveScan exceeded 1.0 D; this did not occur in any eye with Tracey measurements.
Several types of wavefront-sensing devices are used in excimer laser trials of custom corneal ablation. Surprisingly, we are aware of only 1 peer-reviewed study that evaluates the accuracy and reproducibility of these devices in measuring refractive error,8 and we know of no reports that evaluate accuracy or reproducibility in measuring higher-order aberrations.
Wang and coauthors8 evaluated the accuracy of an earlier version of the Tracey device (Tracey-1) in measuring refractive error in pseudophakic eyes. Although they found a mean error of 1.0 D in measuring SE, this version of the Tracey device measured astigmatism accurately and reproducibly, with 70% of eyes showing a mean difference in SE of 0.1 D between 2 consecutive measurements. Pallikaris and coauthors7 evaluated the reproducibility of Tracey-1 by performing 30 consecutive measurements in each of 7 pseudophakic eyes. They found an SD in the measurement of cylinder of 0.14 D.
In the first part of our study, we evaluated the accuracy and reproducibility of the WaveScan device using MR as the standard. Overall, there was excellent agreement between the measurements of the 2 devices as demonstrated by their mean differences, 95% LA values, and aggregate analysis of astigmatic values. The data did vary, with a maximal difference in SE of 1.2 D and a cylinder magnitude of 1.3 D. Reproducibility of the WaveScan device was outstanding as demonstrated by 2 SDs, ICC values, and aggregate analysis of astigmatism.
Comparable results were noted with the Tracey device. The greatest differences between MR and Tracey values were 1.66 D for SE and 1.15 D for cylinder magnitude. Tracey measurements were also highly reproducible. In the eyes measured by both devices, we found that values from both devices had comparable absolute deviations from MR values, with the exception that the deviation of Tracey cylindrical values was larger by 0.14 D on average than that of WaveScan cylindrical values.
Our data raise several issues. In some eyes, there were clinically important differences in the values obtained by MR and the wavefront-sensing device. In these situations, the obvious question is which of the values better reflects the refractive needs of the patient's eye? This presumably would have to be tested by evaluating the patient's vision with each of the 2 corrections. In the future, this might also be evaluated using some form of adaptive optics. This issue is obviously of critical importance in selecting patients for wavefront-guided custom corneal ablation. How should we best evaluate patients in whom there is an obvious discrepancy between values obtained by MR and the wavefront-sensing devices? Fortunately, our study suggests that in most eyes, these differences are sufficiently small to be of no clinical concern.
How do our findings compare with those reported using standard autorefractors? In several studies that compare the accuracy of autorefractors to MR, the mean differences in SE (MR − autorefraction) ranged from −0.16 to +0.26 D and the SDs of the differences ranged from 0.44 to 0.81 D.17–19 Øyo-Szerenyi and coauthors20 compared the Nidek ARK 2000 and Canon RK-3 to the subjective refraction in 48 normal eyes and 78 eyes having photorefractive keratectomy (PRK). The mean differences in normal eyes were +0.09 D and −0.12 D, respectively, for SE and −0.17 D and −0.15 D, respectively, for cylinder, and the SDs ranged from 0.35 to 0.41 D, similar to the SDs of the differences between the MR and the wavefront devices in our study. However, in their study, the mean differences in PRK-treated eyes were +0.75 D and +0.56 D for SE and +0.42 D and +0.41 D for cylinder power, respectively, and the SDs ranged from 0.64 to 1.23 D; these values are greater than the differences that we found between the MR and the wavefront devices. Rosenfield and Chiu21 evaluated the repeatability of subjective and objective refraction of the Canon R-1 autorefractor and found that the 2 SDs of 5 measurements with the subjective and objective techniques were 0.27 D and 0.35 D, respectively, which are comparable to our findings (0.26 D for WaveScan and 0.31 D for Tracey).
Summarizing these findings, we conclude the following: (1) Although the WaveScan and Tracey devices measure slightly more hyperopia or less myopia than the subjective refraction, the SDs of the differences from the MR fall within the lower range reported for autorefractors. (2) The WaveScan and Tracey devices measure eyes after corneal refractive surgery more accurately than autorefractors. (3) The repeatability of WaveScan and Tracey is comparable to that of autorefractors.
In our direct comparison of the 2 devices, differences were minimal, although slightly favoring the WaveScan device. The WaveScan is currently used in clinical trials with the Visx Star S3 laser system. We are unaware of clinical trials in which the data from the Tracey device are linked to an excimer laser. A major limitation of the WaveScan device used in our study was its inability to measure many eyes, particularly those that had had refractive surgery. We presume that these difficulties are in part a limitation of the Hartmann-Shack technology, with large higher-order aberrations causing crossover effects that preclude obtaining measurements.22
Conceivably, each of these devices can undergo further improvements. There may be ways to increase the robustness of the WaveScan by altering the spot pattern or obtaining multiple overlapping measurements that can be fused to create a wavefront measurement for highly aberrated eyes. For the Tracey system, improvements could include more precise means of centering around the entrance pupil; this should improve reproducibility. The Tracey device obtains its measurement over approximately 50 milliseconds. Therefore, a potential disadvantage of this device is that a small amount of eye movement could occur during the time of the measurement.
In conclusion, the WaveScan and Tracey devices showed excellent accuracy and reproducibility in measuring refractive errors in normal and postoperative eyes. Future work will involve analysis of reproducibility of the measurements of higher-order aberrations. We believe that studies such as these are needed for all wavefront-sensing devices used in clinical trials.
1. Liang J, Grimm B, Goelz S, Bille JF. Objective measurement of wave aberrations of the human eye with the use of a Hartmann-Shack wave-front sensor. J Opt Soc Am A Opt Image Sci Vis 1994; 11:1949-1957
2. Guirao A, Porter J, Williams DR, Cox IG. Calculated impact of higher-order monochromatic aberrations on retinal image quality in a population of human eyes: erratum. J Opt Soc Am A Opt Image Sci Vis 2002; 19:620-628
3. Salmon TO, Thibos LN, Bradley A. Comparison of the eye's wave-front aberration measured psychophysically and with the Shack-Hartmann wave-front sensor. J Opt Soc Am A Opt Image Sci Vis 1998; 15:2457-2465
4. Mrochen M, Kaemmerer M, Seiler T. Wavefront-guided laser in situ keratomileusis: early results in three eyes. J Refract Surg 2000; 16:116-121
5. Mrochen M, Kaemmerer M, Mierdel P, et al. Principles of Tscherning aberrometry. J Refract Surg 2000; 16:S570-S571
6. Molebny VV, Panagopoulou SI, Molebny SV, et al. Principles of ray tracing aberrometry. J Refract Surg 2000; 16:S572-S575
7. Pallikaris IG, Panagopoulou SI, Molebny VV. Clinical experience with the Tracey technology wavefront device. J Refract Surg 2000; 16:S588-S591
8. Wang L, Misra M, Pallikaris IG, Koch DD. A comparison of a ray-tracing refractometer, autorefractor, and computerized videokeratography in measuring pseudophakic eyes. J Cataract Refract Surg 2002; 28:276-282
9. Webb RH, Penney CM, Thompson KP. Measurement of ocular local wave-front distortion with a spatially resolved refractometer. Appl Optics 1992; 31:3678-3686
10. MacRae S, Fujieda M. Slit skiascopic-guided ablation using the Nidek laser. J Refract Surg 2000; 16:S576-S580
11. Campbell C. Corneal topography in customized ablation. In: MacRae SM, Krueger RR, Applegate RA, eds, Customized Corneal Ablation; the Quest for SuperVision. Thorofare, NJ, Slack, 2001; 229-236
12. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1:307-310
13. Holladay JT, Moran JR, Kezirian GM. Analysis of aggregate surgically induced refractive change, prediction error, and intraocular astigmatism. J Cataract Refract Surg 2001; 27:61-79
14. Kirshner B, Guyatt G. A methodological framework for assessing health indices. J Chron Dis 1985; 38:27-36
15. Fleiss JL, Cohen J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas 1973; 33:613-619
16. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33:159-174
17. Sunder Raj P, Villada JR, Lewis AE, et al. Comparative evaluation of the Allergan Humphrey 570 and Canon RK-1 autorefractors: I. Objective autorefraction in normal subjects. Eye 1992; 6:284-286
18. Kinge B, Midelfart A, Jacobsen G. Clinical evaluation of the Allergan Humphrey 500 autorefractor and the Nidek AR-1000 autorefractor. Br J Ophthalmol 1996; 80:35-39
19. Mallen EAH, Wolffsohn JS, Gilmartin B, Tsujimura S. Clinical evaluation of the Shin-Nippon SRW-5000 autorefractor in adults. Ophthalmic Physiol Opt 2001; 21:101-107
20. Øyo-Szerenyi KD, Wienecke L, Businger U, Schipper I. Autorefraction/autokeratometry and subjective refraction in untreated and photorefractive keratectomy-treated eyes. Arch Ophthalmol 1997; 115:157-164
21. Rosenfield M, Chiu NN. Repeatability of subjective and objective refraction. Optom Vis Sci 1995; 72:577-579
22. Moreno-Barriuso E, Navarro R. Laser Ray Tracing versus Hartmann-Shack sensor for measuring optical aberrations in the human eye. J Opt Soc Am A Opt Image Sci Vis 2000; 17:974-985