Several methods of refraction have been studied to determine which is more accurate and which is easier to perform and more suited to evaluate refraction and perform mass screening of visual defects in children.1 Among them, photorefraction1-15 has more advantage over autorefractors in that it can be performed at a distance and is, therefore, particularly suited for infants and noncooperative subjects.2,13 Major draw-backs in previous photorefractors were their limited ability to measure astigmatism and their questionable precision and repeatability.3 Therefore, Gekeler et al.4 proposed a new infrared photoretinoscope with six LED segments arranged at 30, 90, and 150°, and their complementary orientations. Weiss and Schaeffel have improved the design proposed by Gekeler et al. and produced a portable photorefractor, the PowerRefractor (Multichannel Systems, Reutlingen, Germany) which is professionally programmed with Windows. A further generation of photorefractors is currently commercially available, the PowerRefractor II (PlusoptiX AG, Nurnberg, Germany). This is technically based on its predecessor and offers an additional Dynamic Scan mode, allowing measurements of temporal changes in pupil size and accommodation.15
This study compares the accuracy and precision of the PowerRefractor to that of the Canon R-50 autorefractor in a large community of children 5- and 6-years-old in Ecuador, a country in which the need for spectacle correction of visual loss due to refractive error is largely unmet. We studied two defined urban areas of the highlands of Ecuador, Ibarra and Quito, which are racially, ethnically, and economically matched and were believed to be representative of the Ecuadorian population. Our goals were to assess the different ocular pathologies found in this population and to provide optometric and ophthalmologic care to the cases in need. The research purpose was to compare the PowerRefractor with autorefraction for assessing refractive error in a large group of children aged 5 to 6 years.
MATERIALS AND METHODS
We evaluated the children attending state school in three locations: two neighborhoods, one in the periphery of Quito and one in the town of Ibarra, and one at a village near Quito, called Carapungo. The local Ethic Committees reviewed and approved the study, which followed the tenets of the Declaration of Helsinki. Agreement was obtained from the directors of the schools in which the study was conducted. The directors sent a communication to the parents of the children before the study. No written consent was obtained from the parents.
Study Sample
Overall, we examined 6141 children, of whom 3855 were from 15 schools in Ibarra, where one school was not assessed because permission was denied by the school director, and 2286 children were from 10 schools in the Quito area. For this study, we limited the sample to children aged 5 or 6 years who completed a cycloplegic autorefraction with the Canon R-50 (CAR), the gold standard or reference test for this study. We examined 1591 children of whom 808 (50.8%) were aged 5 years and 783 (49.2%) were aged 6 years. One hundred eighty-seven (11.8%) out of 1591 children had uncorrected visual acuity 20/40 or worse in at least one eye, all but two of whom underwent cycloplegic examination. Of the remaining 1404 children with better visual acuity, 25 (1.8%) could not have cycloplegic evaluation, usually because their parents withdrew them after the first phase of the screening. Finally, in this study we conducted the analyses on 1564 right eyes of the children aged 5 and 6 years.
Refraction Tests
The first version of the PowerRefractor (Multichannel Systems, Reutlingen, Germany) was used in this study. Its performance without cylcoplegia was compared with noncycloplegic autorefraction (NCAR) with the Canon R-50 autorefractor. The gold standard was CAR in this study. Cycloplegic autorefraction has been shown to be well correlated with the traditional gold-standard assessment of refraction, cycloplegic retinoscopy, in population-based and community-based studies16-19 and in clinical research.20 Furthermore, CAR has been found to be more reliable than cyloplegic retinoscopy,21-23 a fact that led some authors23-24 to recommend it for mass screening, especially if trained technicians perform testing.
In our study the examinations at the schools were conducted by a team of two ophthalmology residents who were specially trained to perform this project. Their work was supervised by a pediatric ophthalmologist (AM), the principal coordinator of this project. The ophthalmic examination of each child consisted of external evaluation, ocular motility evaluation by cover test and Hirschberg test, stereoscopic vision screening with the Lang II test, visual acuity assessed with the HOTV test at 3 m distance without correction and with the child's habitual prescription. The data from these tests were not used in this study. The assessment of refraction followed the sequence of tests described above. The refractive status of each child was evaluated without cycloplegia with a Canon R-50 autorefractor (noncycloplegic autorefraction, NCAR) and then with the PowerRefractor. Thereafter, we achieved cycloplegia by instilling one drop of cyclopentolate 1% onto the inferior conjunctiva twice 10 min apart. A third drop was instilled if inadequate dilation was noted. The autorefraction was performed again 40 min after instillation of the first drop.
For the autorefractor the fixation target was the picture from the machine. The fixation target for the PowerRefractor was the camera, which had flashing lights on it that caught the child's attention. This camera was placed at around 1 m. Refraction was recorded as the mean of five measurements for aurorefraction. For the PowerRefractor, the value obtained after 15 to 30 s of stable fixation and refraction was recorded. All data were recorded on an individual clinical history form for each child.
Because all exams were performed by the same two examiners, they were masked to the results of the reference test (CAR) when performing the two index tests (NCAR and the PowerRefractor), but they were unmasked to the results of the index tests when they performed CAR. Nonetheless, we believe that CAR is a sufficiently standard and objective procedure, so that little bias can be expected because of unmasking.
Finally, examination of the retina was done by indirect ophthalmoscopy.
Data Analysis
We used power vector analysis25,26 to study refractive error. The refraction vector has three components, the spherical equivalent and two Jackson cross-ed-cylinder lenses, one (JCC0) with power J0 at axis α = 0° = 180°and the other (JCC45) with power J45 at axis α = 45°. The square root of the sum of the three squared components gives the total blur, the estimate of the true optical blur independently of the type of refractive error.
Bland-Altman method27 was used to assess the agreement of the PowerRefractor and NCAR as compared to CAR. In short, the mean difference and the 95% confidence limits of agreement are estimated. As the variance of the difference was a function of the average, we chose to present the estimates by stratifying across the subgroups of each vector component of the reference test. With this method, not only the variance of the difference is expected to be more homogeneous within each quintile, but it is also possible to compare, in the same groups of subjects, the mean error and 95% limits of agreement of two methods against a reference test according to the level of refractive error We used the software Stata 9.2 (Stata, College Station, TX) for all calculations. Data are presented for the right eye only, because data obtained in the left eye were similar.
RESULTS
The frequencies of given levels of hyperopia, myopia, and astigmatism are shown in Table 1 for the index and reference tests. It can be seen that few children were myopes.
Mean spherical equivalent was 1.27 D (SD: 1.10) for CAR, the reference test, and was similar for the PowerRefractor (1.30 D, SD: 0.93 D). A myopic shift was evident with NCAR (mean: 0.30, SD:0.88). Mean cylinder absolute value was 0.79 D (SD: 0.87 D) for CAR, and was slightly less with NCAR (mean: 0.37 D, SD: 0.43 D) and with the PowerRefractor (mean: 0.54 D; SD: 0.62 D).
Table 2 presents the overall agreement of the two index methods with the gold-standard. For spherical equivalent, NCAR induced a myopic shift of about 1 D, whereas the PowerRefractor showed virtually no bias. However, the interval between the 95% limits of agreement was 1/3 narrower for NCAR as compared to CAR. For the two Jackson crossed-cylinder components, there was no clinically significant mean error for both methods, but, again, the width of the interval between the 95% limits of agreement was much narrower for NCAR.
As we noticed that the mean difference was a function of the average value of CAR in Bland-Altman plots, which are not shown for brevity, we provide a stratified estimate of agreement in Fig. 1. It can be seen that the PowerRefractor slightly overestimates mild hyperopia and it underestimates high hyperopia, whereas NCAR always underestimates hyperopia. The mean error for the two cylinder components was negligible, possibly because of the narrower range of the measurements. The 95% limits of agreement of the Power Refractor with CAR are particularly wide for hyperopic vectors.
DISCUSSION
Some previous studies have reported on the performance of the version of the PowerRefractor which we used in our study, that is no longer commercially available.5-10,12,13 Although in most of these studies noncycloplegic assessment of refraction was believed to be acceptable as compared to cycloplegic retinoscopy or to an autorefractor, the Vision in Preschoolers Study1 found that its sensitivity was lower than the Retinomax autorefractor and some other vision screening instruments to detect several ophthalmic conditions, including ametropia, in 796 children.
The latest version of the PowerRefractor (PowerRefractor II, Plusoptix, Nurnberg, Germany) was compared to cycloplegic autorefraction by Jainta et al.15 In eight subjects with mild ametropia tested at different distances without cycloplegia, they found that the PowerRefractor II obtained a more hyperopic estimate by 0.25 D when compared with subjective refraction at 5 meters, and by 0.76 D compared to Canon R1 autorefraction, a result similar to the performance of its predecessor according to the authors.15
Our study has compared the performance of the PowerRefractor with that of the Canon R-50 autorefractor in a large number of children aged 5 and 6 from two similar communities in Ecuador. We found that although the PowerRefractor suffers from no overall mean error when measuring spherical equivalent compared to a shift towards minus of about 1 D for the Canon R-50 autorefractor, the width of its limits of agreement is larger (about ±2 vs. ±1.5 D) and even more for hyperopes. For detecting astigmatism, the limits of agreement are also wider (about ±0.8 vs. ±0.3 D for JCC0 and ±0.4 vs. ±0.2 D for JCC45) and again they increase markedly for hyperopic vectors. The widening of the limits of agreement with increasing hyperopia was seen both for the PowerRefractor and for NCAR, but this increase was much more for the PowerRefractor. Choi et al.5 reported that the linear operating range of the PowerRefractor is +4 to -6 D, but its poor performance at high values of hyperopia in our study could have partly be due to the large amount of eyes (40%) exceeding +4 D in the subgroup with spherical equivalent of at least +3 D in our figure. Nonetheless, our sample is expected to be representative of the refractive error distribution in the local population and this limitation of the PowerRefractor must be considered in clinical practice.
Thus, the advantage of better accuracy, because of no myopic shift due to accommodation, is lost due to its worse precision. In fact, although systematic or mean error can be subtracted when it is known in a specific group of patients, poor precision cannot be accounted for due to its randomness. This is expected to result in a worse performance of the PowerRefractor as compared to autorefraction for screening children for ametropia, as found in the Vision in Preschoolers Study.1 In fact, the choice of the cut-point of refractive error is irrelevant in this case, as a yes/no decision about referral to a full ophthalmic examination can be based on any value of refractive error taking into account the mean error. On the contrary, the correctness of such decision will be related to the instrument precision compared to the gold standard assessment.
The strength of our study is the collection of a large sample size at several schools in two locations. We believe that the coverage of the local population was good, given the high attendance rate of schools until age 10 in these areas. Due to the large number of children we could obtain precise estimates of the bias and limits of agreement for different types of refractive error, particularly high hyperopia and astigmatism. The profile of the refractive error distribution was not dissimilar from that found in a U.S. population-based study,1 a requirement for generalizing the results to other subjects. This landmark study, the Vision in Preschoolers Study,1 also suggested a poorer performance of the PowerRefractor as a screening tool for pediatric ophthalmic conditions, mostly refractive error, as compared to other tests, using retinoscopy as a reference test.
In conclusion, the PowerRefractor was accurate but not precise for measuring refractive error as compared to the Canon R-50 autorefractor for the range of refractive errors studied. The precision of the PowerRefractor is worse in children with high hyperopia.
ACKNOWLEDGMENTS
The authors thank Prof. Reinhard Kusel and Prof. Elisabeth Schulz Eppendorf University Eye Clinic, Hamburg Germany for their invaluable support to this study. They also thank the Editors and the Reviewers of the journal Optometry and Vision Science for their helpful comments and editorial assistance in its revision.
This work was supported in part with funds from EC Alfa Project ARL 2.039 (6) and from Progetto CEI 022/99 Formazione di Personale per la Diagnosi e Correzione dell'Handicap Visivo in Ecuador.
Gianni Virgili
Department of Oto-Neuro-Ophthalmological Surgical Sciences
University of Florence
Viale Morgagni 85
Florence 50134, Italy
e-mail: gianni.virgili@unifi.it
REFERENCES
1. The Vision in Preschoolers Study Group. Comparison of preschool vision screening tests as administered by licensed eye care professionals in the vision in preschoolers study. Ophthalmology 2004;111:637-50.
2. Wesemann W, Rassow B. Automatic infrared refractors-a comparative study. Am J Optom Physiol Opt 1987;64:627-38.
3. Thompson AM, Li T, Peck LB, Howland HC, Counts R, Bobier WR. Accuracy and precision of the Tomey ViVA infrared photorefractor. Optom Vis Sci 1996;73:644-52.
4. Gekeler F, Schaeffel F, Howland HC, Wattam-Bell J. Measurement of astigmatism by automated infrared photoretinoscopy. Optom Vis Sci 1997;74:472-82.
5. Choi M, Weiss S, Schaeffel F, Seidemann A, Howland HC, Wilhelm B, Wilhelm H. Laboratory, clinical, and kindergarten test of a new eccentric infrared photorefractor (PowerRefractor). Optom Vis Sci 2000;77:537-48.
6. Schimitzek T, Haase W. Efficiency of a video-autorefractometer used as a screening device for amblyogenic factors. Graefes Arch Clin Exp Ophthalmol 2002;240:710-6.
7. Hunt OA, Wolffsohn JS, Gilmartin B. Evaluation of the measurement of refractive error by the PowerRefractor: a remote, continuous and binocular measurement system of oculomotor function. Br J Ophthalmol 2003;87:1504-8.
8. Abrahamsson M, Ohlsson J, Bjorndahl M, Abrahamsson H. Clinical evaluation of an eccentric infrared photorefractor: the PowerRefractor. Acta Ophthalmol Scand 2003;81:605-10.
9. Suryakumar R, Bobier WR. The manifestation of noncycloplegic refractive state in pre-school children is dependent on autorefractor design. Optom Vis Sci 2003;80:578-86.
10. Allen PM, Radhakrishnan H, O'Leary DJ. Repeatability and validity of the PowerRefractor and the Nidek AR600-A in an adult population with healthy eyes. Optom Vis Sci 2003;80:245-51.
11. Anker S, Atkinson J, Braddick O, Ehrlich D, Hartley T, Nardini M, Wade J. Identification of infants with significant refractive error and strabismus in a population screening program using noncycloplegic videorefraction and orthoptic examination. Invest Ophthalmol Vis Sci 2003;44:497-504.
12. Schittkowski M, Hucks-Sievers S, Krentz H, Guthoff R. [Accuracy of the autorefractor power refractor in clinical work-a comparative study]. Klin Monatsbl Augenheilkd 2005;222:983-92.
13. Schimitzek T, Lagreze WA. Accuracy of a new photo-refractometer in young and adult patients. Graefes Arch Clin Exp Ophthalmol 2005;243:637-45.
14. Blade PJ, Candy TR. Validation of the PowerRefractor for measuring human infant refraction. Optom Vis Sci 2006;83:346-53.
15. Jainta S, Jaschinski W, Hoormann J. Measurement of refractive error and accommodation with the photorefractor PowerRef II. Ophthal Physiol Opt 2004;24:520-7.
16. He M, Zeng J, Liu Y, Xu J, Pokharel GP, Ellwein LB. Refractive error and visual impairment in urban children in southern China. Invest Ophthalmol Vis Sci 2004;45:793-9.
17. Murthy GV, Gupta SK, Ellwein LB, Munoz SR, Pokharel GP, Sanga L, Bachani D. Refractive error in children in an urban population in New Delhi. Invest Ophthalmol Vis Sci 2002;43:623-31.
18. Dandona R, Dandona L, Srinivas M, Sahare P, Narsaiah S, Munoz SR, Pokharel GP, Ellwein LB. Refractive error in children in a rural population in India. Invest Ophthalmol Vis Sci 2002;43:615-22.
19. Choong YF, Chen AH, Goh PP. A comparison of autorefraction and subjective refraction with and without cycloplegia in primary school children. Am J Ophthalmol 2006;142:68-74.
20. Miller JM. Vision in Preschoolers Study. Ophthalmology 2004;111:2313-14.
21. Harvey EM, Miller JM, Dobson V, Tyszko R, Davis AL. Measurement of refractive error in Native American preschoolers: validity and reproducibility of autorefraction. Optom Vis Sci 2000;77:140-9.
22. Zadnik K, Mutti DO, Adams AJ. The repeatability of measurement of the ocular components. Invest Ophthalmol Vis Sci 1992;33:2325-33.
23. Bullimore MA, Fusaro RE, Adams CW. The repeatability of automated and clinician refraction. Optom Vis Sci 1998;75:617-22.
24. Wood MG, Mazow ML, Prager TC. Accuracy of the Nidek ARK-900 objective refractor in comparison with retinoscopy in children ages 3 to 18 years. Am J Ophthalmol 1998;126:100-8.
25. Thibos LN, Horner D. Power vector analysis of the optical outcome of refractive surgery. J Cataract Refract Surg 2001;27:80-5.
26. Naeser K, Hjortdal J. Multivariate analysis of refractive data: mathematics and statistics of spherocylinders. J Cataract Refract Surg 2001;27:129-42.
27. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res 1999;8:135-60.