Purpose. To compare the performance of two handheld auto-refractors, the Retinomax and the Palm-Automatic Refractometer (Palm-AR), for detecting significant vision disorders in pre-school children.
Methods. Children attending Philadelphia PreKindergarten Head Start were screened with the Retinomax and Palm-AR and underwent a gold standard eye examination. The results of cycloplegic retinoscopy, cover testing, and visual acuity were used to classify children as having normal vision or one of four conditions: amblyopia, strabismus, significant refractive error, and reduced visual acuity. Pass/fail criteria for each instrument were selected to maximize overall sensitivity (with specificity set at 90% and at 94%) for detecting targeted disorders. Comparisons of sensitivities between the auto-refractors were performed using the exact McNemar test.
Results. Testability was >99% for both instruments. Test time was similar for the two instruments (median 2 min; p = 0.10). At 90% specificity, the sensitivity for detection of one or more targeted conditions was 74% for the Palm-AR and 78% for the Retinomax. At 94% specificity, the sensitivity for detection of one or more targeted conditions was 66% for both the Palm-AR and the Retinomax. At 90% specificity, the sensitivity for detecting significant refractive error was 84% for both auto-refractors, and at 94% specificity, the sensitivity was 76% for the Palm AR and 75% for the Retinomax. There were high correlations between the instruments for sphere (r = 0.85) and cylinder (r = 0.88) power. The mean difference between instruments was −0.13 diopters (D) (95% limit of agreement: −2.28 to 2.02) for sphere, and −0.15 D (95% limit of agreement: −0.89 to 0.59) for cylinder.
Conclusions. In this pilot study, the Retinomax and Palm-AR appear comparable with respect to testability, sensitivity, and specificity. There was strong agreement in readings of sphere and cylinder indicating that they may perform similarly in a screening setting.
*The members of the VIP Study Group Writing Committee are listed in the Acknowledgments.
Received September 16, 2010; accepted February 8, 2011.
The Pennsylvania College of Optometry at Salus University 8360 Old York Road Elkins Park, Pennsylvania 19027-1598 e-mail: email@example.com
The most prevalent childhood vision disorders are amblyopia, strabismus, and significant refractive error (RE) including myopia, hyperopia, astigmatism, and anisometropia.1–5 Vision screening of children has been shown to be effective in detecting children at risk for having one or more of these disorders.6–8 Handheld auto-refractors have been useful in a screening setting because of their portability, minimal space needed to operate, ease of use, high testability rates in young children, and short administration time.9–26 An added advantage is the ability of non-eyecare professionals and lay screeners to use these instruments successfully.26,27 Comparisons of handheld auto-refractors with table-mounted auto-refractors and portable power refractors have also been reported in the literature.28–33 The investigators of the Vision in Preschoolers Study and others have previously reported on comparisons of the handheld Refractometer Retinomax (“Retinomax”; Nikon, Chiyoda-ku, Tokyo) and SureSight Vision Screener (“SureSight”; WelchAllyn, Skaneateles Falls, NY) for detecting vision disorders in pre-school children.29–31,37,38 A third auto-refractor, the Palm-Automatic Refractometer (Palm-AR; Marco, Jacksonville, FL), is commercially available and offers many of the same features; however, there have been no reports published regarding its performance in pre-school vision screening. Therefore, we undertook a study to provide initial data on the characteristics of the Palm-AR as a screening instrument in comparison with the Retinomax.
Children who were enrolled in Philadelphia area PreKindergarten Head Start programs and participating in the Vision in Preschoolers Study Phase II were included in this pilot study. (n = 190). All children were between the ages of 3 and 5 years as of September 1st of the academic year during which the children were tested. The Institutional Review Board of the Pennsylvania College of Optometry approved the study protocol. Written, informed consent was obtained from the parent or legal guardian of all eligible study participants.
Each participant received a vision screening using both the Retinomax and Palm-AR on one day followed by a standardized comprehensive eye examination on a subsequent day. Screenings took place at the Head Start Center attended by the child in a room designated for the vision screening. Both the Retinomax and the Palm-AR were administered with the order of screening randomly assigned for each child. The currently available Palm-ARK has the same properties as of the Palm-AR that was used in this study. The actual instrument looks the same with the only difference being that the Palm-ARK also provides keratometry readings.a The cur-rently available Retinomax has the same properties as the Retinomax used in this study. An alternative model of the Retinomax, the “Retinomax K plus” is also available that has the ability to record keratometry readings and has the same properties as the Retinomax that was used in this study.b A trained and certified (non-eyecare professional) screener (ACarter) administered the screening tests to all children. Both autorefractions were completed at the same testing session.
Each time the Retinomax is administered, eight, automatic and consecutive readings are taken for each eye. These eight readings are averaged into one overall final reading along with a confidence level ranging from 1 to 10 for each eye. The manufacturer's recommended confidence level is 8 to 10.34 A maximum of three administrations were allowed for each eye to obtain an acceptable confidence level.6,27,35
The Palm-AR provides up to five automatic consecutive readings for each eye with a confidence level for each of the readings. The confidence levels range from 5 to 9. One summary RE reading is provided by the instrument, without a summary confidence level. Summary readings for the Palm-AR were considered acceptable for the purposes of this study when all confidence levels associated with individual readings were ≥7 and the majority of readings were ≥8 (as recommended by the Marco representative at the time the study was conducted). A maximum of three administrations were allowed for each eye to obtain an acceptable summary reading.
Gold Standard Eye Examinations
Standardized Gold Standard Eye (GSE) examinations were performed by optometrists or ophthalmologists who were experienced working with children.6,27 GSE examinations were conducted in a specially equipped vision van parked at each Head Start Center. The tests that were used to determine the presence of significant vision disorders included cycloplegic retinoscopy, distance and near cover test, and monocular threshold visual acuity (VA) using crowded HOTV optotypes on the Electronic Vision Assessment system.36
On the basis of the results of the GSE examinations, children were classified as having one or more of the four types of Vision in Preschoolers (VIP) targeted vision disorders (amblyopia, strabismus, significant RE, and reduced VA) or normal vision. Significant RE included astigmatism, hyperopia, myopia, and anisometropia. Children with more than one disorder (e.g., amblyopia and strabismus) would be included in each category. Vision disorders were also categorized by severity into group 1—very important to detect and treat early; group 2—important to detect early; and group 3—detection clinically useful. Whenever children had more than one condition, they were assigned only to the Group corresponding to their most severe condition. A full description of the hierarchy of these targeted vision disorders has been previously reported, and their definitions are summarized in Table 1.6
The testing characteristics of the Retinomax and Palm-AR were compared, including testability, testing time, and the ability of each instrument to detect VIP-targeted vision disorders. The Exact McNemar test was performed for comparison of the percentage of children testable, and the Wilcoxon signed rank test was used for the comparison of testing time because of the skewed distribution of testing time. For comparison of testing accuracy (sensitivity at predetermined specificity levels), the screening failure criteria that resulted in 90 and 94% specificity were determined for each auto-refractor. Children who were not testable were considered as screening failures.39 The first reading from an eye with an acceptable confidence level was used for data analysis and the best reading was used for eyes with no acceptable confidence levels from three attempted consecutive tests. For each set of failure criteria, the sensitivities for detecting each of VIP-targeted conditions, any GSE condition, and the three hierarchical grouped conditions were calculated. The statistical comparison of sensitivities between the Retinomax and the Palm-AR at the same level of specificity (either 90 or 94%) was performed by the Exact McNemar test for correlated proportions, because the Retinomax and the Palm-AR were performed on the same child and the number of children with a VIP-targeted condition is small. Agreement on RE was assessed with Bland-Altman plots and 95% limits of agreement.40
Of the 190 children who were initially enrolled and screened, 181 (95%) received a GSE examination. The remaining nine children were absent on the day of the GSE examination and were not able to be rescheduled for reasons such as illness or family relocation. Among the 181 with GSE examinations, 99 (55%) of children were 4 years old, with a mean age of 4.3 (range 3 to 5 years) and 94 (52%) were male.
Testability was >99% for each instrument (Table 2; p = 0.32). Acceptable confidence levels were obtained in 98.7% of eyes with the Retinomax and 89.4% of eyes with the Palm-AR (p < 0.0001). Overall, the Retinomax required less repeat testing and had a higher percentage of eyes with an acceptable confidence level after up to three tests (p < 0.0001).
The time to complete testing for each child (Table 3) was also evaluated. Among the 145 children who had the testing time recorded for each auto-refractor, the time ranged from less than 1 to 10 min. There was no difference in the median testing time (2 min each) between the two instruments (p = 0.11, Wilcoxon signed rank test).
The screening failure criteria for each instrument (set to achieve maximum sensitivity to detect any of the targeted conditions when specificity was set at 90 or 94%) are provided in Table 4. Increasing the specificity from 90 to 94% for the Palm-AR required increasing the failure criterion for hyperopia by 0.50 diopter (D), from 1.00 to 1.50 D. For the Retinomax, increasing the specificity resulted in increasing the criterion by 0.25 D for myopia, increasing the criterion by 1.25 D for astigmatism, and decreasing the criterion by 0.25 D for anisometropia.
Sixty-five (35.9%) of the 181 children were determined to have one or more targeted vision conditions (Table 5), resulting in a total of 99 vision disorders among this subset of children. The sensitivities for detecting any of the targeted conditions were similar for the Palm-AR and the Retinomax (74 and 78%, respectively) at 90% specificity and were the same (66%) at 94% specificity. Sensitivities for detecting RE were very similar between the two auto-refractors at both levels of specificity (Table 5). Comparisons of sensitivity between the auto-refractors for the other subgroups of conditions are hampered because of the small sample sizes (<30) and resulting low precision.
When the 90% specificity failure criteria previously derived for lay screeners are applied to the Retinomax screening results from the children included in this study,27 the specificity is 88% and the sensitivity is 71% for any VIP-targeted disorders, 86% for group 1 conditions, 55% for group 2, 67% for group 3 conditions, 83% for amblyopia, 60% for strabismus, 78% for RE, and 50% for reduced VA.
The Bland-Altman plots in Figs. 1 and 2 show the agreement between the two auto-refractors for sphere and cylinder power values. Overall, the correlation coefficient for both sphere and cylinder between the instruments was high (Pearson correlation coefficient =0.85 for sphere and 0.88 for cylinder). The difference between the instruments is −0.13 D (95% limit of agreement: −2.28 to 2.02 D) for sphere, and −0.15 (95% limit of agreement: −0.89 to 0.59) for cylinder.
The results of this study indicate that the Palm-AR and the Retinomax may both be useful instruments in a screening setting for pre-school children. The two instruments appear comparable with respect to administration time, testability, sensitivity and specificity, and provide similar measurements of sphere and cylinder power. The children in this study were a pre-selected group of pre-school children at a high risk for vision disorders.
Vision screening with the Palm-AR required more repeat testing than with the Retinomax. Also, the Palm-AR did not yield a reading with a confidence level deemed acceptable by the manufacturer as often as the Retinomax. Nonetheless, the accuracy for detecting any targeted condition was similar between the two auto-refractors. The RE components were highly correlated (correlation coefficient ≥0.85) between the two instruments, supporting the similarity in screening accuracy. However, the sample size for this pilot assessment of the Palm-AR did not provide sufficient precision to rule out important differences between the two instruments, particularly for detecting children at risk for less prevalent vision disorders such as amblyopia. Further investigation with larger numbers of subjects is needed to precisely estimate the accuracy of the Palm-AR.
This study demonstrated that the Retinomax and Palm-AR auto-refractors appear to be comparable with similar costs in a screening setting in this small sample size study. Future studies focusing on a more general population, larger sample size, and older children can provide further insights into the advantages of each instrument.
This work was supported by Grants from the National Eye Institute, National Institutes of Health, Department of Health & Human Services: U10EY12644; U10EY12547; U10EY12545; U10EY12550; U10EY12534; U10EY12647; and U10EY12648; U10EY12545-04S1, and R21EY018908.
The Vision in Preschoolers (VIP) Writing Committee
Elise Ciner, OD (Chair), The Pennsylvania College of Optometry at Salus University, Ashanti Carter, Gui-Shuang Ying, PhD, University of Pennsylvania, Department of Ophthalmology, Maureen Maguire, PhD, University of Pennsylvania, Department of Ophthalmology, and Marjean Taylor Kulp, OD, MS, The Ohio State University College of Optometry.
VIP Executive Committee
Paulette Schmidt, OD, MS (Chair), Agnieshka Baumritter, MA, Elise Ciner, OD, Lynn Cyert, PhD, OD, Velma Dobson, PhD, Marjean Taylor Kulp, OD, MS, Maureen Maguire, PhD, Bruce Moore, OD, Deborah Orel-Bixler, PhD, OD, Graham Quinn, MD, Maryann Redford, DDS, MPH, Janet Schultz, MA, CPNP, and Gui-Shuang Ying, PhD.
VIP Personnel at PCO
Principal Investigator: Elise Ciner, OD
Project Coordinator: Angela Duson
Asst. Project Coordinator: Lydia Parke, MS
Nurse Coordinator: Lynn Rosenbach, RN
Pediatric Eye Care Practitioners: Mark Boas, OD, Shannon Burgess, OD, Penelope Copenhaver, OD, Michael Gallaway, OD, Graham Quinn, MD, Janet Schwartz, OD, Brandy Scombordi-Raghu, OD, Edward Zikoski, OD, and Sheryl Menacker, MD
Research Intern: Ashanti Carter
Parent Liaison: Geneva Moss
Van Drivers: Jose Figueroa, David Harvey
Elise B. Ciner
The Pennsylvania College of Optometry at Salus University
8360 Old York Road
Elkins Park, Pennsylvania 19027-1598
1. Vision screening in children. U.S. Public Health Service. Am Fam Physician 1994;50:587–90.
2. Moore B. Eye Care for Infants and Young Children. Boston, MA: Butterworth-Heinemann; 1997.
3. Ciner EB, Schmidt PP, Orel-Bixler D, Dobson V, Maguire M, Cyert L, Moore B, Schultz J. Vision screening of preschool children: evaluating the past, looking toward the future. Optom Vis Sci 1998;75:571–84.
4. American Optometric Association Consensus Panel on Pediatric Eye and Vision Examination. Pediatric Eye and Vision Examinations, 2nd ed. St. Louis, MO: American Optometric Association; 2002.
5. American Academy of Ophthalmology Preferred Practice Patterns Pediatric Ophthalmology Panel. Pediatric Eye Evaluations. Preferred Practice Pattern. San Francisco, CA: American Academy of Ophthalmology; 2002.
6. Schmidt P, Maguire M, Dobson V, et al; Vision in Preschoolers (VIP) Study Group. Comparison of preschool vision screening tests as administered by licensed eye care professionals in the Vision In Preschoolers Study. Ophthalmology 2004;111:637–50.
7. Ying GS, Kulp MT, Maguire M, Ciner E, Cyert L, Schmidt P; Vision in Preschoolers (VIP) Study Group. Sensitivity of screening tests for detecting vision in preschoolers-targeted vision disorders when specificity is 94%. Optom Vis Sci 2005;82:432–8.
8. Hartmann EE, Dobson V, Hainline L, Marsh-Tootle W, Quinn GE, Ruttum MS, Schmidt PP, Simons K. Preschool vision screening: summary of a Task Force report. Behalf of the Maternal and Child Health Bureau and the National Eye Institute Task Force on Vision Screening in the Preschool Child. Pediatrics 2000;106:1105–16.
9. Ehrlich MI, Reinecke RD, Simons K. Preschool vision screening for amblyopia and strabismus. Programs, methods, guidelines, 1983. Surv Ophthalmol 1983;28:145–63.
10. el-Defrawy S, Clarke WN, Belec F, Pham B. Evaluation of a hand-held autorefractor in children younger than 6. J Pediatr Ophthalmol Strabismus 1998;35:107–9.
11. Schimitzek T, Wesemann W. Clinical evaluation of refraction using a handheld wavefront autorefractor in young and adult patients. J Cataract Refract Surg 2002;28:1655–66.
12. Wesemann W, Dick B. Accuracy and accommodation capability of a handheld autorefractor. J Cataract Refract Surg 2000;26:62–70.
13. Suryakumar R, Bobier WR. The manifestation of noncycloplegic refractive state in pre-school children is dependent on autorefractor design. Optom Vis Sci 2003;80:578–86.
14. Banks MS. Infant refraction and accommodation. Int Ophthalmol Clin 1980;20:205–32.
15. Simons K. Preschool vision screening: rationale, methodology and outcome. Surv Ophthalmol 1996;41:3–30.
16. Robinson B, Bobier WR, Martin E, Bryant L. Measurement of the validity of a preschool vision screening program. Am J Public Health 1999;89:193–8.
17. Cordonnier M, Dramaix M, Kallay O, de Bideran M. How accurate is the hand-held refractor Retinomax(R) in measuring cycloplegic refraction: a further evaluation. Strabismus 1998;6:133–42.
18. Chan OY, Edwards M. Comparison of cycloplegic and noncycloplegic retinoscopy in Chinese pre-school children. Optom Vis Sci 1994;71:312–8.
19. Harvey EM, Miller JM, Wagner LK, Dobson V. Reproducibility and accuracy of measurements with a hand held autorefractor in children. Br J Ophthalmol 1997;81:941–8.
20. Adams RJ, Dalton SM, Murphy AM, Hall HL, Courage ML. Testing young infants with the Welch Allyn suresight non-cycloplegic autorefractor. Ophthalmic Physiol Opt 2002;22:546–51.
21. Harvey EM, Dobson V, Miller JM, Clifford-Donaldson CE, Green TK, Messer DH, Garvey KA. Accuracy of the Welch Allyn SureSight for measurement of magnitude of astigmatism in 3- to 7-year-old children. J AAPOS 2009;13:466–71.
22. Suryakumar R, Bobier WR, Machan C. Objective assessment of refractive error in preschool children. In: Thorn F, Trolio D, Gwiazda J, eds. Myopia 2000: Proceedings of the VII International Conference on Myopia. Boston, MA: The New England College of Optometry; 200:83–7.
23. Büchner TF, Schnorbus U, Grenzebach UH, Busse H. Examination of preschool children for ametropia: first experiences using a new hand-held autorefractor. Strabismus 2004;12:111–7.
24. Miller JM, Harvey EM, Dobson V. Visual acuity screening versus noncycloplegic autorefraction screening for astigmatism in Native American preschool children. J AAPOS 1999;3:160–5.
25. Cordonnier M, Dramaix M. Screening for abnormal levels of hyperopia in children: a non-cycloplegic method with a hand held refractor. Br J Ophthalmol 1998;82:1260–4.
26. Kemper AR, Keating LM, Jackson JL, Levin EM. Comparison of monocular autorefraction to comprehensive eye examinations in preschool-aged and younger children. Arch Pediatr Adolesc Med 2005;159:435–9.
27. Vision in Preschoolers (VIP) Study Group. Preschool vision screening tests administered by nurse screeners compared with lay screeners in the vision in preschoolers study. Invest Ophthalmol Vis Sci 2005;46:2639–48.
28. Clarke N, Shacks J, Kerr AR, Bottrell CL, Poulsen MK, Yin L. Use of a noncycloplegic autorefractor to perform vision screening in preschools. J Sch Nurs 2008;24:158–63.
29. Prabakaran S, Dirani M, Chia A, Gazzard G, Fan Q, Leo SW, Ling Y, Au Eong KG, Wong TY, Saw SM. Cycloplegic refraction in preschool children: comparisons between the hand-held autorefractor, table-mounted autorefractor and retinoscopy. Ophthalmic Physiol Opt 2009;29:422–6.
30. Borchert M, Wang Y, Tarczy-Hornoch K, Cotter S, Deneen J, Azen S, Varma R; MEPEDS Study Group. Testability of the Retinomax autorefractor and IOLMaster in preschool children: the Multi-ethnic Pediatric Eye Disease Study. Ophthalmology 2008;115:1422–5; 1425.e1.
31. Virgili G, Angi M, Molinari A, Casotto V. Cox regression was used to compare the measurement error of two tests vs. a gold standard. J Clin Epidemiol 2007;60:345–9.
32. Virgili G, Angi M, Heede S, Rodriguez D, Bottega E, Molinari A. PowerRefractor versus Canon R-50 Autorefraction to assess refractive error in children: a community-based study in ecuador. Optom Vis Sci 2007;84:144–8.
33. Wesemann W, Dick B. [Experiences with the hand-held autorefractometer “Retinomax” in adults and children.] Klin Monatsbl Augenheilkd 1997;211:387–94.
34. Nikon Corporation. Nikon Retinomax 2 Autorefractor Operation Manual. Tokyo: Nikon Corporation; 1999.
35. Vision in Preschoolers Study Group. Impact of confidence number on the screening accuracy of the retinomax autorefractor. The Vision in Preschoolers (VIP) Study Group. Optom Vis Sci 2007;84:181–8.
36. Kulp MT, Dobson V, Peskin E, Quinn G, Schmidt P. The electronic visual acuity tester: testability in preschool children. The Vision in Preschoolers (VIP) Study Group. Optom Vis Sci 2004;81:238–44.
37. Cordonnier M, De Maertelaer V. Comparison between two hand-held autorefractors: the Sure-Sight and the Retinomax. Strabismus 2004;12:261–74.
38. Steele G, Ireland D, Block S. Cycloplegic autorefraction results in pre-school children using the Nikon Retinomax Plus and the Welch Allyn SureSight. Optom Vis Sci 2003;80:573–7.
39. Vision In Preschoolers (VIP) Study Group. Children unable to perform screening tests in vision in preschoolers study: proportion with ocular conditions and impact on measures of test accuracy. Invest Ophthalmol Vis Sci 2007;48:83–7.
40. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1:307–10.
a The Palm-AR has been replaced by the Palm-ARK (http://www.marco.com/automated/4.html). Cited Here...
b The Retinomax has been replaced by the Retinomax Series 3 (http://www.rightmedical.com/PRODUCT%20PDFs/RETINOMAX.pdf). Cited Here...
Keywords:© 2011 American Academy of Optometry
auto-refractor; vision screening; preschool; significant vision disorder; refractive error; Retinomax; Palm-AR