Share this article on:

Validity and Reliability of the Revised Convergence Insufficiency Symptom Survey in Children Aged 9 to 18 Years

BORSTING, ERIC J. OD, MS, FAAO; ROUSE, MICHAEL W. OD, MS, FAAO; MITCHELL, G. LYNN MS, FAAO; SCHEIMAN, MITCHELL OD, FAAO; COTTER, SUSAN A. OD, FAAO; COOPER, JEFFREY OD, MS, FAAO; KULP, MARJEAN TAYLOR OD, MS, FAAO; LONDON, RICHARD MA, OD, FAAOand THE CONVERGENCE INSUFFICIENCY TREATMENT TRIAL GROUP

Articles

Purpose. To assess the validity and reliability of the Convergence Insufficiency Symptom Survey (CISS) in children aged 9 to 18 years. The CISS is the primary outcome measure for a pilot study evaluating two different treatments for convergence insufficiency (CI).

Methods. Children with CI were given the CISS twice to assess reliability. CISS scores for the first administration were also compared with scores from children with normal binocular vision to assess the validity of the CISS.

Results. Forty-seven children with CI and 56 children with normal binocular vision participated in the study. Reliability was assessed using intraclass correlation and 95% limits of agreement for the children with CI. For children with CI, the intraclass correlation was 0.77 (95% confidence interval, 0.613 to 0.873), and the 95% limits of agreement were −10.2 to +12.1. The mean (±SD) CISS score was 30.8 ± 8.4 for the children with CI and 8.4 ± 6.4 for the children with normal binocular vision. These means were significantly different (p < 0.0001). Good discrimination (sensitivity, 96%; specificity, 88%) was obtained using a score of ≥16.

Conclusions. Children with CI showed a significantly higher CISS symptom score than children with normal binocular vision. The results of the study indicate that the CISS is a valid and reliable instrument to use as an outcome measure for children aged 9 to 18 who are enrolled in clinical research concerning CI.

Southern California College of Optometry, Fullerton, California (EJB, MWR, SAC), The Ohio State University College of Optometry, Columbus, Ohio (GLM, MK), Pennsylvania College of Optometry, Philadelphia, Pennsylvania (MS), State University of New York, College of Optometry, New York (JC), Pacific University College of Optometry, Forest Grove, Oregon (RL)

Received January 21, 2003;

revision received July 21, 2003.

Eric Borsting

Southern California College of Optometry

2575 Yorba Linda Blvd.

Fullerton, CA 92831

e-mail: eborsting@scco.edu

The Convergence Insufficiency Treatment Trial (CITT) group has been conducting a series of pilot studies in preparation for a randomized clinical trial comparing two treatment modalities for convergence insufficiency (CI). One critical issue faced by the group was developing a method to assess the effect of treatment on a patient’s symptoms. With nonstrabismic binocular vision disorders such as CI, the success or failure of treatment is determined by changes in clinical signs as well as changes in patient symptoms (i.e., does the patient have an improvement in visual comfort and performance after the therapeutic intervention). Although scaled symptom surveys have been developed and used in the past, 1–3 there is no standardized instrument that is designed for assessing changes in symptoms associated with treatments for CI or other nonstrabismic binocular vision disorders. Developing such a survey is essential for determining the success of treatment for nonstrabismic binocular disorders.

CI is a common binocular vision disorder 4–9 and has been associated with symptoms such as visual fatigue, headaches, and double vision primarily in adults. 10–12 The association of CI and symptoms in children has recently been assessed in clinical 13 and population-based 14 samples of children using a 13-item Convergence Insufficiency Symptom Survey (CISS) developed by the Convergence Insufficiency and Reading Study (CIRS) Group. The CISS allows a two-factor analysis of symptoms: first, whether the symptom is present and second, how frequently the symptom occurs. To test the validity of the CISS, a case comparison method 13 was used to compare 14 school-aged children (aged 8 to 13 years) with CI and 14 children with normal binocular vision (NBV) of the same age recruited from a clinic population. Borsting and colleagues 13 found that the children with CI scored significantly higher (i.e., were more symptomatic) than the children with NBV. In a subsequent study, 14 a modified version of the CISS was administered to 392 children, aged 8 to 15 years, who passed a vision screening of visual acuity and refractive status. Eighteen of the children (4.6%) had three signs of CI, and this group had symptom scores that were significantly higher than the NBV group. The results of these two studies indicate that the CISS can discriminate between children with CI and children with NBV in both clinic- and population-based groups of school-aged children aged 8 to 15 years. Borsting and coworkers 15 also investigated the reliability of the CISS by administering the survey twice over a 1- to 2-week period to a group of children with two or three signs of CI. The between-session reliability was found to be excellent (intraclass correlation coefficient of 0.93).

Although our initial studies with the CISS were promising, several issues needed to be addressed before the survey could be used in a clinical trial. The original CISS ranked severity of symptoms using three or four response categories, which is adequate for discriminating between the CI and NBV groups, but may not be sensitive enough to track changes observed during therapeutic interventions. 16 In addition, the reliability of the CISS has not been established with the proposed five response categories. Finally, data on children with NBV are necessary to establish the normal variance of symptom scores in a school-aged population. To address these issues, the CISS was modified and then administered to two groups of 9- to 18-year-old children, one group of children with CI and the other with NBV.

Back to Top | Article Outline

METHODS

Survey Development

The original CISS 13 was modified to broaden the type of near work activities and track changes in symptoms during treatment (See Table 1 for revised CISS). Instead of asking about symptoms during reading and studying, subjects were asked about symptoms present when reading and performing close work because we felt that this included a broader range of activities (e.g., video games, hobbies, and pleasure reading) than only asking about reading and studying. Two items on the original CISS were divided into two separate questions to better clarify the specific symptoms. For example, one question on the original CISS asked whether the child’s eyes were tired or uncomfortable when reading or studying. This was changed to two separate questions, one related to tired eyes and the other to uncomfortable eyes. We also changed the scale for classifying frequency from four to five choices. The new version used the following response choices: never, infrequently, sometimes, fairly often, and always. Increasing the number of response choices to five makes tracking changes during therapeutic intervention more sensitive. 16 In addition, the response option labels were chosen in such a manner as to have equal perceived spacing. The labels were chosen using data on the numeric rating of frequency terms from 20 studies in the social science literature. 17

TABLE 1

TABLE 1

Back to Top | Article Outline

Subjects

Children, aged 9 to 18 years, were recruited from five optometric teaching clinics as part of a pilot study evaluating the efficacy of two different treatments for CI. The inclusion and exclusion criteria for CI subjects are listed in Tables 2 and 3. The diagnosis of CI was made when the child presented with the following three signs: exophoria at near that was at least 4 Δ greater than far, 5 failure of Sheard’s criterion 18 or minimum normative positive fusional vergence (break <15 Δ), 19 and a receded nearpoint of convergence (≥6 cm). 20 Additionally, potential CI subjects were given the original 13-item CISS to determine whether their symptom score met the inclusion criteria of ≥9. 13 Subjects with attention deficit disorder, learning disability, or currently taking a medication that could affect accommodation or vergence were excluded. Children identified as having CI were scheduled for an eligibility examination, which included administration of the revised CISS along with other tests for diagnosing CI.

TABLE 2

TABLE 2

TABLE 3

TABLE 3

Children in the age range of 9 to18 years with NBV were recruited from each of the five optometric teaching clinics. The inclusion criteria are listed in Table 4, and the exclusion criteria, which were essentially the same as for children with CI, are listed in Table 3.

TABLE 4

TABLE 4

Each CITT study site received approval from its affiliated institutional review board (Southern California College of Optometry, Pennsylvania College of Optometry, The Ohio State University College of Optometry, State University of New York College of Optometry, and Pacific University College of Optometry). Centralized human subjects approval was obtained from the Biomedical Sciences Institutional Review Board at The Ohio State University, including approval of the individual informed consent documents. A parent or guardian provided consent, and each child provided assent before any testing was done.

Back to Top | Article Outline

Procedure

The CISS (Table 1) was administered to each of the children with either CI or NBV. To assess reliability, CI subjects were given the CISS a second time when they returned for their initial treatment visit. Subjects with NBV were given the CISS at the eligibility examination, but did not participate in the reliability portion of the study.

Questions from the CISS were read to each subject while he or she looked at a printed copy of the response options. The questions were read, in order, exactly as written and were repeated if the subject did not respond or requested to hear the question again. After each question, the examiner recorded the subject’s response. The survey was scored as follows: never (0), infrequently (1), sometimes (2), fairly often (3), and always (4). The total score was then obtained by summing the points for all 15 items, which could range from 0 to 60.

Back to Top | Article Outline

RESULTS

Subjects

Forty-seven children with CI and 56 children with NBV were enrolled in the study. The mean (±SD) age was 11.5 ± 2.2 years for the CI group and 11.4 ± 2.2 years for the NBV group. In the CI group, 57.5% of the subjects were female, and in the NBV group, 45.5% were female. The distribution of children by center is listed in Table 5. The Pennsylvania College of Optometry center recruited the most CI and NBV subjects, but the symptom score was not significantly different when comparing the Pennsylvania College of Optometry to the other centers (mean at Pennsylvania College of Optometry, 31.56; mean at other sites, 30.79; p = 0.124). The values for near heterophoria, positive fusional vergence, nearpoint of convergence, and accommodative amplitude are shown in Table 6. The CI group had significantly different values than the NBV group on all three signs of CI (p < 0.0001).

TABLE 5

TABLE 5

TABLE 6

TABLE 6

The internal consistency of the survey was assessed using Cronbach’s alpha coefficients. The coefficient was 0.92, and no item was negatively correlated with the total. This indicates that the internal consistency of the CISS was good to excellent and that the items within the survey were not redundant.

Reliability of the CISS for the children with CI was assessed using the intraclass correlation coefficient (ICC) 24 and 95% limits of agreement. 25 The mean time between administration was 14.6 ± 14.7 days. The mean difference between the first and second administration was 0.98 ± 5.7 points, indicating minimum bias between the two administrations (one-sample t-test, 1.14; p = 0.2607). The ICC was 0.77 (95% confidence interval, 0.613 to 0.873), and the 95% limits of agreement were −10.2 to +12.1 (Fig. 1).

FIGURE 1.

FIGURE 1.

The mean score on the CISS at the eligibility visit was 30.8 ± 8.4 for the children with CI and 8.4 ± 6.4 for the NBV children. The children with CI scored significantly higher than the NBV group (t = 15.4, p < 0.0001). In addition, the age of the child did not correlate with the symptom score (r = 0.052, p = 0.728). The distribution of response option on each item for children with CI and children with NBV is shown in Table 1. We have also graphically presented the distribution of the percentage of children with CI and children with NBV responding “fairly often” or “always” on the CISS (Fig. 2).

FIGURE 2.

FIGURE 2.

To assess the ability of the CISS to correctly classify subjects as CI or NBV, sensitivity and specificity values were calculated using various cutoff values for the CISS score. A cutoff value of 16 (i.e., CISS ≥16) yielded a sensitivity of 95.7% and a specificity of 87.5%. This value was also 1 SD above the mean for the NBV group. A list of cutoff values and corresponding sensitivity and specificity values are shown in Table 7. We have also graphically shown the distribution of CISS scores for the CI and NBV groups (Fig. 3).

TABLE 7

TABLE 7

FIGURE 3.

FIGURE 3.

Back to Top | Article Outline

DISCUSSION

The results of this study indicate that the CISS is a valid and reliable instrument for use as a primary outcome measure for 9- to18-year-old children enrolling in the CITT. Children with CI scored significantly higher than the NBV group on the CISS, suggesting that the survey is valid. In addition, an ICC of approximately 0.8 indicates that the CISS has good reliability.

The results of this study are similar to those found by Borsting and colleagues. 13,14 Both studies found that children with CI had significantly higher symptom scores than children with NBV. The total symptom score in this study is higher than that found in both of our previous studies 13,14; this is primarily because of the increased number of questions (13 to 15) and the expanded response categories (3 or 4 to 5) in the modified CISS. Our results are also consistent with those of McKeon and colleagues, 26 who used the Vision Function Scale in patients with intermittent exotropia. The Vision Function Scale has many items that are similar to the revised CISS (e.g., How often do you lose your place?). The intermittent exotropia group was found to have a higher symptom score than the visually normal group.

This study assessed the test-retest reliability of the CISS, which is important for evaluating changes in symptoms occurring before and after a specific treatment. The 95% limits of agreement were −10.2 to +12.1 with a mean bias of 0.98. This means that a change of more than 10 points would be considered clinically meaningful and outside the range of normal variability. For example, a child with CI who scored 32 on the CISS before treatment would have to score ≤21 after treatment for the change to be considered significant. These data allow both the practitioner and the researcher to determine whether a treatment had a clinically meaningful effect on the patient’s symptoms.

The practitioner can use the results of this study to distinguish between children with normal and abnormal levels of symptoms associated with CI by using a symptom score of ≥16. This score is more than 1 SD from the mean of the children with NBV and has high sensitivity (95.7) and specificity (87.5). Using this value, only one CI subject was considered asymptomatic, and seven NBV subjects were considered symptomatic. We can only speculate on why seven NBV subjects were symptomatic given our current data. The NBV subjects may have had a binocular dysfunction that we did not assess (e.g., vergence facility) or an undiagnosed learning disorder.

Although it is has been suggested that CI is not common in children, and the associated symptoms, such as blur and diplopia, can be the result of the child’s interpretation of normal physiological phenomenon, 27 no data have been presented to support this position. However, we can indirectly investigate this claim by looking at our data and comparing the occurrence of blur and diplopia as reported by children with CI and NBV. If blur and diplopia were the result of the child’s interpretation of normal physiological phenomenon, one would expect both groups to report these symptoms with equal frequency. However, we found that the children with CI reported blur as fairly often or always in 40.5% of cases, whereas only 4.4% of NBV children reported blur this frequently. For diplopia, we found that the children with CI reported diplopia as fairly often or always in 36.5% of cases, whereas no child with NBV reported diplopia in the fairly often or always categories. Children with NBV may report blur or diplopia as Wright and Boger 27 suggest, but the occurrence tends to fall into the “infrequently” category (19.6% for blur and 7.1% for diplopia). Thus, in our study, children with CI had a significantly greater occurrence of blur and diplopia as well as all other symptoms on the CISS than children with NBV (Fig. 2).

The CISS appears appropriate to use in children presenting with symptoms associated with convergence insufficiency. Even though this study did not address the use of CISS for other nonstrabismic disorders of accommodation and vergence, the symptoms described in the literature tend to be similar across these vision conditions. 28 For example, headaches and eyestrain are reported in both CI and accommodative dysfunction. This hypothesis is also supported by recent research conducted by Borsting et al., 14 who found that the CISS was able to discriminate between children with accommodative insufficiency and NBV. As a result, future studies should look at the use of the CISS in other disorders of accommodation and vergence.

In conclusion, children with all three signs of CI showed a significantly higher CISS symptom score than children with NBV. This study adds further evidence to support previous research 7,13,14 indicating that CI has a significant number of associated symptoms. In addition, the results of this study demonstrate that the CISS is a valid and reliable instrument for evaluating symptoms in 9- to 18-year-old children. Future studies should evaluate the CISS in adults and also evaluate the use of the CISS in other binocular vision disorders.

Back to Top | Article Outline

ACKNOWLEDGMENTS

This Convergence Insufficiency and Treatment Trial Study Group pilot study was supported by National Eye Institute, National Institutes of Health planning grant EY13164. Presented at the American Academy of Optometry Meeting, December 2001, Philadelphia, PA.

Back to Top | Article Outline

REFERENCES

1. Sheedy JE, Saladin JJ. Phoria, vergence, and fixation disparity in oculomotor problems. Am J Optom Physiol Opt 1977; 54: 474–8.
2. Sheedy JE, Saladin JJ. Association of symptoms with measures of oculomotor deficiencies. Am J Optom Physiol Opt 1978; 55: 670–6.
3. Cooper J, Selenow A, Ciuffreda KJ, Feldman J, Faverty J, Hokoda SC, Silver J. Reduction of asthenopia in patients with convergence insufficiency after fusional vergence training. Am J Optom Physiol Opt 1983; 60: 982–9.
4. Cooper J, Duckman R. Convergence insufficiency: incidence, diagnosis, and treatment. J Am Optom Assoc 1978; 49: 673–80.
5. Daum KM. Convergence insufficiency. Am J Optom Physiol Opt 1984; 61: 16–22.
6. Letourneau JE, Ducic S. Prevalence of convergence insufficiency among school children. Can J Optom 1988; 50: 194–7.
7. Rouse MW, Hyman L, Hussein M, Solan H, Convergence Insufficiency and Reading Study (CIRS) Group. Frequency of convergence insufficiency in optometry clinic settings. Optom Vis Sci 1998; 75: 88–96.
8. Rouse MW, Borsting E, Hyman L, Hussein M, Cotter S, Flynn M, Scheiman M, Gallaway M, De Land PN, Convergence Insufficiency and Reading Study (CIRS) group. Frequency of convergence insufficiency among fifth and sixth graders. Optom Vis Sci 1999; 76: 643–9.
9. Porcar E, Martinez-Palomera A. Prevalence of general binocular dysfunctions in a population of university students. Optom Vis Sci 1997; 74: 111–3.
10. White JW, Brown HW. Occurrence of vertical anomalies associated with convergent and divergent anomalies: a clinical study. Arch Ophthalmol 1939; 21: 999–1009.
11. Hirsch MJ. A study of forty-eight cases of convergence insufficiency at the near point. Am J Optom Arch Am Acad Optom 1943; 20: 52–8.
12. Kent PR, Steeve JH. Convergence insufficiency, incidence among military personnel and relief by orthoptic methods. Milit Surgeon 1953; 112: 202–5.
13. Borsting E, Rouse MW, De Land PN, Convergence Insufficiency and Reading Study (CIRS) Group. Prospective comparison of convergence insufficiency and normal binocular children on CIRS symptom survey Optom Vis Sci 1999; 76: 221–8.
14. Borsting E, Rouse MW, Deland PN, Hovett S, Kimura D, Park M, Stephens B. Association of symptoms and convergence and accommodative insufficiency in school-age children. Optometry 2003; 74: 25–34.
15. Borsting E, Rouse MW, De Land PN, Convergence Insufficiency and Reading Study (CIRS) Group. Survey instruments in the convergence insufficiency and reading study. Optom Vis Sci 1998; 75( Suppl): 10.
16. Kirshner B, Guyatt G. A methodological framework for assessing health indices. J Chronic Dis 1985; 38: 27–36.
17. Mosteller F, Youtz C. Quantifying probabilistic expressions. Stat Sci 1990; 5: 2–34.
18. Sheard C. Zones of ocular comfort. Am J Optom Arch Am Acad Optom 1930; 7: 9–25.
19. Scheiman M, Herzberg H, Frantz K, Margolies M. A normative study of step vergence in elementary schoolchildren. J Am Optom Assoc 1989; 60: 276–80.
20. Hayes GJ, Cohen BE, Rouse MW, De Land PN. Normative values for the nearpoint of convergence of elementary schoolchildren. Optom Vis Sci 1998; 75: 506–12.
21. Morgan MW. Analysis of clinical data. Am J Optom Arch Am Acad Optom 1944; 21: 477–91.
    22. Wesson MD. Normalization of prism bar vergences. Am J Optom Physiol Opt 1982; 59: 628–34.
      23. Hofstetter HW. A comparison of Duane’s and Donders’ tables of the amplitude of accommodation. Am J Optom Arch Am Acad Optom 1944; 21: 345–63.
        24. Fleiss JL. The Design and Analysis of Clinical Experiments. New York: Wiley, 1986.
        25. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1: 307–10.
        26. McKeon C, Wick B, Aday LA, Begley C. A case-comparison of intermittent exotropia and quality of life measurements. Optom Vis Sci 1997; 74: 105–10.
        27. Wright JD Jr, Boger WP III. Visual complaints from healthy children. Surv Ophthalmol 1999; 44: 113–21.
        28. Scheiman M, Wick B. Clinical Management of Binocular Vision: Heterophoric, Accommodative and Eye Movement Disorders, 2nd ed. Philadelphia: Lippincott Williams & Wilkins, 2002.
        Keywords:

        convergence insufficiency; symptom survey; reliability; validity; children

        © 2003 American Academy of Optometry