RYAN, BARBARA Bsc(Hons), MCOptom; COURT, HELEN Bsc(Hons); MARGRAIN, TOM H. PhD, Bsc, MILT, MCOptom
The growing number of people with a visual impairment1 and fragmented nature of low vision services2 recently prompted the Welsh Assembly Government to establish an all Wales Low Vision Service (WLVS),3 which is now operating in over one third of optometry practices throughout the country. The service has responded to the views of patients4 and enshrines many of the features recommended by the UK Low Vision Services Consensus Group.5 That is, the service has good links with other services (secondary health care, social services, education professionals, and voluntary organizations), it is provided close to where it is needed, waiting times are short, and people with low vision may be referred to the service via a number of routes including self-referral.3
Although the new service may be more user friendly than traditional UK hospital-based services at this time, it is not clear whether it is effective. All the 145 practitioners providing the service have been trained and accredited, but there are differences in prior experience and the number of patients they see (some will assess <20 people a year). Consequently, there may be variations in the effectiveness of the service in different locations. How then should the performance of a large-scale, multi-center, low vision service be evaluated? Clinical outcomes and logistical measures are being audited but these measures do not necessarily reflect the ability of the service to help people.
Like other areas of optometry and ophthalmology,6 quality-of-life has become an important measure when trying to determine the effectiveness of a low vision service7-11 or intervention.12,13 However, despite over 95% of people attending modern UK low vision services reporting that they use the devices loaned,7,13 improvements demonstrated in quality-of-life following low vision intervention have been very modest.7-9,11 Such modest findings may be attributed to insensitive outcome measures and/or reflect the fact that current forms of low vision service provision have little effect on overall quality-of-life. Whatever the reason, the use of patient-based outcome measures that are unresponsive to the intervention is inappropriate.14 From a service evaluation perspective, the use of unresponsive outcome measures is highly problematic because they cannot differentiate between services that are offering a high standard of care from those that are not.
In the area of health care assessment, quality-of-life is a concept about which there is little agreement.14,15 However within the area of vision rehabilitation, the notion that it is a multidimensional concept (which includes a functional, a physical, a social, and a psychological dimension) is currently popular.16 Evidence suggests that, current forms of low vision service provision have their greatest effect on the functional dimension.9,11 More specifically, low vision service provision has repeatedly been shown to help people with particular near, reading, and distance vision tasks9,11,17 i.e., it reduces disability for some near, reading, and distance tasks. This is not surprising because one of the primary aims of low vision service provision is to help people overcome the disabling effects of visual impairment. For example, people with low vision frequently report that they have difficulty reading. By providing magnifiers, improved lighting, developing viewing strategies and/or using a large print alternative, reading ability for some tasks is often improved.
The need for sensitive instruments to evaluate low vision service outcomes has meant that the development of questionnaires in the low vision field is possibly one of the most advanced in any area of optometry or ophthalmology. Of particular note has been the use of Rasch analysis to improve the design, sensitivity, and validity of questionnaires.9,12,17-25
Most of the visual function assessment and quality-of-life questionnaires used in optometry and ophthalmology use Likert (ordinal) scales.22 Typically, the category labels used (e.g., agree, disagree) are assigned numerical values (e.g., 1 to 5) which are then added together to provide a domain and/or instrument score. However, it is not appropriate to perform mathematical operations on values which originate from ordinal scales because the true relationship between response categories is unknown. In addition, this method of analysis assumes that items within each subscale are of equal difficulty; each subscale is of equal difficulty; and patients' responses to one item correspond to patients' responses to another item. Rasch analysis can be used to overcome many of the limitations associated with Likert scales.21,23
The WLVS is the first attempt at a large-scale, national, government funded, multi-center low vision service in the United Kingdom. It is vital that the effectiveness of the service is monitored continuously and services are compared on a center-by-center basis. Therefore, the primary aim of this study was to develop a short, sensitive, user centered, outcome measure and to describe its psychometric properties using Rasch analysis. In accordance with government funded National Health Service (NHS) Health Technology recommendations, the measure should be: appropriate, responsive, valid, precise, easy to interpret, reliable, acceptable to patients, and feasible in terms of the burden associated with its administration.14 A secondary aim was to determine if postal implementation and self-completion might bias data collection.
METHODS
Questionnaire Selection
Development of an entirely new instrument was outside the scope of this study and there are many good examples already available.17,22,26,27 In March 2004, all questionnaires that had been used to evaluate low vision service outcomes were reviewed. Questionnaires were considered, if published information regarding their development and psychometric properties was available, they measured one or more dimensions of quality-of-life as defined by Aaranson16, and they had been reported to be sensitive to at least one low vision intervention. At that time, only three questionnaires met the criteria: the Visual Quality of Life Core Measure questionnaire (VCM1); NEI VFQ; and the Low Vision Quality of Life questionnaire (LVQoL).
Of the three questionnaires reviewed, the NEI VFQ was considered to be the most suitable for several reasons: a change in quality-of-life had been demonstrated in three low vision services by two independent research teams8,9; it had been used to study populations in the United Kingdom28; and single item Rasch scores have been published which facilitated the assessment of individual questions.9
Questionnaire Design
The original NEI-VFQ consisted of 51 items10,29 but a shorter 25-item version, with 14 additional optional items, was later developed.30 Given the scale of the WLVS, and our desire to monitor the service continuously on a center-by-center basis, we were concerned that even the 25-item version of the NEI-VFQ was too long. Significantly, assessment of single item Rasch scores following attendance at two particularly comprehensive low vision services for veterans in America suggested that only seven items were sensitive to low vision service intervention.9 These questions are relevant to service users, they fit Aaranson's definition of the functional dimension of quality-of-life, and are common to other low vision service outcome measures.11,17,31,32 The WLVS focuses on providing magnifiers, lighting, tints, and advice. Therefore, the seven items include aspects of functioning that one would reasonably expect the new NHS funded WLVS to have an impact on. They do not include items such as mobility and daily living skills, which are covered by the local authority funded social services rehabilitation programs in Wales, with which the NHS service links, but which are not the focus of this appraisal. Therefore, we chose to base our outcome measure on these seven items (Table 1). Although the inclusion of all the items on the NEI VFQ 25 would have allowed us to measure quality-of-life, we were concerned that it would have been an unjustifiable burden to thousands of patients and hundreds of services when evidence suggested it would not provide any additional information about the services we wished to assess.
The information sheet for the NEI VFQ was revised to Anglicize the language, include the use of low vision aids in the instructions as recommended by Stelmack et al.9 and incorporate a consent form.
It has been shown that people with a visual impairment can self complete questionnaires if they are provided in large bold print.11 Therefore, a 16 point, bold version was piloted with two focus groups of people with low vision at Cardiff Institute for the Blind. Both groups found the questionnaire relevant and easy to complete. However, on their recommendation, the wording of the question about television was changed from seeing and enjoying programs on television to watching television.
The response options were identical to those on the original NEI-VFQ 25, i.e., the choices included: no difficulty, a little difficulty, moderate difficulty, extreme difficulty, stopped doing this because of my eyesight, and stopped doing this for other reasons/not interested.33
Study Design and Population
During the establishment of the WLVS (July 2004 to 2005), newly accredited practitioners were asked to distribute questionnaires along with a stamped addressed envelope to adults (>17 year of age) who booked an appointment for a low vision assessment. Patients were asked to complete the questionnaire at home, before their appointment, and to return the completed questionnaire, in a stamped addressed envelope to a central NHS administration team where the information was transferred to a centralized database. Practitioners were also supplied with standard record cards, which included both biographical and clinical information, and were required to fax these (via a secure fax) to the central NHS administration team following the low vision assessment. Data from patients who provided informed consent were incorporated into the database. All procedures adhered to the tenets of the Declaration of Helsinki and ethical approval was obtained from the all Wales Research Ethical Committee.
Statistical Analysis
Biographical and clinical information and questionnaire data were transferred from the centralized database to SPSS 12.0.1 for analysis. Data from record cards were matched with returned questionnaires on the basis of date of birth and name. Questionnaire data were recoded and, in accordance with previous studies of the NEI VFQ, the last category stopped doing this for other reasons or not interested in doing this was treated as missing data.9
To look for response bias, the demographic and visual function characteristics of patients who returned completed questionnaires (<3 missing items) were compared to those who did not return a completed questionnaire. Following evaluation of the distributional characteristics of the data set (Shapiro-Wilks test), the Mann-Whitney U test was used to evaluate age and visual acuity data. χ2 tests were used to determine differences in gender, prevalence of age-related macular degeneration (AMD) and living situation.
Completed questionnaires were evaluated using the Andrich Rating Scale model which was implemented using Winsteps Version. 3.58.1.34 Rasch analysis assumes that only one underlying variable is being measured by the questionnaire, i.e., that the scale is unidimensional. The analysis applies a probabilistic logistic model to the data set to produce Logit values which describe both item difficulty and person ability. Summary fit statistics (Infit and Outfit mean square) describe how well both item scores and person responses fit the Rasch model. Infit statistics are weighted to give more importance to those people who are closer to the item mean. Outfit statistics are not weighted and so, are more sensitive to outlying scores. Items which fit perfectly to the unidimensional scale have an expected Infit and Outfit mean square statistic of 1. Data quality was also investigated using traditional statistical tests, i.e., to describe the distribution of responses, calculating the percentage of missing data, and assess ceiling effects.
RESULTS
Data were collected from the record cards of 1154 adults who consented to take part in the research. Of these, 490 returned a completed questionnaire, giving a minimum response rate of 42.5%. The demographic and visual functioning characteristics of the sample (n = 1154) are presented in Table 2. There were no significant differences in the age (Mann-Whitney U, p = 0.291), gender (χ2, p = 0.750), distance binocular VA (Mann-Whitney U, p = 0.862), living situation (χ2, p = 0.336) or prevalence of macular degeneration (χ2, p = 0.311) between those who returned a completed questionnaire and those who did not.
Initial Questionnaire Description
The Winsteps program34 enables item difficulty and person ability to be visualized along a linear scale (like a ruler) known as a person-item map (Fig. 1). The symbols to the left of the vertical line describe the ability of individual respondents. The mean difficulty of each item is denoted to the right of the vertical line with items at the top of the map (e.g., difficulty watching television) best able to discriminate between those people with poorer vision and items at the bottom of the map (e.g., reading small print) best able to discriminate between those people with better vision. The mean item difficulties range from -1.21 to 1.38 logits; item 6 (reading bills) has the overall mean item difficulty and is therefore located at 0 logits. The accuracy with which items are placed along the ruler is described by the root mean square standard error (i.e., the average standard error of the items) and for this analysis it was 0.08 logits. The precision of these item difficulties is indicated by the high separation reliability coefficient (0.99), which ranges from 0 to 1.
Inspection of the person-item map indicates that the items have levels of difficulty that are oriented toward more able people. This analysis shows that the mean person ability is 1.70 logits from the mid point of the ruler. The precision with which people are located on the ruler is described by the root mean square standard error and in this case it was 0.74 logits. The person separation reliability coefficient describes the reliability of person ordering and is similar to the conventional Cronbach alpha coefficient. Person separation reliability is 0.85 for this sample. The person separation ratio expresses the reliability of the scale to discriminate between people of different abilities. It was 2.39 for this sample, which exceeds the minimum recommended value of two.35
Therefore, inspection of Fig. 1 shows a substantial range in person ability, a relatively limited range in item difficulty, and a mismatch between person and item means.
Response Scale Analysis
Initial inspection of category usage showed that all categories had been used, but category 0 (no difficulty at all) far less so than the others (0 to 8%). Therefore, in accordance with standard practice36 and the work of others,37 we decided to combine category 0 with category 1 (a little difficulty) in the analysis. This improved category utilization of the end category from 0 to 8% to 4 to 22%, and had the added benefit of improving the targeting of questions to persons. The mismatch between persons and items reduced from 1.7 to 0.86 logits. The root mean square standard error of person ability and item difficulty estimates were 0.78 and 0.09, respectively. Regarding the ability of the scale to discriminate between patients of different visual ability, person separation reduced slightly from 2.39 to 2.32 after category combination, however, it still remained above two.35
Item Reduction
Descriptive data and Rasch fit statistics are shown in Table 3. Before inspecting the items, person fit was examined to check for rogue responses which may contribute to poorly fitting items. Questionnaires completed by people with poor fit statistics were examined but there were no apparent inconsistencies, so all the questionnaire data were used.
To ensure measurement validity, Rasch analysis was used to identify any items which misfit the Rasch model. Pesudovs et al.35 presented a framework to assess the functioning of items, including examination of traditional statistics. This assessment suggested that all questions performed well. Only one aspect of the performance of question 4 (Infit mean square 1.42) was just outside the conservative criteria proposed by Pesudovs et al. However, Linacre34 indicates that items with Infits of up to 1.5 are productive of measurement and hence we decided to retain this item.
Descriptive data and Rasch model fit statistics for the final questionnaire are described in Table 3 and the person item map is shown in Fig. 2. The final seven-item, four response category questionnaire has good measurement precision expressed by the high person and item reliability measures, 0.84 and 0.99, respectively.
DISCUSSION
The WLVS is a national, multicenter, low vision service. It is a flagship initiative of the Welsh Assembly Government and enshrines many of the principles identified by the UK Low Vision Services Consensus Group.5 Given the pioneering nature of the service in Wales and the substantial investment of public money it is vital that it is evaluated in a rigorous fashion. However, although it is generally accepted that low vision service provision helps people with low vision, currently there is little evidence to support this notion7-9,11,17 and even less is understood about what influences the success of a service.13,17 Evaluation of the WLVS is complicated because there are over 160 service delivery points, and outcomes may vary from center to center. Therefore, it was thought essential to ascertain if all the services were having an impact and if there was variation in their effectiveness. This has dictated that the initial service assessment should include an outcome measure at each location, was centralized, and pragmatic.
Our approach to question selection breaks with tradition in as much that we did not select items on the basis of focus group discussions. Rather we opted to build on the work of others to inform reduction by retaining questions which have been field tested on people with low vision,8,9,28 and which have also been shown to be responsive.9 Some might suggest that by choosing items that are known to be responsive that the relationship between the instrument and the service is circular and demonstration of success is a self fulfilling prophesy. That is, the instrument can only measure what the service does and this may have no real meaning for patients, and hence it could produce a positive result but one that is meaningless in terms of patient benefit. Clearly, for an outcome measure to have any real value it must measure something that matters to patients. While it is certainly the case that the seven-item NEI VFQ only measures that part of a patient's visual disability that low vision service provision can do something about there is strong evidence to suggest that what is being measured matters to patients. First, low vision rehabilitation addresses the needs of the individual and people with low vision typically ask for help with tasks such as: reading bills, cooking, sewing, fixing things around the house, seeing prices in shops, looking at photos, watching television, going out, reading the newspaper etc. That is, they ask for help with the types of task coved by the questions on the seven-item NEI VFQ. Second, the original NEI VFQ, from which the seven items are drawn, was developed in response to issues raised by people with chronic eye diseases which can cause low vision (e.g., cataract, age-related macular degeneration, glaucoma, and diabetic retinopathy). Third, the overlap with items on other questionnaires, which have been developed in response to issues raised specifically by people with low vision, and which feature in the leading vision related quality-of-life instruments, is striking (see Table 1).11,29-32 Hence, there is very strong evidence to suggest that the seven items on this questionnaire measure something that matters to people with low vision.
Low vision service provision does not seek to give people back the vision they have lost, i.e., it is unlikely to improve their overall visual ability. Rather, it offers a range of interventions which aim to minimize disability by making specific tasks easier to perform. Massof et al. have identified 337 tasks that are relevant to people with low vision and which could be included in a customized questionnaire.38,39 Tailor made questionnaires are arguably the ideal method of quantifying an individuals disability because they only include items that are relevant to the person concerned. However, tailor made questionnaires need to be administered by an interviewer, require the use of specialist software, and are relatively time consuming to implement. Hence, such questionnaires are not well suited to the evaluation of a large-scale community-based low vision service. Similarly the associated resources required to use any of the excellent validated instruments (19 items and above) which have been developed specifically to measure the outcome of visual rehabilitation programs (IVI and VFQ 48 LVQol) would be prohibitive to evaluate the performance (for thousands of people within a couple of days of making an appointment all over the country) of each of the 145 practitioners that provide the WLVS.
The seven items clearly do not cover all the issues that may be relevant to an individual with low vision and so the questionnaire would not be well suited to the task of quantifying an individuals overall level of disability or exploring the holistic impact of a service on an individual. However, Rasch model fit statistics suggest that they measure a single unidimensional construct, i.e., it is more than the sum of its parts. This observation is consistent with previous reports which have shown that items in the NEI VFQ which use difficulty ratings work together to measure a single latent variable.18 Although it could be argued that the questions are measuring one dimension of quality-of-life (i.e., the functional dimension) it is more appropriate to describe the seven items from the NEI VFQ as a visual disability scale.40 However, even this description is not complete because the seven questions were selected on the basis that they have previously been shown to be responsive to low vision service intervention. Therefore, the seven items from the NEI VFQ only try to measure aspects of vision related disability that are addressed by low vision service intervention. Consequently, it is targeted toward measurement of low vision service outcomes and should allow differentiation between the performances of low vision service providers. Low vision service provision may have an effect on constructs such as QoL, vision related QoL, or overall disability but at the very least, it should have an effect on the aspects of disability that it claims to be able to do something about. We suggest this is the construct measured by the seven-item NEI VFQ and in that respect; it should be a highly responsive service outcome measure.
Inspection of the person item map for the raw seven-item NEI-VFQ (Fig. 1) indicated that while the questions were good at discriminating between people with relatively good vision, they were less well suited to those with poorer vision. However, by combining the first and second response categories in the analysis (no difficulty at all and a little difficulty) both targeting and category usage were improved without compromising person separation (Fig. 2). Hence, we recommend that these response categories are collapsed in the analysis of the seven-item NEI-VFQ. Although this change improved targeting, there is still a bias toward slightly more able people (0.86 logits). However, this is unlikely to be problematic because the seven-item NEI VFQ item difficulties range from -4.48 to 4.94 logits when the influence of category structure is accounted for, i.e., it can reliably measure a wide range of person ability. There is no evidence to suggest that the presentation of the questionnaire should be changed.
Questionnaires with a reduced number of items are advantageous in as much that they reduce response burden and facilitate efficient implementation. However, short form questionnaires run the risk of compromising measurement precision.41 The statistics used to describe the performance of our questionnaire are the real not the model summary statistics and hence describe the worst case performance of the instrument. However, even this conservative estimate indicates that the seven-item NEI VFQ is a fully functional questionnaire.
In the United Kingdom, the NHS Health Technology Assessment Programme has established a set of criteria designed to aid the selection of appropriate patient-based outcome measures.14 That is, questionnaires should be: appropriate, responsive, valid, precise, easy to interpret, reliable, acceptable to patients, and feasible in terms of the burden associated with its administration. Patient outcome measures differ in their ability to meet each of these criteria. Indeed, there are trade offs between criteria, e.g., questionnaires containing a large number of items may be precise but they are less likely to be feasible. There is also a trade off between the concept being measured and responsiveness, i.e., the more distal the concept is from the intervention the less likely it is to be responsive.14 While we might expect low vision service provision to have a beneficial effect on quality-of-life it is arguably a distal concept and hence is likely to be less responsive. Although the responsiveness of the seven-item NEI VFQ has yet to be established, judged against the criteria listed above, we believe it is an appropriate instrument with which to measure the outcomes of this large scale multi-center low vision service.
In order to identify and advance best practice in low vision service provision, we present the Rasch analysis derived scoring key for the seven-item NEI VFQ in Table 4. By substituting the Likert scores for each item with those in the table, the questionnaire is converted to a true linear measurement scale. Use of this scoring key will enable other investigators to use the seven-item NEI VFQ and compare their results to those obtained from the WLVS.
By obtaining a measure of visual disability before and after low vision service intervention it will be possible to monitor the performance of each service center in the WLVS. This will provide a measure of quality assurance for the public, enable the identification of best practice, and identify services where remedial action may be required. It is hoped that future investigations will allow more in-depth assessment of the outcomes of a sample of the services once the variation in provision has been determined and explored.
Although 1154 adults consented to take part in research, we only received completed questionnaires from 490 people giving an apparent response rate of 42.5%. However, this figure must be treated with caution because we cannot be sure that everyone who consented was given a questionnaire, e.g., from time to time optometrists may have forgotten to give the patient a questionnaire before their appointment. Hence, this response rate is the minimum response rate.
Although, comparison of some key demographic and visual functioning characteristics of those who completed and returned the seven-item NEI VFQ with those who did not failed to provide any evidence to suggest response bias for this implementation method, e.g., there is no evidence that people who have poorer vision are less likely to return this postal questionnaire, we cannot be sure that response bias does not exist. Data are rarely, if ever, missing at random.42 Although missing data are unlikely to have had a significant effect on our analysis of the psychometric properties of the seven-item NEI VFQ (i.e., it is most unlikely we have missed an important group of patients), it may play a significant role in generating response bias in a before and after study. For example, people who have a negative experience may be less (or in some cases, more) likely to return the post intervention questionnaire. Hence, it is vital that the response rate for the follow-up post intervention questionnaire is high.
Since the end of the first year, the questionnaire return rates have been monitored and encouragement has been given to all practitioners and practice staff to ensure that everyone using the service is given a questionnaire.
CONCLUSIONS
A description of the psychometric properties of the seven-item NEI-VFQ 25 using Rasch analysis in a population of adults attending the WLVS shows that it provides a meaningful measure of the ability of a person with low vision to perform everyday tasks and hence it is a valid tool with which to assess a large scale multicenter low vision service. The questionnaire is made up of items that are important for people with low vision but, it does not include them all. The seven-item NEI VFQ only measures that aspect of visual disability that low vision service provision claims to be able to do something about. Therefore, it should be a highly sensitive service outcome measure. The adoption of such a short highly focused outcome measure is a pragmatic approach to the evaluation of the WLVS.
ACKNOWLEDGMENTS
This research was funded by the Welsh Assembly Government and Cardiff University.
Tom H. Margrain
School of Optometry and Vision Sciences
Cardiff University, Cardiff CF10 3AN, United Kingdom
e-mail: margrainth@cardiff.ac.uk
REFERENCES
1. Bunce C, Wormald R. Leading causes of certification for blindness and partial sight in England & Wales. BMC Public Health 2006;6:58.
2. Culham LE, Ryan B, Jackson AJ, Hill AR, Jones B, Miles C, Young JA, Bunce C, Bird AC. Low vision services for vision rehabilitation in the United Kingdom. Br J Ophthalmol 2002;86:743-7.
3. Margrain TH, Ryan B, Wild JM. A revolution in Welsh low vision service provision. Br J Ophthalmol 2005;89:933-4.
4. Ryan B, McCloughan L. Our Better Vision: What People Need from Low Vision Services in the UK. London: Royal National Institute for the Blind; 1999.
5. Low Vision Services Consensus Group. Recommendations for Future Low Vision Service Delivery in the UK. London: Royal National Institute for the Blind; 1999.
6. Fielder AR, Bentley C, Moseley MJ. Recent advances: ophthalmology. BMJ 1999;318:717-20.
7. Hinds A, Sinclair A, Park J, Suttie A, Paterson H, Macdonald M. Impact of an interdisciplinary low vision service on the quality of life of low vision patients. Br J Ophthalmol 2003;87:1391-6.
8. Scott IU, Smiddy WE, Schiffman J, Feuer WJ, Pappas CJ. Quality of life of low-vision patients and the impact of low-vision services. Am J Ophthalmol 1999;128:54-62.
9. Stelmack JA, Stelmack TR, Massof RW. Measuring low-vision rehabilitation outcomes with the NEI VFQ-25. Invest Ophthalmol Vis Sci 2002;43:2859-68.
10. Mangione CM, Lee PP, Pitts J, Gutierrez P, Berry S, Hays RD. Psychometric properties of the National Eye Institute Visual Function Questionnaire (NEI-VFQ). NEI-VFQ Field Test Investigators. Arch Ophthalmol 1998;116:1496-504.
11. Wolffsohn JS, Cochrane AL. Design of the low vision quality-of-life questionnaire (LVQOL) and measuring the outcome of low-vision rehabilitation. Am J Ophthalmol 2000;130:793-802.
12. Smith HJ, Dickinson CM, Cacho I, Reeves BC, Harper RA. A randomized controlled trial to determine the effectiveness of prism spectacles for patients with age-related macular degeneration. Arch Ophthalmol 2005;123:1042-50.
13. Reeves BC, Harper RA, Russell WB. Enhanced low vision rehabilitation for people with age related macular degeneration: a randomised controlled trial. Br J Ophthalmol 2004;88:1443-9.
14. Fitzpatrick R, Davey C, Buxton MJ, Jones DR. Evaluating patient-based outcome measures for use in clinical trials. Health Technol Assess 1998;2:1-74. Available at:
http://www.hta.nhsweb.nhs.uk/fullmono/mon214.pdf. Accessed October 4, 2007.
15. Garratt A, Schmidt L, Mackintosh A, Fitzpatrick R. Quality of life measurement: bibliographic study of patient assessed health outcome measures. BMJ 2002;324:1417-9.
16. Aaronson NK. Quality of life: what is it? How should it be measured? Oncology 1988;2:69-76.
17. Stelmack JA, Szlyk JP, Stelmack TR, Demers-Turco P, Williams RT, Moran D, Massof RW. Measuring outcomes of vision rehabilitation with the Veterans Affairs Low Vision Visual Functioning Questionnaire. Invest Ophthalmol Vis Sci 2006;47:3253-61.
18. Massof RW, Fletcher DC. Evaluation of the NEI visual functioning questionnaire as an interval measure of visual ability in low vision. Vision Res 2001;41:397-413.
19. Stelmack J, Szlyk JP, Stelmack T, Babcock-Parziale J, Demers-Turco P, Williams RT, Massof RW. Use of Rasch person-item map in exploratory data analysis: a clinical perspective. J Rehabil Res Dev 2004;41:233-41.
20. Babcock-Parziale J, McKnight PE, Head DN. Evaluating psychometric properties of a clinical and a self-report blind rehabilitation outcome measure. J Rehabil Res Dev 2005;42:487-98.
21. Massof RW. The measurement of vision disability. Optom Vis Sci 2002;79:516-52.
22. Massof RW, Rubin GS. Visual function assessment questionnaires. Surv Ophthalmol 2001;45:531-48.
23. Pesudovs K. Patient-centred measurement in ophthalmology-a paradigm shift. BMC Ophthalmol 2006;6:25.
24. Court H, Greenland K, Margrain TH. Content development of the optometric patient anxiety scale. Optom Vis Sci 2007;84:729-37.
25. Pesudovs K, Burr JM, Harley C, Elliott DB. The development, assessment, and selection of questionnaires. Optom Vis Sci 2007;84:663-74.
26. de Boer MR, Moll AC, de Vet HC, Terwee CB, Volker-Dieben HJ, van Rens GH. Psychometric properties of vision-related quality of life questionnaires: a systematic review. Ophthal Physiol Opt 2004;24:257-73.
27. Raasch TW, Leat SJ, Kleinstein RN, Bullimore MA, Cutter GR. Evaluating the value of low-vision services. J Am Optom Assoc 1997;68:287-95.
28. Smeeth L, Fletcher AE, Hanciles S, Evans J, Wormald R. Screening older people for impaired vision in primary care: cluster randomised trial. BMJ 2003;327:1027-30A.
29. Mangione CM, Berry S, Spritzer K, Janz NK, Klein R, Owsley C, Lee PP. Identifying the content area for the 51-item National Eye Institute Visual Function Questionnaire: results from focus groups with visually impaired persons. Arch Ophthalmol 1998;116:227-33.
30. Mangione CM, Lee PP, Gutierrez PR, Spritzer K, Berry S, Hays RD. Development of the 25-item National Eye Institute Visual Function Questionnaire. Arch Ophthalmol 2001;119:1050-8.
31. Stelmack JA, Szlyk JP, Stelmack TR, Demers-Turco P, Williams RT, Moran D, Massof RW. Psychometric properties of the Veterans Affairs Low-Vision Visual Functioning Questionnaire. Invest Ophthalmol Vis Sci 2004;45:3919-28.
32. Weih LM, Hassell JB, Keeffe J. Assessment of the impact of vision impairment. Invest Ophthalmol Vis Sci 2002;43:927-35.
33. Mangione CM. The National Eye Institute Visual Functioning Questionnaire-25 (NEI VFQ-25) Scoring Algorithm: Version 2000. Santa Monica, CA: RAND; 2000.
34. Linacre JM. WINSTEPS Rasch Measurement [Computer Program]. Chicago:
Winsteps.com; 2005.
35. Pesudovs K, Garamendi E, Keeves JP, Elliott DB. The Activities of Daily Vision Scale for cataract surgery outcomes: re-evaluating validity with Rasch analysis. Invest Ophthalmol Vis Sci 2003;44:2892-9.
36. Bond TG, Fox CM. Applying The Rasch Model: Fundamental Measurement in the Human Sciences. Mahwah, NJ: L. Erlbaum; 2001.
37. Pesudovs K, Garamendi E, Elliott DB. The quality of life impact of refractive correction (QIRC) questionnaire: development and validation. Optom Vis Sci 2004;81:769-77.
38. Massof RW, Hsu CT, Baker FH, Barnett GD, Park WL, Deremeik JT, Rainey C, Epstein C. Visual disability variables. I: the importance and difficulty of activity goals for a sample of low-vision patients. Arch Phys Med Rehabil 2005;86:946-53.
39. Massof RW, Hsu CT, Baker FH, Barnett GD, Park WL, Deremeik JT, Rainey C, Epstein C. Visual disability variables. II: the difficulty of tasks for a sample of low-vision patients. Arch Phys Med Rehabil 2005;86:954-67.
40. La Grow SJ. Predicting perceived quality of life scores from the National Eye Institute 25-item Visual Function Questionnaire. Optom Vis Sci 2007;84:785-8.
41. Mallinson T, Stelmack J, Velozo C. A comparison of the separation ratio and coefficient alpha in the creation of minimum item sets. Med Care 2004;42:I17-I24.
42. Little RJA, Rubin DB. The analysis of social-science data with missing values. Sociol Methods Res 1989;18:292-326.