Are the measures of functional performance that are used within populations of adults with dementia psychometrically sound?
More specifically, the objective is to:
Review the reliability, validity and responsiveness of common measures of functional performance that are used in exercise interventions for adults with dementia.
In 2010, it was estimated that globally over 35 million people were living with a diagnosis of dementia, equating to 21% of the population aged 60 years or older, with this figure projected to rise to over 115 million people by 2050.1 In the same year, nearly 7.7 million new cases of dementia were diagnosed, an average of one every four seconds, leading to an estimated financial impact of $604 billion, and over 1% of Gross Domestic Profit (GDP) worldwide.1 A source of major concern within the Australian health and aged care system, dementia is receiving much attention from the government, researchers and health workers alike. In 2011, it was estimated that almost 300,000 Australians were living with a diagnosis of dementia, accounting for almost 1.5% of the total population and approximately 9% of older adults (>65years of age). In 2011 alone, 63,300 people were diagnosed with dementia, equating to nearly seven people every hour, with dementia having a direct cost to the Australian Government of $2 billion annually.2 In the next two decades, dementia will be become the third highest source of health and residential aged care spending, equating to almost 1% of GDP.3 In the next 30 years, the number of Australian adults with dementia is expected to rise to 900,000.2 Consequently, considerable strain on the health and aged care sectors is to be expected.
For the individual, dementia is a significant health and wellbeing burden, as well as a disruption to the lives of friends and family, with the government estimating that over 1 million Australians will be providing unpaid and informal care to adults with dementia.1 Compounding this is the significant loss in cognitive and physical function, behavioral changes such as aggressiveness, agitation, anxiousness, paranoia and wandering common to people with dementia.4 Furthermore, on average people with dementia live with five or more comorbid diseases, including depression, with prevalence rates of 50% in aged care facilities.2 Adding these factors together, living with dementia can prevent individuals from enjoying a quality of life that they were once used to.
Exercise may be part of the solution. It has the potential to improve, or at least attenuate, a decline in physical, cognitive, behavioral and psychological impairments often associated with dementia.5–7 However, confidence in conclusions can only be drawn from measurement tools that have sound psychometric properties for the population with which they are used. A recent systematic review highlighted this issue, that for older adults with cognitive impairment, 10 out of 11 included studies did not report the validity and reliability of chosen measures, or protocols were unclear.5 More recently, the systematic review by Fox et al.8 reported that within a dementia specific population, measures of functional ability were poorly justified, with researchers often citing validity or reliability studies from dissimilar clinical populations, such as those having bronchitis or healthy older adults without dementia. This led the authors to conclude that functional performance measures were poorly justified for dementia specific populations. While some measurement tools, such as the Timed Up and Go (TUG), have undergone a rudimentary assessment for reliability in dementia specific populations, seemingly more comprehensive and rigorous psychometric evaluation is absent. In addition, results from pilot work by our group showed poor reliability and measurement error of two commonly employed measures of functional performance in a sample of adults with dementia residing in permanent care.9,10 A recent search of two databases, including PROSPERO, identified no systematic reviews of this exact nature. A similar systematic review has been completed by Sikkes et al.11 However, that review was limited to activities of daily living scales, not quantitative measurement of functional performance, which is the objective of the current review. Several systematic reviews of measurement properties have been completed on measures of functional performance,12–16 but none to date are dementia specific. This current systematic review extends upon previous work and covers a significant research gap in order to review what psychometric justification is available for measures of functional performance specific to adults with dementia.
Types of participants
Participants will be adults, aged 65 years and older, with a confirmed medical diagnosis of dementia. While no specific aetiology of disease will be targeted, any study that provides such details will be reported on. Studies that include dementia as only part of the sample will be included in this review, but will be differentiated from dementia specific studies.
Types of intervention(s)/phenomena of interest
The following measures have been identified as common measures used within exercise-based interventions for older adults with dementia from a recent JBI systematic review.8
General physical function
Performance Orientated Mobility Assessment (POMA)17
Senior Fitness Test18
Short Physical Performance Battery (SPPB)19,20
1RM Leg Press21
Standing from Floor23
BERG Balance Test24
1 Leg Balance25
Gait parameters of Stride Length, Cadence and Double Limb Support Time27
Timed Up and Go (TUG)29,30
Sit and Reach18
Back Scratch Test18
These are the main outcome measures that are of interest to this review because they are commonly employed and presented on in dementia and exercise intervention literature.
Types of outcomes
Studies that examined the reliability, validity and sensitivity/responsiveness of measurement tools will be included in this review. The COSMIN study31 aimed to reach a consensus on the taxonomy and terminology of reliability, validity and responsiveness. This review will use the COSMIN taxonomy to define the outcomes of interest. While the COSMIN study is clear in its definitions, the Trinitarian approach to validity, as presented by COSMIN,31 is largely outdated in psychometrics. The pre-eminence of construct validity is more widely accepted, as modern interpretations of validity envelope the three aforementioned concepts of validity under the single concept of construct validity.32 It is important to note this distinction so as to identify how individual studies semantically define the psychometric property of interest and whether a classical or modern interpretation is preferred.
Types of studies
Desired studies will be observational and cross-sectional in nature, examining one or more of the psychometric properties of pre-defined outcome measures of functional performance. Papers which use included measures as a criterion to establish concurrent validity for an undefined measurement tool will not be included in this review.
The search strategy aims to find both published and unpublished studies. A three-step search strategy will be utilized in this review. An initial limited search of PUBMED, will be undertaken, followed by an analysis of the text words contained in the title and abstract, and of the index terms used to describe the article. This process was guided by work already completed by COSMIN to develop sensitive and precise search filters in PUBMED,33 to identify key words and index terms. A second search using all identified keywords and index terms will then be undertaken across all included databases. Thirdly, the reference list of all identified articles will be searched for additional studies.
Due to resource implications, non-English studies will not be included. No date range has been included for this review, to ensure capturing any study looking to validate the measures. The databases to be searched include:
MEDLINE, EMBASE, CINAHL and ISI Web of Science
The search for unpublished, grey literature studies will include:
ALOIS (Cochrane Dementia and Cognitive Impairment Group), Google Scholar and, ProQuest (Dissertations and Theses Databases),
Initial keywords to be used will include:
#1 construct search
“physical per*” OR “physical func*” OR “func* abil*” OR “gait” OR “strength” OR “balance” OR “geriatric assessment” OR “func*”
#2 population search
“dementia” OR “demented” OR “alzhiemer's disease” OR “cog* func*” OR “cog* impair*”
#3 instrument search
“performance orientated mobility assessment” OR “short physical performance battery” OR “senior fitness test” OR “wal*” OR “GAITrite” OR “Bessou” OR “stair*” OR “timed up and go”
#4 measurement search
“reproducib*” OR “methods” OR “valid*” OR “reliab*” OR “sensitive*” OR “psychometrics” OR “precision”
The search strategy has been developed to capture the validity, reliability and responsiveness of pre-identified measures, in a population of adults (aged 65 years and older) with a confirmed diagnosis of dementia.
Finally, pre-identified measures not found by the described search strategy will be searched for in original protocol or validation papers. This will include the protocol papers cited by this protocol. The use of the “citation map” function from ISI Web of Science, which creates a two generation map of all “cited” and “citing” articles from the chosen publication will be used to find literature in dementia specific populations.
There is potential that the measurement properties of some tools have not been assessed within a dementia specific population and this will be reported on.
Assessment of methodological quality
Assessment of the methodological quality will be undertaken through the COSMIN checklist (Appendix I).34 This is a four-point scoring system developed by COSMIN specifically for use in systematic reviews and will be completed by two reviewers independently. Any disagreement between reviewers will be conferred by a third independent reviewer. Interested parties should read Terwee34 for more information regarding the development of the checklist for use in the health sciences. The advantage of this checklist is the inclusion of psychometric assessment from an Item Response Theory approach.
The statistical output from reliability, validity and responsiveness to change will be collected. This includes, but is not limited to, Intra-Class Correlations (ICC), Kappa, Cronbach's Alpha, Chi Squared, Standard Error of Measurement (SEM), Minimal Detectable Change (MDC), Limits of Agreement (LoA), and factor analysis. Extraction will be completed by two reviewers independently. Reviewers will confer afterwards and a third reviewer will be consulted upon in case of any disagreement. Attempts will be made to contact original authors in the case of missing data. Standardized Joanna Briggs Institute data collection forms will be used to extract data from included studies (Appendix II). The data extracted will include specific details about populations, study design, study methods and statistics,
Data synthesis will be in the form of a narrative review of measurement properties of the tools used to measure functional performance. Quantitative meta-analysis using pooled, weighted averages will be conducted where possible, using Microsoft Excel 2013. However, the type, quantity and quality of data may limit such analysis. In order to overcome this, authors of included studies will be contacted to provide original data sets for analysis. A consumer table of ratings of quality of evidence from included studies for all types of outcomes will also be presented.35 An example of such is presented in Appendix III, which is taken from Schellingerhout et al.36
Conflicts of interest
The authors have no conflict of interest to declare.
The authors would like to acknowledge Alzheimer's Australia Dementia Research Foundation for financial support for the Chief Investigator, in the form of a PhD scholarship.
1. World Health Organisation. Dementia: A public Health Priority. Geneva: World Health Organisation, 2012.
2. Australian Institute of Health and Welfare. Dementia in Australia. Canberra: AIHW, 2012.
3. Alzheimer's Australia. Keeping Dementia front of mind: incidence and prevalance 2009-20502009.
4. Hope T, Keene J, Fairburn C, McShane R, Jacoby R. Behaviour changes in dementia 2: Are there behavioural syndromes? Int J Geriatr Psychiatry. 1997;12(11):1074-8.
5. Hauer K, Becker C, Lindemann U, Beyer N. Effectiveness of physical training on motor performance and fall prevention in cognitively impaired older persons: a systematic review. Am J Phys Med Rehab. 2006;85(10):847-57.
6. Heyn P, Abreu BC, Ottenbacher KJ. The effects of exercise training on elderly persons with cognitive impairment and dementia: A meta-analysis. Arch Phys Med Rehab. 2004;85(10):1694-704.
7. Heyn PC, Johnson KE, Kramer AF. Endurance and strength training outcomes on cognitively impaired and cognitively intact older adults: A meta-analysis. Journal of Nutrition Health & Aging. 2008;12(6):401-9.
8. Fox B, Hodgkinson B, Parker D. The effects of physical exercise on older adults, aged 65yrs and older, with a diagnosis of dementia: a systematic review protocol. The JBI Database of Systematic Reviews and Implementation Reports. 2013;11(7):414-21.
9. Fox B, Henwood T, Neville C. Reliability of functional performance in older people with dementia. Australas J Ageing. 2013;32(4):248-9.
10. Fox B, Henwood TR, Neville C, Keogh J. Relative and absolute reliability of functional performance measures for adults with dementia living in residential aged care. Int Psychogeriatr. 2014:1-9.
11. Sikkes SAM, de Lange-de Klerk ESM, Pijnenburg YAL, Scheltens P, Uitdehaag BMJ. A systematic review of Instrumental Activities of Daily Living scales in dementia: room for improvement. J Neurol Neurosurg Psychiatry. 2009;80(1):7-12.
12. Bartels B, de Groot JF, Terwee CB. The Six-Minute Walk Test in Chronic Pediatric Conditions: A Systematic Review of Measurement Properties. Phys Ther. 2013;93(4):529-41.
13. Kroman SL, Roos EM, Bennell KL, Hinman RS, Dobson F. Measurement properties of performance-based outcome measures to assess physical function in young and middle-aged people known to be at high risk of hip and/or knee osteoarthritis: a systematic review. Osteoarthritis Cartilage. 2014;22(1):26-39.
14. Silva PFS, Quintino LF, Franco J, Faria C. Measurement properties and feasibility of clinical tests to assess sit-to-stand/stand-to-sit tasks in subjects with neurological disease: a systematic review. Brazilian Journal of Physical Therapy. 2014;18(2):99-110.
15. Terwee CB, Mokkink LB, Steultjens MPM, Dekker J. Performance-based methods for measuring the physical function of patients with osteoarthritis of the hip or knee: a systematic review of measurement properties. Rheumatology. 2006;45(7):890-902.
16. Voshaar M, ten Klooster PM, Taal E, van de Laar M. Measurement properties of physical function scales validated for use in patients with rheumatoid arthritis: A systematic review of the literature. Health Qual Life Outcomes. 2011;9.
17. Tinetti ME. Performance-Oriented Assessment
of Mobility Problems in Elderly Patients. J Am Geriatr Soc. 1986;34(2):119-26.
18. Rikli RC, Jones J. Senior Fitness Test Manual - 2nd Edition. Champaign, IL: Human Kinetics; 2012.
19. Thomas VS, Hageman PA. A preliminary study on the reliability of physical performance measures in older day-care center clients with Dementia. Int Psychogeriatr. 2002;14(1):17-23.
20. Guralnik JM, Simonsick EM, Ferrucci L, Glynn RJ, Berkman LF, Blazer DG, et al. A short physical performance battery assessing lower extremity function: association with self-reported disability and prediction of mortality and nursing home admission. J Gerontol. 1994;49(2):M85-94.
21. American College of Sports Medicine. ACSM's Guidelines for Exercise Testing and Prescription: Eighth Edition. Philadelphia, PA: Lippincott Williams & Wilkins; 2010. 380 p.
22. Roberts HC, Denison HJ, Martin HJ, Patel HP, Syddall H, Cooper C, et al. A review of the measurement of grip strength in clinical and epidemiological studies: towards a standardised approach. Age Ageing. 2011;40(4):423-9.
23. Andreotti RA, Oukuma SS. Validação de uma bateria de testes de atividades da vida diária para idosos fisicamente independentes. Revista Paulista de Educação Fisica. 1999;13(1):46-66.
24. Berg K, Wood-Dauphinee S, Williams JI, Gayton D. Measuring balance in the elderly: preliminary development of an instrument. Physiother Can. 1989;41(6):304-11.
25. Vellas BJ, Rubenstein LZ, Ousset PJ, Faisant C, Kostek V, Nourhashemi F, et al. One-leg standing balance and functional status in a population of 512 community-living elderly persons. Aging. 1997;9(1-2):95-8.
26. McGavin CR, Gupta SP, McHardy GJ. Twelve-minute walking test for assessing disability in chronic bronchitis. Br Med J. 1976;1(6013):822-3.
27. Bessou P, Dupui P, Montoya R, Pages B. Simultaneous Recording of Longitudinal Displacements of Both Feet during Human Walking. J Physiol (Paris). 1988;83(2):102-10.
28. van Iersal MB, Benraad CE, Olde Rikkert MGM. Validity and reliability of quantitative gait analysis in geriatric patients with and without dementia. J Am Geriatr Soc. 2007;55(4):632-4.
29. Large J, Gan N, Basic D, Jennings N. Using the Timed Up and Go Test to stratify elderly inpatients at risk of falls. Clin Rehabil. 2006;20(5):421-8.
30. Podsiadlo D, Richardson S. The timed “up & go”: a test of basic functional mobility for frail elderly persons. Journal of the American Geriatric Society. 1991;39(2):142-8.
31. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010;63(7):737-45.
32. Rowe DA, Mahar MT. Validity. In: Wood TM, Zhu W, editors. Measurement Theory and Practice in Kinesiology. 1. Champaign, IL: Human Kinetics; 2006.
33. Terwee C, Jansma E, Riphagen I, Vet HW. Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments. Qual Life Res. 2009;18(8):1115-23.
34. Terwee C, Mokkink L, Knol D, Ostelo RJG, Bouter L, Vet HW. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012;21(4):651-7.
35. Terwee CB, Bot SDM, de Boer MR, van der Windt DAWM, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34-42.
36. Schellingerhout J, Verhagen A, Heymans M, Koes B, Vet H, Terwee C. Measurement properties of disease-specific questionnaires in patients with neck pain: a systematic review. Qual Life Res. 2012;21(4):659-70.
Appendix I: COSMIN checklist
The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments
The COSMIN checklist (COnsensus-based Standards for the selection of health status Measurement INstruments) is provided below in its published format (www.cosmin.nl). For the purpose of this review the COSMIN checklist with the four-point scale will be used to assess the methodological quality of the studies. The COSMIN checklist will be used as prescribed (see COSMIN Checklist Manual; www.cosmin.nl), with the option of minor modifications in the process as required by the types of studies retrieved for review.
The COSMIN checklist was developed for evaluating measurement instruments of health related patient reported outcomes (HR-PROs). The use of the COSMIN checklist in this review is valid because the criteria for what constitutes good measurement properties for HR-PROs are equivalent for collaboration measurement instruments.
Appendix II: Data extraction instruments
MAStARI data extraction instrument
Appendix III: Example of a consumer table of evidence36
(reproduced from Schellingerhout et al.36 under the terms of Creative Commons Attribution Non- commercial License)