Secondary Logo

Reliability, validity and relevance of needs assessment instruments for informal dementia caregivers: a psychometric systematic review protocol

Kipfer, Stephanie1,4; Eicher, Manuela2; Oulevey Bachmann, Annie2,3,4; Pihet, Sandrine1

JBI Database of Systematic Reviews and Implementation Reports: February 2018 - Volume 16 - Issue 2 - p 269–286
doi: 10.11124/JBISRIR-2017-003367
SYSTEMATIC REVIEW PROTOCOLS
Free

Review question/objective: The objective of this psychometric review is to identify needs assessment instruments for informal dementia caregivers which are:

  1. Reliable and valid in measuring the needs of informal dementia caregivers
  2. Relevant for clinical practice, research and informal caregivers

More specifically, the aim is to present an overview and an evaluation of the available needs assessment instruments, including: i) their psychometrics (reliability and validity) when available, and ii) their relevance according to the instrument characteristics, namely, their purpose, application method, administration burden, number of items and domain structure.

1School of Health Sciences Fribourg, University of Applied Sciences and Arts Western Switzerland, Fribourg, Switzerland

2Faculty of Biology and Medicine, Institute of Higher Education and Research in Healthcare (IFS), University of Lausanne, Lausanne, Switzerland

3School of Health Sciences Lausanne, University of Applied Sciences and Arts Western Switzerland, Lausanne, Switzerland

4Bureau d’Echange des Savoirs pour des pratiques exemplaires de soins (BEST): a Joanna Briggs Institute Centre of Excellence, Lausanne, Switzerland

Correspondence: Stephanie Kipfer, stephanie.kipfer@hefr.ch

There is no conflict of interest in this project.

Back to Top | Article Outline

Introduction

Dementia is characterized by a progressive decline of cognitive and social functions. This limits the autonomy of those affected by dementia and makes it difficult for them to cope with daily life. They become increasingly dependent on the care of others, particularly, informal caregivers. Informal caregivers are individuals who regularly provide unpaid care, assistance and/or supervision to a close person with reduced autonomy.1 In the EUROFAMCARE study the informal caregivers reported memory problems in 46% of all caring situations and some behavioral problems in 34% of the care recipients.2 Caring for someone with behavioral problems was stated as most burdensome for the informal caregivers. Studies show that informal dementia caregivers spend more time providing care over a longer time period than caregivers of people without dementia. The caregiving time increases even more if the person with dementia shows behavioral symptoms.3 In the United States (US), for example, 31.1% of informal dementia caregivers provide care for two to three years, 18.5% for four to five years and 38.4% for six or more years.4 In a survey conducted by Alzheimer Europe almost half of the informal dementia caregivers spent more than 10 hours per day providing care.5 This is comparable to data from the US, where informal dementia caregivers spend on average nine hours per day caregiving.6 In 2015 the estimated economic value of unpaid care provided by informal dementia caregivers in the US was $221.3 billion.4 In Switzerland, as an example of a Western European country, informal caregivers contributed 64 million of unpaid hours, which saved the Swiss health care system 3.55 billion CHF, a 22% increase since 2010.7 Informal caregivers contribution is expected to further increase due to the rising care needs of an aging population and the growing prevalence of (multiple) chronic conditions, in particular, dementia.7 In addition the number of available formal carers is not expected to increase in line with the rising care needs of the aging population.8 This shows that supporting informal caregivers and recognizing their valuable contribution is a significant public health issue. Informal caregivers play a key role not only for the people with dementia but also for society by promoting the sustainability of the health care system.9-11

Caring for a person with dementia is a challenging experience and the burden on informal dementia caregivers is higher in comparison to informal caregivers of persons with other chronic conditions.12,13 Straining continuous care, an unpredictable course and neuropsychiatric symptoms of the person with dementia can cause high levels of stress which often lead to physical, psychological, emotional, social and financial problems.3,13-15 In addition, family caregivers often have no experiences in performing care, feel unprepared and are lacking the required knowledge and support from health care providers to deliver appropriate care.4,16 Informal caregivers report feelings of tiredness, stress, helplessness, and loneliness, and show a high prevalence of depression and anxiety.3 Due to the nature of dementia, informal caregivers also struggle with feelings of guilt, ambivalence, grief and loss. Identified physical problems can create an increased risk for vascular disease, impaired wound healing, decreased immunity, and reduced likelihood to engage in preventive health behavior.3,17 Poor physical and psychological health conditions not only impair the quality of life of informal dementia caregivers, but also affect the ability to provide care to the person with dementia, and to sustain the own social support network, which leads to social isolation.3,17-19 Burden and health deterioration of informal dementia caregivers are core predictors of early institutionalization and mistreatment of their care recipient.17,20

Due to the challenges of caregiving and the associated burden, informal dementia caregivers report significant unmet needs at all stages of the disease. Their needs cover very diverse areas, such as information about the illness and support resources, support for own emotional concerns, support to communicate with the care recipient, the family or the service providers, practical support in daily care and respite, or financial support.3,21-25

Informal dementia caregivers often report that health care providers do not attend and adapt to their multiple needs sufficiently, resulting in care fragmentation and poor coordination, ultimately increasing stress and underutilization of support services despite the needs.26-28 Underutilization of health care and other support resources contributes to the exhaustion of the informal dementia caregivers and precipitated institutionalization of their care recipient, and thereby increases health care costs.20,29,30 Informal caregivers do not always spontaneously express how their needs can be met.31 Therefore the evaluation of their needs by a professional is crucial in order that their needs can be met in a person centered way, the quality of life of the caregiver and the person affected can be enhanced, and the caring situation at home can be maintained.

Most studies on informal dementia caregivers needs have used qualitative study designs. Existing quantitative questionnaires have limitations: very few items for caregivers,2,32 poor validation,21,22,33,34 or lack of empirical evidence about need dimensions (factor structure). This limits their use in both research and clinical practice. In addition many of the assessment instruments, such as semi-structured interviews, are time-intensive (e.g. assessment alone lasts on average two hours25; or 90 minutes22). Furthermore, most of the collected information in interviews is qualitative, as such it is usually extensive, and more time is needed to prepare the information to make it available for the caregiver or other service providers (e.g. transcriptions). In view of the growing economic pressure on the social system and rising support needs associated with population aging, such resources are impossible to manage on a large scale.

A preliminary search in MEDLINE, CINAHL, JBI Database of Systematic Reviews and Implementation Reports and the Cochrane Database of Systematic Reviews was performed to identify completed and in progress systematic reviews on needs assessment instruments for informal dementia caregivers. To our knowledge there is no review which evaluates and compares the different needs assessment instruments for informal dementia caregivers according to their reliability, validity and relevance for clinical practice, research and informal caregivers. This information would be invaluable for researchers and clinicians to help them make informed decisions about the best instruments available for their specific purpose and clarify whether a new comprehensive needs assessment instrument is required.

Back to Top | Article Outline

Inclusion criteria

The COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) guidelines for systematic reviews of measurement properties38 recommend the following inclusion criteria: i) the instrument should aim to measure the construct of interest (informal dementia caregiver needs), ii) the study sample should concern the target population of interest (informal dementia caregivers), iii) the study should concern the type of measurement instrument of interest (self-reported or professionally interviewed), iv) the aim of the study should be the development of a measurement instrument or the evaluation of one or more of its measurement properties (see types of studies). The development of the inclusion criteria for this review was guided by the COSMIN guidelines and the JBI template.

Back to Top | Article Outline

Participants

This review will consider studies that include informal caregivers of persons with dementia living at home. Informal caregivers are individuals who regularly provide unpaid care, assistance and/or supervision to a close person with reduced autonomy, in this context with dementia.1

Back to Top | Article Outline

Intervention

This review will consider studies that report on needs assessment instruments for assessing the needs of informal dementia caregivers. Needs can be defined as “a condition that is important to the subject and that is not being satisfied in the subject's present environment”.35(p.772) The application method of the instruments can either be self-reported or professionally interviewed. For example, studies about the Carers’ Needs Assessment for Dementia (CAN-D), the Care Needs Assessment Pack for Dementia (CARENAPD) or the John Hopkins Dementia Care Needs Assessment (JHDCNA).

Back to Top | Article Outline

Outcomes

This review will consider studies that include the following outcomes:

Psychometric properties:

  • Reliability (test-retest reliability, inter-rater reliability, internal consistency)
  • Validity (content validity, construct validity, structural validity, sensitivity to change, criterion-related validity).

Instrument characteristics:

  • Purpose (original intended use)
  • Application method (self-reported, professionally interviewed)
  • Administration burden (training for clinicians, time for completion and evaluation for the clinicians and the informal dementia caregivers)
  • Number of items and domain structure.

As not every article will provide data for all of these outcomes, articles which report at least on one outcome regarding reliability or validity will be considered. Criterion-related validity will only be considered if there is a reasonable gold-standard available (in accordance with COSMIN guidelines38).

Back to Top | Article Outline

Types of studies

This psychometric review will consider psychometric studies namely instrument development or instrument evaluation studies. Other types of studies (in which only needs assessment instruments are used) will be considered to identify eligible instruments and their responsible authors. Responsible authors will then be contacted in search for unpublished psychometric studies or testing of the instrument.

Back to Top | Article Outline

Methods

Search strategy

The search strategy aims to find both published and unpublished studies. A three-step search strategy will be utilized in this review. An initial limited search of MEDLINE and CINAHL will be undertaken followed by an analysis of the text words contained in the title and abstract, and of the index terms used to describe article. A second search using all identified keywords and index terms will then be undertaken across all included databases. Thirdly, the reference list of all identified reports and articles will be searched for additional studies. A complementary search will be performed in the included databases using the names of the needs assessment instruments identified in the three foregoing steps. Studies published in English, German and French will be considered for inclusion in this review. There will be no limitation regarding the publication time.

Back to Top | Article Outline

Information sources

The databases to be searched include: MEDLINE, OVID Nursing and PsycINFO via OVID, and CINAHL.

The search for unpublished studies will include: Google Scholar, ProQuest Dissertations and Theses, Researchgate (contact with relevant researchers), homepages with information about needs assessment/outcome tools and homepages of dementia associations or organizations.

We will also identify relevant researchers during the literature research and contact them by email to obtain information about unpublished psychometric studies or instrument testing of relevant instruments. The number of articles retrieved through this approach and the process itself will be described in the section “Search strategy” of the review report.

Initial keywords to be used will be: dementia; informal caregivers; caregiver needs; needs assessment; needs evaluation; instrument; interview; assessment; questionnaire; validation; psychometrics; reliability; validity.

Back to Top | Article Outline

Assessment of methodological quality

Papers selected for retrieval will be assessed by two independent reviewers for methodological validity prior to inclusion in the review. To assess the methodological quality the COSMIN checklist will be used (Appendix I). The COSMIN checklist is a standardized tool which is recommended to use in systematic reviews of measurement properties.36 This tool meets the specific needs of a psychometric review and has been suggested before in another Johanna Briggs Institute review protocol.37 It will therefore be preferable over the standardized critical appraisal instruments from the Joanna Briggs Institute. The checklist consists of 12 different boxes with five to 18 items per box. Seven of these boxes consider psychometric properties, namely, internal consistency, reliability, measurement error, content validity, structural validity, criterion-related validity and responsiveness. The general methodological quality will be assessed with the box, hypotheses testing. There is also one box for additional methodological standards for studies using item response theory models, one for studies investigating interpretability and one for cross-cultural validity. These three boxes will not be used in this review as they focus on more advanced properties. The remaining box concerning generalizability is recommended to use for data extraction. The COSMIN checklist is a modular tool. The selection of boxes to be used is determined by the measurement properties evaluated in the reviewed study. The checklist contains a four-point response option to evaluate the different items: excellent, good, fair and poor. However some items have only two or three response options. A methodological quality score per box can be obtained by taking the lowest rating of any item in the box (“worst score count”). The lowest score of any box presents the overall score of the reviewed study. Studies with poor scores in all boxes will be excluded from the review. Any disagreements that arise between the reviewers will be resolved through discussion, or with a third reviewer.

Back to Top | Article Outline

Data extraction

Data will be extracted from papers included in the review using a standardized data extraction tool developed for this review (Appendix II). This tool is inspired by the different elements of the standardized data extraction tools from Joanna Briggs Institute System for the Unified Management, Assessment and Review of Information (JBI SUMARI)and adapted to the specific elements of a psychometric review. The data extracted will include specific details about: i) the study characteristics, ii) the instrument characteristics, and iii) the outcomes of significance to the review question and specific objectives.

Study characteristics: citation details, aim of the study, study design and method, setting, population characteristics, definition of informal caregivers and needs.

Instrument characteristics: name of the instrument, purpose, target population, application method, respondent and/or administrative burden, number of items and domain structure, range of scores, response options/format.

Psychometric outcomes: reliability (test-retest reliability, inter-rater reliability, internal consistency), validity (content validity, construct validity, structural validity, criterion-related validity, sensitivity to change).

As mentioned above, the box concerning generalizability of the COSMIN checklist will also be used. Data will be extracted independently by two reviewers. Disagreement between the reviewers will be resolved by discussion, or with a third reviewer. If necessary, the responsible researchers of an instrument will be contacted for missing or additional data.

Back to Top | Article Outline

Data synthesis

The main aim of the data synthesis is to compare outcomes to provide recommendations on the most suitable instrument for research, clinical use and informal caregivers. The findings about reliability, validity and the instrument characteristics will be compared and presented in narrative form including tables and figures to aid data presentation. A content comparison will give an overview of the content of each instrument and the similarities and differences on an item level.38 However, if we can identify substantially more than one published or unpublished psychometric study per instrument, summary measures will be computed for this instrument, for example, the average Cronbach's alpha with its 95% confidence interval. The information gathered in the generalization box can be used to assess the similarity of the participants in these studies.38 To judge the psychometric outcomes of the different instruments, the quality criteria from Terwee et al. will be used.39 These criteria will allow us to judge: content validity, internal consistency, criterion-related validity, construct validity, test-retest reliability (agreement), inter-rater reliability (reliability) and sensitivity to change (responsiveness) in terms of positive rating, indeterminate rating, negative rating, no information available and doubtful design or method (Appendix III). Floor and ceiling effects and interpretability will not be rated as there is no gold-standard. Only the most common and basic properties will be evaluated in this review. For structural validity a quality criteria will be developed if necessary. The results of this appraisal will be presented in a narrative form.

Back to Top | Article Outline

Acknowledgements

We thank Prof. Dr. Dawn Carnes (School of Health Sciences Fribourg, University of Applied Sciences and Arts Western Switzerland) for comments that greatly improved the protocol.

Back to Top | Article Outline

Appendix I: Appraisal instrument

Adapted from the COSMIN checklist40

Figure

Figure

NB. Item 1 is used to determine whether internal consistency is relevant for the instrument under study. It is not used to rate the quality of the study

Figure

Figure

Figure

Figure

Figure

Figure

Figure

Figure

Figure

Figure

Figure

Figure

Figure

Figure

Figure

Figure

Back to Top | Article Outline

Appendix II: Data extraction instrument

Figure

Figure

Figure

Figure

Figure

Figure

Box Generalizability40,41

Figure

Figure

Figure

Figure

Back to Top | Article Outline

Appendix III: Quality criteria for measurement properties of questionnaires

Adapted version, for complete version see Terwee et al.39

Figure

Figure

Figure

Figure

Back to Top | Article Outline

References

1. Swiss Confederation. Support for family caregivers: An analysis of the situation and the required measures in Switzerland [Soutien aux proches aidants: Analyse de la situation et mesures requises pour la Suisse]. Bern: Federal Council; 2014.
2. EUROFAMCARE. EUROFAMCARE - Services for Supporting Family Carers of Older Dependent People in Europe: Characteristics, Coverage and Usage. Brussels: EUROFAMCARE Consortium; 2006.
3. Thompson GN, Roger K. Understanding the needs of family caregivers of older adults dying with dementia. Palliat Support Care 2014; 12 3:223–231.
4. Alzheimer's Association. 2016 Alzheimer's Disease Facts and Figures: Caregiving. Alzheimer's & Dementia 2016; 12 4:31–34.
5. Alzheimer Europe. Who cares? The state of dementia care in Europe. Luxembourg: Alzheimer Europe; 2006.
6. Fisher GG, Franks MM, Plassman BL, Brown SL, Potter GG, Llewellyn D, et al. Caring for individuals with dementia and cognitive impairment, not dementia: findings from the aging, demographics, and memory study. J Am Geriatr Soc 2011; 59 3:488–494.
7. De Pietro C, Camenzind P, Sturny I, Crivelli L, Edwards-Garavoglia S, Spranger A, et al. Switzerland - Health system review. Brussels: European Observatory on Health Systems and Policies; 2015.
8. Zwaanswijk M, Peeters JM, van Beek AP, Meerveld JH, Francke AL. Informal caregivers of people with dementia: problems, needs and support in the initial stage and in subsequent stages of dementia: a questionnaire survey. Open Nurs J 2013; 7:6–13.
9. Kesselring A. Caring for family members at home [Angehörige zu Hause pflegen: Anatomie einer Arbeit]. Swiss Medical Journal [Schweizerische Ärztezeitung] 215 2004; 85 10:504–506.
10. Rosa E, Lussignoli G, Sabbatini F, Chiappa A, Di Cesare S, Lamanna L, et al. Needs of caregivers of the patients with dementia. Arch Gerontol Geriatr 2010; 51 1:54–58.
11. Kraft E, Marti M, Werner S, Sommer H. Cost of dementia in Switzerland. Swiss Med Wkly 2010; 140:w13093.
12. Butcher HK, Holkup PA, Buckwalter KC. The experience of caring for a family member with Alzheimer's disease. West J Nursing Res 2001; 23 1:33–55.
13. de la Cuesta C. The craft of care: family care of relatives with advanced dementia. Qual Health Res 2005; 15 7:881–896.
14. Vellone E, Sansoni J, Cohen MZ. The experience of Italians caring for family members with Alzheimer's disease. J Nurs Scholarsh 2002; 34 4:323–329.
15. Knapp M, Prince M, Albanese E, Banerjee S, Dhanasiri S, Fernandez J-L, et al. Dementia UK. London: Alzheimer's Society; 2007.
16. Reinhard SC, Given B, Petlick NH, Bemis A. Hughes RG. Supporting family caregivers in providing care. Patient safety and quality: An evidence-based handbook for nurses. Rockville (MD): Agency for Healthcare Research and Quality (US); 2008. 341–404.
17. Schulz R, Martire LM. Family caregiving of persons with dementia: prevalence, health effects, and support strategies. Am J Geriatr psychiatry 2004; 12 3:240–249.
18. Perrig-Chiello P, Höpflinger F, Schnegg B. SwissAgeCare: Nursing relatives of older people in the Switzerland. Bern: Universität Bern; 2010.
19. Brodaty H, Donkin M. Family caregivers of people with dementia. Dialogues Clin Neurosci 2009; 11 2:217–228.
20. Gaugler JE, Kane RL, Kane RA, Clay T, Newcomer R. Caregiving and institutionalization of cognitively impaired older people: utilizing dynamic predictors of change. Gerontologist 2003; 43 2:219–229.
21. Black BS, Johnston D, Rabins PV, Morrison A, Lyketsos C, Samus QM. Unmet needs of community-residing persons with dementia and their informal caregivers: findings from the maximizing independence at home study. J Am Geriatr Soc 2013; 61 12:2087–2095.
22. Ducharme F, Kergoat MJ, Coulombe R, Levesque L, Antoine P, Pasquier F. Unmet support needs of early-onset dementia family caregivers: a mixed-design study. BMC Nurs 2014; 13 1:49.
23. Bass DM, Judge KS, Snow AL, Wilson NL, Morgan R, Looman WJ, et al. Caregiver outcomes of partners in dementia care: effect of a care coordination program for veterans with dementia and their family members and friends. J Am Geriatr Soc 2013; 61 8:1377–1386.
24. Afram B, Verbeek H, Bleijlevens MH, Hamers JP. Needs of informal caregivers during transition from home towards institutional care in dementia: a systematic review of qualitative studies. Int psychogeriatr 2015; 27 6:891–902.
25. Freudiger Pittet S, Jordan A. Evaluation de la charge et des besoins des proches aidants. Lausanne: Association vaudoise d’aide et de soins à domicile (AVASD); 2012.
26. Brodaty H, Thomson C, Thompson C, Fine M. Why caregivers of people with dementia and memory loss don’t use services. Int J Geriatr Psychiatry 2005; 20 6:537–546.
27. Bass DM, Judge KS, Snow AL, Wilson NL, Looman WJ, McCarthy C, et al. Negative caregiving effects among caregivers of veterans with dementia. Am J geriatr psychiatry 2012; 20 3:239–247.
28. Karlsson S, Bleijlevens M, Roe B, Saks K, Martin MS, Stephan A, et al. Dementia care in European countries, from the perspective of people with dementia and their caregivers. J Adv Nurs 2015; 71 6:1405–1416.
29. Gaugler JE, Kane RL, Kane RA, Newcomer R. Unmet care needs and key outcomes in dementia. J Am Geriatr Soc 2005; 53 12:2098–2105.
30. Gaugler JE, Kane RL, Kane RA, Newcomer R. Early community-based service utilization and its effects on institutionalization in dementia caregiving. Gerontologist 2005; 45 2:177–185.
31. van der Roest HG, Meiland FJ, Maroccini R, Comijs HC, Jonker C, Droes RM. Subjective needs of people with dementia: a review of the literature. Int psychogeriatr 2007; 19 3:559–592.
32. Reynolds T, Thornicroft G, Abas M, Woods B, Hoe J, Leese M, et al. Camberwell Assessment of Need for the Elderly (CANE): Development, validity and reliability. BJPsych 2000; 176 5:444–452.
33. Wancata J, Krautgartner M, Berner J, Alexandrowicz R, Unger A, Kaiser G, et al. The Carers’ Needs Assessment for Dementia (CNA-D): development, validity and reliability. Int psychogeriatr 2005; 17 3:393–406.
34. McWalter G, Toner H, McWalter A, Eastwood J, Marshall M, Turvey T. A community needs assessment: the care needs assessment pack for dementia (CarenapD)--its development, reliability and validity. Int J Geriatr Psychiatry 1998; 13 1:16–22.
35. Hileman JW, Lackey NR, Hassanein RS. Identifying the needs of home caregivers of patients with cancer. Oncol nurs forum 1992; 19 5:771–777.
36. Terwee CB, Mokkink LB, Knol DL, Ostelo RWJG, Bouter LM, de Vet HCW. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res 2012; 21 4:651–657.
37. Simpelaere I, White A, Bekkering GE, Geurden B, Van Nuffelen G, De Bodt M. Patient-reported and proxy-reported outcome measures for the assessment of health-related quality of life among patients receiving enteral feeding: a systematic review protocol. JBI database of systematic reviews and implementation reports 2016; 14 7:45–75.
38. Terwee CB. Protocol for systematic reviews of measurement properties. Amsterdam: VU University Medical Center Knowledgecenter Measurement Instruments; 2011.
39. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007; 60 1:34–42.
40. Terwee CB. COSMIN checklist with 4-point scale. Amsterdam: University Medical Center, Department of Epidemiology and Biostatics, EMGO Institute for Health and Care Research; 2011.
41. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. COSMIN checklist manual. Amsterdam: University Medical Center, Department of Epidemiology and Biostatics, EMGO Institute for Health and Care Research; 2012.
Keywords:

dementia; informal caregivers; instrument; needs assessment; psychometrics

© 2018 by Lippincott williams & Wilkins, Inc.