In an aging population, there is no more relevant chronic cardiovascular health problem than heart failure (HF), which is estimated to currently affect 23 million individuals worldwide and will continue to be a major public health issue into the future.1 Associated with HF, cognitive impairment is closely entwined with poor clinical outcomes, including engagement in self-care, medication adherence and increased mortality.2–5 The HF-cognition paradigm is a contentious issue with reported prevalence rates from 25%6 up to 80%2,7 of HF patients, warranting greater research endeavors to inform clinical practice and tailoring management plans to individual need. Studies involving HF patients residing independently in their community demonstrate that the cognitive impairment reported may constitute mild cognitive impairment (MCI). Referred to as the predementia phase, MCI is associated with accelerated rate of cognitive demise,8 with 10% to 15% individuals developing dementia and Alzheimer’s disease annually.9 In HF patients, the added effects of even MCI are substantial, increasing mortality, morbidity, and the risk of developing more severe cognitive impairment that warrants early detection.2,4
Cognitive screening as part of routine clinical assessment may identify HF patients vulnerable to poor health outcomes enabling disease management strategies to be appropriately tailored to individual needs.10 Attesting to the significance of cognitive impairment in HF, 7 systematic reviews of the literature have been conducted6,7,11–15 focusing on the prevalence, type and severity of cognitive impairment. These reviews did not examine the diagnostic test accuracy of cognitive screening measures applied in HF studies. Consequently, there is a gap in the knowledge base about the diagnostic accuracy of cognitive screening measures applied in HF research with ramifications for the clinical setting. Furthermore, there is no consensus on screening methods to identify cognitive impairment in HF, and different measures have been applied in studies resulting in disparate findings.16,17 For example, when both the Montreal Cognitive Assessment (MoCA) and Mini Mental State Examination (MMSE) have been administered, 73% of HF patients were classified as MCI, scoring below thresholds on both measures. However, in screening for MCI, the level of agreement between the MMSE and MoCA was low.18 This raises the question of how to best screen for and measure cognitive impairment in HF. This systematic review attempts to address this question by determining the diagnostic test accuracy of cognitive screening instruments in the detection of MCI among HF patients.
Cognitive functioning can be considered along a continuum with, at one end, individuals performing normally for their age and, at the other, progressive cognitive demise associated with dementia.19 Evidence of neurocognitive impairment, as determined by a cognitive screening instrument, is insufficient for a formal diagnosis but warrants the need for further assessment. A formal clinical examination is required to diagnose a neurocognitive disorder (including MCI, dementia, and delirium). The diagnostic criteria for neurocognitive impairment are met if there is a measurable decline in 1 or more cognitive domains.20 Cognitive decline is based on (1) a concern about the individual’s cognitive abilities and (2) performance on a battery of neuropsychological tests that is equal to or greater than 1.5 Standard Deviation (SD) less than the age and education standardized means.21 There is growing awareness that cognitive impairment, which is abnormal for aging, does not always constitute the severe end of the spectrum such as Alzheimer’s disease. Indeed, significant attention has been directed to the ramifications of delirium and MCI. Delirium is considered an acute, neuropsychological syndrome resulting in impaired general cognition and attention, which must be ruled out in order to diagnose dementia.22 In contrast, the consensus criteria for MCI are as follows: a recent change in cognition, impairment in 1 or more cognitive domains that is abnormal for the patient’s age and educational level, ability to remain functionally independent, and not demented.23 Therefore, MCI is mutually exclusive to dementia and encompasses cognitive impairment that does not interfere with instrumental activities of daily living.24 In recent times, screening instruments have been specifically developed to identify MCI such as the MoCA25 and the Saint Louis University Mental Status Examination (SLUMS),26 but these have yet to be psychometrically evaluated in HF populations with clear scoring thresholds to delineate between MCI and dementia in this population.
Therefore, the aim of this systematic review was to examine the diagnostic accuracy of cognitive screening instruments when applied in HF. Addressing this issue may help formulate evidence-based recommendations for cognitive screening methods relevant to HF research and clinical practice.27
The objective of this article is to synthesize from a systematic review of the literature evidence for the diagnostic accuracy of cognitive screening instruments for detecting MCI in HF patients.
The following electronic bibliographic databases were searched from January 1999 to June 2013: MEDLINE, EMBASE, The Cochrane Library, CINAHL, psychARTICLES, and PsycINFO. Terms used in the search strategy included “heart failure,” “mild cognitive impairment,” “cognitive impairment,” “cognitive dysfunction,” “cognitive deficit,” “cognitive disorder,” “memory disorder,” “neuropsychological,” “neurocognitive battery,” “neurocognitive test,” and “screening.” Database limitations were age 18 years or older, published as full studies in English, full text of original research, and published after 1999. Bibliographies of relevant studies were manually scanned to identify additional studies. The World Health Organisation International Clinical Trials Registry Platform was searched using the following terms: “heart failure” and “cognitive impairment.” Further details of the search strategies can be obtained from the authors.
The inclusion criteria were as follows:
- full-text original research publications containing primary data that examined cognitive functioning in adults (aged ≥18 years) with HF; and
- cognitive impairment defined as a decline in 1 or more cognitive domains that is greater than expected with normal aging and identified by
- (a) administration of a cognitive screening instrument in its entirety,
- (b) administration of a neuropsychological test battery that assessed at least 4 cognitive domains (also used as the reference standard), and28
- (c) an “interindividual” approach to determine abnormal cognitive function; that is, neuropsychological test scores were characterized as a performance of equal to or greater than 1.5 SDs less than that of population normative data or an appropriate comparison group.29
Studies were excluded if dementia or delirium was the primary target condition under investigation and if a cognitive screening instrument was used to rule out dementia, so that cognitive function was assessed only in HF patients scoring greater than a predetermined cutoff score.
Two investigators (J.C., C.E.K.) independently assessed titles and abstracts of publications identified by the searches and the methodological quality of the studies and independently extracted data using a data extraction form designed and pilot tested by the review authors.
The full-text articles identified as potentially eligible for inclusion were retrieved and assessed for inclusion using an eligibility proforma. If a publication did not contain sufficient information for a decision to be made about its eligibility, further information was sought from the study investigators. The 2 review authors worked independently to identify which publications met the inclusion criteria.
Assessment of Methodological Quality
Methodological quality of the studies was assessed using a combination of 2 bias risk instruments: the QUality Assessment of Diagnostic Accuracy Studies (QUADAS-2) instrument30 and the STAndards for the Reporting of Diagnostic accuracy studies (STARD).31 While both instruments assess potential for bias in the accuracy and comprehensiveness of reporting studies of diagnostic accuracy, the use of STARD and QUADAS-2 combined is considered to provide greater rigor when evaluating the quality of studies.32 The QUADAS-2 was developed to critique the methodological rigor of a study and consists of 14 items assessed as “yes,” “no,” or “unclear,” which refer to internal validity.30 The STARD checklist has 25 items, regarding quality of the study design (eg, participant recruitment, data collection). Moreover, “the flexibility of both instruments allows them to be adapted to the purpose of each study.”31(p5) Disagreements about publication eligibility or in the assessment of methodological quality were resolved by discussion or by consulting a third author (C.F.S.).
Following data extraction, 2 attempts were made to contact the principal researchers of eligible studies requesting data to populate a 2 × 2 contingency table and calculate sensitivity and specificity of the screening instruments used. Four MCI categories were used to characterize cognitive function: (1) amnestic single cognitive domain, (2) amnestic multiple cognitive domains, (3) nonamnestic single cognitive domain, or (4) nonamnestic multiple cognitive domains.33 Sensitivity (proportion of HF patients characterized as MCI who screened positive for it), specificity (proportion of HF patients characterized as cognitively normal who were screened as such), positive predictive values (the proportion of HF patients with a positive MMSE screening who were characterized as MCI), and negative predictive values (the proportion of HF patients with a negative MMSE screening who were characterized as non-MCI) were generated and presented in a 2 × 2 contingency table. These data were generated from the following Web site: http://www.medcalc.org/calc/diagnostic_test.php. The 95% confidence interval (CI) for each variable was included in the generated 2 × 2 contingency table. A receiver operating characteristic curve was computed using IBM SPSS statistics 21, to determine the sensitivity, specificity, and appropriate cutoff score for MCI for the screening instrument. The receiver operating characteristic curve was constructed with impairment on any 1 cognitive domain on the neuropsychological test battery as the positive test.
Of the 593 references retrieved from the search, 557 were removed (115 duplicates, 442 failed to meet initial inclusion criteria), 36 abstracts were retrieved as full-text articles, and 8 were eligible for inclusion (Figure 1).
Reasons for study exclusion are illustrated in Figure 1. In the 8 included publications,34–41 the cognitive screening questionnaire had been administered as a component of the comprehensive assessment of cognitive function.
In 5 of the 8 included publications,36–40 the MMSE42 was administered as an overall assessment of global cognitive functioning. Three publications34,35,41 had used the Modified Mini Mental State Examination (3MS),43 which incorporates 4 additional items to the MMSE questionnaire and has a revised scoring format ranging from 0 to 100, instead of the original 0 to 30. There was a lack of consistency in the choice of neuropsychological battery of tests; however, the cognitive domains assessed included attention, memory (working, immediate and delayed), language, executive functioning, motor speed, and visuospatial and processing abilities (Table 1). Between 7.7%34 and 70%36 of HF participants exhibited cognitive impairment in at least 1 cognitive domain.
The results of bias and applicability evaluation using the STARD and QUADAS-2 criteria are presented in Figures 2A to C. None of the 8 publications examined the diagnostic test accuracy of the cognitive screen, indicating a high risk of quality bias (Figure 2A). All publications adequately presented the sample demographics, and the ages of HF patients ranged from 53 years36 to 69 years.34 All of the publications included relatively stable HF outpatients recruited from specialty HF clinics and cardiac rehabilitation programs. Seven studies used cross-sectional data, and none reported on longitudinal cognitive changes. Most studies administered the cognitive screening instrument at the same interview as the neuropsychological battery. However, it was unclear as to whether the cognitive screening had been interpreted without knowledge of the reference standard results, representing a potential risk of bias (Figure 2B).
Each publication assessed a select sample of HF patients specific to their research question. As a result of differing research questions, there was disparity between the studies with respect to the severity and age of the HF patients selected, for example, only HF patients with New York Heart Association classes II and III, patients awaiting cardiac transplant, or patients 50 years or older. This suggests there is a high risk of bias across the publications (Figure 2C).32
One of the 8 researchers provided data that enabled calculation of the sensitivity and specificity of the MMSE and for the positive predictive and negative predictive values for MCI to be computed. The MMSE had been administered to a select sample of 249 HF patients37,38; however, for the accuracy analysis, MMSE scores for 3 patients were missing and could not be included in the analysis. The demographics of the HF group are presented in Table 2.
According to the study investigators,37,38 cognitive impairment on the neuropsychological tests had been defined from healthy individuals’ Z scores using the seventh percentile. Patients performing at or below the seventh percentile were regarded as cognitively impaired in that domain. Only 53 HF patients (22%) had no cognitive impairment and were correctly characterized as such by the MMSE (true negative). Just over half the HF patients (57%) were potentially incorrectly characterized by MMSE as being cognitively intact (false negative; Table 3).
In this sample of HF patients, the sensitivity of the MMSE in screening for MCI was 26% (95% CI, 20%–33%), and the specificity was 95% (95% CI, 85%–98%). The positive predictive value of the MMSE in screening for MCI was good (94%), and the negative predictive value was low (27%; Table 4).
When the positive state was “MCI,” the area under the curve was 0.69 (asymptomatic 95% CI, lower = 0.62, upper = 0.76, P < .01) (Figure 3). The coordinates of the receiver operating characteristic curve (Figure 3) showed that an MMSE score of 28 or less provided the optimal cutoff (sensitivity = 62% and 1-specificity = 30%) in screening for MCI.
Despite the widespread prevalence of cognitive impairment in HF, our findings indicate that cognitive screening instruments have not been evaluated for their diagnostic test accuracy in detecting MCI in this specific population. In 5 of the 8 studies included in this review, the MMSE was the chosen cognitive screen,36–40 and the 3MS had been administered in the other 3 publications.34,35,41 Both screening questionnaires have established validity in screening for dementia in the general and aging populations44,45; however, their accuracy has not been established in HF populations.
This systematic review and diagnostic test accuracy analysis, utilizing data from 1 study, indicates the low sensitivity and high specificity of the MMSE had better properties for ruling in, rather than ruling out MCI.46 This finding is clinically important as it indicates that the MMSE is helpful in identifying MCI when it is present. In this manner, further in-depth cognitive assessment can be arranged, and disease management and surveillance strategies can be more appropriately targeted to individual need. However, findings also suggest that in general HF populations potentially three-quarters of individuals with MCI will be missed. Several studies involving aging populations have also demonstrated limitations of the MMSE in identifying MCI because of its low sensitivity.24,47–49
In cognitive screening of elderly veterans in a primary care setting, Donnelly and colleagues49 compared the diagnostic accuracy of 4 MCI screening measures: the MMSE, the Clock Drawing Test, the Hopkins Verbal Learning Test, and Trail Making A and B Test. Similar to our study findings, the authors reported the sensitivity of the MMSE was low (20%; 95% CI, 6–44), and the specificity was high (93%; 95% CI, 84–97).49 Of the 4 screening measures examined, the Trail Making B Test produced the best predictive validity results across the 8 diagnostic indicators assessed.49 Furthermore, in a study of 91 nondemented community-residing participants with cerebrovascular disease, the MMSE only achieved a sensitivity greater than 70% when a cutoff score was used less than 29.48 In contrast, both the MoCA and Addenbrooke’s Cognitive Examination–Revised had superior sensitivity and specificity than the MMSE in screening for amnestic MCI. The authors noted that optimal cutoff score for cognitive instruments will depend on their intended use: high sensitivity for screening or high specificity for diagnosis.48
Notwithstanding this, the SLUMS and MoCA have been developed to overcome the shortcomings of the MMSE in screening for MCI. The questionnaires focus less on language items and include assessment of executive functioning, which is considered a cardinal feature of MCI.25,26 This systematic review of the literature had identified 9 studies that administered the MoCA, and 3 had implemented the SLUMS. However, none of these met the study inclusion criteria because of either the absence of a neuropsychological battery or they had not met the operationalized definition for cognitive impairment. Despite the development of screening measures that are more sensitive to the assessment of MCI, only studies that had used the MMSE or 3MS met the inclusion criteria for this review. This is somewhat surprising considering the body of evidence for screening questionnaires, which are considered to be more sensitive to MCI, and draws attention to the need for future studies to examine the accuracy of cognitive screening measures in HF populations.
The cognitive deficits exhibited by HF patients often constitute the mild spectrum of impairment and therefore unless specifically assessed are often not detected by clinicians.2,10 In a sample of 251 veteran outpatients, 58% were identified as having MCI that would otherwise not have been identified by clinicians. Furthermore, MCI was a significant predictor of medication adherence.3 Therefore, even though mild in nature, these deficits render patients vulnerable to adverse health outcomes, including poor medication adherence and a blunted response to recognize and respond to deleterious changes in HF symptoms, which increase burden and healthcare resource utilization.3,4 The adverse health outcomes of cognitive impairment among patients diagnosed with HF underscore the importance for accurate cognitive screening instruments for detecting in this population this significant comorbidity. Therefore, it is imperative for future studies to consider examining the accuracy of cognitive screening measures in HF populations. In doing so, consensus and evidence-based guidelines can be developed with recommendations on how we screen for, characterize, and manage cognitive impairment in HF populations.
As with all systematic reviews, there were strict inclusion criteria, especially regarding the manner in which MCI had been operationalized. We had used published criteria21 whereby cognitive performance is standardized against an appropriate comparison group and impairment was operationalized as falling 1.5 SDs less than an appropriate comparison. On this basis, 19 published studies were excluded. As such, we were able to examine in only 1 select HF sample the diagnostic test accuracy of the MMSE and were unable to examine pooled estimates of its accuracy from the other 7 publications. Therefore, the sensitivity, specificity, and positive and negative predictive values calculated will be influenced by the selected sample and may not be generalizable to other settings with different patient profiles. As such, the specificity and positive predictive value we report for the MMSE may be somewhat inflated as the frequency of HF patients with impaired cognitive abilities may be higher than what would otherwise be found in the general population.46 Future studies are needed that examine the diagnostic test accuracy of cognitive screening instruments in broader HF samples with a good representation of women and a broad range of cognitive abilities. As memory loss is common in HF, there is a need to conduct prospective studies that focus on determining the best cognitive screening instrument in detecting both mild and major neurocognitive impairment.
This systematic review found 8 publications, representing 5 studies that met the strict inclusion criteria. The diagnostic test accuracy of the MMSE in screening for MCI was shown to have low sensitivity (26%) and high specificity (95%) suggesting that the MMSE is more useful in ruling in, rather than ruling out MCI, among HF patients. Issues of sample bias in all the included studies illustrate that a more diverse sample of HF patients is needed in future test assessment studies. To close the “evidence gap,” future studies need to consider examining the test accuracy of cognitive screening instruments administered to broader HF samples. This would enable definitive recommendations to be made regarding optimal methods in screening HF patients for cognitive impairment. Evidence is also needed to confirm whether screening for cognitive impairment can also improve outcomes for these vulnerable patients.
What’s New and Important?
- This article highlights a dearth in test accuracy studies examining optimal screening practices for MCI in HF patients.
- To close the “evidence gap,” future studies need to consider examining the test accuracy of cognitive screening instruments for MCI administered to broader HF samples.
- There is a need to develop consensus on how to characterize, assess and manage MCI in HF patients. This evidence would then be able to be translated into contemporary HF management programs.
The authors acknowledge the input of Ms Jane Reid in initial discussions about the conduct of the systematic review. They are grateful for the funding support from the Australian Government’s Collaborative Research Networks program.
1. Bui AL, Horwich TB, Fonarow GC. Epidemiology and risk profile of heart failure
. Nat Rev Cardiol
2. Cameron J, Worrall-Carter L, Page K, Riegel B, Lo SK, Stewart S. Does cognitive impairment predict poor self-care in patients with chronic heart failure
? Eur J Heart Fail
3. Hawkins LA, Kilian S, Firek A, Kashner TM, Firek CJ, Silvet H. Cognitive impairment and medication adherence in outpatients with heart failure
. Heart Lung
4. Lee CS, Gelow JM, Bidwell JT, et al. Blunted responses to heart failure
symptoms in adults with mild cognitive dysfunction. J Cardiovasc Nurs
5. McLennan SN, Pearson SA, Cameron J, Stewart S. Prognostic importance of cognitive impairment in chronic heart failure
patients: does specialist management make a difference? Eur J Heart Fail
6. Pressler SJ. Cognitive functioning and chronic heart failure
: a review
of the literature (2002–July 2007). J Cardiovasc Nurs
7. Bennett SJ, Sauve MJ. Cognitive deficits in patients with heart failure
. A review
of the literature. J Cardiovasc Nurs
8. Stephan BC, Matthews FE, McKeith IG, Bond J, Brayne C. Early cognitive change in the general population: how do different definitions work? J Am Geriatr Soc
9. Grundman M, Petersen RC, Ferris SH, et al. Mild cognitive impairment
can be distinguished from Alzheimer disease and normal aging for clinical trials. Arch Neurol
10. Cameron J, Ski CF, Thompson DR. Cognitive impairment in chronic heart failure
and the need for screening
. Am J Cardiol
11. Vogels RL, Scheltens P, Schroeder-Tanka JM, Weinstein HC. Cognitive impairment in heart failure
: a systematic review
of the literature. Eur J Heart Fail
12. Almeida OP, Flicker L. The mind of a failing heart: a systematic review
of the association between congestive heart failure
and cognitive functioning. Intern Med J
13. Jurgens CY, Faulkner KM, Lee CS. Phenotypic profiling of cognitive impairment risk among patients with heart failure
: a literature review
of the usefulness of cardiac-related variables. Eur J Cardiovasc Nurs
14. Davis KK, Allen JK. Identifying cognitive impairment in heart failure
: a review
measures. Heart Lung
15. Currie K, Rideout A, Lindsay G, Harkness K. The association between mild cognitive impairment
and self-care in adults with chronic heart failure
: a systematic review
and narrative synthesis. J Cardiovasc Nurs
. 2014. [Epub ahead of print]
16. Cameron J, Pressler SJ, Ski CF, Thompson DR. Cognitive impairment in heart failure
: towards a consensus on screening
. Eur J Heart Fail
17. Bauer LC, Johnson JK, Pozehl BJ. Cognition in heart failure
: an overview of the concepts and their measures. J Am Acad Nurse Pract
18. Cameron J, Worrall-Carter L, Page K, Stewart S, Ski CF. Screening
for mild cognitive impairment
in patients with heart failure
: Montreal Cognitive Assessment versus Mini Mental State Exam. Eur J Cardiovasc Nurs
19. Petersen RC. Mild cognitive impairment
as a diagnostic entity. J Intern Med
20. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders
. 5th ed. Arlington, VA: American Psychiatric Association; 2013.
21. Gorelick PB, Scuteri A, Black SE, et al. Vascular contributions to cognitive impairment and dementia. Stroke
22. Inouye SK, Westendorp RGJ, Saczynski JS. Delirium in elderly people. Lancet
23. Albert MS, DeKosky ST, Dickson D, et al. The diagnosis of mild cognitive impairment
due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement
24. Lin JS, O’Connor E, Rossom RC, et al. Screening for Cognitive Impairment in Older Adults: An Evidence Update for the U.S. Preventive Services Task Force
. Rockville, MD: Agency for Healthcare Research and Quality (US); 2013.
25. Nasreddine ZS, Phillips NA, Bedirian V, et al. The Montreal Cognitive Assessment, MoCA: a brief screening
tool for mild cognitive impairment
. J Am Geriatr Soc
26. Tariq SH, Tumosa N, Chibnall JT, Perry MH III, Morley JE. Comparison of the Saint Louis University Mental Status Examination and the Mini-Mental State Examination for detecting dementia and mild neurocognitive disorder—a pilot study. Am J Geriatr Psychiatry
27. Holly C. Systematic reviews: toward best practice in patient care. Heart Lung
28. Hachinski V, Iadecola C, Petersen RC, et al. National Institute of Neurological Disorders and Stroke–Canadian Stroke Network vascular cognitive impairment harmonization standards. Stroke
29. Nelson AP, O’Connor MG. Mild cognitive impairment
: a neuropsychological perspective. CNS Spectr
30. Whiting PF, Rutjes AWS, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med
31. Bossuyt PM, Reitsma JB, Bruns DE, et al. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med
32. Oliveira MR, Gomes Ade AC, Toscano CM. QUADAS and STARD: evaluating the quality of diagnostic accuracy studies. Rev Saude Publica
33. Petersen RC, Negash S. Mild cognitive impairment
: an overview. CNS Spectr
34. Alosco ML, Spitznagel MB, Raz N, et al. The 2-minute step test is independently associated with cognitive function in older adults with heart failure
. Aging Clin Exp Res
35. Alosco ML, Spitznagel MB, van Dulmen M, et al. The additive effects of type-2 diabetes on cognitive function in older adults with heart failure
. Cardiol Res Pract
36. Mapelli D, Bardi L, Mojoli M, et al. Neuropsychological profile in a large group of heart transplant candidates. PLoS One
37. Pressler SJ, Subramanian U, Kareken D, et al. Cognitive deficits and health-related quality of life in chronic heart failure
. J Cardiovasc Nurs
38. Pressler SJ, Subramanian U, Kareken D, et al. Cognitive deficits in chronic heart failure
. Nurs Res
39. Steinberg G, Lossnitzer N, Schellberg D, et al. Peak oxygen uptake and left ventricular ejection fraction, but not depressive symptoms, are associated with cognitive impairment in patients with chronic heart failure
. Int J Gen Med
. 2011;4: 879–887.
40. Vogels RL, Oosterman JM, van Harten B, et al. Profile of cognitive impairment in chronic heart failure
. J Am Geriatr Soc
41. Alosco ML, Spitznagel MB, Raz N, et al. Dietary habits moderate the association between heart failure
and cognitive impairment. J Nutr Gerontol Geriatr
42. Folstein MF, Folstein SE, McHugh PR. “Mini-Mental State”: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res
43. Teng EL, Chui HC. The Modified Mini-Mental State (MMS) examination. J Clin Psychiatry
44. Bravo G, Hebert R. Age- and education-specific reference values for the Mini-Mental and Modified Mini-Mental State Examinations derived from a non-demented elderly population. Int J Geriatr Psychiatry
45. Tombaugh TN, McDowell I, Kristjansson B, Hubley AM. Mini-Mental State Examination (MMSE) and the Modified MMSE (3MS): a psychometric comparison and normative data. Psychol Assess
46. Peat J, Barton B, Elliott E. Statistics Workbook for Evidence-based Health Care
. Chichester: Wiley-Blackwell; 2009.
47. Mitchell AJ. A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment
. J Psychiatr Res
48. Pendlebury ST, Mariz J, Bull L, Mehta Z, Rothwell PM. MoCA, ACE-R, and MMSE versus the National Institute of Neurological Disorders and Stroke–Canadian Stroke Network vascular cognitive impairment harmonization standards neuropsychological battery after TIA and stroke. Stroke
49. Donnelly K, Donnelly JP, Cory E. Primary care screening
for cognitive impairment in elderly veterans. Am J Alzheimers Dis Other Demen