Several studies have reported high prevalence of cognitive impairment in HIV+ patients, with the highest estimates exceeding 50%.1,2 However, a large proportion of individuals scoring below specified thresholds on cognitive testing and meeting the criteria for HIV-associated neurocognitive disorder (HAND) have no interference with everyday functioning and are categorized as having asymptomatic neurocognitive impairment (ANI) by the Frascati criteria.3 Others have used participant self-report methods such as the modified Lawton & Brody Instrumental Activities of Daily Living Scale (IADLS) to discriminate mild neurocognitive disorder (MND) from ANI.2,4–6 Deterioration, if defined as a change from ANI to MND, is based solely on the assessment of difficulties with activities of daily living (ADL), therefore discerning this change is highly important. Functional decline and cognitive symptoms may also be the main driver of patients coming to clinicians' attention in clinical settings.
Some studies have suggested limited validity of patient self-report with commonly used questionnaires when used to predict objectively measured cognitive function.7–10 But even if symptoms are only weakly correlated with cognitive impairment, they are in some sense highly important to the patient. There are likely to be psychological and other factors underlying reported symptoms and understanding these factors may steer the clinician away from investigations such as brain imaging and lumbar puncture and toward the patients' more pertinent needs. In this analysis, we aimed to measure the strength of association between self-reported decline in ADL, self-reported symptoms of cognitive impairment, and objectively measured cognitive function. We then sought to determine which other factors were associated with these symptoms of cognitive difficulties.
The Cognitive Impairment in People with HIV in the European Region (CIPHER) study is a European study of cognitive function in HIV+ adult outpatients (2 clinics in London, United Kingdom; one clinic in each of Copenhagen, Denmark; Minsk, Belarus; Rome, Italy). We recently reported, along with detailed psychosocial and clinical cofactors, a 35% prevalence of cognitive impairment, of whom more than half met Frascati criteria for ANI and a quarter had confounding conditions, with the remaining 6% having MND or HAD.11 Participants were recruited opportunistically from May 2011 to January 2013 regardless of symptoms, treatment, and comorbidities. All questionnaires were self-completed in a written format by the participant in the local language, and participants had to be sufficiently cognitively intact to complete study procedures.
The study was approved by the UK National Research Ethics Service—London Hampstead Committee (11/LO/0077), De Videnskabsetiske Komiteér for Region Hovedstaden (H-4-2011-105), the Ethics Committee of the Minsk Municipal Infectious Diseases Hospital (04/2011), and the Comitato Etico of Istituto Nazionale Malattie Infettive (66/2011). All study participants gave written informed consent before commencing the study.
Assessment of Symptoms and Self-Reported Function
Declining function in ADLs was measured using the Modified Lawton & Brody IADLS, which required participants to indicate their current and, retrospectively, their “best ever” level of function on 16 activities, according to a 3- or 4-point scale (see Supplemental Digital Content 1, http://links.lww.com/QAI/B55). Where their current level was worse than their best ever level, participants recorded the perceived cause as “primarily cognitive problems,” “primarily physical problems,” or “equally cognitive and physical problems.” The questionnaire has been used in large published studies of HAND.2,12,13
Symptoms of cognitive impairment were elicited using 4 questions from the Medical Outcomes Study in HIV (MOS-HIV)14 (see Supplemental Digital Content 2, http://links.lww.com/QAI/B55). Participants were asked to indicate how much of the time in the past 4 weeks they had experienced difficulty reasoning and solving problems, forgetfulness, trouble keeping their attention, and difficulty in concentration. Possible responses were scored: 0, none of the time; 1, a little bit of the time; 2, some of the time; 3, a good bit of the time; 4, most of the time; 5, all of the time. Adding the 4 responses gave a total score of 0–20.
Participants completed the 9-item Patient Health Questionnaire (PHQ-9) for depression15 and the 7-item Generalized Anxiety Disorder (GAD-7) questionnaire for anxiety.16 These questionnaires elicit symptom frequency in the past 2 weeks: 0, not at all; 1, several days; 2, more than half the days; 3, nearly every day. PHQ-9 scores ranged from 0 to 27 and were graded: 0–4, none/minimal; 5–9, mild; 10–14, moderate; 15–19, moderate-severe; 20–27, severe. GAD-7 scores ranged from 0 to 21 and were graded: 0–4, none/minimal; 5–9, mild; 10–14, moderate; 15–21, severe.
The study used Cogstate, software adapted from standard neuropsychological (NP) tasks for assessment of multiple cognitive domains (http://cogstate.com/academic-2/measurement-of-cognition)17–19 and 2 pen-and-paper NP tests. Cogstate is mainly nonlanguage based, and the stimuli vary randomly between assessments, reducing the influence of cultural, educational, and practice effects. Several studies have used Cogstate in HIV+ patients in anglophone countries,20–26 with work demonstrating good construct validity and correlation with traditional pen-and-paper NP batteries,21–23 and the method has been used in children and adults in diverse settings in Africa and Asia27–29 (although not in all countries where CIPHER was based). Ten tests were organized into 5 domains: psychomotor speed (Detection and Identification); verbal memory (Shopping List learning and recall); executive function (Groton Maze learning and recall; these tests also assess visuospatial learning and memory); working memory (One-Back speed and accuracy); verbal fluency (2 pen-and-paper tests: Controlled Oral Word Association Test and Category Fluency Test). Testing included a full practice run (not scored) on the same day as the main assessment.30
Each cognitive test was converted to an age-, sex-, and education-standardized Z-score using normative mean values and SD. Normative data for the computerized tests were provided by Cogstate from Europe, United States of America, South East Asia, and Australia/New Zealand, frequency-matched for age (18–34, 35–50, and 51+ years; minimum cell size n = 145), sex (157 female, 377 male), and education (university/further education, n = 243, or secondary school, n = 291). The normative population excluded individuals with clinical or functional impediments to test performance, and subsets of these data had been used previously to measure cognitive function in HIV+ adults.20–22,24,26 Published norms were used for the Controlled Oral Word Association Test and Category Fluency Test.31,32 A continuous measure of neurocognitive function was calculated from the mean of all 5 neuropsychological domain Z scores (NPZ-5).
Other Data Collection
Drug and alcohol use, educational attainment, and socioeconomic information were obtained from self-report. Medical and psychiatric histories were obtained from both patient report and clinical notes. Clinical information included current HIV viral load, CD4 count, antiretroviral therapy history, HIV-related complications, laboratory markers, and neurological and psychiatric comorbidities. Hepatitis C virus (HCV) status was defined as the most recent HCV RNA assay or the most recent antibody result in the absence of an RNA result. Participants with any of a prespecified list of comorbid conditions (published as an appendix to the original criteria3)—including depressive disorders, historical traumatic brain injury, developmental disabilities, alcohol and substance use disorders, certain HIV-related and non–HIV-related neurologic conditions, and systemic disease likely to affect cognition—were noted. We adopted the nomenclature used by the CHARTER group, where contributing conditions were those with potential for mild cognitive impairment, whereas confounding conditions were those likely to have a substantial effect on cognitive function.2
The first outcome measure was defined, using the IADLS, as self-reported decline in 2 or more ADLs of the possible 16 (designated “functionally dependent” by Obermeit et al7) and attribution to cognitive problems by the participant. To harmonize our results with Obermeit and others, we included ADLs that had been omitted in our previous analysis,11 namely planning social activities, reading/watching television, and childcare. The associations of age, sex, ethnic group (white versus other ethnic groups), study site, educational attainment (as ordered categories of primary, secondary, and higher education), employment status (as a categorical variable of in employment, unable to work, or unemployed for other reasons), cognitive function (as a continuous variable by domain and as a mean score), clinical data, and psychosocial variables were estimated by odds ratios (ORs) and 95% confidence intervals (CIs). All cognitive function scores and factors found to be at least moderately associated with the outcome (at a threshold of P < 0.1) were included in separate logistic regression models with adjustment for demographic variables (age, sex, ethnic group, study site, and education). Education, site, and ethnicity have been identified as factors associated with cognitive test scores in an earlier analysis of the same study sample.11
The second outcome measure was symptom score from the MOS-HIV questionnaire in 5 ordered categories. The base category was defined as asymptomatic (0 points on the questionnaire) and subsequent categories corresponded to 4 quartiles of the remaining values. Ordered logistic regression was used to calculate ORs and CIs for each factor in the same way as the ADL decline outcome measure. The ordered logistic model assumes the same effect size for each unit increase in the outcome. Multivariable regression models were also constructed for factors with P < 0.1 in the bivariate analyses, adjusted for demographic variables, in the same way as the first outcome measure.
Finally, we assessed the possible application of the IADLS and MOS-HIV symptom questionnaires as diagnostic or screening tools for cognitive impairment. Two reference standards were used, defined as low performance in 2 or more cognitive domains to a level of Z ≤−1 (mild cognitive impairment) or Z ≤−2 (severe cognitive impairment). Receiver operator curves were generated from sensitivity and specificity statistics for all possible thresholds on each scale (number of ADLs of difficulty when attributed by the patient to cognitive problems and MOS-HIV cognitive symptom score). Test accuracy statistics [positive and negative likelihood ratios (LRs) and diagnostic odds ratios (DORs)] were calculated.
Of 486 patients recruited, 448 (92%) completed all cognitive assessments [London 292 (65%); Rome 90 (20%); Copenhagen 42 (9.4%); Minsk 24 (5.4%)], 15 participants only completed the practice assessment, and 23 completed fewer than 5 cognitive domains. The sample was 87% white, 84% male, mean 45.8 years of age (SD 9.6), 89% on antiretroviral therapy, and 81% with viral load <50 copies/mL. Participant characteristics are shown in Table 1 (breakdown by study site has already been published11).
Factors Associated With Self-Reported Decline in ADL
Self-reported functional decline in 2 or more ADLs was present in 138 participants (30.8%). This was attributed to cognitive problems, with or without physical problems, by 96 (69.6% of those reporting decline). The most frequent difficulties were socializing, work, and housekeeping, affecting 55%–72% of functionally impaired patients and 5%–15% of patients with fewer than 2 areas of difficulty (Fig. 1). Information on specific ADLs was not available for participants at the Italian site (n = 90), and 134 of the remainder (37.4%) left the childcare item blank or responded “I do not have children”; these individuals were assumed not to have declined in this activity.
Several factors were associated with self-reported decline in ADLs attributed to cognitive impairment (Table 2). In models adjusted for age, sex, ethnic group, education, and study site, a decline in ADLs was associated with speed/reaction time [adjusted odds ratio (aOR) 0.68 per Z +1, 95% CI: 0.55 to 0.83] and attention/working memory (aOR 0.57 per Z +1, 95% CI: 0.42 to 0.78). Other associated factors were as follows: ability to afford basic needs most of the time (aOR 2.71, 95% CI: 1.52 to 4.80) or some of the time (aOR 4.44, 95% CI: 2.01 to 9.81) compared with those who were always able to afford basic needs; being unable to work (aOR 3.99, 95% CI: 1.94 to 8.22) or unemployed for other reasons (aOR 2.21, 95% CI: 1.26 to 3.86) compared with those in employment; depressive symptoms (aOR 2.06 per grade on the PHQ-9 scale, 95% CI: 1.66 to 2.55); anxiety symptoms (aOR 2.20 per grade on the GAD-7 scale, 95% CI: 1.69 to 2.88); and HIV diagnosis 5–10 years ago (aOR 3.18, 95% CI: 1.42 to 7.13) or ≥10 years ago (aOR 3.62, 95% CI: 1.62 to 8.09).
To explore the interdependence of the effects of mood, cognition, and functional abilities, we analyzed whether adjusting for relevant cognitive domains would change the association between PHQ-9 grade and self-reported decline in ADLs. The effect after including the speed domain was an aOR of 1.95 (95% CI: 1.58 to 2.41), and the effect after including attention and working memory was an aOR of 1.96 (95% CI: 1.59 to 2.42), both being similar to the results of the simpler model. In addition, there was no evidence of an interaction between depression and functional decline in their effects on the speed domain (P = 0.37) or on the attention/working memory domain (P = 0.84).
Factors Associated With Self-Reported Symptoms of Cognitive Impairment
MOS-HIV scores were available for 440 participants, with a median score of 4 (IQR 1–8). Numbers in each of the 5 ordered categories were as follows: 86 asymptomatic (0 points, 19.6%); 115 with grade 1 (1–3 points, 26.1%); 70 with grade 2 (4–5 points, 15.9%); 85 with grade 3 (6–9 points, 19.3%); and 84 with grade 4 (10–20 points, 19.1%). In multivariable ordered logistic regression models adjusted for age, sex, ethnic group, education, and study site (Table 3), increasing grade of symptom frequency was associated with impairment of speed/reaction time (aOR 0.68 per +1 in Z score, 95% CI: 0.58 to 0.80), attention/working memory (aOR 0.60, 95% CI: 0.47 to 0.77), verbal memory (aOR 0.87, 95% CI: 0.77 to 0.98), and verbal fluency (aOR 0.84, 95% CI: 0.71 to 1.00). Other associated factors were as follows: being educated to only primary school level compared with completing higher education (aOR 1.89, 95% CI: 1.17 to 3.05); confounding (aOR 2.07, 95% CI: 1.31 to 3.27) or contributing medical conditions (aOR 2.02, 95% CI: 1.22 to 3.36); ability to afford basic needs most of the time (aOR 1.91, 95% CI: 1.25 to 2.90), some of the time (aOR 3.56, 95% CI: 1.86 to 6.82), or not at all (aOR 3.80, 95% CI: 1.85 to 7.81); being unable to work (aOR 4.68, 95% CI: 2.55 to 8.59) or unemployed for other reasons (aOR 2.11, 95% CI: 1.41 to 3.14) compared with those in employment; depressive symptoms (aOR 2.75 per grade on the PHQ-9, 95% CI: 2.28 to 3.32); anxiety symptoms (aOR 3.24 per grade on the GAD-7, 95% CI: 2.58 to 4.07); and ≥10 years since HIV diagnosis (aOR 1.82, 95% CI: 1.12 to 2.97).
Accuracy of the Questionnaires as Tests for Cognitive Impairment
After excluding participants with incomplete data, the IADLS had sensitivity 40%, specificity 78% when used as a test for severe cognitive impairment (prevalence 30/356, 8.4%) with a cutoff of decline in ≥2 ADLs (Fig. 2). There were 12 true positives, 253 true negatives, 73 false positives, and 18 false negatives equating to positive LR 1.79, negative LR 0.77, and DOR 2.31. The questionnaire had worse sensitivity and better specificity when used with a cutoff of a higher number of ADLs, and LRs and DORs were consistently poor (close to 1).
The most balanced trade-off between sensitivity and specificity on the MOS-HIV symptom scale was at a cutoff of ≥7 points, giving sensitivity 48%, specificity 69% as a test for severe cognitive impairment (prevalence 48/440, 10.9%). There were 23 true positives, 271 true negatives, 121 false positives, and 25 false negatives giving positive LR 1.55, negative LR 0.75, and DOR 2.06. The questionnaire had better accuracy overall at higher cutoffs at the expense of poor sensitivity (high false-negative rates). At lower cutoffs in the range of 2–5 points, sensitivity was improved at 50%–73%, but specificity was markedly down at 27%–57%, and both positive LR and DOR were consistently <2.
Both questionnaires were, in general, less sensitive and less specific when compared with a reference standard of mild cognitive impairment. We conducted exploratory analyses with exclusion of the most severely anxious or depressed participants, and this made no improvement to the diagnostic accuracy statistics for either outcome measure (IADLS or MOS-HIV score).
In this study of HIV+ outpatients at 5 clinics in Europe, 80% reported cognitive symptoms in the past 4 weeks, and just over 20% reported decline in their ADLs and attributed this to cognitive impairment. There were associations between poorer cognitive function (especially reaction time, speed, and attention) and both self-reported decline in ADLs and increasing frequency of cognitive symptoms. However, symptoms of depression and anxiety, longer time since HIV diagnosis, unemployment, and difficulty affording basic needs were also strongly associated with cognition-related symptoms and functional decline. In addition, lower educational attainment and comorbid medical conditions were associated with cognitive symptoms. All of our observed associations may have multiple explanations, and causality could be in either direction (particularly in the relationships between employment and financial means and the outcomes of interest). The accuracy of both questionnaires, modeled as diagnostic tests for cognitive impairment, was relatively poor and below a level likely to be acceptable for individual patient use.
These results imply that patients who report symptoms of cognitive impairment, or declining everyday function, should be assessed for depression, anxiety, concomitant medical conditions, and financial difficulties. Failure to recognize these important elements of patients' lived experiences risks diagnostic delay, failure to address important needs, unnecessary investigations, and further anxiety. This is similar to the conclusions of a large cohort study of HIV-negative older adults in Amsterdam: “when an older person complains about his or her memory and does not show actual cognitive impairment or cognitive decline, one should be aware that these complaints might reflect psycho-affective or health problems. Adequate intervention in these psycho-affective or health problems may improve quality of life.”33
The relationships between cognitive impairment, low mood, and functional decline are complex and multidirectional. Secondary analyses suggested that the effects of mood and cognition on everyday function were relatively independent, although we cannot exclude other explanations. In our recently published report of the same study population, we showed that severe depressive symptoms were associated with cognitive impairment, particularly on timed tests.11 A controlled intervention to improve patients' depression or anxiety symptoms might tease out the effect of low of mood on subjective and objective cognitive outcomes.
Limitations to CIPHER study methodology and Cogstate in general have been discussed in earlier reports,11,25 but some deserve further mention here. Notably, selection biases may have arisen as a result of our opportunistic sampling strategy, with patients choosing to participate in the study on the basis of their own concerns and self-perceived cognitive symptoms. Also, the largely computerized neuropsychological methods did not provide as comprehensive an assessment as traditional assessment conducted by a neuropsychologist. Importantly, the lack of HIV-negative controls prevents us from determining whether our findings are specific to PLWH. However, there is a body of literature from the general population concluding that psychological and psychiatric factors are as important as cognitive function to subjective memory complaints. In their conceptual framework for research into subjective cognitive decline in preclinical Alzheimer disease, Jessen et al34 note that mood- and anxiety-related symptoms may be both a confounder of the relationship between subjective cognitive decline and subsequent dementia and an early symptom of dementia. An analogous situation might be at work in HIV-related cognitive impairment.
Despite initially positive results with a three-question symptom tool1 and the MOS-HIV cognitive questions,35 several studies have reported poor test accuracy of ADL and cognitive symptom scales in HIV+ patients.7–10 In 290 PLWH over 50 years in Great Britain and Ireland and 97 seronegative controls, other patient-reported outcome measures, including depression, falls, sexual function, and general health, correlated poorly with cognitive impairment.8 Thames et al36 divided 107 HIV-positive patients into 4 groups, on the basis of whether they were impaired or unimpaired, and whether they were accurate or inaccurate in their self-assessment of this. Notably, only 38% were accurate, 33% were impaired but did not report a decline, and 25% reported a decline but did not have cognitive impairment. In similar studies, it was reported that only 3337 and 37%38 of patients gave answers to symptom questions that concurred with their objective cognitive test scores. In an analysis of CNS HIV Anti-Retroviral Therapy Effects Research (CHARTER) cohort patients (n = 277) in the United States, impaired everyday function was associated with depression, unemployment, findings on neuromedical examination, and physician-assigned Karnofsky score. Although patients were able to rate their own overall level of ability, they were poor at correctly attributing their difficulties to cognitive versus physical problems.7
Patients' symptoms are important because they guide clinical decisions, but clearly there are reasons other than neurological disease why patients and study participants may report symptoms. First, PLWH may have psychiatric problems, excessive substance use, or some other reversible cause for their symptoms. Second, depression or anxiety may cause them to perceive a functional decline that is not present. Third, they may be hypervigilant for symptoms or worried about normal everyday experiences, although not having any psychiatric disorder. Fourth, there may be desirability bias: they say what they think the clinician or researcher wants to hear. Fifth, there may be secondary gain as a result of misreporting their symptoms. And sixth, the measuring instruments may lack construct validity and fail to measure what they purport to measure.
Differentiation of asymptomatic from symptomatic HAND relies solely on estimating daily function at ADL, and most published research in this area relies predominantly on participants' self-report. There are limited data on more accurate and reliable measures of everyday performance, such as third-party observation or performance-based measures.12,13 This has crucial implications for the interpretation of longitudinal studies in which the risk of change from ANI to MND is measured, such as the United States CHARTER cohort,39 the Canadian OHTN Cohort,6 and the French Aquitaine cohort.40 For example, in the Ottawa cohort, depression was a major risk factor for change from ANI to MND.6 Because self-reported functional impairment is associated with depression, change from ANI to MND does not always indicate decline in neurocognitive abilities. In addition, change from ANI to MND in longitudinal studies should not be viewed as progressive, unidirectional deterioration (as is implied using Kaplan–Meier plot analyses). The CHARTER cohort used a more robust method of both self-reported and performance-based assessments to discriminate ANI from MND.39 These issues should be taken into consideration in the design and analysis of future HAND cohort studies.
The authors acknowledge all study participants for contributing to the study. The CIPHER Study Group includes Andrea Antinori, Pietro Balestra, Tina Bruun, Simon Collins, Marina Daskalopoulou, Jan Gerstoft, Richard Gilson, Lewis Haddow, John Harrison, Graham Hart, Andrzej Horban, Igor Karpov, Fiona Lampe, Rosanna Laverick, Jens Lundgren, Jeffrey McDonnell, Lars Nielsen, Andrew Phillips, Alison Rodger, Lorraine Sherr, Andrew Speakman, and Anna Vassilenko. The authors also thank Adebiyi Aderonke, Nataliya Brima, Christina Broussard, Tina Bruun, Bill Burman, Simon Edwards, Jonathan Elford, Anna-Maria Geretti, Simon Gilson, Jaqueline Hjetstedt, Anne Johnson, Margaret Johnson, Paul Maruff, Alec Miners, Robert Pralat, Winnie Ann Ryholt, Adrian Schembri, Colette Smith, Rita Trombin, and Sonali Wayal.
1. Simioni S, Cavassini M, Annoni JM, et al. Cognitive dysfunction in HIV
patients despite long-standing suppression of viremia. AIDS. 2010;24:1243–1250.
2. Heaton RK, Clifford DB, Franklin DRJ, et al. HIV
-associated neurocognitive disorders persist in the era of potent antiretroviral therapy: CHARTER Study. Neurology. 2010;75:2087–2096.
3. Antinori A, Arendt G, Becker JT, et al. Updated research nosology for HIV
-associated neurocognitive disorders. Neurology. 2007;69:1789–1799.
4. Bonnet F, Amieva H, Marquant F, et al. Cognitive disorders in HIV
-infected patients: are they HIV
-related? AIDS. 2013;27:391–400.
5. Vassallo M, Durant J, Biscay V, et al. Can high central nervous system penetrating antiretroviral regimens protect against the onset of HIV
-associated neurocognitive disorders? AIDS. 2014;28:493–501.
6. Rourke SB, Gill J, Rachlis A, et al. Asymptomatic neurocognitive impairment (ANI) is associated with progression to symptomatic HIV
-associated neurocognitive disorders (HAND) in people with HIV
: results from the Ontario HIV
Treatment Network (OHTN) cohort study. 8th IAS Conference on HIV
Pathogenesis, Treatment and Prevention; July 19–22, 2015; Vancouver, Canada. Abstract WEPEB326.
7. Obermeit LC, Beltran J, Casaletto KB, et al. Evaluating the accuracy of self-report for the diagnosis of HIV
-associated neurocognitive disorder (HAND): defining “symptomatic” versus “asymptomatic” HAND. J Neurovirol. 2017;23:67–78.
8. Underwood J, De Francesco D, Post FA, et al. Associations between cognitive impairment and patient-reported measures of physical/mental functioning in older people living with HIV
9. Muñoz-Moreno JA, Prats A, Pérez-Álvarez N, et al. A brief and feasible paper-based method to screen for neurocognitive impairment in HIV
-infected patients: the NEU screen. J Acquir Immune Defic Syndr. 2013;63:585–592.
10. Fasel D, Kunze U, Elzi L, et al. A short tool to screen HIV
-infected patients for mild neurocognitive disorders—a pilot study. BMC Psychol. 2014;2:21.
11. Haddow LJ, Laverick R, Daskalopoulou M, et al. Multicenter European prevalence study of neurocognitive impairment and associated factors in HIV
positive patients. AIDS Behav. 2017. doi:10.1007/s10461-017-1683-z.
12. Heaton RK, Marcotte TD, Mindt MR, et al. The impact of HIV
-associated neuropsychological impairment on everyday functioning. J Int Neuropsych Soc. 2004;10:317–331.
13. Gandhi NS, Skolasky RL, Peters KB, et al. A comparison of performance-based measures of function in HIV
-associated neurocognitive disorders. J Neurovirol. 2011;17:159–165.
14. Wu AW, Rubin HR, Mathews WC, et al. A health status questionnaire using 30 items from the Medical Outcomes Study. Preliminary validation in persons with early HIV
infection. Med Care. 1991;29:786–798.
15. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16:606–613.
16. Spitzer RL, Kroenke K, Williams JB, et al. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166:1092–1097.
17. Weaver Cargin J, Maruff P, Collie A, et al. Mild memory impairment in healthy older adults is distinct from normal aging. Brain Cogn. 2006;60:146–155.
18. Harrison J, Maruff P. Measuring the mind: assessing cognitive change in clinical drug trials. Expert Rev Clin Pharmacol. 2008;1:471–473.
19. Lim YY, Ellis KA, Harrington K, et al. Use of the cogstate brief battery in the assessment of Alzheimer's disease related cognitive impairment in the Australian imaging, Biomarkers and Lifestyle (AIBL) study. J Clin Exp Neuropsychol. 2012;34:345–358.
20. Garvey LJ, Surendrakumar V, Winston A. Low rates of neurocognitive impairment are observed in neuro-asymptomatic HIV
-infected subjects on effective antiretroviral therapy. HIV
Clin Trials. 2011;12:333–338.
21. Cysique LAJ, Maruff P, Darby D, et al. The assessment of cognitive function in advanced HIV
-1 infection and AIDS dementia complex using a new computerised cognitive test battery. Arch Clin Neuropsychol. 2006;21:185–194.
22. Overton ET, Kauwe JS, Paul R, et al. Performances on the Cogstate and standard neuropsychological batteries among HIV
patients without dementia. AIDS Behav. 2011;15:1902–1909.
23. Maruff P, Thomas E, Cysique LA, et al. Validity of the Cogstate brief battery: relationship to standardized tests and sensitivity to cognitive impairment in mild traumatic brain injury, schizophrenia, and AIDS dementia complex. Arch Clin Neuropsychol. 2009;24:165–178.
24. Winston A, Puls R, Kerr SJ, et al. Dynamics of cognitive change in HIV
-infected individuals commencing three different initial antiretroviral regimens: a randomized, controlled study. HIV
25. McDonnell J, Haddow LJ, Daskalopoulou M, et al. Minimal cognitive impairment in UK HIV
positive men who have sex with men: effect of case definitions, and comparison with the general population and HIV
negative men. J Acquir Immune Defic Syndr. 2014;67:120–127.
26. Ashby J, Foster CJ, Garvey LJ, et al. Cerebral function in perinatally HIV
-infected young adults and their HIV
-uninfected sibling controls. HIV
Clin Trials. 2011;2011:81–87.
27. Bangirana P, Sikorskii A, Giordani B, et al. Validation of the CogState battery for rapid neurocognitive assessment in Ugandan school age children. Child Adolescent Psychiatry Mental Health. 2015;9:38.
28. Yamashita Y, Mukasa A, Anai C, et al. Summer treatment program for children with attention deficit hyperactivity disorder: Japanese experience in 5 years. Brain Dev. 2011;33:260–267.
29. Zhong N, Jiang H, Wu J, et al. Reliability and validity of the CogState battery Chinese language version in schizophrenia. PLoS One. 2013;8:e74258.
30. Falleti MG, Maruff P, Collie A, et al. Practice effects associated with the repeated assessment of cognitive function using the Cogstate battery at 10-minute, one week and one month test-retest intervals. J Clin Exp Neuropsychol. 2006;28:1095–1112.
31. Tombaugh TN, Kozak J, Rees L. Normative data stratified by age and education for two measures of verbal fluency: FAS and animal naming. Arch Clin Neuropsychol. 1999;14:167–177.
32. Ruff RM, Light RH, Parker SB, et al. Benton controlled oral Word association test: reliability and updated norms. Arch Clin Neuropsychol. 1996;11:329–338.
33. Comijs HC, Deeg DJH, Dik MG, et al. Memory complaints; the association with psycho-affective and health problems and the role of personality characteristics: a 6-year follow-up study. J Affect Disord. 2002;72:157–165.
34. Jessen F, Amariglio RE, van Boxtel M, et al. A conceptual framework for research on subjective cognitive decline in preclinical Alzheimer's disease. Alzheimer's Dement. 2014;10:844–852.
35. Knippels HM, Goodkin K, Weiss JJ, et al. The importance of cognitive self-report in early HIV
-1 infection: validation of a cognitive functional status subscale. AIDS. 2002;16:259–267.
36. Thames AD, Becker BW, Marcotte TD, et al. Depression, cognition, and self-appraisal of functional abilities in HIV
: an examination of subjective appraisal versus objective performance. Clin Neuropsychol. 2011;25:224–243.
37. Barber TJ, Bansi L, Pozniak A, et al. Low levels of neurocognitive impairment detected in screening HIV
-infected men who have sex with men: the MSM Neurocog Study. Int J STD AIDS. 2017;28:715–722.
38. Hinkin CH, Van Gorp WG, Satz P, et al. Actual versus self-reported cognitive dysfunction in HIV
-1 infection: memory-metamemory dissociations. J Clin Exp Neuropsychol. 1996;18:431–443.
39. Grant I, Franklin DR, Deutsch R, et al. Asymptomatic HIV
-associated neurocognitive impairment increases risk for symptomatic decline. Neurology. 2014;82:2055–2062.
40. Dufouil C, Richert L, Thiebaut R, et al. Diabetes and cognitive decline in a French cohort of patients infected with HIV
-1. Neurology. 2015;85:1065–1073.