Kalichman, Seth C. PhD*; Cherry, Chauncey MPH*; Kalichman, Moira O. MSW*; Amaral, Christina BA*; White, Denise MA*; Grebler, Tamar BA*; Eaton, Lisa A. PhD*; Cruess, Dean PhD*; Detorio, Mervi A. MS†; Caliendo, Angela M. MD‡; Schinazi, Raymond F. PhD†
HIV epidemics remain concentrated in the poorest and most disadvantaged communities. Significant advances in antiretroviral therapy (ART) result in better management of HIV infection, but these benefits are not equally shared by all people living with HIV.1,2 Among factors known to impede medication adherence are poor health literacy skills, particularly, difficulty reading and interpreting medical information.3 Although the most prevalent reasons for missing medications are based on memory lapses, poor-literacy skills preclude the use of written reminders and other verbal systems commonly used to enhance adherence.4 Patients with lower literacy skills may not understand the repercussions of nonadherence, which can lead to intentionally missing medications to relieve side effects, taking drug holidays, or cleansing their body.5–9 Indeed, impediments to adherence may account for the poorer health outcomes consistently observed in lower literacy medical populations.10,11
The association between health literacy and ART adherence seems quite robust7,12–18 Unfortunately, few adherence interventions have addressed literacy skills, and we are not aware of any ART adherence interventions tailored for limited literacy populations tested in controlled trials.19–21 The purpose of the present research was to test a pictograph-guided patient education and skills-building intervention to improve ART adherence for people with marginal and lower literacy skills. We designed a counseling intervention for use in clinical care with minimal demands on reading skills, and pictographically presented treatment-relevant information. We tested the pictograph-guided intervention in comparison to both standard adherence counseling and time-matched general health improvement counseling. We hypothesized that for patients with lower literacy skills, pictograph-guided counseling would result in greater use of adherence skills, HIV suppression, and ART adherence compared with both standard adherence and general health improvement counseling. We also hypothesized that the benefits of pictograph-guided counseling would not be observed in patients with marginal literacy skills.
Participants and Setting
Participants were men and women recruited from AIDS services and community outreach in Atlanta, GA, a city with among the fastest growing HIV epidemics in the United States.22 The study commenced November 2008, enrollment ended in April 2011, and follow-ups were completed April 2012.
This trial was registered with ClinicalTrials.gov, identifier NCT01061762. All study protocols were approved by the University of Connecticut Institutional Review Board, and a Federal Certificate of Confidentiality was obtained from the National Institutes of Health. There were 2 adverse events that were unrelated to study activities: participant injuries sustained while coming to the study.
Overview of Intervention Conditions
The 3 experimental conditions in this trial were implemented using matched operational protocols and procedures. All interventions were conducted at the same community-based research site. The 3 counseling conditions were grounded in Social-Cognitive Theory of behavior change23,24 and designed for use in HIV treatment settings. The interventions were formatted to deliver two 60-minute one-on-one counseling sessions over 2 weeks and a third 30-minute booster session 2 weeks later. The same interventionists delivered all 3 manual-based and patient-education flip chart–driven counseling conditions. All counselors received extensive training in each condition and attended weekly supervision.
Pictograph-Guided Adherence Counseling
After the initial formative study period, we designed a treatment adherence intervention tailored for people with lower health literacy skills. This intervention was extensively pilot tested in a phase-1 trial described elsewhere.25 The content of the intervention relied on pictographic information particularly relevant to an individual's medication regimen. The intervention concentrated on delivering the most relevant information for treatment adherence, including the importance of following prescribed instructions for each drug. We also included motivational enhancement techniques, including providing direct feedback on participant health status and training in self-monitoring skills for changes in adherence and viral load. We tailored medication instructions to lower levels of reading literacy, including the use of memory cues for fitting medications into daily routines using strategies described in earlier research.26
A primary aim of the intervention was to integrate intensive interactions with pictograph-guided instructions.27 The intervention materials were developed with minimal words, modeled after similar interventions that have been effective in other areas of health promotion.27,28 The intervention sessions were guided by a tabletop flip chart that moved the counselor and participant together through each intervention component. In addition, a pocket-sized pamphlet was developed to represent the participant's medication regimen, as well as dosing times and administration instructions. The counselor and participant therefore created an individualized adherence plan within the context of 2 counseling sessions. Participants were given an array of adherence tools of their choosing, including pillboxes, watch alarms, reminder notes, and other such tools. The adherence tools were discussed in the context of current medications and current efforts to remain adherent. Planning and problem solving skills were central to the goals of the counseling sessions. The third and final session was a booster that applied problem solving strategies to challenging situations that occurred since the previous session (2 weeks earlier). Situations in which medications may have been missed or not taken on schedule were recreated and role-played for problem solving with the aim of improving future adherence.
Standard Adherence Counseling
As a comparison condition, we included interactive counseling that delivered educational information about HIV treatments, viral suppression, side effects, and the role of adherence in preventing viral resistance. The standard counseling was guided by a participant education flip chart that included brief verbal descriptions of concepts that the participant and counselor could read together as they talked. Text was used throughout the materials that included illustrations of HIV infection processes and comic strips depicting the HIV disease process as an alien invasion. Problem solving skills were applied to situations that create challenges for medication adherence. Participants were given a pillbox and discussed how they could use it and other tools to improve adherence. The third session was a brief booster guided by challenges to adherence that the participant experienced during the previous two weeks.
General Health Improvement Counseling
The control arm was contact-matched noncontaminating health improvement counseling for individuals living with HIV. This condition concentrated on improving general health and well-being in relation to living with HIV. The first session focused on understanding nutrition in terms of food groups, including how to read a food label and relate nutritional information to diet and food choices. The second session focused on stress reduction, relaxation, and exercise to improve health and well-being. The session ended with participants setting personal health goals and selecting health improvement tools, such as pillboxes, pedometers, hand-squeeze balls, and nutrition guides. The third session was a brief booster that discussed and problem-solved barriers to achieving personal health improvement goals.
Reading literacy was assessed at screening with the reading comprehension scale of the Test of Functional Health Literacy in Adults (TOFHLA).29,30 The scale is timed and includes 50 multiple-choice items, in which selecting the correct word among 4 options completes sentences from standard medical instructions. Scores ranged from 0 to 50 with the percent correct computed for the total score.
We also administered the TOFHLA numeracy scale that assesses numerical reasoning for medical instructions.30 For the purposes of the present study, the numeracy scale was used to internally validate literacy groups based on the TOFHLA reading comprehension scale.
Participants were asked a series of questions regarding their vision and use of corrective lenses. Participants who complained of blurred vision during reading were offered nonprescription reading glasses.
For self-report instruments, participants completed 30-minute audio-computerized self-interviewing at baseline and at 3-month and 9-month postintervention follow-ups.13,14 Participants reported demographic information, the date they tested HIV positive, and their income/disability status. We also assessed 14 HIV-related symptoms of 2-week duration.15 Adherence strategies and skills were also assessed to serve as secondary outcomes.31 Specifically, participants indicated whether they had used 13 common memory-based strategies for improving medication adherence.32,33
Baseline Viral Load and CD4 Counts
We used a participant-assisted method for collecting baseline chart abstracted viral load and CD4 cell counts from participants' medical records. Participants were given a form that requested their doctor's office to provide results and dates of their most recent viral load and CD4 cell counts. The form included a place for the provider's office stamp or signature to assure authenticity.
Primary Outcomes: HIV RNA Viral Load and ART Adherence
HIV RNA Viral Load
Participants provided blood specimens to test for HIV (RNA) viral load at the final follow-up assessment. Blood samples were provided at the project offices using standard phlebotomy and were couriered to the laboratory for processing. Whole blood specimens in EDTA tube (Becton Dickinson) were centrifuged at 500g for 10 minutes within 4 hours of collection. The plasma was recovered and aliquoted into 1-mL samples and stored at−70°C. Before August 2010, HIV-1 viral load was determined using the ultrasensitive version of the Amplicor HIV-1 Monitor Test (Roche Diagnsotics, Indianapolis, IN), with a lower limit of quantification of 50 copies/mL. From August 2010 forward, HIV-1 viral load was measured using the RealTime HIV-1 assay (Abbott Molecular) with a lower limit of quantification of 40 copies/mL. For consistency across assays and baseline chart values, we defined undetectable viral load as <50 copies/mL.
HIV treatment adherence was monitored with monthly unannounced telephone-based pill counts. Unannounced pill counts are a reliable and valid measure of ART adherence when conducted in participants' homes and through the telephone,34,35 including demonstrated reliability and validity among people with poor literacy skills.36–38 Unannounced pill counts are an objective measure of adherence, not subject to reporting biases. In order for participants to fabricate their pill counts, they would have to mentally calculate missed doses into missed pills since the last unannounced call which occurred a month earlier, while also knowing how many pills they should have taken and how many pills they had counted on the previous call, an exceedingly difficult task. Participants were provided with a free cell phone that restricted service for project contacts and emergency use. After the initial office-based training in the pill counting procedure, participants were called every 21–35 days at unscheduled times by a phone assessor. Pharmacy information from pill bottles was also collected to verify the number of pills dispensed between calls. Adherence was calculated as the ratio of pills counted relative to pills prescribed, taking into account the number of pills dispensed. The first 3 pill counts occurred before the baseline assessment, allowing us to calculate preintervention adherence.
A moderate effect size (d = 0.35) was used to calculate statistical power for both primary end points.39,40 We assumed 80% retention and estimated a sample of 140 for each primary outcome to achieve 90% chance of detecting differences between groups.
Recruitment and Enrollment
We notified AIDS service providers and infectious disease clinics throughout Atlanta about the study opportunity. Interested persons phoned the research site to schedule an intake appointment. People living with HIV and taking ART were enrolled in a run-in study to screen for literacy skills. Participants with TOFHLA scores <90% correct were recruited for participation in the trial. We selected this liberal cutoff to screen out individuals with higher reading ability, while minimizing the exclusion of lower literacy participants. The additional study entry criteria were (1) age: 18 years or older and (2) proof of positive HIV status and current use of ART with a photo ID matching a current ART prescription bottle.
Within the sample, we defined marginal literacy as scoring between 85% and 90% correct on the TOFHLA and lower literacy as scoring <85% correct29 marginal/lower literacy was treated as a blocking variable in all outcome analyses.
Randomization and Blinding
After the baseline assessment and the first 3 unannounced phone assessments, the Project Manager randomly assigned participants to conditions. Allocation was accomplished using an automated randomization generator accessed at www.randomizer.org. Randomization was not breached throughout the trial. Recruitment, screening, office-based assessment, and telephone assessment staff remained blinded to condition throughout the study, and interventionists never conducted assessments.
We first examined differences between conditions on demographic and health characteristics using analyses of variance for continuous measures and contingency table χ2 tests for categorical variables. We also used procedures suggested by Jurs and Glass41 to test baseline equivalence between conditions and effects of attrition on dependent measures.
Primary and secondary outcome analyses used an intent-to-treat approach where all available follow-up data from participants were included in the analyses, regardless of their exposure to the intervention sessions. Primary outcome analyses for adherence and viral load used generalized estimating equations (GEEs) with unstructured working correlation matrixes. All outcome analyses controlled for baseline values. Counseling condition, literacy level, time of assessment, and all interactions were entered as model effects. Planned contrasts with least significant difference adjustment were used to test for simple effects. Adherence outcomes represent overdispersed count data and therefore used Poisson distribution. To simplify interpretation of results, we report the 95% ART adherence outcomes at each assessment point. For viral load outcomes, we performed GEE models for both the log values (continuous scale) and dichotomously coded detectable/undetectable (binomial) values. Finally, secondary outcomes for adherence strategies at 3- and 9-month postintervention follow-ups were analyzed using logistic regression models for use or nonuse of each strategy among participants who did not report using the strategy at baseline. We also created a composite score for total aggregated adherence strategies reported at the 3- and 9-month follow-ups. Differences between intervention and literacy groups for aggregated strategies were tested using analyses of variance, controlling for number of strategies used at baseline. All main outcome analyses and planned comparisons defined statistical significance as P < 0.05.
Preliminary analyses showed that there were no differences between intervention conditions at baseline (Table 1). Participants' ART regimens included nucleoside reverse transcriptase inhibitors (N = 404, 90%), nonnucleoside reverse transcriptase inhibitors (N = 51, 11%), protease inhibitors (N = 296, 66%), integrase inhibitors (N = 38, 9%), and multiclass single pills (N = 83, 18%). ART classes were proportional across conditions, as were the number of pills and doses taken per day, and there were no differences on baseline measures of adherence, viral load, or behavioral skills. Marginal literacy participants were younger (M = 46.0, SD = 8.0) and had more years of education (M = 12.5, SD = 1.6) than lower literacy participants [M = 48.0, SD = 7.9, t(444) = 2.6, P < 0.01; M = 11.6, SD = 1.8, t(444) = 5.09, P < 0.01, respectively]. The literacy groups did not differ on any other demographic characteristics.
As shown in Figure 1, counseling session attendance was proportional across conditions: 96% of participants attended at least one counseling session and 86% attended all 3 sessions. The trial retained >92% of participants randomized to conditions for audio-computerized self-interviewing and 90% for monthly telephone assessments; 78% (n = 348) completed all 9 monthly postintervention assessment calls (mean = 8.3, SD = 1.6). Two participants were known to have died during the trial, 3 withdrew and 3 moved out of state. Attrition was proportional for the 2 conditions at 3- and 9-month follow-ups. Planned attrition analyses did not find differences between participants retained and lost by condition.
FIGURE1. Flowchart o...Image Tools
HIV RNA Viral Load Primary Outcomes
The mean log values for HIV RNA viral load at baseline and follow-ups for the counseling conditions and literacy groups are shown in Table 2. Analyses indicated that there were no main effects at the follow-up viral load testing for counseling conditions, Wald χ2 (2) = 0.45, P > 0.1, or literacy groups, Wald χ2 (1) = 1.88, P > 0.1. However, there was a significant counseling condition by literacy group interaction, Wald χ2 (2) = 6.80, P < 0.03. The condition by literacy group interaction effect was also significant for participants achieving undetectable viral loads 9 months after counseling, Wald χ2 (2) = 2.05, P < 0.01 (Fig. 2). Among the marginal literacy participants who had a detectable viral load at baseline, 40% of the pictograph-guided and 45% of the standard adherence counseling conditions achieved an undetectable viral load at the follow-up, compared with 33% of the general health improvement condition. In contrast, for the lower literacy participants, only 28% who received pictograph-guided counseling and had detectable viral loads at baseline achieved an undetectable viral load at the follow-up compared with 35% of participants in the standard adherence counseling and 40% of those in the general health improvement condition.
Treatment Adherence Primary Outcomes
Table 2 shows the percentages of participants in each condition within literacy groups who achieved >95% adherence at each time point. GEE models for monthly unannounced pill count adherence indicated that there were no main effects for condition or literacy group, although there was a main effect for time, Wald χ2 (8) = 19.72, P < 0.01. Paralleling the viral load outcomes, there was a significant intervention condition by literacy group interaction effect, Wald χ2 (2) = 5.93, P < 0.05. Planned comparisons showed that significant differences between conditions were observed at the 1-month follow-up, Wald χ2 (2) = 4.66, P < 0.05, 2-month follow-up, Wald χ2 (2) = 4.73, P < 0.05, and 4-month follow-up, Wald χ2 (2) = 4.75, P < 0.05. Results showed that among marginal health literacy participants, the pictograph-guided and standard adherence counseling conditions demonstrated greater adherence compared with the general health improvement counseling. The difference between the pictographic and standard conditions was not significant. In contrast, lower literacy participants in the general health improvement counseling condition demonstrated greater adherence than those in the pictograph-guided and standard adherence counseling.
Behavioral Adherence Strategy Secondary Outcomes
Results of logistic regression models testing the use and nonuse of 13 behavioral adherence strategies controlling for baseline use and including literacy group, indicated significant intervention condition differences at the 3-month follow-up (Table 3). Analysis of the behavioral strategy composite measure indicated a main effect for condition, F2,426 = 3.40, P < 0.05; the 2 adherence counseling interventions used significantly more strategies than the health improvement counseling condition. There was also a significant intervention condition by literacy group interaction, F2,426 = 3.1, P < 0.05; lower literacy participants in the pictograph-guided and standard counseling conditions used more strategies than the health improvement condition. However, among the marginal literacy participants, the pictograph-guided counseling condition reported greater use of adherence strategies compared with the standard and health improvement counseling conditions.
Similar results were observed at the 9-month follow-up. The pictograph-guided counseling condition reported the greatest use of strategies. The main effect for intervention condition was not significant, F2,409 = 2.70, P < 0.06. However, the intervention condition by literacy group interaction was significant, F2,409 = 3.40, P < 0.05; lower literacy participants in the pictograph-guided and standard adherence counseling interventions again reported more use of adherence strategies than the general health counseling condition, whereas among marginal literacy participants, the pictograph-guided counseling condition reported more adherence strategies than the standard adherence and general health counseling conditions.
Similar to past research, we observed poor ART adherence among persons with marginal and lower health literacy.18 To our knowledge, this is the first randomized clinical trial of an ART adherence improvement intervention for people with poor literacy skills. We designed an intervention using established principles for enhancing health communication and education for medical patients with lower health literacy.10,27 The experimental ART counseling was guided by pictographic representations of HIV disease processes, actions of ART in suppressing HIV, and consequences of ART nonadherence. The intervention underwent extensive preliminary testing and demonstrated promising outcomes in a pilot study.25 For participants with marginal levels of health literacy, the present findings failed to show any added benefit of the pictograph-guided counseling for adherence improvement beyond those observed from a standard approach to adherence counseling. Among participants with lower health literacy, neither the pictographic nor the standard adherence counseling demonstrated positive outcomes. This unexpected pattern of results suggests that individuals who demonstrate modest health literacy deficits can benefit from brief and focused adherence counseling. However, persons who experience more difficulty reading and understanding health information may require more intensive provider-directed approaches to adherence.
The present trial was conducted in a city in the southeastern United States that may not be generalizable to other cities and regions. Generalizability was also limited by recruiting with outreach and referral procedures. Thus, although our sample extends across multiple clinics, our convenience sample cannot be considered representative of people living with HIV receiving care. Another limitation of the study was our use of self-reported measures of medication adherence strategies. The primary behavioral end point in this study was ART adherence measured by unannounced phone-based pill counts, which has not shown evidence of assessment reactivity.37 Nevertheless, we cannot rule out the potential for monthly assessment calls prompting participant adherence across conditions. It is also possible that the literacy groups were confounded by unmeasured characteristics including neurocognitive disorders. In addition, we did not differentiate changes in viral load attributable to poor adherence from viral load increases resulting from treatment failure. With these limitations in mind, we believe that the present trial results have implications for HIV treatment adherence interventions for limited literacy adults.
HIV infections are most prevalent in low-income disadvantaged communities. Poor health literacy likely plays a predominant role in HIV treatment outcomes and health disparities.42 Results from the present study encourage screening patients for basic health literacy skills in risk assessments for difficulties adhering to treatment. Patients who do not experience difficulty reading and those who are less proficient readers may benefit from brief skills-based adherence counseling. However, patients who are unable to read and those who read with greater difficulty will require closer clinical monitoring and may benefit from more intensive approaches to adherence such as modified direct observation therapies, blister packs, and mobile medication alert systems.43 Literacy skills are associated with verbal memory, planning skills, motor speed, and other neurocognitive functions that can all interfere with adherence.44 Interventions are therefore still needed to achieve optimal ART adherence and positive treatment outcomes for patients with lower health literacy skills.
1. Bangsberg DR, Ragland K, Monk A, et al.. A single tablet regimen is associated with higher adherence and viral suppression than multiple tablet regimens in HIV+ homeless and marginally housed people. AIDS. 2010;24:2835–2840.
2. Raboud J, Li M, Walmsley S, et al.. Once daily dosing improves adherence to antiretroviral therapy. AIDS Behav. 2011;15:1397–1409.
3. Zarcadoolas C, Pleasant A, Greer DS. Understanding health literacy: an expanded model. Health Promot Int. 2005;20:195–203.
4. Velthoven MH, Brusamento S, Majeed A, et al.. Scope and effectiveness of mobile phone messaging for HIV/AIDS care: a systematic review. Psychol Health Med. 2013;18:182–202.
5. Graham J, Bennett IM, Holmes WC, et al.. Medication beliefs as mediators of the health literacy-antiretroviral adherence relationship in HIV-infected individuals. AIDS Behav. 2007;11:385–392.
6. Osborn CY, Davis TC, Bailey SC, et al.. Health literacy in the context of HIV treatment: introducing the Brief Estimate of Health Knowledge and Action (BEHKA)-HIV version. AIDS Behav. 2010;14:181–188.
7. Wolf MS, Davis TC, Tilson HH, et al.. Misunderstanding of prescription drug warning labels among patients with low literacy. Am J Health Syst Pharm. 2006;63:1048–1055.
8. Wolf MS, Davis TC, Osborn CY, et al.. Literacy, self-efficacy, and HIV medication adherence. Patient Educ Couns. 2007;65:253–260.
9. Berkman ND, Sheridan SL, Donahue KE, et al.. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. 2011;155:97–107.
10. Berkman ND, Dewalt DA, Pignone MP, et al.. Literacy and health outcomes. Evid Rep Technol Assess (Summ). 2004;(87):1–8.
11. Ownby RL. Medication adherence and health care literacy: filling in the gap between efficacy and effectiveness. Curr Psychiatry Rep. 2005;7:1–2.
12. Osborn CY, Paasche-Orlow MK, Davis TC, et al.. Health literacy: an overlooked factor in understanding HIV health disparities. Am J Prev Med. 2007;33:374–378.
13. Waldrop-Valverde D, Jones DL, Jayaweera D, et al.. Gender differences in medication management capacity in HIV infection: the role of health literacy and numeracy. AIDS Behav. 2009;13:46–52.
14. Kalichman SC, Rompa D. Functional health literacy is associated with health status and health-related knowledge in people living with HIV-AIDS. J Acquir Immune Defic Syndr. 2000;25:337–344.
15. Kalichman SC, Ramachandran B, Catz S. Adherence to combination antiretroviral therapies in HIV patients of low health literacy. J Gen Intern Med. 1999;14:267–273.
16. Davis TC, Wolf MS, Bass PF III, et al.. Low literacy impairs comprehension of prescription drug warning labels. J Gen Intern Med. 2006;21:847–851.
17. Davis TC, Wolf MS, Bass PF III, et al.. Literacy and misunderstanding prescription drug labels. Ann Intern Med. 2006;145:887–894.
18. Paasche-Orlow MK, Cheng DM, Palepu A, et al.. Health literacy, antiretroviral adherence, and HIV-RNA suppression: a longitudinal perspective. J Gen Intern Med. 2006;21:835–840.
19. van Servellen G, Carpio F, Lopez M, et al.. Program to enhance health literacy and treatment adherence in low-income HIV-infected Latino men and women. AIDS Patient Care STDs. 2003;17:581–594.
20. Khachani I, Harmouche H, Ammouri W, et al.. Impact of a psychoeducative intervention on adherence to HAART among low-literacy patients in a resource-limited setting: the case of an Arab country–Morocco. J Int Assoc Physicians AIDS Care (Chic). 2012;11:47–56.
21. Smith SA, Moore EJ. Health literacy and depression in the context of home visitation. Matern Child Health J. 2012;16:1500–1508.
22. Murri R, Cingolani A, De Luca A, et al.. Asymmetry of the regimen is correlated to self-reported suboptimal adherence: results from AdUCSC, a cohort study on adherence in Italy. J Acquir Immune Defic Syndr. 2010;55:411–412.
23. Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977;84:191–215.
24. Bandura A. Self-Efficacy in Changing Societies. New York, NY: Cambridge University Press; 1995:xv, 334 pp.
25. Kalichman SC, Cherry J, Cain D. Nurse-delivered antiretroviral treatment adherence intervention for people with low literacy skills and living with HIV/AIDS. J Assoc Nurses AIDS Care. 2005;16:3–15.
26. Jolly BT, Scott JL, Feied CF, et al.. Functional illiteracy among emergency department patients: a preliminary study. Ann Emerg Med. 1993;22:573–578.
27. Houts PS, Bachrach R, Witmer JT, et al.. Using pictographs to enhance recall of spoken medical instructions. Patient Educ Couns. 1998;35:83–88.
28. Jacobson TA, Thomas DM, Morton FJ, et al.. Use of a low-literacy patient education tool to enhance pneumococcal vaccination rates. A randomized controlled trial. JAMA. 1999;282:646–650.
29. Parker RM, Baker DW, Williams MV, et al.. The test of functional health literacy in adults: a new instrument for measuring patients' literacy skills. J Gen Intern Med. 1995;10:537–541.
30. Baker DW, Williams MV, Parker RM, et al.. Development of a brief test to measure functional health literacy. Patient Educ Couns. 1999;38:33–42.
31. Amico KR, Barta W, Konkle-Parker DJ, et al.. The information-motivation-behavioral skills model of ART adherence in a Deep South HIV+ clinic sample. AIDS Behav. 2009;13:66–75.
32. Catz SL, Kelly JA, Bogart LM, et al.. Patterns, correlates, and barriers to medication adherence among persons prescribed new treatments for HIV disease. Health Psychol. 2000;19:124–133.
33. Kalichman SC, Cain D, Cherry C, et al.. Pillboxes and antiretroviral adherence: prevalence of use, perceived benefits, and implications for electronic medication monitoring devices. AIDS Patient Care STDS. 2005;19:49–55.
34. Bangsberg DR, Hecht FM, Charlebois ED, et al.. Comparing objective measures of adherence to HIV antiretroviral therapy: electronic medication monitors and unannounced pill counts. AIDS Behav. 2001;5:275–281.
35. Kalichman SC, Amaral CM, Cherry C, et al.. Monitoring antiretroviral adherence by unannounced pill counts conducted by telephone: reliability and criterion-related validity. HIV Clin Trials. 2008;9:298–308.
36. Weiser SD, Bangsberg DR, Kegeles S, et al.. Food insecurity among homeless and marginally housed individuals living with HIV/AIDS in San Francisco. AIDS Behav. 2009;13:841–848.
37. Simbayi LC, Cloete A, Strebel A, et al.. HIV/AIDS risk reduction and domestic violence intervention for South African men: theoretical foundations, development, and test of concept. Int J Men's Health. 2008;44:494–600.
38. Kalichman SC, Amaral CM, Stearns H, et al.. Adherence to antiretroviral therapy assessed by unannounced pill counts conducted by telephone. J Gen Intern Med. 2007;22:1003–1006.
39. Crepaz N, Lyles C, Wolitski R, et al.. Do prevention interventions reduce HIV risk behaviours among people living with HIV? A meta-analytic review of controlled trials. AIDS. 2006;20:143–157.
40. Amico KR, Harman JJ, Johnson BT. Efficacy of antiretroviral therapy adherence interventions: a research synthesis of trials, 1996 to 2004. J Acquir Immune Defic Syndr. 2006;41:285–297.
41. Jurs S, Glass G. The effect of experimental mortality on the internal and external validity of the randomized comparative experiment. J Exp Educ. 1971;40:62–66.
42. Paasche-Orlow MK, Wolf MS. Promoting health literacy research to reduce health disparities. J Health Commun. 2010;15(suppl 2):34–41.
43. Haberer JE, Kahane J, Kigozi I, et al.. Real-time adherence monitoring for HIV antiretroviral therapy. AIDS Behav. 2010;14:1340–1346.
44. Waldrop-Valverde D, Ownby RL, Wilkie FL, et al.. Neurocognitive aspects of medication adherence in HIV-positive injecting drug users. AIDS Behav. 2006;10:287–297.
© 2013 Lippincott Williams & Wilkins, Inc.