Antiretroviral therapy (ART) reduces morbidity and mortality related to HIV infection. Sustained adherence to ART improves individual outcomes and reduces secondary transmission, because low viral load (VL) is associated with reduced HIV transmission1–3 and earlier ART initiation reduces sexual transmission by 96%.4 It is important to identify efficient ways to support medication adherence over a lifetime, as ART is now recommended in the United States for all persons living with HIV (PLWH) regardless of CD4 count.5 However, only an estimated 77% of US patients on ART have suppressed VLs.6 Reducing transmission risk behaviors among PLWH (prevention with positives) is a longstanding public health goal.7 The chronicity of HIV infection may be accompanied by continued or increased sexual risk behaviors for some individuals; however, not all providers routinely address HIV transmission risk reduction with their HIV-positive patients.8–10
Scalable strategies are needed to optimize ART adherence and to reduce secondary transmission of HIV. Meta-analyses show that interventions to support ART adherence11,12 and reduce secondary HIV transmission risk13,14 are efficacious. Because these interventions have been largely research-based, and staff- and resource-intensive,15,16 population-level implementation may not occur.17,18
We hypothesized that a computer-delivered intervention could support ART adherence and reduce HIV transmission risk by PLWH. We evaluated such an intervention called Computer Assessment & Rx Education for HIV-positive people (CARE+).
Study participants were recruited from a university-affiliated public HIV clinic and a large AIDS service organization in Seattle, WA. Eligibility criteria included age ≥18 years, on ART, able to understand spoken English; exclusions included thought disorders and current participation in ART adherence or prevention-with-positives studies. Written consent was obtained before randomized assignment. All procedures were approved by the University of Washington Human Subjects Division, 06-1198-C. Participants received $20 at the first 3 sessions and $40 at the final session.
The computerized-counseling intervention was evaluated in a prospective 2-arm randomized controlled trial. The study sample of n = 240 was assigned through an automated pseudo-random number generation algorithm, disallowing any exposure to intervention by controls. The experimental group received CARE+ (audio-narrated assessment, tailored feedback, skill-building videos, health plan, and printout) on a tablet computer and standard care, whereas controls received assessment only on tablets and standard care. Each group underwent 4 sessions specific to assigned arm at 3-month intervals over 9 months. Sessions were scheduled on same day as clinic visits wherever possible.
CARE+ is a NET-based (Microsoft, Redmond, WA) custom software application on touch-screen computers. Intervention content is based on theoretical frameworks: information–motivation–behavior including “importance” and confidence' scales around ART use and transmission risk-reduction,19 transtheoretical including stage of change questions around condoms,20 social cognitive role-modeling with peers demonstrating healthy behaviors in videos,21 and motivational interviewing, including messages acknowledging ambivalence around behavior change and highlighting user's commitment.22 The tool incorporates evidence-based elements shown in randomized controlled trials to improve ART adherence or reduce sexual risk,23,24 such as feedback, including consequence-framed messages (eg, Unprotected sex may expose you to STDs)25 and videos.26 Content recommendations were obtained through 30 qualitative interviews conducted with PLWH.27 Final CARE+ content was reviewed by an expert panel for face validity.
Figure 1 summarizes the CARE+ session. Users received tailored feedback based on risk assessment responses and viewed video versions for heterosexuals or for men who have sex with men showcasing skills around HIV disclosure, ART adherence, safer sex, substance abuse, male/female condoms, condom use negotiation, working with providers, and HIV natural history and ART mechanisms. Users develop a plan for ART adherence or safer sex (user choice at the first session and switched at the third session). A personalized printout summarized feedback, health plan, and referral phone numbers.
The control condition comprised computerized risk-assessment only (sexual behaviors, substance use, mental health, ART regimen, side effects, adherence in last 7 and 30 days).
In both study arms, the tool flagged reports of severe depression by Patient Health Questionnaire (PHQ-9 score of ≥20),28 intimate partner violence (IPV), or suicidal ideation; as outlined in the consent, study staffers notified case managers for appropriate follow-up. At repeat sessions, these participants were asked how referrals went. All intervention participants were reminded of their last plan and asked to continue or make a new health plan. Software usability was evaluated among an additional 30 HIV-positive clients, and 1-week test–retest reliability assessment was performed to establish psychometric performance of key tool variables.29
The primary biological outcome was HIV-1 RNA VL, determined using a 500 mL plasma specimen in a TaqMan real-time polymerase chain reaction assay with 30 copies per microliter as the lower limit of quantification for detectable HIV-1. HIV-1 VL was assessed using specimens drawn on day of study interview or as part of patient care within a month before study visit. The same laboratory was used for all study and clinical VLs and was determined by personnel blinded to study arm. Primary behavioral outcome measures collected by self-report in both arms consisted of a composite variable of sexual transmission risk—no condom use (unprotected sex) or condom use with problems/errors (ie, vaginal or anal sex either without a condom or where a condom was used but HIV exposure may have occurred due to mechanical or user failure)—and ART adherence by 30-day visual analog scale (VAS).30,31 Accurate reporting was encouraged during enrollment, consent, and sessions through normalizing language and reiteration that the study would not share self-reported data to providers, with sole exception of IPV, severe depression, and suicidality, which (as noted in the consent) prompted appropriate provider referral. Secondary outcome measures included changes in ART/condom stage of change and HIV disclosure.
Fisher exact and Wilcoxon rank-sum tests assessed differences between intervention and control groups and between study sites in baseline study population characteristics. We used generalized estimating equation (GEE) models with a Gaussian link and an unstructured correlation structure to compare changes in VL (log10-transformed) and ART adherence (30-day VAS) between intervention and control groups and from baseline to 9-month follow-up. GEE models with a logit link and an unstructured correlation structure compared odds of undetectable VL and sexual transmission risks between intervention and control groups and from baseline to 9-month follow-up. All GEE models included main effects for intervention condition and linear trend, as well as an interaction between these terms to capture differences in change between intervention conditions. The analysis was intent-to-treat. Covariates in the models included likely depression diagnosis by PHQ-9 because this was the only variable that differed significantly between study arms at baseline, as well as education, condom use with main partner at baseline, and study site. These analyses were performed for the whole sample and for the subgroup that had detectable VL at baseline. Estimates for these “detectables” were obtained by including relevant main and interaction effects for an indicator variable specifying whether each participant had detectable VL at baseline. When modeling odds of VL being undetectable within this subgroup, only the 3 follow-up assessments were included. Analyses were performed using R,32 including geepack33 for GEE, doBy34 for postestimation and ggplot235 for result visualization.
We approached 301 individuals at 2 study sites; 240 enrolled (80% acceptance), 239 completed baseline, and 87% (209/239) were retained for 9-month study duration (Fig. 2). Participants were consented, enrolled, and then randomized. The 30 lost to follow-up had similar numbers and reasons across arms, suggesting nondifferential dropout (P = 0.56 for attrition by arm).
Table 1 shows participant characteristics at baseline by study arm. At baseline intervention, participants were less likely than controls to obtain a positive screening result for likely depression diagnosis (n = 14 vs. 25, P = 0.032). There were several significant differences at baseline between clinic- and organization-recruited participants, including proportion of men who have sex with men, proportion incarcerated more than 1 night, condom use, self-reported ART adherence (VAS), proportion with resistant virus, and proportion victimized by IPV (see Table S1, Supplemental Digital Content,https://links.lww.com/QAI/A504).
Figure 3 shows 95% confidence intervals (CIs) for outcome means by time and study condition. Detailed GEE results are in Supplemental Digital Content Tables S2-S4 (see https://links.lww.com/QAI/A504) showing impact on VL, proportion with undetectable VL, ART adherence, and sexual transmission risks in the full sample (see Table S2, Supplemental Digital Content,https://links.lww.com/QAI/A504) and the subset of participants with detectable VL at baseline (see Table S1, Supplemental Digital Content,https://links.lww.com/QAI/A504), as well as contrasts between intervention and control conditions at each time point and between baseline and 9-month time points within each condition, for full and subset samples (see Table S4, Supplemental Digital Content,https://links.lww.com/QAI/A504).
Figure 4 summarizes main outcomes of interest. The first 4 contrasts in each figure are intervention versus control at each time point, whereas the last 2 contrasts in each figure compare baseline with 9-month follow-up within each study arm.
HIV-1 VL Effect
Figure 4A shows 95% CIs for log10 VL point-in-time study condition mean differences and change within each group from baseline to 9-month follow-up. Figure 4B shows 95% CIs for point-in-time study condition differences and change within each group from baseline to 9-month follow-up in the odds of having undetectable VL. There were marginally significant differences in change from baseline to the 9-month follow-up between study arms in log10 VL (P = 0.053) and in the odds of having undetectable VL (P = 0.090). CARE+ intervention participants overall had an average decrease of 0.17 log10 VL (P = 0.112; 95% CI: −0.39 to 0.04), whereas control participants had an increase of 0.13 (P = 0.250; 95% CI: −0.09 to 0.35) (Fig. 4A, right). Relative to baseline, the odds of having undetectable VL at the 9-month follow-up were increased in the CARE+ condition [odds ratio (OR) = 1.57; P = 0.037; 95% CI: 1.03 to 2.39] but reduced in the control condition (OR = 0.98; P = 0.925; 95% CI: 0.71 to 1.37) (Fig. 4B, right). At the 9-month follow-up, CARE+ intervention participants were lower than controls in log10 VL (−0.06; P = 0.741; 95% CI: −0.4 to 0.30) and had increased odds of undetectable VL (OR = 1.03; P = 0.920; 95% CI: 0.58 to 1.81), but neither of these differences were significant.
Among the subgroup that had detectable VL at baseline, CARE+ intervention participants had an average decrease of 0.60 log10 VL (P = 0.004; 95% CI: −1.01 to −0.19) (Fig. 4A, left), whereas control participants had an increase of 0.15 log10 VL (P = 0.641; 95% CI: −0.48 to 0.78). At the 9-month follow-up, CARE+ intervention participants were lower than controls in log10 VL (−0.73; 95% CI: −1.42 to −0.03), a significant difference (P = 0.041) (Fig. 4A, left). CARE+ intervention participants also had higher odds of undetectable VL than controls at the 9-month follow-up (OR = 2.32; 95% CI: 0.85 to 6.34), although this difference was only marginally significant (P = 0.101) (Fig. 4B, left). For the subgroup with detectable VL at baseline, in Figure 4B we do not show ORs for the baseline time point nor changes from baseline to the 9-month follow-up time point within each group, because all of the participants in this subgroup had detectable VL at baseline. For the log10 VL outcome, the 3-way interaction between baseline detectable VL, study arm, and time was significant (P = 0.034), indicating CARE+ was more effective than control for those with detectable VL at baseline [Supp. 6].
ART Adherence Effect
Figure 4C shows 95% CIs for point-in-time VAS mean differences between groups and mean change within each group from baseline to 9-month follow-up. There was a statistically significant difference in change from baseline to the 9-month follow-up between study arms (P = 0.046) in self-reported ART adherence by 30-day VAS (see Table S2, Supplemental Digital Content,https://links.lww.com/QAI/A504). CARE+ intervention participants had an average increase of 4.71 points in the percentage of medication doses taken (P = 0.014; 95% CI: 0.95 to 8.48), whereas control participants had a decrease of 1.39 points (P = 0.556; 95% CI: −6.03 to 3.24) (Fig. 4C, right). At the 9-month follow-up, CARE+ intervention participants were higher than controls in ART adherence (4.77; 95% CI: −0.79 to 10.33), but this difference was not significant (P = 0.093) (Fig. 4C, right).
Among those with detectable VL at baseline, CARE+ intervention participants had an average VAS adherence increase of 8.00 points (P = 0.040; 95% CI: 0.37 to 15.62), whereas control participants had a decrease of 1.53 points (P = 0.822; 95% CI: −14.84 to 11.78) (Fig. 4C, left). At the 9-month follow-up, CARE+ intervention participants were higher than controls in ART adherence (13.44; 95% CI: 0.73 to 26.14), a significant difference (P = 0.038) (Fig. 4C, left).
Secondary HIV Transmission Risk Effect
Figure 4D shows CIs for point-in-time study condition differences and change within each group from baseline to 9-month follow-up in self-reported transmission risks, defined as sex without a condom or condom use with errors. There was a statistically significant difference in change from baseline to the 9-month follow-up between study arms in self-reported transmission risks (P = 0.040). Among CARE+ intervention participants, the odds of transmission risks were 0.55 times lower at the 9-month follow-up than at baseline (P = 0.020; 95% CI: 0.34 to 0.91), whereas for control participants, the odds of transmission risks increased over time (OR = 1.10; P = 0.664; 95% CI: 0.72 to 1.67) (Fig. 4D, right). At the 9-month follow-up, CARE+ intervention participants had a reduced odds of transmission risks when compared with controls (OR = 0.46; 95% CI: 0.25 to 0.84), a significant difference (P = 0.012) (Fig. 4D, right).
Clinic Site and Detectable VL at Baseline as Effect Modifiers
None of the 3-way interactions involving study arm, linear trend, and clinic site were statistically significant, suggesting similar intervention effects in the university-affiliated and community-based organization sites (see Table S5, Supplemental Digital Content,https://links.lww.com/QAI/A504). Detectable VL at baseline was a modifier of study arm effects on log10 VL, as indicated by a significant 3-way interaction between study arm, linear trend, and detectable VL at baseline (P = 0.034). Three-way interactions involving study arm, linear trend, and detectable VL at baseline were not statistically significant for other outcomes (see Table S6, Supplemental Digital Content,https://links.lww.com/QAI/A504).
Health Promotion Behavior Plans
CARE+ intervention participants made a concrete plan for ART adherence or transmission risk reduction (controls did not make plans). Many individuals (78%) indicated at baseline that they had an approach that was working for them, which they detailed with specific steps in the CARE+ session; 12% made a new plan. Common plans for ART adherence were to “keep doing what I am doing” (n = 32) “use reminders” (31), and “get support” (25). Common plans for transmission risk reduction were to “not have sex” (31), “use condoms” (27), “have fewer or only 1 sex partner(s)” (22), or “only have sex with people who are also positive (7).”
Intervention participants' confidence in their plan success increased over time, from 66% at 3 months to 80% at 9 months (McNemar χ2P = 0.02). Confidence in their ability to not transmit HIV increased over time: 0–10 ascending scale for confidence, mean 8.43 (SD, 2.27) at baseline and 9.14 (SD, 1.53) at 9 months, P = 0.02.
At baseline, 41 referrals were made for intervention participants and 52 for controls for reported severe depression (37%), IPV (9%), or suicidal ideation (53%); at 9 months total referrals for these conditions were 21 and 32, respectively (P = 0.10).
Nearly all (97%) CARE+ intervention participants found the tool easy to use; 99% rated session length as “just right”; 97% felt they had “enough privacy” during the session. Most (93%) felt the CARE+ session helped them as much or more than face-to-face counseling with a staff person, and 75% said they would prefer the computer over a human counselor in the future. No harms or unintended effects were noted in either arm of the study.
We found that a computerized counseling tool was effective at helping PLWH improve ART adherence and reduce HIV transmission risk behaviors, as measured by improvement in self-reported adherence, reduction in VL, and improvement in reported correct and consistent condom use, compared with controls receiving usual care. The adherence effect was most pronounced among those whose plasma HIV-1 was not suppressed at baseline. The reduced VL and fewer sexual transmission risk behaviors seen among those undergoing the intervention both may contribute to decreasing HIV transmission to sexual partners.
In our study population area, chart audits found that fewer than half of HIV-positive clients were assessed for sexual risks, STD testing, or referral.37 Another study assessing 26 HIV clinics across the United States found that providers reported delivering prevention-with-positives counseling at 67% of initial visits but only 53% of subsequent regular visits.38 Computerized counseling may lack some advantages offered by a highly skilled human counselor, but it is delivered consistently with fidelity,39 without need for staff time or training. In our study, it proved highly acceptable and had an efficacious impact on priority behaviors and objective measures of VL response.
Multiple studies have used computers to assess ART nonadherence40–42 or HIV transmission risk43–49 among PLWH, but fewer have been used to influence patient behavior.50,51 Lightfoot found that computer-assisted self-monitoring of transmission risk behaviors can be a strategy for PLWH.52,53 Fisher et al54 found that computerized counseling supported ART adherence though this study did not find a VL impact. Others have used computerized counseling to reduce HIV acquisition risk,55–57 which meta-analyses have found to be effective.58
Economic evaluation models have found that adherence interventions with modest effectiveness may provide survival benefit to patients and be cost effective.59 The intervention we tested that does not require staff time, training, and monitoring may be easier to introduce into busy practice settings.
Study limitations include the fact that two-thirds of our population already had suppressed VL at baseline and 60% did not engage in sexual activities at any time point. Both limitations present conservative biases to the null. Mirroring the Seattle HIV epidemic, the sample was predominantly male. This makes extension of these results to females, or to those living outside the United States, less generalizable. These were heavily treatment- and intervention-experienced populations. Half the sample was from an HIV clinic whose approach to ART adherence support was itself found to reduce log10 VL.60 The potential for detecting intervention effect, and magnitude of effect, may be greater in populations with lower ART adherence or higher sexual risk at baseline.61 The study was well powered to detect a clinically meaningful intervention effect, that is, a mean half-log10 change in VL. However, given that this is one study, examining intervention effects in additional diverse HIV-positive samples would contribute to important next steps of replication and generalization. Future research could include examination of the interplay of multiple HIV behaviors such as nonadherence and sexual risk. The intervention may have influenced risk behavior reporting, though computerized approaches can reduce social desirability bias62 and the VL differences seen is consistent with differences in reported adherence behavior. In previous work with the CARE platform, users reported that it was easier to be honest with the computer and that the session allowed them to reflect on recent risk behaviors.63 Self-reports of sexual transmission risk outcomes and of ART adherence are limitations. Follow-up of 9 months did not allow evaluation of longer-term impact or effect duration. Though the study was conducted in a period when there were fewer ART regimens available, inconsistent ART adherence even to current simplified regimens continues to be a major challenge.64
The study was strengthened by including community- and clinic-based samples, which had similar intervention effects, increasing generalizability.
Computer-delivered counseling had a modest, but significant, positive impact on HIV-1 VL—a primary driver of morbidity and genital compartment infectivity65—and on self-reported HIV transmission risks. This was particularly the case for those who had nonsuppression of VL at baseline—precisely the highest-need group in whom an intervention can have impact. This group's average VL at baseline declined by a clinically meaningful reduction of approximately 0.5 log10 a reduction that has implications for the person's own health and infectiousness. Their reported ART adherence increased by around 10% (76% at baseline to 85% at 9 months), whereas controls started at 74% mean adherence and showed no improvement over time. The intervention's relatively modest absolute changes were enough to get this vulnerable group into better ranges of medication adherence, as seen by VL impact. The importance of supporting treatment adherence has been highlighted by Gardner and others who have shown how few HIV-positive individuals even in care are virally suppressed in the United States.66 Interventions to support adherence have tended to show relatively small effects, highlighting the need for efficacious interventions that can be implemented without straining health system resources,64 as is the promise of computerized counseling tools such as CARE+.
As far as we know, this is the first ART adherence and secondary HIV transmission risk intervention to find biological effect (VL) and behavioral impact among PLWH. The computer format was highly acceptable and facilitated delivery in busy settings. Such an approach warrants further evaluation to determine utility in improving HIV treatment outcomes and reducing secondary HIV transmission among PLWH.
The authors acknowledge Jim Larkin and Tycen Hopkins of Resources Online, CARE+ software developers; and J. Dennis Fortenberry, MD, MS and C. Kevin Malotte, DrPH for their contribution to the original CARE platform. The authors appreciate the assistance of Carol Glenn, RN, Robert D. Harrington, MD, and Thomas M. Hooton, MD of the Harborview Medical Center HIV Clinic affiliated with the University of Washington; David Richart and Hal Garcia-Smith of Lifelong AIDS Alliance; and Peter Tarczy-Hornoch, University of Washington Biomedical and Health Informatics. The authors thank Nok Chhun for her expert assistance with article production.
1. Barroso PF, Schechter M, Gupta P, et al.. Adherence to antiretroviral therapy and persistence of HIV RNA in semen. J Acquir Immune Defic Syndr. 2003;32:435–440.
2. Donnell D, Baeten JM, Kiarie J, et al.. Heterosexual HIV-1 transmission after initiation of antiretroviral therapy: a prospective cohort analysis. Lancet. 2010;375:2092–2098.
3. Quinn TC, Wawer MJ, Sewankambo N, et al.. Viral load and heterosexual transmission of human immunodeficiency virus type 1. Rakai Project Study Group. N Engl J Med. 2000;342:921–929.
4. Cohen MS, Chen YQ, McCauley M, et al.. Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med. 2011;365:493–505.
5. Panel on Antiretroviral Guidelines for Adults and Adolescents. Guidelines for the use of antiretroviral agents in HIV-1 infected adults and adolescents. Department Health Hum Serv. 2013. Available at http://aidsinfo.nih.gov/contentfiles/lvguidelines/adultandadolescentgl.pdf
6. CDC. Vital Signs: HIV prevention through care and treatment—United States. Morbidity Mortality Weekly Rep (MMWR). 2011;60:1621. Available at http://www.cdc.gov/mmwr/preview/mmwrhtml/mm6047a4.htm
7. CDC, the Health Resources and Services Administration, the National Institutes of Health, and the HIV Medicine Association of the Infectious Diseases Society of America. Incorporating HIV prevention into the medical care of persons living with HIV. MMWR Recomm Rep. 2003;52:1–24.
8. Marks G, Richardson JL, Crepaz N, et al.. Are HIV care providers talking with patients about safer sex and disclosure?: a multi-clinic assessment. AIDS. 2002;16:1953–1957.
9. Gilliam PP, Straub DM. Prevention with positives: a review of published research, 1998-2008. J Assoc Nurses AIDS Care. 2009;20:92–109.
10. Rose CD, Koester KA, Kang Dufour MS, et al.. Messages HIV clinicians use in prevention with positives interventions. AIDS Care. 2012;24:704–711.
11. Amico KR, Harman JJ, Johnson BT. Efficacy of antiretroviral therapy adherence interventions: a research synthesis of trials, 1996 to 2004. J Acquir Immune Defic Syndr. 2006;41:285–297.
12. Simoni JM, Pearson CR, Pantalone DW, et al.. Efficacy of interventions in improving highly active antiretroviral therapy adherence and HIV-1 RNA viral load. A meta-analytic review of randomized controlled trials. J Acquir Immune Defic Syndr. 2006;43(suppl 1):S23–S35.
13. Johnson BT, Carey MP, Chaudoir SR, et al.. Sexual risk reduction for persons living with HIV: research synthesis of randomized controlled trials, 1993 to 2004. J Acquir Immune Defic Syndr. 2006;41:642–650.
14. Crepaz N, Lyles CM, Wolitski RJ, et al.. Do prevention interventions reduce HIV risk behaviours among people living with HIV? A meta-analytic review of controlled trials. AIDS. 2006;20:143–157.
15. Bell SG, Newcomer SF, Bachrach C, et al.. Challenges in replicating interventions. J Adolesc Health. 2007;40:514–520.
16. Johnson MO, Charlebois E, Morin SF, et al.. Effects of a behavioral intervention on antiretroviral medication adherence among people living with HIV: the healthy living project randomized controlled study. J Acquir Immune Defic Syndr. 2007;46:574–580.
17. Gordon CM. Commentary on meta-analysis of randomized controlled trials for HIV treatment adherence interventions. Research directions and implications for practice. J Acquir Immune Defic Syndr. 2006;43(suppl 1):S36–S40.
18. Glasgow RE, Klesges LM, Dzewaltowski DA, et al.. The future of health behavior change research: what is needed to improve translation of research into health promotion practice? Ann Behav Med. 2004;27:3–12.
19. Fisher JD, Fisher WA, Amico KR, et al.. An information-motivation-behavioral skills model of adherence to antiretroviral therapy. Health Psychol. 2006;25:462–473.
20. Prochaska JO, Redding CA, Harlow LL, et al.. The transtheoretical model of change and HIV prevention: a review. Health Educ Q. 1994;21:471–486.
21. Bandura A. Health promotion by social cognitive means. Health Educ Behav. 2004;31:143–164.
22. Miller WR, Rollnick S. Motivational Interviewing. New York, NY: Guildford Press; 1991.
23. Haddad M, Inch C, Glazier RH, et al.. Patient support and education for promoting adherence to highly active antiretroviral therapy for HIV/AIDS. Cochrane Database Syst Rev. 2000;3:CD001442.
24. Fisher JD, Cornman DH, Osborn CY, et al.. Clinician-initiated HIV risk reduction intervention for HIV-positive persons: formative Research, Acceptability, and Fidelity of the Options Project. J Acquir Immune Defic Syndr. 2004;37:S78–S87.
25. Richardson JL, Milam J, McCutchan A, et al.. Effect of brief safer-sex counseling by medical providers to HIV-1 seropositive patients: a multi-clinic assessment. AIDS. 2004;18:1179–1186.
26. O'Donnell CR, O'Donnell L, San Doval A, et al.. Reductions in STD infections subsequent to an STD clinic visit. Using video-based patient education to supplement provider interactions. Sex Transm Dis. 1998;25:161–168.
27. Kurth A, Clausen M, Moore A. Formative Research for a Computer Counseling Intervention to Support Antiretroviral Adherence. Paper presented at: A State of the Science Meeting on Intervention Research to Improve Anti-Retroviral Adherence, Yale University; November 11, 2005; New Haven, CT.
28. Thibault JM, Steiner RW. Efficient identification of adults with depression and dementia. Am Fam Physician. 2004;70:1101–1110.
29. Skeels MM, Kurth A, Clausen M, et al.. CARE+ user study: usability and attitudes towards a tablet pc computer counseling tool for HIV+ men and women. AMIA Annu Symp Proc. 2006;729–733.
30. Walsh JC, Mandalia S, Gazzard BG. Responses to a 1 month self-report on adherence to antiretroviral therapy are consistent with electronic data and virological treatment outcome. AIDS. 2002;16:269–277.
31. Lu M, Safren SA, Skolnik PR, et al.. Optimal recall period and response task for self-reported HIV medication adherence. AIDS Behav. 2007;1:86–94.
32. Core TR. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2013. Available at: http://www.R-project.org
. ISBN: 3-900051-07-0. Accessed November 15, 2013.
33. Halekoh U, Højsgaard S, Yan J. The R Package geepack for generalized estimating equations. J Stat Softw. 2006;15:1–11.
34. Højsgaard S, Halekoh U, Robison-Cox J, et al.. doBy—Groupwise summary statistics, general linear contrasts,population means (least-squares-means), and other utilities. 2012. Available at: http://CRAN.R-project.org/package=doBy
. R package version 4.5–5. Accessed November 15, 2013.
35. Wickham H. ggplot2: Elegant Graphics for Data Analysis. New York, NY: Springer; 2009.
36. Cumming G, Fidler F, Vanx DC, et al.. Error bars in experimental biology. J Cell Biol. 2007;177:7–11.
37. Kahle E, Zhang Q, Golden M, et al.. Trends in evaluation for sexually transmitted infections among HIV-infected people, King County, Washington. Sex Transm Dis. 2007;34:940–946.
38. Myers JJ, Rose CD, Shade SB, et al.. Sex, risk and responsibility: provider attitudes and beliefs predict HIV transmission risk prevention counseling in clinical care settings. AIDS Behav. 2007;11(suppl 5):S30–S38.
39. Wantland DJ, Portillo CJ, Holzemer WL, et al.. The effectiveness of Web-based vs. non-Web-based interventions: a meta-analysis of behavioral change outcomes. J Med Internet Res. 2004;6:e40.
40. DiIorio C, Resnicow K, McDonnell M, et al.. Using motivational interviewing to promote adherence to antiretroviral medications: a pilot study. J Assoc Nurses AIDS Care. 2003;14:52–62.
41. Johnson MO, Catz SL, Remien RH, et al.. Theory-guided, empirically supported avenues for intervention on HIV medication nonadherence: findings from the Healthy Living Project. AIDS Patient Care STDS. 2003;17:645–656.
42. Bangsberg DR, Bronstone A, Chesney MA, et al.. Computer-assisted self-interviewing (CASI) to improve provider assessment of adherence in routine clinical practice. J Acquir Immune Defic Syndr. 2002;31(suppl 3):S107–S111.
43. Grimley DM, Bachmann LH, Jenckes MW, et al.. Provider-delivered, theory-based, individualized prevention interventions for HIV positive adults receiving HIV comprehensive care. AIDS Behav. 2007;5:S39–S47.
44. Zuniga ML, Baldwin H, Uhler D, et al.. Supporting Positive Living and Sexual Health (SPLASH): a clinician and behavioral counselor risk-reduction intervention in a university-based HIV clinic. AIDS Behav. 2007;11(suppl 5):S58–S71.
45. Smith DM, Drumright LN, Frost SD, et al.. Characteristics of recently HIV-infected men who use the Internet to find male sex partners and sexual practices with those partners. J Acquir Immune Defic Syndr. 2006;43:582–587.
46. Wight RG, Rotheram-Borus MJ, Klosinski L, et al.. Screening for transmission behaviors among HIV-infected adults. AIDS Educ Prev. 2000;12:431–441.
47. Gross M, Holte SE, Marmor M, et al.. Anal sex among HIV-seronegative women at high risk of HIV exposure. The HIVNET Vaccine Preparedness Study 2 Protocol Team. J Acquir Immune Defic Syndr. 2000;24:393–398.
48. Gilbert P, Ciccarone D, Gansky SA, et al.. Interactive “Video Doctor” counseling reduces drug and sexual risk behaviors among HIV-positive patients in diverse outpatient settings. PLoS One. 2008;3:e1988.
49. Serovich JM, Reed S, Grafsky EL, et al.. An intervention to assist men who have sex with men disclose their serostatus to casual sex partners: results from a pilot study. AIDS Educ Prev. 2009;21:207–219.
50. Naar-King S, Outlaw AY, Sarr M, et al.. Motivational Enhancement System for Adherence (MESA): pilot randomized trial of a brief computer-delivered prevention intervention for youth initiating antiretroviral treatment. J Pediatr Psychol. 2013;38:638–648.
51. Remien RH, Mellins CA, Robbins RN, et al.. Masivukeni: Development of a multimedia based antiretroviral therapy adherence intervention for counselors and patients in South Africa. AIDS Behav. 2013;17:1979–1991.
52. Lightfoot M, Rotheram-Borus MJ, Comulada S, et al.. Self-monitoring of behaviour as a risk reduction strategy for persons living with HIV. AIDS Care. 2007;19:757–763.
53. Lightfoot M, Rotheram-Borus MJ, Comulada WS, et al.. Efficacy of brief interventions in clinical care settings for persons living with HIV. J Acquir Immune Defic Syndr. 2010;53:348–356.
54. Fisher JD, Amico KR, Fisher WA, et al.. Computer-based intervention in HIV clinical care setting improves antiretroviral adherence: the LifeWindows Project. AIDS Behav. 2011;15:1635–1646.
55. Grimley DM, Hook EW III. A 15-minute interactive, computerized condom use intervention with biological endpoints. Sex Transm Dis. 2009;36:73–78.
56. Marsch LA, Grabinski MJ, Bickel WK, et al.. Computer-assisted HIV prevention for youth with substance use disorders. Subst Use Misuse. 2011;46:46–56.
57. Peipert JF, Redding CA, Blume JD, et al.. Tailored intervention to increase dual-contraceptive method use: a randomized trial to reduce unintended pregnancies and sexually transmitted infections. Am J Obstet Gynecol. 2008;198:630.e1–630.e8.
58. Noar SM. Computer technology-based interventions in HIV prevention: state of the evidence and future directions for research. AIDS Care. 2011;23:525–533.
59. Freedberg KA, Hirschhorn LR, Schackman BR, et al.. Cost-effectiveness of an intervention to improve adherence to antiretroviral therapy in HIV-infected patients. J Acquir Immune Defic Syndr. 2006;43(suppl 1):S113–S118.
60. Frick PTK, Grant P, Novotny M, et al.. The effect of a multidisciplinary program on HAART adherence. AIDS Patient Care and STDS. 2006;20:511–524.
61. Remien RH, Exner TM, Morin SF, et al.. Medication adherence and sexual risk behavior among HIV-infected adults: implications for transmission of resistant virus. AIDS Behav. 2007;11:663–675.
62. Beauclair R, Meng F, Dreprez N, et al.. Evaluating audio computer assisted self-interviews in urban South African communities: evidence for good suitability and reduced social desirability bias of a cross sectional survey on sexual behaviour. BMC Med Res Methodol. 2013;13:11.
63. Mackenzie SL, Kurth AE, Spielberg F, et al.. Patient and staff perspectives on the use of a computer counseling tool for HIV and sexually transmitted infection risk reduction. J Adolesc Health. 2007;40:572e9–e16.
64. Simoni JM, Amico KR, Smith L, et al.. Antiretroviral adherence interventions: translating research findings to the real world clinic. Curr HIV/AIDS Rep. 2010;7:44–51. 2007;40:572.e9–572.e16.
65. Kalichman SC, Di Berto G, Eaton L. Human immunodeficiency virus viral load in blood plasma and semen: review and implications of empirical findings. Sex Transm Dis. 2008;35:55–60.
66. Mugavero MJ, Amico KR, Horn T, et al.. The state of engagement in HIV care in the United States: from cascade to continuum to control. Clin Infect Dis An Official Publ Infect Dis Soc America. 2013;57:1164–1171.