HIV disproportionately affects Latinx sexual minority men (LSMM) and
transgender women (LTGW). In Los Angeles County, home to one of the country's largest HIV epidemics and location of this study, the estimated HIV prevalence is 15% among LSMM and 17% among LTGW. The U.S. Centers for Disease Control and Prevention (CDC) estimates that Latinx adults are less likely than other groups to have their infections diagnosed, 1 posing a serious public health problem because the undiagnosed population contributes to nearly 40% of new HIV infections. 2 A 2019 CDC report underscores the need to increase HIV testing among populations at high risk for HIV and to promote testing at least once every 3 months. 3 3
and other studies 3,4 have shown that LSMM and LTGW face challenges in accessing HIV prevention information and testing services. A key barrier is lack of access to regular medical care, a problem more prevalent among undocumented populations, who may avoid perceived authority figures, such as health care providers, fearing that interaction with the health care system could increase scrutiny of their immigration status. Using text messages offers a unique opportunity to remain in contact with LSMM and LTGW, vulnerable communities that can be difficult to reach with traditional means. Mobile technologies have demonstrated potential to increase HIV knowledge and testing frequency, 5–12 but there is increasing evidence that LSMM and LTGW refuse to engage with these technologies 13–18 or abandon them quickly, 5 a phenomenon frequently referred to as “technology fatigue.” 19–21
Despite the CDC advocating for increased screening of HIV for people at heightened risk for HIV,
often testing is sought only after an experience that puts an individual at increased risk for HIV (such as an episode of unprotected sex). Engaging adults when they come in to get tested for HIV—especially undocumented LSMM and LTGW who may be more likely to avoid health care services because of their documentation status—presents a unique opportunity to connect with individuals at high risk for HIV and create an ongoing line of communication. 3
Behavioral economics (BEs), a strand of economics that incorporates insights from psychology, holds promise to address technology fatigue and increase the effectiveness of text messages aiming to improve critical prevention behaviors. BE studies people's decision-making environment and identifies sources for systematic errors or “biases.” Two such biases help explain why some people fail to regularly test for HIV. First, HIV prevention behaviors are not salient. Individuals at risk of HIV over time may “forget” about the threat of acquiring HIV as other concerns absorb their attention.
Second, present bias occurs when people value current benefits disproportionally more than future benefits. 22 This bias is particularly detrimental for health prevention behaviors whose benefits are reaped in the future (eg, improved life expectancy from avoiding an HIV infection) but whose costs, such as going to the clinic, occur in the present. 23 Moreover, biases tend to be exacerbated when stress levels are higher, 22 and LSMM and LTGW often experience high levels of stress because of discrimination and stigma. 24–26
In this study, we developed and piloted Mobile Technology and Incentives (MOTIVES), an intervention that uses the principles of BE to target lack of salience and present bias, aims to remain in contact with clients coming for HIV tests, provides them with up-to-date prevention information, and encourages them to return for HIV testing at least once every 3 months. We enrolled HIV-negative LSMM and LTGW to receive weekly messages with information on HIV prevention and reminders to test for HIV to counter salience. We randomly assigned half the participants to also be eligible for prize drawings conditional on HIV testing every 3 months to counter present bias. Participants in this group further received weekly quizzes about the information they received through text messages to keep the prize drawings salient, and correct responses increased the odds of winning a prize at testing visits, motivating participants to engage with the mobile intervention. We compared MOTIVES intervention participants to clients of our study partner Bienestar, who were not offered the intervention but had the same eligibility criteria.
Information on the study and the parallel design used have been published elsewhere
although all details from the CONSORT checklist 3,27,28 are described briefly here. We partnered with Bienestar Human Services, Inc (hereafter referred to as Bienestar), a primarily Latinx-focused organization located across Los Angeles County. Bienestar provides a range of services including case management, substance use, mental health, HIV testing, and other prevention services [such as pre-exposure prophylaxis (PrEP) navigators to help link interested clients to clinics prescribing PrEP as Bienestar does not prescribe it]. Bienestar staff recruited participants coming to Bienestar for HIV testing. Potentially eligible participants who tested negative for HIV and met eligibility criteria were approached by study staff (all of whom were HIV testers at Bienestar) to provide an in-depth description of the study and enroll interested individuals. The HIV testers conducted surveys, and RAND staff sent the text messages and quizzes and conducted the prize drawings remotely. All participating institutions received approval from their institutional review boards, a National Institute of Mental Health certificate of confidentiality was obtained, and the study was judged to be low risk. The trial was registered on May 5, 2017, with the 29 registry (NCT03144336). ClinicalTrials.gov
We recruited 218 participants from 6 Bienestar locations during routine visits. Eligibility criteria included being HIV negative, owning or having regular mobile phone access, self-identifying as SMM or TGW and Latinx, 18 years of age or older, and fluency in English or Spanish. Written consent was obtained from all study participants in the intervention arms; however, no consent was required to obtain testing data for individuals in the comparison group.
On completing HIV tests, study staff enrolled 218 individuals in the 2 intervention groups; before recruitment, preassigned identification number (IDs) were stratified by a Bienestar site and by LSMM or LTGW. Blocks of 20 for 80 LSMM and 40 for LTGW IDs were randomized to the 2 groups with a 1:1 ratio using a random machine generator. Recruitment took place late May 2017 through early April 2018.
We enrolled participants into 1 of 2 intervention arms. Participants in the information only (IO) group (99 assigned, n = 91 analyzed) received weekly text messages with HIV prevention information to keep the risk of HIV salient, that is, high on an individual's mental priority list. The information plus (IP) arm layered on components rooted in BEs to make information more salient and to counter present bias. Participants in the IP group (119 assigned, n = 107 analyzed) also had a chance to win small prizes (eg, $50 gift cards from Target, and other stores that participants requested during formative interviews
—importantly participants had asked that gift cards be used for fun purchases rather than practical needs). Participation in the prize drawings was conditional on testing at least once every 3 months. The objective of the incentives was to counter present bias by rewarding behavior in the present that usually does not provide benefit until years into the future (ie, improved life expectancy and absence of HIV-related disease). 28 In addition, participants were asked to reply by text to short quizzes on the texted information to make the information more salient. Correctly answering questions improved the chance of winning a prize at the next testing visit from 1:10 if no correct answers to 1:5 if all correct. We had HIV testing data on participants regardless of whether they continued the intervention, and we analyzed data for all clients even if they stopped engaging with or dropped out of the intervention (n = 52); of these, 29 never responded to the initial text message to actively begin the study, and 11 had wrong or missing phone numbers. The only reason we did not analyze data from any of the 218 clients assigned to an intervention arm was missing records in the HIV testing data (n = 12 in IP and n = 8 in IO). This is likely the results of entry errors in the medical record ID that made it so we could not link our participants to their testing data. We expect these errors to be uncorrelated with HIV testing, and therefore, excluding these cases should not bias our results. 30 Figure 1 describes the total sample from recruitment through analysis. FIGURE 1.:
CONSORT flow diagram.
To construct a comparison group, we used electronic medical records to identify 540 clients who came to Bienestar for HIV testing during the intervention period who had the same eligibility criteria as study participants but were not offered the intervention. We used coarsened exact matching to select a sample from the 540 with characteristics that were highly correlated with HIV testing (transgender, US citizenship, and identify as White).
We conducted analyses using both the full comparison group (reported) and the coarsened exact matching matched comparison group, with no significant differences in the 2 approaches. 31
Our main analysis compares the probability of HIV testing at any Bienestar site between the comparison group and the 2 treatment arms. We first pooled all postrecruitment intervals to assess treatment effects averaged over all time periods. We also separately estimated treatment effects in each 3-month interval (1–3, 4–6, 7–9, and 10–12 months) to assess whether and how the intervention effect evolved over time. We used linear probability models for both analyses; we clustered standard errors by individual to adjust for correlation of repeated observations within individuals over time.
We conducted one unadjusted and one adjusted analysis; the latter controlled for age, classification as being at high risk for HIV (extracted from medical records and defined as SMM reporting sex without a condom, SMM reporting meth use, anyone reporting having sex with an HIV-positive methamphetamine use partner, and/or anyone reporting injection drug use), race, sexual orientation, gender identity, insurance status, and citizenship. 32
We measured change in HIV prevention knowledge using the HIV Knowledge Questionnaire (HIV-KQ-18)
which covers a range of HIV prevention topics directly related to the information provided through our text messages (eg, HIV transmission risks based on behaviors, presence of sexually transmitted infection, and level of viral load; PrEP and post-exposure prophylaxis; and differences between oral and anal sex practices). We used this questionnaire at baseline and 12 months and used ordinary least squares regression to compare the number of correct responses (out of 13) between the intervention arms. We do not have this outcome for the comparison group, who did not fill out surveys. Only participants who remained enrolled in the study and completed a final survey after 12 months were included in this analysis (n = 198). 33 RESULTS
MOTIVES intervention participants were similar to the comparison group in key demographic and clinical characteristics (
Table 1). The intervention groups were more likely to self-identity as Latinx and White and identify as TGW. As a result, we control for these variables in our adjusted models. Both treatment arms had relatively high levels of HIV prevention knowledge at baseline, correctly answering nearly 80% of questions, with no change over time in the IO group (77.8%–79.2%; P = 0.678) but a significant 8.9% point increase in the IP group (79.2%–88.1%; P = 0.007; Table 1).
TABLE 1. -
Summary Statistics by the Treatment Arm
High risk for HIV
Gay, lesbian, queer
Full time employed
Change in HIV knowledge
from pre to post (percentage points) ‡ N/A
Tested for HIV in 3-month interval
Data are from medical records.
*All participants identified as Latino or Hispanic, but there was variation in ethnic identification within these categories. †High risk status was determined by Bienestar staff based on the client's risk factors including SMM reporting sex without a condom, SMM reporting meth use, anyone reporting having sex with an HIV-positive partner, and/or anyone reporting injection drug use. ‡HIV knowledge is measured as the difference in the share of questions answered correctly at baseline and endline. §Difference between pre and post is significant at the 5% level. ‖Tested for HIV in a 3-month interval is the average across all postrecruitment 3-month intervals.
MOTIVES intervention participants were significantly more likely to test for HIV in any given 3-month interval compared with the comparison group.
Figure 2A shows that 22.0% of participants in the IO group and 24.9% of the IP group tested at a Bienestar site in a given quarter after the intervention started compared with 13.0% in the comparison group. This represents unadjusted relative risk ratios (RRRs) of 1.69 for the IO group [95% confidence interval (CI): 1.25 to 2.10; P < 0.01] and 1.91 for the IP group (95% CI: 1.51 to 2.31; P < 0.01), respectively. The adjusted model produced RRRs of 1.90 for the IO group (95% CI: 1.29 to 2.51; P < 0.01) and 2.41 for the IP group (95% CI: 1.83 to 2.98; P < 0.01), respectively. The pilot study was not powered to detect differences between the 2 treatment arms, but differences were clinically important, with unadjusted RRRs of 1.13 (95% CI of RRRs: 0.80 to 1.46; P = 0.428) and adjusted RRRs of 1.25 (95% CI of RRRs: 0.87 to 1.63; P = 0.190). The effect of MOTIVES diminished slightly over time but remained positive and significant even in the last 3-month interval ( Fig. 2B). However, we were not powered to detect statistical differences between time periods and cannot reject that there is no change in the effect over time. MOTIVES intervention participants were also more likely than the comparison group to have multiple HIV tests in the 12 months after recruitment: only 13.7% of the comparison group tested multiple times, compared with 26.3% in the IO group and 30.8% in the IP group. FIGURE 2.:
HIV testing results (per 3-month interval). A, The share of participants that tested for HIV at a Bienestar site in a given 3-month interval (all 4 intervals pooled). Brackets are 95% CIs of the difference between the treatment arms and the comparison arm, estimated using a linear probability model with standard errors clusters by individual. B, The difference between each treatment arm and the comparison arm in each of the 3-month intervals with 95% CIs estimated in the same way as in A.
Our study found the frequency of HIV testing was higher in both intervention groups compared with the comparison group, demonstrating that a simple, low-cost intervention may help increase HIV testing frequency among LSMM and LTGW. The text messages contained information about HIV prevention and risk. We found no evidence that knowledge of HIV risk behaviors improved in the IO group; however, it did increase in the IP group. This finding suggests that the intervention may have affected HIV testing either because it reinforced existing knowledge or because engaging with the new information made HIV risk factors more salient. The finding also suggests that quizzes and associated rewards may be helpful tools for effectively providing HIV information. Of note, although not explicitly included as an outcome in our pilot, our results show a statistically significant increase in PrEP uptake. Although it is possible that the information provided through text messages helped participants recognize the importance of initiating PrEP, further research—including biological reporting of PrEP outcomes rather than by self-report—is needed to better understand these results.
The BE components associated with the IP arm are encouraging. The difference in HIV testing frequency between the IP and IO arms was clinically important (25% increase in the probability of testing at a Bienestar site in a given 3-month interval in the adjusted model). This is consistent with other work that documents the power of incentives in the context of HIV.
However, in this pilot study, we were not powered to detect statistically significant differences between treatment arms, and therefore, we cannot reject that the BE components do not provide extra benefit. Future work should test the intervention arms at scale to assess the added value of the BE components more precisely. 34,35
Several limitations should be noted. First, we do not have data on out-of-network testing, limiting our ability to measure if and when people tested for HIV outside of Bienestar clinics. However, our anecdotal evidence from HIV testers at Bienestar suggest that out-of-network testing among Bienestar clients is rare. Second, we do not have a randomly assigned comparison group and selection into the treatment arms could have been correlated with testing frequency. Adding variables that are predictive of testing, using a statistical approach based on matching of observables, and limiting our analysis to those in the comparison group with the same eligibility criteria as MOTIVES intervention participants make our results more convincing. Relatedly, we noted that
transgender women were more likely to be in the intervention arms. This may be due to the fact that transgender individuals face a range of structural challenges (eg, unemployment and housing instability), increasing the likelihood of participation compared with other communities (such as SMM) because their need for supportive programs are substantial. Third, the comparison group was not informed of the study (data were deidentified secondary data), and thus part of our results could be explained by the Hawthorne effect because only the treatment arms knew they were being observed. Fourth, we do not have information on PrEP use in the comparison arm as the information was not collected at the time of the study in Bienestar's electronic medical record system. It is likely that PrEP use was similar across arms, although biological markers (by urine or blood spots) should be collected in future research to more accurately evaluate its uptake as being on PrEP can influence HIV testing frequency. Only a study at scale including a true, randomized control group can overcome these limitations. Finally, we were missing HIV testing data on 12 of our study participants (likely the result of data entry errors). Although we expect these data entry errors to be random, nonrandom missingness could bias our estimates in unpredictable ways. 36
Our findings suggest that simple text messages combined with small incentives based on BE insights may have the potential to positively impact critical prevention behaviors such as HIV testing, warranting a fully‐powered investigation to determine the intervention's full effect.
1. Division of HIV and STD Programs, Los Angeles County Department of Public Health. Transgender Population Estimates 2-12-13. Los Angeles County, CA: Los Angeles County Department of Public Health; 2013. Available at:
. Accessed May 29, 2020.
2. Centers for Disease Control and Prevention. Recommendations for HIV Screening of Gay, Bisexual, and Other Men who have Sex with Men‐United States, 2017. Available at:
. Accessed May 29, 2020.
3. Barreras J, Linnemayr S, MacCarthy S. “We have a stronger survival mode”: exploring knowledge gaps and culturally sensitive messaging of PrEP among Latino men who have sex with men and Latina
in Los Angeles, CA. AIDS Care. 2019;31:1221–1227.
4. Brooks RA, Landrian A, Nieto O, et al. Experiences of anticipated and enacted pre-exposure prophylaxis (PrEP) stigma among latino MSM in Los Angeles. AIDS Behav. 2019;23:1964–1973.
5. Brown JL, Sales JM, DiClemente RJ. Combination HIV prevention interventions: the potential of integrated behavioral and biomedical approaches. Curr HIV/AIDS Rep. 2014;11:363–375.
6. Galindo GR, Walker JJ, Hazelton P, et al. Community member perspectives from
and men who have sex with men on pre-exposure prophylaxis as an HIV prevention strategy: implications for implementation. Implement Sci. 2012;7:116.
7. Mantell JE, Sandfort TG, Hoffman S, et al. Knowledge and attitudes about preexposure prophylaxis (PrEP) among sexually active men who have sex with men and who participate in New York City Gay Pride events. LGBT Health. 2014;1:93–97.
8. Krakower DS, Mimiaga MJ, Rosenberger JG, et al. Limited awareness and low immediate uptake of pre-exposure prophylaxis among men who have sex with men using an internet social networking site. PLoS One. 2012;7:e33119.
9. Bauermeister JA, Meanley S, Pingel E, et al. PrEP awareness and perceived barriers among single young men who have sex with men in the United States. Curr HIV Res. 2013;11:520.
10. Jaiswal J, Griffin M, Singer SN, et al. Structural barriers to pre-exposure prophylaxis use among young sexual minority men: the P18 cohort study. Curr HIV Res. 2018;16:237–249.
11. Lelutiu-Weinberger C, Golub SA. Enhancing PrEP access for Black and Latino men who have sex with men. J Acquir Immune Defic Syndr. 2016;73:547.
12. Rhodes SD, Hergenrather KC, Vissman AT, et al. Boys must be men, and men must have sex with women: a qualitative CBPR study to explore sexual risk among African American, Latino, and white gay men and MSM. Am J Mens Health. 2011;5:140–151.
13. Lim MSC, Hocking JS, Hellard ME, et al. SMS STI: a review of the uses of mobile phone text messaging in sexual health. Int J STD AIDS. 2008;19:287–290.
14. Sun CJ, Stowers J, Miller C, et al. Acceptability and feasibility of using established geosocial and sexual networking mobile applications to promote HIV and STD testing among men who have sex with men. AIDS Behav. 2015;19:543–552.
15. Newell A. A mobile phone text message and Trichomonas vaginalis. Sex Transm Infect. 2001;77:225.
16. Muessig KE, Pike EC, LeGrand S, et al. Mobile phone applications for the care and prevention of HIV and other sexually transmitted diseases: a review. J Med Internet Res. 2013;15:e1.
17. Sullivan PS, Grey JA, Rosser BRS. Emerging technologies for HIV prevention for MSM: what we’ve learned, and ways forward. J Acquir Immune Defic Syndr. 2013;63:S102–S107.
18. Dhar J, Leggat C, Bonas S. Texting—a revolution in sexual health communication. Int J STD AIDS. 2006;17:375–377.
19. Pop-Eleches C, Thirumurthy H, Habyarimana JP, et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders. AIDS. 2011;25:825–834.
20. Free C, Whittaker R, Knight R, et al. Txt2stop: a pilot randomised controlled trial of mobile phone-based smoking cessation support. Tob Control. 2009;18:88–91.
21. Bramley D, Riddell T, Whittaker R, et al. Smoking cessation using mobile phone text messaging is as effective in Maori as non-Maori. N Z Med J. 2005;118:U1494.
22. Linnemayr S. HIV prevention through the lens of behavioral economics. J Acquir Immune Defic Syndr. 2015;68:e61–e63.
23. Coomes CM, Lewis MA, Uhrig JD, et al. Beyond reminders: a conceptual framework for using short message service to promote prevention and improve healthcare quality and clinical outcomes for people living with HIV. AIDS Care. 2012;24:348–357.
24. James SE, Herman JL, Rankin S, et al. The Report of the 2015 US Transgender Survey. Washington, DC: National Center for Transgender Equality; 2016.
25. Rodriguez A, Agardh A, Asamoah BO. Self-reported discrimination in health-care settings based on recognizability as transgender: a cross-sectional study among transgender U.S. citizens. Arch Sex Behav. 2018;47:973–985.
26. Cruz TM. Assessing access to care for transgender and gender nonconforming people: a consideration of diversity in combating discrimination. Soc Sci Med. 2014;110:65–73.
27. Linnemayr S, MacCarthy S, Kim A, et al. Behavioral economics-based incentives supported by mobile technology on HIV knowledge and testing frequency among Latino/a men who have sex with men and
: protocol for a randomized pilot study to test intervention feasibility and acceptability. Trials. 2018;19:540.
28. MacCarthy S, Barreras JL, Mendoza-Graf A, et al. Strategies for improving mobile technology-based HIV prevention interventions with Latino men who have sex with men and Latina
. AIDS Educ Prev. 2019;31:407–420.
29. CONSORT. The CONSORT Statement. Ottawa, ON: The CONSORT Group; 2010. Available at:
. Accessed May 29 2020.
30. White JS, Dow WH. Intertemporal choices for health. Behav Econ Public Health. 2015;27:62.
31. Iacus SM, King G, Porro G. Causal inference without balance checking: coarsened exact matching. Polit Anal. 2012;20:1–24.
32. Cameron AC, Miller DL. A practitioner's guide to cluster-robust inference. J Hum Resour. 2015;50:317–372.
33. Carey MP, Schroder KE. Development and psychometric evaluation of the brief HIV Knowledge Questionnaire. AIDS Educ Prev. 2002;14:172–182.
34. Bor J, Thirumurthy H. Bridging the efficacy–effectiveness gap in HIV programs: lessons from economics. J Acquir Immune Defic Syndr. 2019;82:S183–S191.
35. Galárraga O, Genberg BL, Martin RA, et al. Conditional economic incentives to improve HIV treatment adherence: literature review and theoretical considerations. AIDS Behav. 2013;17:2283–2292.
36. McCambridge J, Witton J, Elbourne DR. Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol. 2014;67:267–277.