Behavioral interventions for HIV prevention have advanced greatly over the past 25 years. Reviews and meta-analyses of the literature demonstrate that efficacious individual-level and group-level interventions to increase condom use and reduce incident sexually transmitted diseases (STD) have been developed for numerous at-risk and infected populations [1–4]. Virtually all of these interventions, however, require human facilitators for their implementation. For example, a recent review of 18 ‘best evidence’ interventions revealed that all 18 necessitated a counselor, health educator, or some other facilitator [4]. Given the documented fact that clinical and community settings often do not have the human resources to devote to such interventions [5–7], the ability to disseminate efficacious interventions into many practice settings remains limited. Additional challenges to disseminating interventions include barriers such as length or format [7–9] and translational challenges with regard to intervention fidelity [10–13]. To date, these barriers and challenges have limited the public health impact of HIV prevention behavioral interventions.
A promising avenue of research with the capacity to correct for many of these problems is computer technology-based HIV prevention interventions. Such interventions are those that use computer technology as the primary or sole medium from which to deliver an intervention [14]. These applications hold much promise for several reasons, including the cost of implementing such interventions once they have been developed is minimal compared with those requiring significant human resources; intervention fidelity is maintained through the standardization of content; computerized interventions can individually customize intervention content through the use of computer algorithms; computer technologies include features such as interactivity and multimedia, which may aid in the fostering of behavioral change; and computerized interventions tend to be brief as well as flexible in terms of dissemination channels, which might include community-based agencies, clinical settings, and the Internet [15,16].
An increasing number of applications of computer technology to HIV prevention have been tested in a series of recent randomized, controlled trials. This literature contains a variety of applications, including interventions tailored to individual risk characteristics [17,18], those targeted on group characteristics [19], and interactive video interventions, which simulate ‘virtual’ decision-making experiences [20,21]. Such interventions have been tested with heterosexual adolescents and young adults [17,19,22], men who have sex with men (MSM) [21,23], and at-risk women [18,24]. These interventions have been delivered on screen, over the Internet, and via computer-tailored print materials. Such studies have provided the first tests of the potential efficacy of computer technology-based interventions in modifying HIV-related sexual risk behavior.
A critical question is whether these computer-based interventions are efficacious in reducing sexual risk behavior. If so, such interventions would hold much promise for increasing the delivery of HIV prevention messages to a variety of at-risk audiences. Thus, the purpose of the current study was to conduct a meta-analysis of these intervention trials. Three research questions were posed. First, are computer technology-based interventions efficacious in changing sexual risk behavior? Second, how does the mean-weighted effect size of such interventions compare to that of human-delivered HIV prevention interventions? And third, what are the moderators of efficacious computer technology-based interventions? Considered here were a number of participant, intervention, and methodological factors that have the potential to moderate the efficacy of these interventions.
Method
Search strategy
In order to ensure a comprehensive search, a detailed strategy to search for published and unpublished studies relevant to this meta-analysis was undertaken. The intent was to locate all studies through March of 2008 that were applicable to this meta-analysis. First, comprehensive searches of the Medline and PsycINFO databases were conducted. Numerous keywords were used in combination in the search, including computer, technology, Internet, cell phone, personal digital assistant (PDA), chat room, tailor, HIV, AIDS, prevention, condom use, safe(r) sex, program, and intervention. Second, forward citation searches were conducted on all articles located in the PsycINFO and Medline searches, using the Social Science Citation Index. These searches identified any published studies that cited any of these previously located studies. Third, review articles in this area [6,14,25,26] were examined and any studies that had the possibility of being relevant to this meta-analysis were located. Fourth, once the final set of articles was identified for this study, each article's reference list was examined for possible additional relevant articles that might apply here.
In order to potentially include unpublished work in this meta-analysis, an e-mail message was developed soliciting unpublished studies for the project. An e-mail list of researchers who were (or were potentially) conducting work in this area was developed from all of the studies identified in all of the searches. The message was sent to a total of 44 researchers, and resulted in a total of 17 replies with accompanying information. This process ultimately resulted in five (largely unpublished) manuscripts, which contained the results of six studies, being added to the meta-analysis [23,24,27,28] (S. Bull, C. Ortiz, D. Vallejos, K. Pratte, M. McFarlane, submitted for publication).
All studies identified through the above searches were considered for inclusion in this meta-analysis. In order to be included, a study had to meet the following inclusion criteria. Studies had to:
- test the efficacy of an HIV prevention behavioral intervention focused on changing sexual risk behavior(s) in individuals of HIV-negative or unknown serostatus;
- measure condom use or unprotected sex as a dependent variable;
- use computer technology in the development or, delivery of the intervention, including desktop or laptop computers, the Internet, interactive video, cell phones or PDAs; and
- utilize an experimental design in which individuals were randomized to conditions (at least one computerized condition and one comparison condition).
Initial searches resulted in hundreds of abstracts that were examined for relevance. Approximately 49 articles that had the potential to be included in the meta-analysis were located and closely examined for relevance. Of these:
- 18 studies (37%) were excluded because they did not include any measures of unprotected sex or condom use as a behavioral outcome, for example [29];
- seven studies (14%) were excluded because they did not utilize a randomized design that included a control or comparison group, for example [30];
- six studies (12%) were excluded because they did not utilize computer technology, for example [31];
- five studies (10%) were excluded because they were not intervention studies, for example [32]; and
- one study (2%) was excluded because it was an intervention with HIV positive persons [33].
As a result, a final set of 12 studies (25%) met all the criteria and was included in the meta-analysis.
Article coding
Articles were coded on numerous dimensions by two independent coders. Features coded included demographic and sample characteristics; intervention characteristics such as intervention type, content, delivery, dose, and theoretical framework(s); and methodological characteristics including length of follow-up, study retention, and type of comparison group.
The coders and the first author met to discuss each article after it was coded to compare the two coders' work and discuss any discrepancies that were present. Intercoder reliability was calculated for each characteristic that was coded. Percentage agreement was calculated by dividing the number of agreed upon coding instances by the total, and was calculated for each coding category. Cohen's κ for intercoder reliability [34] was also calculated. Mean percentage agreement across all coding categories was 98%, or κ = 0.95. These figures indicated very good agreement among the coders. All discrepancies between coders were resolved through discussion between the two coders and the first author.
Effect size extraction and calculation
Cohen's d, or the difference in treatment and control means divided by the pooled standard deviation, was used as the effect size indicator [35–37]. Effect sizes were calculated from data reported in the article (for example, summary statistics, t-test) using appropriate formulas [37]. Articles that reported results in terms of percentages were converted to odds ratios (OR) and then to d following published guidelines [38]. Adjusted statistics were used in calculating effect sizes to ensure that any baseline differences among the groups were taken into account. Where adjusted statistics were not reported, group differences were adjusted for using baseline data contained in the study report.
In most cases, only one comparison condition existed for purposes of computing effect size estimates. In three cases, however, two potential comparison conditions existed. In those cases, the most-minimal intervention condition was used in order to most accurately estimate the ‘absolute’ effect of the computer-based intervention condition. All effect sizes were calculated from data based upon the longest term follow-up assessment for which data were provided. This allowed a meaningful period of time (in most cases 3–6 months) to pass after the intervention was implemented [4]. In order to keep effect sizes consistent, all studies in which the computerized condition outperformed the comparison were given a positive sign (+), whereas a negative sign (−) indicated that the reverse of this was true.
Effect sizes were calculated on the four outcomes reported in the studies – condom use (or unprotected sex), frequency of sexual behavior, number of sex partners, and incident STD. Outcomes reported in the studies varied greatly, and the key outcome reported across all the studies was condom use (k = 11) or unprotected sex (k = 3). Consistent with meta-analytic convention in this area [2,39], effect size was calculated by taking into account all reported measures of condom use and unprotected sex that were available. Given that most studies only reported data on condom use, this variable is hereafter referred to as condom use.
Meta-analytic approach
Effect sizes were weighted by sample size and combined using standard fixed and random effects meta-analytic procedures and are presented along with their 95% confidence intervals (CI) [37,40,41]. Because results were very similar in both sets of analyses, only fixed effects results are presented. The Q statistic was used to examine whether significant heterogeneity existed among the effect sizes. Effect sizes for the a priori hypothesized categorical moderators were calculated along with their 95% CI, and those effect sizes were statistically compared to one another using the QB statistic. In addition, in the case of continuous (i.e., interval-level) moderator variables, correlations were calculated between particular moderator variables and effect size. All analyses were conducted using Comprehensive Meta-Analysis software, Version 2.0, and Statistical Package for the Social Sciences (SPSS), Version 15.
Results
The k = 12 studies had a cumulative N = 4639 (median N per study = 319) and were published (or presented) between 2002 and 2008 (median = 2006; Table 1). All studies were conducted in the United States, except for one study conducted in the Netherlands [23]. Ten of 12 studies (83%) were of heterosexually active samples, whereas the remaining k = 2 were of MSM. Six studies (50%) were of mixed male/female samples, whereas k = 4 were female-only and k = 2 were male-only studies. Mean age across the 12 studies was 22.52 years (SD = 6.76).
;)
Table 1: Characteristics of studies included in the meta-analysis.
The most common intervention type was an individually tailored intervention (k = 6). This was followed by GROUP TARGETED (k = 3), virtual decision-making (k = 2), group targeted (k = 2) and mixed-type interventions (k = 1). Most interventions were delivered on-screen using a computer located on site (k = 8), whereas the remainder were delivered over the Internet (k = 3) or via computer-generated print materials (k = 1). Most of the interventions (k = 10) were theory based, with nine of the 10 (90%) being based on behavioral theories that have been commonly applied in HIV prevention [42]. The most commonly applied theory among these studies was the Stages of Change or ‘Transtheoretical’ Model [43]. Mean retention across the study trials was 70%.
Efficacy of interventions: condom use
All 12 studies reported outcomes on condom use, and the sample size-weighted mean effect size for this variable was d = 0.259 (95% CI = 0.201, 0.317; Z = 8.74, P < 0.001; N = 4639). This indicated that computer technology-based interventions have had statistically significant protective effects on condom use behavior (see Fig. 1). When the two studies with low retention (defined as <60%) were removed, the weighted mean effect size increased d = 0.308 (95% CI = 0.238–0.377; Z = 8.64; P < 0.001; N = 3243).
Fig. 1: Forest plot displaying effect sizes and 95% confidence intervals for condom use.
To examine the possibility that publication bias inflated the condom use effect size, two fail-safe N values were calculated. Orwin's method [41] suggested that 51 studies with nonsignificant findings would need to exist to reduce the d = 0.259 to a trivial effect size of d = 0.05, whereas Rosenthal's fail-safe N method [37] suggested that 203 studies with nonsignificant findings would need to exist in order to reduce the effect size to the point of nonsignificance (P > 0.05). These results suggest that a large number of null unpublished studies (that are not already included in this meta-analysis) would need to exist in order to nullify the current effect size.
How does this condom use effect size compare to meta-analytic effect sizes of human-delivered HIV prevention interventions? A recent analysis of meta-analyses of HIV prevention interventions across a number of at-risk populations [3] found that the mean weighted effect size typically achieved in interventions for condom use ranges from an odds ratio (OR) of 1.13 to 1.64. The condom use effect size found in the current study, d = 0.259, coverts to an OR = 1.54. In addition, the largest single meta-analysis of HIV prevention interventions to date [1] found an effect size for condom use of d = 0.18 (which can be compared to the current d = 0.259). These results suggest that the current effect size compares quite favorably with effect sizes from previous meta-analyses.
Efficacy of interventions: other outcomes
Three studies reported on frequency of sexual behavior and three reported on incident STD. The sample size-weighted mean effect size for sexual behavior was d = 0.427 (95% CI = 0.251, 0.602; Z = 4.78, P < 0.001; N = 515) and for incident STD was d = 0.140 (95% CI = 0.035, 0.245; Z = 2.61, P < 0.01; N = 1403). This indicated that interventions have reduced both frequency of sexual behavior and incident STD. Finally, two studies reported outcomes for number of sex partners. The sample size-weighted mean effect size for this variable was d = 0.422 (95% CI = 0.116, 0.728; Z = 2.70, P < 0.01; N = 168), indicating an effect of the interventions on reducing numbers of sexual partners (Table 2).
Table 2: Effect sizes, N, and 95% confidence intervals for studies in the meta-analysis.
Heterogeneity and intervention moderators
Next, heterogeneity of effect sizes of the computer-based interventions was examined. Statistical testing indicated significant heterogeneity among the studies with regard to the condom use outcome, Q11 = 25.62, P = 0.007. Thus, we next computed analyses to explore the potential impact of the a priori determined moderating variables on intervention efficacy for condom use (Table 3).
Table 3: Sample size-weighted effect sizes by categorical moderating variables.
Participant moderators were examined first. Correlations between condom use effect size and sex [r (11) = 0.07, P = 0.82], age [r (11) = 0.24, P = 0.57], and race [r (11) = 0.51, P = 0.09] failed to reach statistical significance at the P < 0.05. However, they suggested trends in that interventions were more efficacious among samples with fewer men, young people, and white participants. Categorical analysis of sex groups found that effect sizes for female-only interventions were the largest, followed by male-only, and finally by mixed sex groups. This contrast was statistically significant (QB = 8.62, degree of freedom (DF) = 2, P < 0.01). Comparison of heterosexually active and MSM populations found a slightly greater effect size for interventions directed at MSM. The contrast between these effect sizes was not statistically significant, however.
Intervention moderators were examined next. Interventions were significantly more efficacious when they used individualized tailoring (QB = 14.74, DF = 1, P < 0.001) and a Stages of Change model (QB = 13.51, DF = 1, P < 0.001). Analysis of intervention types suggested that group targeted interventions had the smallest effect sizes, followed by virtual decision-making interventions and then by tailored interventions. The contrast between these effect sizes was statistically significant, QB = 15.73, DF = 2, P < 0.001. Interventions with more sessions were also found to be significantly more efficacious than those with fewer sessions, QB = 4.54, DF = 1, P < .05. Use of behavioral theory and provision of skills training had no significant impact on the efficacy of interventions, however. Motivation and skills content [39] could not be examined as moderators because these were included in virtually every intervention.
Methodological characteristics were examined next. Length of follow-up was negatively correlated with effect size [r (11) = −0.18, P = 0.57], suggesting that a nonsignificant trend in that studies with longer follow-up periods had smaller effects. Study retention also had a nonsignificant correlation of [r (11) = 0.35, P = 0.30] with effect size, suggesting stronger effects in studies with greater retention. Finally, a comparison of studies using HIV intervention comparison groups versus no treatment control groups found that the studies that used HIV intervention comparison groups had smaller effect sizes than those that did not. This difference was not statistically significant, however (Table 3).
Finally, unpublished studies were compared with published ones in order to examine whether effect sizes differed between these two groups. Results indicated no statistically significant difference between the two types of studies (Table 3).
Discussion
The results of the current meta-analysis indicate that computer-based interventions have been efficacious in increasing condom use and reducing sexual activity, numbers of sexual partners, and incident STD. Results also suggest that these types of interventions have been as efficacious as many commonly utilized human-delivered interventions in HIV prevention. In addition, three factors make it likely that the condom use effect size observed in the current study is not an overestimate: the comprehensive search that yielded several published and unpublished studies; the analysis that demonstrated similar effect sizes among published and unpublished studies; and the publication bias analysis that suggested that a very large number of null unpublished studies would need to exist in order to nullify the effect size found here. Thus, this meta-analysis indicates that computer-mediated interventions hold much promise for the future of HIV prevention efforts.
Moreover, it is important to point out that computer technology-based interventions have many advantages when compared to human-delivered interventions. These include lower cost to deliver, greater intervention fidelity, and greater flexibility in dissemination channels, which might include in person (for example, clinic setting), mail, Internet, cell phones, or other delivery channels. This is very promising in the context of recent discussion with regard to increasing the dissemination of efficacious HIV prevention interventions [9–13]. In fact, it is worth noting that the studies included in the current meta-analysis were quite diverse in nature and included heterosexual, MSM, men, women, urban, rural, minority and majority populations. The fact that interventions were successful with a number of such populations may bode well for the broad application of these types of interventions.
Moderators of intervention efficacy were also examined in the current meta-analysis. Interventions directed at a single sex were found to be more efficacious than those directed at both sexes. This finding may reflect the fact that interventions directed at a single sex may be more highly targeted to that audience segment than those developed to be efficacious with both sexes [3]. Interventions were also found to be more efficacious when they utilized individualized tailoring and a Stages of Change model. Computer-tailored interventions, which assess individual characteristics and tailor content at the individual (rather than group) level, have been successfully utilized in cancer prevention for far more than a decade. Reviews of such interventions have found them to be efficacious in changing a number of health-related behaviors [44–47]. The individually tailored interventions included in the current meta-analysis represent the first applications of these types of interventions to HIV prevention. The results clearly indicate that more applications of tailoring to HIV prevention are warranted.
A final issue worth noting is the distinction between computer-based trials versus those that were Internet-based trials. Although most studies in the current meta-analysis did not use the Internet as a delivery mode, three studies did. It is worth noting that the two trials with poor retention were also the two Internet-based trials (the third intervention [22] recruited individuals in schools). Given that the Internet has emerged as a conduit for individuals to seek and find high-risk sexual partners, developing and testing interventions that can proactively reach out to such populations is a high priority. Thus, future research to improve ways of conducting randomized trials online is warranted.
Conclusion
The HIV prevention behavioral literature appears to be shifting from a focus on development of efficacious interventions to translation and dissemination efforts [9–13]. Although this is a promising development, numerous challenges to dissemination of current interventions have been encountered [9,48,49]. Computer technology-based interventions represent a relatively new and promising intervention type that may have great potential for dissemination. Continued development, testing, and ultimate dissemination of such interventions can increase the public health impact of HIV behavioral interventions and potentially avert new infections.
Acknowledgements
We would like to thank Tony Roberto, Stephen Read, Colleen Redding, Julie Downs, and Sheana Bull for providing additional data for use in this meta-analysis.
S.M.N. conceptualized and supervised the study. H.G.B. and L.B.P. conducted the literature search and coded the studies. S.M.N. conducted the analysis and wrote the first draft of the article. H.G.B. and L.B.P. contributed to writing and revision of the article draft.
References
1. Albarracin D, Gillette JC, Earl AN, Glasman LR, Durantini MR, Ho MH. A test of major assumptions about behavior change: a comprehensive look at the effects of passive and active HIV-prevention interventions since the beginning of the epidemic. Psychol Bull 2005; 131:856–897.
2. Crepaz N, Lyles CM, Wolitski RJ, Passin WF, Rama SM, Herbst JH,
et al. Do prevention interventions reduce HIV risk behaviours among people living with HIV? A meta-analytic review of controlled trials. AIDS 2006; 20:143–157.
3. Noar SM. Behavioral interventions to reduce HIV-related sexual risk behavior: review and synthesis of meta-analytic evidence. AIDS Behav 2008; 12:335–353.
4. Lyles CM, Kay LS, Crepaz N, Herbst JH, Passin WF, Kim AS,
et al. Best-evidence interventions: findings from a systematic review of HIV behavioral interventions for US populations at high risk, 2000–2004. Am J Public Health 2007; 97:133–143.
5. Ward H. One-to-one counselling for STI prevention: not so much whether as how. Sex Transm Infect 2007; 83:1–11.
6. Rietmeijer CA. Risk reduction counselling for prevention of sexually transmitted infections: how it works and how to make it work. Sex Transm Infect 2007; 83:2–9.
7. Jemmott LS, Jemmott JB, Hutchinson MK, Cederbaum JA, O'Leary A. Sexually transmitted infection/HIV risk reduction interventions in clinical practice settings. J Obstet Gynecol Neonatal Nurs 2008; 37:137–145.
8. Botvin GJ, Griffin KW. Life skills training: empirical findings and future directions. J Prim Prev 2004; 25:211–232.
9. Solomon J, Card JJ, Malow RM. Adapting efficacious interventions: advancing translational research in HIV prevention. Eval Health Pro 2006; 29:162–194.
10. Bell SG, Newcomer SF, Bachrach C, Borawski E, Jemmott JB 3rd, Morrison D,
et al. Challenges in replicating interventions. J Adolesc Health 2007; 40:514–520.
11. Kelly JA, Spielberg F, McAuliffe TL. Defining, designing, implementing, and evaluating phase 4 HIV prevention effectiveness trials for vulnerable populations. J Acquir Immune Defic Syndr (1999) 2008; 47(Suppl 1):S28–S33.
12. Neumann MS, Sogolow ED. Replicating effective programs: HIV/AIDS prevention technology transfer. AIDS Educ Prev 2000; 12:35–48.
13. Harshbarger C, Simmons G, Coelho H, Sloop K, Collins C. An empirical assessment of implementation, adaptation, and tailoring: the evaluation of CDC's national diffusion of VOICES/VOCES. AIDS Educ Prev 2006; 18(Suppl A):184–197.
14. Ybarra ML, Bull SS. Current trends in Internet- and cell phone-based HIV prevention and intervention programs. Curr HIV/AIDS Rep 2007; 4:201–207.
15. Cassell MM, Jackson C, Cheuvront B. Health communication on the Internet: an effective channel for health behavior change? J Health Commun 1998; 3:71–79.
16. Bickmore T, Giorgino T. Health dialog systems for patients and consumers. J Biomed Inform 2006; 39:556–571.
17. Kiene SM, Barta WD. A brief individualized computer-delivered sexual risk reduction intervention increases HIV/AIDS preventive behavior. J Adolesc Health 2006; 39:404–410.
18. Scholes D, McBride CM, Grothaus L, Civic D, Ichikawa LE, Fish LJ,
et al. A tailored minimal self-help intervention to promote condom use in young women: results from a randomized trial. AIDS 2003; 17:1547–1556.
19. Lightfoot M, Comulada WS, Stover G. Computerized HIV preventive intervention for adolescents: indications of efficacy. Am J Public Health 2007; 97:1027–1030.
20. Downs JS, Murray PJ, Bruine de Bruin Wn, Penrose J, Palmgren C, Fischhoff B. Interactive video behavioral intervention to reduce adolescent females STD risk: a randomized controlled trial. Soc Sci Med 2004; 59:1561–1572.
21. Read SJ, Miller LC, Appleby PR, Nwosu ME, Reynaldo S, Lauren A,
et al. Socially optimized learning in a virtual environment: reducing risky sexual behavior among men who have sex with men. Hum Commun Res 2006; 32:1–34.
22. Roberto AJ, Zimmerman RS, Carlyle KE, Abner EL. A computer-based approach to preventing pregnancy, STD, and HIV in rural adolescents. J Health Commun 2007; 12:53–76.
23. Davidovich U, de Wit JBF, Stroebe W. Using the Internet to reduce risk of HIV-infection in steady relationships: a randomized controlled trial of a tailored intervention for gay men. In: Davidovich E, editor. Liaisons dangereuses – HIV risk behavior and prevention in steady gay relationships. Amsterdam: Roel & Uigeefprojecten; 2006. pp. 95–122.
24. Peipert J, Redding CA, Blume J, Allsworth JE, Iannuccillo K, Lozowski F,
et al. Design of a stage-matched intervention trial to increase dual method contraceptive use (Project PROTECT). Contemp Clin Trials 2007; 28:626–637.
25. McFarlane M, Kachur R, Klausner JD, Roland E, Cohen M. Internet-based health promotion and disease control in the 8 cities: successes, barriers, and future plans. Sex Transm Dis 2005; 32:S60–S64.
26. Portnoy DB, Scott-Sheldon LAJ, Johnson BT, Carey MP. Computer-delivered interventions for health promotion and behavioral risk reduction: a meta-analysis of 75 randomized controlled trials, 1988–2007. Prev Med 2008; 47:3–16.
27. Redding CA, Prochaska JO, Rossi JS, Armstrong K, Coviello D, Pallonen U,
et al. Effectiveness of a randomized clinical trial of stage-matched interventions targeting condom use and smoking in young urban females [abstract]. Ann Behav Med 2002; 24:S061.
28. Redding CA, Morokoff PJ, Rossi JS, Meier KS, Hoeppner BB, Mayer K,
et al. Effectiveness of a computer-delivered TTM-tailored intervention at increasing condom use in at-risk heterosexual adults: RI Project Respect [abstract]. Ann Behav Med 2004; 27:S113.
29. Di Noia J, Schinke SP, Pena JB, Schwinn TM. Evaluation of a brief computer-mediated intervention to reduce HIV risk among early adolescent females. J Adolesc Health 2004; 35:62–64.
30. Bellis JM, Grimley DM, Alexander LR. Feasibility of a tailored intervention targeting STD-related behaviors. Am J Health Behav 2002; 26:378–385.
31. Chernoff RA, Davison GC. An evaluation of a brief HIV/AIDS prevention intervention for college students using normative feedback and goal setting. AIDS Educ Prev 2005; 17:91–104.
32. Bolding G, Davis M, Sherr L, Hart G, Elford J. Use of gay Internet sites and views about online health promotion among men who have sex with men. AIDS Care 2004; 16:993–1001.
33. Kurth AE, Severynen A, Clausen M, Frick P, Bangsberg DR, Simoni JM,
et al.
Computer counseling to support antiretroviral therapy adherence and HIV transmission risk reduction: baseline and six-month follow-up.
Paper presented at the 2007 CDC National HIV Prevention Conference; Atlanta, Georgia; 2007.
34. Carlson KD, Schmidt FL. Impact of experimental design on effect size: Findings from the research literature on training. J Appl Psychol 1999; 84:851–862.
35. Schmidt FL, Hunter JE. Development of a general solution to the problem of validity generalization. J Appl Psychol 1977; 62:529–540.
36. Cohen J. Statistical power analysis for the behavioral sciences. revised ed. Hillsdale, NJ, England: Lawrence Erlbaum Associates; 1977.
37. Rosenthal R. Meta-analytic procedures for social research. rev. ed. Thousand Oaks, CA, USA: Sage; 1991.
38. Sanchez-Meca J, Marin-Martinez F, Chacon-Moscoso S. Effect-size indices for dichotomized outcomes in meta-analysis. Psychol Methods 2003; 8:448–467.
39. Johnson BT, Carey MP, Chaudoir SR, Reid AE. Sexual risk reduction for persons living with HIV research synthesis of randomized controlled trials, 1993 to 2004. J Acquir Immune Defic Syndr 2006; 41:642–650.
40. Cooper H, Hedges LV. The handbook of research synthesis. New York, NY, US: Russell Sage Foundation; 1994.
41. Lipsey MW, Wilson DB. Practical meta-analysis. Thousand Oaks, CA, USA: Sage; 2001.
42. Noar SM. An interventionist's guide to AIDS behavioral theories. AIDS Care 2007; 19:392–402.
43. Prochaska JO, Redding CA, Harlow LL, Rossi JS, Velicer WF. The transtheoretical model of change and HIV prevention: a review. Health Educ Q 1994; 21:471–486.
44. Noar SM, Benac CN, Harris MS. Does tailoring matter? Meta-analytic review of tailored print health behavior change interventions. Psychol Bull 2007; 133:673–693.
45. Kroeze W, Werkman A, Brug JA. systematic review of randomized trials on the effectiveness of computer-tailored education on physical activity and dietary behaviors. Ann Behav Med 2006; 31:205–223.
46. Sohl SJ, Moyer A. Tailored interventions to promote mammography screening: a meta-analytic review. Prev Med 2007; 45:252–261.
47. Redding CA, Morokoff PJ, Rossi JS, Meier KS. A TTM-tailored condom use intervention for at-risk men and women. In: Edgar T, Noar SM, Freimuth VS, editors. Communication perspectives on HIV/AIDS for the 21st century LEA's communication series. New York: Lawrence Erlbaum Associates/Taylor & Francis Group; 2008. pp. 423–428.
48. Glasgow RE, Lichtenstein E, Marcus AC. Why don't we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. Am J Public Health 2003; 93:1261–1267.
49. Eke AN, Neumann MS, Wilkes AL, Jones PL. Preparing effective behavioral interventions to be used by prevention providers: the role of researchers during HIV Prevention Research Trials. AIDS Educ Prev 2006; 18(4 Suppl A):44–58.
