LaCroix, Jessica M. MS^*,†; Snyder, Leslie B. PhD^‡; Huedo-Medina, Tania B. PhD^†,§; Johnson, Blair T. PhD^*,†

INTRODUCTION

Since the 1960s, mass media-delivered health interventions, also known as public communication campaigns or social marketing campaigns,^1–3 have been an important strategy for many health behavior change topics, including heart disease, smoking, family planning, and HIV/AIDS prevention.^4–9 Mass media–based interventions have been an integral part of HIV prevention since the mid-1980s, when many nations began sponsoring national campaigns,¹⁰ and community-based groups began developing local interventions. Using the mass media within an HIV prevention intervention provides such advantages as wide reach, standardization and repetition of messages, and the ability to use different content formats, including entertainment, news, and short advertisements or announcements. The international nature of the HIV prevention campaign literature allows for analysis of the differential impact of mass media–based interventions in different parts of the world.

With regard to HIV prevention campaigns, mass media channels can be used on their own (eg, television, newspapers, radio) or in combination with interpersonal sources of information (eg, outreach workers, peer educators). In addition, HIV prevention campaigns may incorporate condom pricing and distribution strategies,^1,2 community organizing, or advocacy for policy change.^11,12 The number of channels used in a campaign may predict the extent to which that campaign influences the target population; multichannel approaches are often seen as more effective than single channel interventions.⁵

Duration of HIV prevention campaigns may also predict campaign effects, although results from past research are conflicting. One meta-analysis suggested that US health promotion campaigns lasting a year or less were more effective than longer campaigns,⁸ whereas another review suggested that in developing countries, health promotion campaigns lasting 18 months or longer were more effective.¹³ It might be, for example, that in developing countries, mass media messages take a relatively long time to reach and to have an impact on the knowledge, attitudes, and behavior of a substantial portion of the target population, whereas the target population in developed countries may have more and faster media access.

Three previous systematic reviews have investigated the nature of mass media campaigns to combat HIV/AIDS. The first review reported characteristics for 41 campaigns published in peer-reviewed journals from 1986 to 1998.¹⁴ Of the 41 campaigns reviewed, 26 had a name or slogan, 28 reported rates of exposure, and only 7 used a controlled comparison group design. Of the studies reporting exposure, 50% of the audience was exposed to intervention messages, and televised campaigns had the highest rates of exposure. Building on the first review, a more recent publication systematically reviewed HIV/AIDS mass communication campaigns published in peer-reviewed journals between 1998 and 2007.¹⁵ Of the 34 new campaigns, 28 had a name or slogan, and 28 reported exposure, the mean of which was 77% of the audience. Although neither review was able to use meta-analytic techniques because of the lack of methodological rigor in intervention evaluations, Noar et al¹⁵ reported improvements in safer sex behavior and intention for 8 of the 10 campaigns that used quasi-experimental designs. A third review—focused exclusively on campaigns in developing countries—identified 24 articles published in peer-reviewed journals from 1990 to 2004.¹⁶ Half of the campaigns used television or radio and half used small media (eg, brochures, leaflets). The authors reported positive results for condom use and knowledge, but concluded that more research is needed to establish the magnitude of effects and identify which campaign elements contribute to success.

The purpose of the present meta-analysis was to expand on previous systematic reviews to examine the overall effectiveness of mass media–delivered HIV interventions and to identify predictors of changes in condom use and HIV-related knowledge. We define mass media–delivered interventions as those wherein the intervention message is delivered in a natural setting through a mass media channel to which individuals may or may not attend (eg, radio, television, newspaper, magazine, or mass distribution or mailing of printed materials). In addition to testing the role of campaign duration and number of channels, we also explored whether date of data collection, country of campaign, level of country development, and message and sample characteristics predicted effect size magnitude. This review is the first to analyze the results of mass media–delivered HIV interventions using meta-analytic techniques (ie, pooling effect sizes and analyzing possible moderator variables of effect size magnitude), and the first to attempt to locate and include unpublished reports. This study contributes to global public health by informing both HIV prevention work and interventions in general in terms of the effectiveness of mass media–delivered health interventions.

METHODS

The current meta-analysis conformed with the standards implied by Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guidelines.¹⁷

Inclusion Criteria

Studies were included if they (1) had an HIV/AIDS prevention focus, (2) targeted youth or the general population, (3) evaluated a specific intervention delivered through an audio, visual, or printed mass media channel in a natural setting, (4) quantitatively evaluated the target population using a pre-/post-campaign design, (5) measured condom use and/or HIV/AIDS-related transmission or prevention knowledge, (6) reported sufficient information to calculate effect sizes, and (7) were available in English. Interventions that solely targeted high-risk groups (eg, injection drug users, commercial sex workers), used forced exposure to messages, only sampled individuals who had all been exposed to some campaign component, or used interpersonal communication supplemented by small media (eg, brochures delivered by outreach workers) were excluded.

Search Strategy

Potentially relevant reports were located through (1) systematically searching PsycINFO, CINAHL Plus, Medline/PubMed, ProQuest Dissertations and Theses; (2) hand searching relevant Web sites (eg, Population Services International, MEASURE Evaluation), the Synthesis of HIV/AIDS Research Project's (SHARP) collection of sexual risk reduction interventions, relevant journals (eg, AIDS, AIDS and Behavior), the reference lists of previous reviews and included studies; and (3) personal communication with authors to locate additional reports (Fig. 1; see also Appendix A, Supplemental Digital Content, http://links.lww.com/QAI/A540, for details of the search and selection process). Each database was searched as far back in time as available. Two authors independently screened potentially relevant reports for eligibility; disagreement was resolved through discussion. Studies that fulfilled the search criteria and were available by September 30, 2013 were included.

FIGURE 1

Coding Study Characteristics

Trained raters coded characteristics describing the study reports (eg, publication status), interventions (eg, duration, number of channels, message branding), and evaluation samples (eg, age, gender, proportion sexually active). A subset of interventions (65%) was double coded to ensure coding accuracy; interrater reliability was high (mean r = 0.94, κ = 0.76, 92% agreement). All disagreements were resolved through discussion. In addition, interventions were matched by country and year of initial data collection to the corresponding Human Development Index (HDI), a measure of nation-level social and economic development that combines indices related to life expectancy, educational attainment, and income.^18,19 HDI values range between 0 and 1, and countries are described as scoring high (HDI of 0.80–1.00), medium (HDI between 0.50 and 0.79), or low (HDI <0.50). In instances when HDI values were not available for the exact year of initial data collection, values from the closest year available were used.

Calculation and Analyses of Effect Sizes

To gauge the success of each mass media-delivered intervention, 2 independent researchers calculated the standardized mean difference, d, for each study sample, k, by subtracting the pre-campaign mean from the post-campaign mean and dividing by the pooled standard deviation.²⁰ In the absence of mean values and standard deviations, other statistical information (eg, F-values) was used with the appropriate transformations.^21,22 If the study reported dichotomous outcomes, we calculated an odds ratio and transformed it to d using the Cox transformation.²³ Effect sizes were corrected for sample size bias. The primary outcome variable was condom use (eg, frequencies and proportions of protected sex). Secondary outcomes included HIV/AIDS transmission knowledge (eg, HIV is transmitted through unprotected intercourse) and HIV/AIDS prevention knowledge (eg, condoms prevent the sexual transmission of HIV). When multiple measures of the same outcome were reported, effects were averaged. Multiple effect sizes were calculated from individual studies when they had more than one intervention condition or when the outcomes were separated by sample characteristics (eg, gender). Effect size calculation disagreements were resolved through discussion. Positive effect sizes indicate more sexual risk reduction after the intervention compared with pre-campaign results (eg, increased condom use, decreased unprotected intercourse, improved HIV knowledge).

Analyses were conducted following random-effects assumptions using published macros²⁴ in Stata 11.2 (StataCorp, College Station, TX).²⁵ The homogeneity statistic index, I ², its corresponding 95% confidence intervals (CIs), and the inference test of homogeneity Q were calculated to determine the extent to which study outcomes were consistent; heterogeneity is inferred when CIs do not include zero.^26–28 Asymmetries in distribution of the different effect size outcomes, which can suggest reporting bias, were examined with Begg technique, trim and fill, and Egger technique.^29–31 The relation between sample, methodological, or campaign characteristics and the magnitude of the effects were examined using a modified weighted least squares regression analysis with weights equivalent to the inverse of the sample and population variance. The moving constant technique was used to estimate mean effect sizes (d ₊) and CIs for specific moderator values of interest.³²

RESULTS

Report, Intervention, and Sample Characteristics

In total, 433 reports were obtained and screened; 54 reports containing evaluations of 72 separate interventions met the selection criteria and were included.^33–86 Interventions were evaluated at pre-/post-campaign using a total of 93 separate sample comparisons drawn from the target populations. Table 1 presents descriptive characteristics of the individual reports (k = 54), interventions (k = 72; see Table S1, Supplemental Digital Content, http://links.lww.com/QAI/A540, for additional characteristics), and samples (k = 93; eg, results were split by gender). Reports were published between 1986 and 2010 (mean = 1999), were retrieved primarily from journals (69%) or organization Web sites (28%), and most studies were conducted in Africa (50%) or Asia (17%). Interventions were generally conducted nationally (49%) or based in individual communities (43%). Many campaigns pretested message content before campaign launch (39%), and 45% were theory based. Of the 72 interventions, 15% used only 1 channel, and of these, 4 used small media, 3 used radio, 3 used newspapers or magazines, and 1 used television. All other campaigns (83%) used 2 or more channels including signage (72%), radio (70%), television (57%), educational literature (51%), newspapers or magazines (33%), and promotional materials (21%). Overall, 47% of the interventions included an interpersonal component. Most interventions focused on condom promotion (76%), and more than a third used an entertainment–education strategy (39%). Most campaigns had an identifying logo, slogan, or brand (57%), and 42% included condom distribution. The median intervention duration was approximately 8 months, though duration varied widely.

TABLE 1-a

TABLE 1-b

All reports (k = 54) evaluated interventions using pre- vs. post-campaign designs; 2 reports evaluated multiple campaigns,^39,43 5 reports evaluated campaigns at multiple sites,^{36,44,51,72,80} and 2 reports^47,53 evaluated campaigns in intervention and comparison areas wherein the comparison areas were exposed to specific campaign elements and were coded separately. Only 8 reports provided results for a no-exposure comparison group at pre- and post-campaign.^{36,55,59,72,77,80,82,86} The 72 interventions were evaluated using a total of 96 samples; median sample size was about 600 participants. Among the individual samples, the mean proportion female was 0.48, mean proportion married was 0.39, mean proportion sexually active was 0.69, mean age was 24.25 years, and mean campaign exposure was 59%.

Overall Efficacy of the Interventions

Overall, analyses indicate significant increases in condom use (d ₊ = 0.25, 95% CI = 0.18 to 0.21, k = 57; Fig. 2) and significant improvements in HIV-related transmission knowledge (d ₊ = 0.30, 95% CI = 0.18 to 0.41, k = 47) and prevention knowledge (d ₊ = 0.39, 95% CI = 0.25 to 0.52, k = 65) following mass media interventions compared with pre-campaign assessments. Nonetheless, the hypothesis of homogeneity was rejected for each outcome; the effects on each of these dimensions were characterized by large heterogeneity (I ² > 91, Ps < 0.001; see Table S2, Supplemental Digital Content, http://links.lww.com/QAI/A540), necessitating the use of a priori determined effect modifiers to explain the variation in study effects. For published articles, trim-and-fill results showed asymmetries for prevention knowledge, suggesting publication bias, but Begg test results and Egger test results were nonsignificant for asymmetries on all 3 outcomes (see Table S2).

FIGURE 2

Condom Use

Several dimensions were related to the magnitude of effect sizes gauging condom use (Table 2). Greater increases in condom use occurred following interventions conducted in African nations, in countries with lower HDI scores (Fig. 3), following longer campaigns, when message content was reportedly matched to the target audience, and when refusal rates were low. Condom use also increased to the extent that campaigns increased knowledge of transmission (β = 0.56, P = 0.009, k =16) and prevention (β = 0.30, P = 0.03, k = 41).

TABLE 2

FIGURE 3

Transmission and Prevention Knowledge

Increases in HIV/AIDS transmission and prevention knowledge were largest in Asian nations. Improvement in transmission knowledge was greater to the extent that the nation had a lower HDI and a larger proportion of the sample reported exposure to the campaign. Increases in prevention knowledge were larger following longer campaigns. Interventions that reportedly included demonstrations of condom use did not significantly improve prevention knowledge compared with interventions that did not report condom demonstrations. No other coded report characteristics (eg, publication status), campaign characteristics (eg, number of communication channels, message branding, reported theoretical background, condom distribution) or sample characteristics (eg, age, gender, proportion sexually active, pre-campaign rate of condom use) were significant predictors of effect size magnitude for any outcome of interest.

DISCUSSION

Results from the current meta-analysis of 54 reports evaluating 72 interventions demonstrated that mass media-delivered interventions increase condom use and improve transmission and prevention knowledge. To the best that we know, our meta-analysis is the first to examine published and unpublished mass media reports across the world. Average effect sizes in this study were of small to medium magnitude (0.25–0.39; Table 1); yet, results were marked by heterogeneity, and several study characteristics moderated intervention efficacy.

First, as previous meta-analytic work related to HIV prevention has demonstrated, this meta-analysis supports the conclusion that interventions achieve better results where the need is greatest,^87–89 as demonstrated by the linkage between HDI and outcome measures (Table 2). Investing resources into areas of the world with lower social and economic development seems to yield greater returns than investing resources into those with greater development. In contrast, the CI results in Table 2 and in Figure 3 that include the value of 0 suggest that mass media interventions failed, on average, in relatively developed nations. The differential impact between nations scoring higher and lower on the HDI may be because of reduced “media clutter” in less developed nations, and differences in incidence rates and perceived risk.

Second, longer campaign duration related to increased condom use; the CI results in Table 2 suggest that mass media interventions failed, on average, in short duration campaigns. With regard to knowledge, the longer the campaign duration, the more prevention knowledge increased, and the higher the proportion of the sample that was exposed to intervention messages, the more transmission knowledge increased. Effect size magnitude did not vary because of age or gender. Increases in HIV/AIDS-related knowledge were also positively related to increased condom use. More recent campaigns demonstrated greater increases in transmission knowledge, but contrary to previous findings, we found no significant linkage between exposure and year of publication.¹⁵ Although the moderator for amount of exposure did not achieve formal significance levels, the CI results in Table 2 suggest that mass media interventions failed, on average, for those who had little exposure to the intervention, a pattern that appeared for condom use and the 2 knowledge dimensions.

Because most interventions included in this meta-analysis used multiple communication channels, it was impossible to tease apart the effects of different channels used in the interventions. Interventions often use as many communication channels as they can afford, consistent with studies that show reach and impact increasing as numbers of media and interpersonal channels increase.^1,4,5 We found no evidence that number of channels was related to greater impact in the present meta-analysis. Unfortunately, studies tend to show the combined impact of all channels and messages; the way in which the media are used within an intervention may differ, and it would inform future interventions to know which types of media approaches work under what conditions.

Previous reviews of mass media–delivered interventions to prevent HIV/AIDS have addressed the variability in research design, methodology, and reporting of intervention development, content, and delivery.^14–16 The most persistent hurdle in determining the efficacy of mass media campaigns is the lack of available comparison groups in the evaluation of these interventions; only 8 of the 72 interventions included in the present meta-analysis provided valid comparison group data. Because of the inability to reliably control access to mass media, it may be both difficult and costly to use true or quasi-experimental designs. As a result, many campaign evaluations compare pre- and post-campaign data or compare individuals who were or were not exposed to the campaign. Both of these evaluation methods have limitations. The former does not allow for direct attribution of changes in knowledge or behavior to the campaign message alone, although it is notable that in the current investigation, campaigns were more successful to the extent that more respondents reported exposure. Additional factors unrelated to a specific campaign, such as the social or political climate, public policy changes, current events, and the presence of multiple campaigns in a targeted area can influence HIV/AIDS-related knowledge and sexual risk behaviors, making it impossible to isolate intervention effects. Evaluations based solely on exposure are also limited in that exposure to mass media may be related to individual characteristics such as income, gender, relationship status, and age.^90,91

As with other reviews, the results of the present meta-analysis are limited by the available studies and reflect the empirical state of the field. The intervention literature in general would benefit tremendously from the public availability of more quantitative evaluations (ie, using pre-campaign comparison groups or quasi-experimental designs), described in greater detail (eg, specifying media channels used). In cases where reporting space is limited, researchers can provide additional information regarding campaign details as online supplements to publications, provide references to more descriptive papers, or invite readers who desire additional information to contact them directly. It would also be beneficial to the literature to have studies include cost information to facilitate cost-effectiveness analyses. Additional information would allow for more nuanced theorizing about what sorts of mass media interventions may work best in different contexts.

Another limitation of the present study is that our database generally focused on short-term effects of the interventions, when media effects are most likely to be at their peak. In media research, effects tend to dissipate over time,^92,93 and analysis of the rate of decay of HIV media interventions is best addressed in a meta-analysis designed to examine such effects. Still, intensive interpersonal HIV prevention strategies have been shown to have larger effects at extended periods (1–3 years) compared with briefer interpersonal interventions.⁹⁴ Finally, the present article does not explore other aspects of campaign content and message design, and although campaigns from more than 30 countries were included, we limited our search to reports available in English.

Past meta-analyses of HIV prevention literature have focused almost exclusively on trials evaluating relatively intensive strategies that demand interpersonal interaction, often in small-group sessions that continue over a series of weeks or months.^94–98 The current studies, in contrast, use media that often are brief, and only sometimes include interpersonal communication in clinics or with outreach workers. Despite the differences in channels between the current and previous meta-analyses, the effects on behavior change are similar in magnitude—small to medium, depending on moderating factors. Despite their modest size in absolute terms, for statewide or national media interventions the scope of their impact is quite large in absolute terms. Media campaigns are a means of “going to scale”—taking an intervention to large numbers of people.²²

In summary, the present results provide strong testimony to the power that mass media campaigns can have for people living in nations most at need for HIV prevention and other health promotion interventions. Results also suggest that such campaigns generally lack effectiveness in relatively developed countries, where the need for health promotion is generally lessened except in particular locales.

REFERENCES

Supplemental Digital Content

• QAI_2014_06_09_JOHNSON_V14368_SDC1.pdf; [PDF] (106 KB)