Leveraging Crowdsourcing for Human Immunodeficiency Virus Testing Posters: A Visual Content Analysis and Cognitive Responses Among Chinese Men Who Have Sex With Men

Abstract

Introduction 

Extensive marketing and advertising research has informed a deep understanding of the link between visual design and consumer behaviors, providing a useful framework for assessing associations between human immunodeficiency virus (HIV)-related health posters and viewer responses.

Methods 

Crowdsourced posters included finalist submissions from a series of nationwide crowdsourcing contests. Centers for Disease Control (CDC) images were sampled from an online poster database maintained by the National Center for acquired immune deficiency syndrome (AIDS). Once coded according to a set of 27 visual features, posters were shown to an online sample of Chinese men who have sex with men—a group currently experiencing the highest HIV incidence in China—to assess their viewer response.

Results 

The CDC posters were more likely to use positive facial expressions (65%; 95% confidence interval [CI], 40.9–83.7] vs 12.5%; 95% CI, 2.2–4%) and an educational messaging style (85%; 95% CI, 61.1–96% vs 31.3%; 95% CI, 12.1–58.5). Crowdsourced posters exhibited better craftsmanship than CDC posters (more design simplicity, image diversity, color choice, design quality, and moderate use of text) used more visual metaphors (56.3%; 95% CI, 30.6–79.2 vs 5%; [95% CI, 0.2–26.9%]). Several differences in visual complexity were identified but these lacked statistical significance.

Conclusion 

Crowdsourced posters were of higher craftsmanship, possibly due to their ability to recruit skills of professional designers. The CDC posters' use of positive visual reinforcement (smiling faces) and educational messaging may be a legacy of their role in the early days of the epidemic in disseminating basic HIV/AIDS knowledge and dispelling misinformation. Crowdsourcing posters' used more metaphors, suggesting better ability to leverage in-group codes and language.

In Brief

Content analysis of human immunodeficiency virus posters shows that crowdsourced posters are better designed and use more coded language than government posters but do not differ in terms of visual persuasiveness.

Posters have been widely used to disseminate human immunodeficiency virus (HIV) prevention messages and spur behavior change.^1,2 With relatively low production costs,³ this tool lends itself well to display in public areas, as well as to mass distribution in the digital age. Though posters are a common component of many HIV-risk behavior change interventions, their content has been an infrequent object of study. The few studies that have considered content have examined abstract qualities, such as visual clarity,⁴ humorousness,⁵ or use of fear tactics.⁶ Extensive marketing and advertising research provides a suitable framework for understanding the link between specific visual design features and ensuing consumer behaviors,⁷ providing a useful framework for assessing associations between the visual content of HIV posters and viewer's cognitive responses to them.

Design and distribution of visual health promotion materials is traditionally one of the many tasks under the purview of publicly funded disease control agencies⁸ which often maintain full control over poster messaging, design, production, and dissemination.^9,10 In China, where our research takes place, health departments—or Centers for Disease Control (CDC) as they are referred to locally—often rely on in-house staff to design promotional materials, most of whom have limited formal design training. Design professionals may be contracted where budgets permit, though ultimate decision-making power regarding content and messaging remains with health authorities. Mixed results of traditional HIV poster campaigns² along with the fact that half the global population of people living with HIV are yet aware of their status¹¹ have mobilized a rethinking of traditional approaches to health promotion design. Chief among the novel strategies proposed is the use of crowdsourcing,¹² a process in which a group of experts and nonexperts collaborate to solve problems, then share the subset of exceptional solutions with the public. Crowdsourcing has been used on several occasions in Chinese settings to generate promotional posters urging HIV testing as part of larger health campaigns.^13,14 In contrast to the top-down approach of health departments, posters designed through crowdsourcing incorporate a diversity of views through a bottom-up approach, often directly involving members of the same communities targeted by these campaigns.¹² This contrast between traditional and crowdsourced approaches to poster design presents an opportunity to identify key features that may be better suited to changing attitudes toward HIV testing in populations at risk.^15,16

Because the process of visual influence—ranging from visual exposure to cognitive response and ultimately to behavior change—is multistaged and complex, assessment of a poster's potential to influence behavior is best conducted at an elemental level. Such an approach breaks down a poster's visual features to key design elements commonly used in visual communication research. Marketing and communication research—based on visual rhetoric and visual persuasion theory—have identified visual features already known to universally maximize viewer appeal or elicit favorable cognitive responses in viewers.^7,17 Borrowing from this broad evidence base can provide some insights into the visual features potentially salient to better “selling” of healthy behaviors.

Individual responses to a visual stimulus are critical for initial stages of persuasive message processing.^7,18,19 The elaboration likelihood model developed by Petty and Cacioppo²⁰ in 1986 provides a framework for examining how external stimuli are processed and can translate into attitude and behavior change. The model specifies 2 pathways through which stimuli are processed and identifies 3 key types of stimuli that contribute to change: attention, relevance, and elaboration.²¹ Attention refers to the ability of effective messaging to attract viewers, thus giving them the ability to process the message.^22,23 Attention is sensitive to visual features, such as complexity and aesthetics, including both amount of information shown and its presentation.^7,24 Relevance, or the perception that the message is intended for that particular viewer that impacts motivation to process the message, has been shown to be swayed by the presence of humans or characters whose portrayal aligns with a viewer's identity.²² Lastly, elaboration, or the process issue-relevant thinking that allows for forming associations between new information and prior knowledge,^20,25,26 is thought to be triggered by the use of symbols or visual metaphors, the artful depiction with socially constructed meanings.^27–29

Using these visual content analyses and conceptualizations of viewer response, our analysis investigates the influencing potential of HIV posters to answer the following 3 questions. First, do HIV testing posters created by lay participants in crowdsourcing events differ visually from those made or commissioned by local CDCs? Second, are crowdsourced posters more likely to elicit viewer response than CDC posters? And last, regardless of how posters are made, what specific visual features are predictive of eliciting a positive viewer response, defined as affirmative answers to questions regarding attention, relevance, and elaboration? To answer these questions, we collected and coded a sample of HIV testing posters created both via crowdsourcing and by the CDC system, which were then showed to an online sample of men who have sex with men—a group currently experiencing the highest HIV incidence in China—to assess the relationship between visual poster features and viewer responses.

MATERIALS AND METHODS

Data collection and generation took part in 2 phases: the first involving sampling and coding of posters related to HIV testing and the second, the recruitment and solicitation of viewer responses from an online sample of Chinese MSM.

Phase 1: Visual Coding of Posters

The CDC posters were sampled from an online repository of over 300 HIV-related health promotion materials maintained by the Chinese Center for Disease Control & Prevention (hereafter the “CDC,” http://ncaids.chinacdc.cn/zsk/). Authors C.L. and R.F. downloaded and cataloged all posters which were then reviewed by authors M.K.S. and B.C. to identify the subset containing content related to HIV testing. Divergent designations between MKS and BC in terms of whether each poster was relevant to HIV testing were resolved through group discussion, which reduced the total number of eligible CDC posters to 79. The analysis sample of 20 CDC posters was then drawn from the total of 79 using a random number generator approach. The sample size of 20 posters was selected to match the total number of crowdsourced posters available for analysis.

Crowdsourced posters were sourced from all finalist submissions across 3 nationwide crowdsourcing events held between 2014 and 2016,^12,15,30 amounting to 20 posters. Competing individuals or teams, made of both MSM and non–queer-identified individuals, were provided instructions for developing promotional materials related to HIV testing including posters, which were then reviewed by expert panels of judges including researchers, community health leaders, public health and marketing experts, and business leaders. Criteria for selection included experts' opinions on whether entries exhibited potential to (1) reach yet untested individuals and (2) generate excitement and responsiveness to the local epidemiological and social content. Although the full sample of 20 crowdsourced posters were originally included in the analysis, 4 were later removed after authors determined that the content was not directly related to HIV testing, resulting in a final sample size of 16 crowdsourced posters A small subset of each of these two poster types are shown in Figure 1.

Figure 1: Subset of posters created by CDC's (left panel) and through crowdsourcing (right panel).

Visual coding of posters took place through 3 iterative rounds (2 trainings and a final coding round) to guide codebook development and improve intercoder reliability. The initial codebook was informed by a review of the visual design literature^31s and then expanded to adapt codes to the relevant cultural setting (mainland China) and subject matter (HIV testing). The final selection of 27 visual features were interpreted according to conventions of visual design research with additional interpretations created through group discussion (M.K.S., R.F., Y.Z., B.C., and A.J.L.; Table S1, https://links.lww.com/OLQ/A511). Coding was conducted by 3 coders—2 research assistants (R.F. and Y.Z.) and a communications expert (B.C.). The 2 training sessions were conducted using samples of CDC posters that were not in the final analysis sample. Discrepancies in coding designations were discussed by the same group members as for code interpretations and used to inform iterative revisions of the codebook, the final version of which was used for the final coding round. The interrater reliability Kappa statistic following the 2 sessions improved from 63.1% to 71.6% and then to 66.1% in the final coding session.

Phase 2: Viewer Response Survey

Eligible participants were at least 16 years of age, had ever had anal sex with another man, and were born biologically male (respondents of all gender identities were included in the analysis). The survey was self-administered online as part of a parent study on sexual health issues affecting MSM. During piloting, we initially showed each participant the full sample of posters (20 CDC posters and 16 crowdsourced posters) but learned that volunteers suffered substantial survey fatigue with so many posters. In response, we administered the final survey by only showing each a set of only 12 posters randomly drawn from the full sample of 36 posters. A subset of respondents was excluded from the final analysis due to the fact that they provided the same answer for every poster (details in Results section).

For each poster, participants were asked a set of 3 questions corresponding to the 3 viewer response constructs. The first, attention^32s, was assessed using the following statement: “This poster grabbed my attention;” the second, perceived relevance,^33s,34s with the statement, “This poster seemed to be designed personally for me;” and the third, cognitive elaboration,^35s with the statement, “This poster made me think about HIV and HIV testing.” For each statement participants indicated their level of agreement on a 4-point Likert scale (“strongly disagree” to “strongly agree”). Accuracy of the English-to-Chinese translation was discussed by bilingual members of the team (authors M.K.S., B.C., C.L., R.F., and Z.Y.), and participants of the survey pilot were invited to submit comments on comprehensibility of questions (no comments were received).

Statistical Analysis

To answer our first research question of whether crowdsourcing and CDC posters differed visually, we compared the proportions of posters exhibiting each of the 27 visual features across poster type using overlap of 95% confidence intervals (CI) of sample proportions.

To determine whether crowdsourced posters were more likely to elicit viewer response as compared with CDC posters (our second research question), we used generalized estimating equation (GEE) regression models to estimate the effect of poster type (ie, crowdsourced versus CDC) on each of the 3 viewer response constructs (ie, attention, relevance, elaboration). We used an exchangeable working correlation structure to account for clustering at the subject level. Given our reduced statistical power due to measures taken to reduce survey fatigue (ie, each subject evaluated fewer posters than in the initial study design), we collapsed the outcome from a 4-point scale to a binary outcome coded as “1” if respondents to a given question with “agree” or “strongly agree”, and a “0” if they indicated “disagree” or “strongly disagree.” The main exposure of poster type was coded as crowdsourced = “1” and CDC = “0.”

Our final question sought to identify which of the 27 visual features were associated with a positive viewer response regardless of the poster type. Univariable logistic models were used to identify visual features predictive of each of the 3 key response constructs. Confounding between viewer response and poster type was addressed by random assignment of posters to all study participants; hence there was no need for adjustment in the final model.

RESULTS

Poster Characteristics

A comparison of visual design features between the 2 poster types (Table 1) showed that crowdsourced posters displayed more features of higher craftsmanship (ie, design simplicity, image diversity, color choice, design quality, and use of text). In terms of messaging, however, CDC posters made more frequent use of positive facial expressions (65%; 95% CI, 40.9–83.7) and an educational style (12.5%; 95% CI, 2.2–4.0%).

TABLE 1: Visual Content Characteristics of CDC and Crowdsourced HIV Testing Posters

Online Survey Respondent Characteristics

A total of 507 participants took part in our online survey among whom 414 (81.7%) provided useable answers for analysis (Table 2). Overall the sample was younger (median age, 26 years), not currently married (ie, to a woman; 91.5%), and of high socioeconomic status (that is, 78.7% had at least a high school education, 79.4% were classified as being of higher income according domestic census conventions,^36s and 67.9% had higher skill jobs). The majority (75.4%) identified as gay, and similar proportions (86%) reported having a male gender identity (as opposed to “female,” “transgender,” or “other”) and to having sex with another man in the last 6 months (74.9%). About a third indicated having a stable male sexual partner at the time of the survey, defined in the survey as being in a monogamous sexual partnership with another man. Nearly 40% reported having rural residency status, a higher rate than the overall estimate of rural migrants living in Chinese cities.^37s

TABLE 2: Descriptive Characteristics of the 414 Participants of the Online Viewer Response Survey

A comparison between the analysis sample (the 414 participants who provided useable answers to poster-related questions) and the full sample (the 507 eligible participants) indicated that the 2 groups were comparable in terms of all known characteristics (Table S2, https://links.lww.com/OLQ/A511).

Association Between Poster Features and Viewer Response

We used univariable logistic regression to test our hypothesis that crowdsourced posters would be associated with any or all of the viewer constructs, as compared with CDC posters. In these analyses, we found no significant association between poster type (crowdsourced vs. CDC) and any of the viewer constructs, whether it be attention (odds ratio [OR], 0.93; 95% CI, 0.81–1.03), relevance (OR, 0.96; 95% CI, 0.86–1.08), or elaboration (OR, 1.03; 95% CI, 0.93–1.13; results not shown in tables).

We then used another set of univariable logistic regressions to assess whether each of the 27 visual features examined by our coding team were associated with the same 3 response constructs, regardless of whether these visual features were found in crowdsourced or CDC posters (Table 3). Several factors were found to simultaneously increase odds of a positive response for all 3 enhance viewer responses, including depiction of the HIV ribbon in the poster (OR, 1.22; 95% CI, 1.12–1.34 for attention; OR, 1.13; 95% CI, 1.03–1.23 for relevance; and OR, 1.11; 95% CI, 1.02–1.21 for elaboration) and use of a motivational tone (OR, 1.43; 95% CI, 1.27–1.60 for attention; OR, 1.12; 95% CI, 1.01–1.25 for relevance; and OR, 1.27; 95% CI, 1.15–1.42 for elaboration). In terms of visual features that detracted from the viewer responses, use of an admonishing tone (OR, 0.69; 95% CI, 0.58–0.81; OR, 0.76; 95% CI, 0.66–0.89 for relevance; and OR, 0.61; 95% CI, 0.53–0.72 for elaboration), depiction of famous people in the poster (OR, 0.64; 95% CI, 0.52–0.79; OR, 0.78; 95% CI, 0.66–0.93 for relevance; and OR, 0.76; 95% CI, 0.65–0.90 for elaboration), and inclusion of nonhuman figures (OR, 0.70; 95% CI, 0.61–0.81; OR, 0.87; 95% CI, 0.77–0.98 for relevance; and OR, 0.78; 95% CI, 0.68–0.88 for elaboration) all decreased odds of a positive response for all 3 constructs.

TABLE 3: Associations Between Visual Features of Posters and Each of the 3 Response Constructs

DISCUSSION

Our analysis of HIV testing posters and viewer responses to them indicated that although crowdsourced and CDC posters differ visually, their capacity to elicit positive viewer responses did not vary substantially, as evidenced by the lack of association between poster type (crowdsourced vs. CDC) and viewer response (attention, relevance, and elaboration). Past evaluations of crowdsourcing efficacy to expand HIV testing in Chinese MSM have found the method to be scalable and cost-effective.¹⁵ Our findings build on this evidence by demonstrating that crowdsourcing is capable of producing health promotion materials of comparable visual persuasiveness as CDC posters but at a fraction of the programmatic costs.

A comparison of the visual characteristics revealed several substantive design differences between the 2 poster types. First, more crowdsourced posters exhibited features of good craftsmanship as defined through by the conventions of design research (details on specific measures provided in Table S1, https://links.lww.com/OLQ/A511). This finding may suggest that not only are crowdsourced posters more cost effective but that they can be adopted without sacrificing quality. Second, crowdsourced posters were more likely to use metaphorical language or imagery than CDC posters, possible evidence of ways in which marginalized communities (eg, MSM) utilize “secret coding” to convey hidden messages intended for other members of their subgroup.^38s Third, CDC posters were more likely to use positive emotional representations (ie, smiling faces) as well as a more educational messaging style than CDC posters. Both features may be stylistic remnants of Chinese social development campaigns in the 1950s and 1960s which expressed values of national optimism and well-being characteristic of socialist realism.^39s,40s

Associations between specific visual features and viewer response constructs provided useful insights, particularly when they were significantly associated with all 3 response constructs (attention, relevance, and elaboration). One such feature, the HIV ribbon, was found to enhance viewer responsiveness, possibly due to its function as a reference to influential organizations, such as the Joint United Nations Program on HIV/AIDS or the World Health Organization. Such associations may confer more gravitas and credibility on the poster content, thus serving as a peripheral cue for expertise and trustworthiness.²¹ One of the features that detracted from viewer response was the depiction of famous people such as celebrities or political figures. Previous work by Ohanian et al^41s has demonstrated ways in which the credibility of celebrity endorsements relate to their perception as attractive, trustworthy, and having expertise. Though our survey did not measure attitudes regarding the depicted individuals, the dominance of political figures within our poster sample suggests that members of the political elite may not be ideal endorsers for causes, such as HIV testing. The presence of nonhuman characters also detracted from viewer response, possibly because many were cartoons—that is, nonrealistic or semi-realistic depictions—which may have been perceived as childish or infantilizing. Though cartoons and other youth-oriented techniques effectively elicit response in adolescents, the same research suggests that they may have no effect on adult viewers.^42s Lastly, the tone assumed by these posters significantly predicted viewer response, though in divergent ways: motivational tones enhanced viewer responsiveness, whereas authoritative tones detracted from it. This finding is notable in light of past findings of youth aversion to authoritative messaging styles in the prevention of tobacco^43s and alcohol,^44s and in light of the fact that crowdsourced posters were far more likely to use an admonishing tone compared with CDC posters.

Of note, metaphors were not found to significantly affect viewer response and even detracted in the case of elaboration. This finding was both unexpected and noteworthy given the tendency for crowdsourced posters to use these techniques as well as existing evidence of the efficacy of such techniques in inducing positive inferences or cognitive elaboration in experimental subjects ^28,45s. The lack of association between metaphor use and a positive viewer response in our sample may, however, be evidence of a comprehension gap between designers and viewers, a common outcome in the use of abstract metaphors or those with high conceptual tension ^46s,47s.

Results reported here should be considered in light of several limitations. First, our sample size was suboptimal given the limited number of posters viewed by each person and because of the not insignificant portion of observations (18.3%) excluded due to low reliability. Although a smaller sample size weakened the precision of our estimates, it is important not to conflate our failure to detect significant differences with evidence of no difference. A second limitation was the subjective nature of many of our visual codes (eg, appropriate color choice, poster craftsmanship, etc.) for which agreement may have been more difficult to achieve even in spite of training. Indeed, intercoder reliability was relatively low compared with other studies ^32s, even after 2 trainings. The fact that coder trainings were conducted using only CDC posters (a decision informed by our need to preserve the limited sample of crowdsourced posters for the full analysis) may also have biased the refinement of these codes in favor of content more commonly found government posters. However, because the original codes were informed by established theories of visual content analysis, we feel this bias is likely minimal. Third, our decision to apply visual codes borrowed from the field of marketing assumed that visual features optimized for retail and commerce can be functionally applied to effective public health messaging. Although visual complexity, persuasive imagery, and aesthetic evaluations have been explored in some health contexts,^23,32s,48s more rigorous experimentation around the efficacy of specific design features for public health messaging will need to test this assumption. Lastly, our sample appears to have had a relatively high proportion of people with rural residency status (39.1%) as compared with commonly cited population census estimates^37s (about one third of city dwellers are thought to be rural migrants). However, a past survey of rural-to-urban male migrants found that a higher proportion of them reported same-sex behaviors as compared with their nonmigrant counterparts in both rural and urban areas, suggesting that MSM born in rural areas are more likely to migrate to cities, possibly in search of a more sexually tolerant norms.^49s If this theory holds, our sample may be representative of other urban MSM populations in China.

Both crowdsourced and CDC posters used features associated with viewer response including the HIV ribbon and a motivational tone. Both, however, also utilized famous people and cartoon characters, features shown to detract from viewer response. Crowdsourced posters further utilized authoritative messaging and pictorial analogies, also associated with diminished response. These findings underscore the merit of each poster making approach, one which our findings further suggest can lower costs¹⁵ without sacrificing design quality. Intersectoral collaboration may also better align HIV testing poster design with viewer preferences, a critical step for improving HIV test uptake particularly for those at risk of or living with HIV.^50s Results presented here may also be informative in the design of future visual health communication materials where the direct effects on health behaviors will need to be evaluated using a rigorous study design, such as a randomized control trial.