Secondary Logo

Journal Logo

RESEARCH ARTICLES

Gearing Up for a Vaccine Requirement: A Mixed Methods Study of COVID-19 Vaccine Confidence Among Workers at an Academic Medical Center

Mahoney, Megan MD; Winget, Marcy PhD; Brown-Johnson, Cati PhD; de Borba, Lindsay; Veruttipong, Darlene; Luu, Jacklyn; Jones, David; Bohman, Bryan; Vilendrer, Stacie MD

Author Information
Journal of Healthcare Management: May-June 2022 - Volume 67 - Issue 3 - p 206-220
doi: 10.1097/JHM-D-21-00226

Abstract

INTRODUCTION

Vaccination programs are considered one of the greatest achievements in public health. They have contributed to the worldwide decline of mortality and morbidity from various infectious diseases, including influenza, polio, and smallpox (Toner, 2017). The spread of the COVID-19 virus has been an unprecedented economic and humanitarian crisis that prompted the rapid development of vaccines. Numerous studies have shown the ability of vaccines to reduce the transmission and severity of infection from the COVID-19 virus (Polack et al., 2020; Sadoff et al., 2021; Zare et al., 2021). Successful immunization programs, however, rely not only on robust safety data but also on acceptance by the general public and healthcare community to confer population-level protection (Hardt et al., 2016). A lack of confidence in these new vaccines and complacency about completing the COVID-19 vaccine series, in particular, challenge the success of vaccination programs among healthcare workers (HCWs) (Milter et al., 2021). Assessing barriers to vaccination among HCWs may be particularly important given their leadership in the healthcare field and their respective communities.

Vaccine confidence in general is influenced by various factors, including the role of healthcare providers and parents, vaccine development and testing, public health communication strategies, and overall perceptions of vaccine safety and efficacy (Díaz Crescitelli et al., 2020; Schuster et al., 2015; Smith et al., 2021). Researchers have also identified barriers to vaccination that are unique to COVID-19, including concerns following the rapid development of these vaccines and emergency use authorization status by the U.S. Food and Drug Administration (Puri et al., 2020). In addition, the internet has become a primary source of health information and misinformation about COVID-19, and harmful misinformation on social media channels can further diminish public confidence in vaccines (Hughes et al., 2021; Jennings et al., 2021). Furthermore, demographic characteristics such as younger age, female gender, member of a racial-ethnic group, and low socioeconomic status have been associated with vaccine hesitancy (Fisher et al., 2020; Robinson et al., 2021). Finally, nonmainstream attitudes and beliefs about vaccination may also influence vaccine hesitancy (Soveri et al., 2021).

Healthcare providers are considered one of the most reliable sources of health information by patients and the general population and play a core role in influencing patient decision-making regarding COVID-19 vaccination (Katzman & Katzman, 2021). As health experts, they serve as trusted role models to patients and are distinctly positioned to provide patient education about the benefits and safety of the vaccine (Jennings et al., 2021). While the vast majority of HCWs support COVID-19 vaccination, negative attitudes, beliefs, and general skepticism remain among some, resulting in a delay or decline in vaccination (Karlsson et al., 2019). Consequently, HCWs with lower vaccine confidence are less likely to recommend the vaccine to patients (Dzieciolowska et al., 2021). Unvaccinated HCWs are also at greater risk of contracting COVID-19 and can become a health threat to both the patient and healthcare workforce population (Cavanaugh et al., 2021). Assessing and addressing vaccine hesitancy in the healthcare workforce can therefore create a safer work environment for HCWs themselves and may improve vaccination rates in the broader patient and general population.

Several efforts have been made to promote vaccination uptake in HCWs and the general population, and some evidence suggests that multicomponent, dialogue-based interventions may be most effective (Jarrett et al., 2015). Promotional messaging that relies on principles from psychology and behavioral economics has also recently been explored for COVID-19, though the evaluation of such efforts is nascent (Milkman et al., 2021; Wood & Schulman, 2021). Understanding how HCWs respond to COVID-19 vaccination promotion efforts is needed. This research used mixed methodologies to explore the perspectives of HCWs regarding COVID-19 vaccine confidence.

METHODS

We developed a survey to assess factors related to COVID-19 vaccine hesitancy. The survey was conducted at Stanford Medicine, a large multisite academic medical center. Questions covered information sources and messaging that influenced a respondent’s decision to be vaccinated and barriers to vaccination for those who were unvaccinated. The survey design was informed by consumer and behavioral economics research on COVID-19 vaccine promotion strategies (Wood & Schulman, 2021). The survey was administered to all Stanford Medicine employees, including clinical and nonclinical faculty and staff (referred to as HCWs) between April 16, 2021, and June 9, 2021, and was offered in English and Spanish. The following information was collected: demographics; profession and years in the profession; vaccine status; factors that led to receiving a vaccine for those vaccinated and reasons for delay for those not vaccinated or those who delayed vaccination; sources of information and messages that might influence vaccine decision-making; and attitudes about scheduling convenience/access, vaccine safety, and vaccine clinical appropriateness. (See the survey in English and in Spanish, provided as an appendix to this article, published as Supplemental Digital Content at https://links.lww.com/JHM/A71.) Optional free-text comment boxes assessed reasons for delay in vaccination and sources of information and messages that might have influenced vaccine decision-making among HCWs who were unvaccinated. Comments without meaningful content (i.e., “not applicable”) [n = 17] and rare positive comments [n = 2] were not included in the analysis. Data were collected and managed in a REDCap (Research Electronic Data Capture) secure research database, hosted at Stanford University (Harris et al., 2019; Harris et al., 2009).

ANALYSIS

Participants who selected one of the following options were classified as “not vaccinated”: had not scheduled a vaccine, did not intend to receive a vaccine, or were undecided. The remaining participants were classified as “vaccinated.” We calculated frequencies and percentages in total and by vaccination status. A χ2 test, Fisher’s exact test, or Monte Carlo estimate for the Fisher’s exact test was used to assess statistical significance; p < .05 was considered statistically significant. Statistical analyses were performed using SAS (Version 9.4). The figures were created using R3.6.3 software (Wickham, 2016).

Our qualitative analysis research lead (M.M.) reviewed the data to develop an initial codebook; the team then consolidated these initial codes into seven broad codes. We conducted the thematic analysis using a combined inductive/deductive approach. Deductive codes were derived from the survey content and vaccine hesitancy literature (Biswas et al., 2021; Ciardi et al., 2021; Harrison et al., 2021; O’Brien et al., 2021). We used an electronic spreadsheet to code 10% of the responses from each question collaboratively and adjusted codes accordingly with consensus approaches (Cohen & Crabtree, 2008; Johnson et al., 2020). Two coders (L.B., J.L.) independently coded another 10% of responses, with greater than 90% agreement on primary codes. The two coders then evenly split the coding of the remaining data. Disagreements or uncertain codings were resolved by consensus in weekly meetings (M.M., C.B.J., S.V., J.L., L.B.). Per best practices for group consensus, we emphasized internal disagreement with the qualitative team. Previous studies have shown that eliciting and exploring minority opinions in scientific endeavors result in higher quality output (Park & Deshon, 2010).

Themes were identified that corresponded with, challenged, and/or elaborated on the quantitative survey findings. We then integrated the findings from both quantitative and qualitative methods to finalize the interpretation using a mixed methods approach (Fetters et al., 2013; O’Cathain et al., 2010). The local institutional review board determined that this evaluation was nonresearch (protocol 60841).

RESULTS

Survey Participant Characteristics

A total of 5,917 employees completed the survey (5,896 English and 21 Spanish), for a response rate of approximately 20%. Compared to the entire workforce, the sample population was older, with 5% greater representation in the 45 to 54-year-old and 55 and older categories. The sample population had 10% greater representation of employees identifying as White. The representation of nurses and physicians in the sample was 10% lower than that in the entire employee population. A large majority of respondents (93%) were vaccinated. We included 198 free-text comments from unvaccinated employees in the qualitative data analysis.

Table 1 shows the demographic characteristics of the survey participants, personal experience with a COVID-19 diagnosis, and distribution of professions by vaccination status. The majority of respondents were female (76%) and slightly more than half (55%) had worked in their profession for 10 or more years. Compared to vaccinated participants, unvaccinated participants were younger (60% younger than 44 years vs. 48%, respectively). In addition, more unvaccinated respondents were from a non-Asian minority group (48% vs. 22%), and unvaccinated respondents were more likely to be nonclinical (57% vs. 46%). Further, unvaccinated respondents were more likely to have had COVID-19 or a family member diagnosed with COVID-19 than those who were vaccinated (55% vs. 33%).

TABLE 1 - Demographic Characteristics of all Participants, by Vaccination Status
Total (N = 5,917) Vaccinated (n = 5,503) Not Vaccinated (n = 414)
Demographics No. % No. % No. % p
Age, years << .001
 < 35 1,352 23 1,224 22 128 31
 35–44 1,557 26 1,436 26 121 29
 45–54 1,367 23 1,293 23 74 18
 ≥ 55 1,498 25 1,436 26 62 15
 Missing 143 2 114 2 29 7
Gender << .001
 Female 4,508 76 4,197 76 311 75
 Male 1,262 21 1,196 22 66 16
 Other 29 0 17 0 12 3
 Missing 118 2 93 2 25 6
Race/ethnicity << .001
 White 2,747 46 2,614 48 133 32
 Asian 1,657 28 1,602 29 55 13
 Latinx 818 14 715 13 103 25
 Black 216 4 181 3 35 8
 Other 368 6 309 6 59 14
 Missing 111 2 82 1 29 7
Diagnosed with COVID-19? << .001
 Yes—me 319 5 248 5 71 17
 Yes—household family member 456 8 377 7 79 19
 Yes—non-household family member 1,578 27 1,437 26 141 34
 No 3,852 65 3,666 67 186 45
Years in profession .001
 < 5 1,337 23 1,226 22 111 27
 5–9 1,245 21 1,145 21 100 24
 ≥ 10 3,243 55 3,058 56 185 45
 Missing 92 2 74 1 18 4
Profession << .001
 Physician, advanced practice provider 896 15 883 16 13 3
 Registered nurse 1,075 18 1,016 18 59 14
 Other clinical 1,115 19 1,028 19 87 21
 Nonpatient facing 2,756 47 2,519 46 237 57
 Missing 75 1 57 1 18 4

Sources of Vaccination Information

Among unvaccinated respondents, 53% indicated that they would be influenced by their healthcare provider. Table 2 shows that unvaccinated respondents were more influenced by social media (38%), family/friends (11%), peers (12%), and community/church leaders (7%) compared with vaccinated respondents.

TABLE 2 - Sources of Information That Influenced Vaccine Decision-Making, by Vaccination Status
Total (N = 5,917) Vaccinated (n = 5,503) Not Vaccinated (n = 414)
Sources of Information No. % No. % No. %
Social media/website 237 4 79 1 158 38
Peers 668 11 617 11 51 12
Friends/family 371 6 326 6 45 11
Community leaders 30 1 2 0 28 7
Town halls 219 4 191 3 28 7
Supervisor 83 1 74 1 9 2
Healthcare provider 221 4 a a 221 53
CEO or dean 26 0 a a 26 6
Brochure, small group, video 87 1 a a 87 21
Concern for others 5,123 87 5,123 93 a a
Assist with scheduling 20 0 20 0 a a
Other 222 4 222 4 a a
aNot an option for the vaccination status group.

Preferred Messaging Around Vaccination

Although none of the messaging options to support vaccination resonated with a majority of respondents, 36% selected “appointment with a nurse or physician to address how the vaccine might interact with your unique health conditions” as an approach that might influence them to get vaccinated. One participant commented, “If the risk is low enough, I will get it and if my hematologist agrees.” The second most frequent response (26%) was “one or more days off of work to be vaccinated with fun home care package to recover” (Figure 1). Many of the related comments involved suggestions regarding time off for recovery: “I’ve heard of some extreme side effects so [I] would like guaranteed time to recover if needed instead of using PTO. I’m a new employee so haven’t accrued much PTO yet.”

F1
FIGURE 1:
Distribution of Vaccine Messages Among Unvaccinated Healthcare Workers (n = 414)

Barriers Among Unvaccinated Participants

The most common reason for delaying vaccination was the desire for more studies showing safety of the vaccine (67%) (Figure 2). Similarly, only 41% agreed/strongly agreed with the statement, “I have all the information I need to make a decision about the COVID-19 vaccine.”

F2
FIGURE 2:
Distribution of Reasons for Vaccine Delay Among Unvaccinated Healthcare Workers (n = 414)

Almost 30% of unvaccinated respondents provided written comments to elucidate their reasons for not receiving the vaccine. Key barriers in order of prevalence included the need for more long-term research, a belief in the right to make an individual choice, mistrust, a desire for greater public health information, personal health concerns, unique circumstances such as prior COVID-19 infection, and access issues. The need for more research was the dominant theme of provided comments. These comments focused on concerns regarding the lack of long-term research on this vaccine, including the belief that the vaccine has the ability to change DNA. One participant commented, “If it does change my DNA, what problems may I face down the road?” Many cited the lack of U.S. Food and Drug Administration approval and need for a better public education campaign. Others relayed an emotional component; the decision to get the vaccine was perceived as scary, and participants were fearful of making the wrong decision, preferring to wait for others to go first to assess long-term risks. One respondent stated, “It’s unethical and negligent to say the vaccine is safe when there’s zero long-term studies done on an experimental vaccine. The mRNA vaccine has never been done before like this and people shouldn’t be manipulated into taking part [in] a clinical vaccine trial.”

Of note, 19% of unvaccinated respondents reported that nothing would influence them to get vaccinated. When examining the comments left by these individuals, we identified themes of right to make an individual choice and mistrust. Many voiced a desire for ownership of the decision. One wrote, “The vaccination no longer feels like an option but the only choice that is becoming a forced solution instead of an opportunity.” Another person noted that “the right to choose should always remain. My body. My choice.”

Comments reflecting a mistrust in some combination of government, the health system, and the pharmaceutical industry were also prevalent; respondents expressed concerns that the vaccines have been politically and/or financially motivated. Individuals commented that they did not want to be a guinea pig, had discomfort with vaccines in general, or preferred to build their own natural defenses against COVID-19 without the vaccine. One individual commented, “None of the messaging would help me decide to get vaccinated. There is currently no data that shows the long-term effect of the vaccine.” Another respondent noted, “I feel that this is a way to try and scare people into getting vaccinated or brainwash people to get vaccinated. Everything you see makes it seem like you can go back to having a normal life so long as you get vaccinated first!” Some strongly resented public and occupational pressure to be vaccinated, seeing it as a personal attack: “I’ve felt bullied, singled out, and harassed for personal choices that are honestly no one’s business.… Nothing will convince me but time and data.”

Such mistrust was related to the next most prevalent theme: concerns about the public dialogue surrounding vaccination. Several comments expressed a concern that potential side effects from the vaccine were not adequately represented and discussed: “Take the side effects and deaths from the vaccine seriously. Educate that [COVID-19 vaccination] is risky. Review VAERS [Vaccine Adverse Event Reporting System] facts and then let people decide.” Without frank conversations about potential side effects, some felt they were unable to make an informed choice: “It makes me suspicious of why no one wants to address it [occurrence of side effects]. Address it so I can make a more informed decision on long-term side effects of both vaccine and virus and weigh them in the balance.” Some respondents commented that other considerations were largely ignored in the public dialogue, such as prior immunity following infection from the wild virus itself, “the fact that young people have minimal risk from COVID,” or criticism of the perceived public message that the vaccine is the cure for COVID.

Survey respondents also expressed concerns relating to how the vaccine might affect personal health. Their comments pertained to family planning issues such as pregnancy, immune-related disorders including autoimmune disease and severe allergies, and cardiac or hematologic conditions. A handful of comments did not describe a specific condition but pointed to a “personal health concern” or “complex medical background.” Many felt their condition necessitated further data: “My disorder is extremely rare. I need to wait for more information given the deaths/serious consequences in people with relevant/similar conditions.” One individual reported being advised to wait until further information presented itself: “I have a robust medical team and have been advised to wait based on my personal medical condition. My family is vaccinated.”

Some comments were not related to an underlying medical condition. They predominantly came from individuals with a history of COVID-19 infection, many of whom felt a vaccination was no longer clinically indicated: “It literally makes no sense to get vaccinated after you’ve already had COVID.” Another respondent suggested, “No one is talking about the natural immunity process for those of us who previously had COVID-19. If the vaccine only protects against certain strands and new strands have been identified, then why the push for this vaccine if I already had COVID and recovered?” Other respondents expressed concerns that those who lived alone could not be monitored for vaccine side effects, and a few reported that loved ones experienced a severe adverse effect to the vaccine.

Finally, a few practical implementation barriers were noted in the comments, such as difficulty obtaining an appointment for either a preferred vaccine or any vaccine at all and an inability to obtain the vaccine at one’s place of work. Other participants described wanting to get vaccinated outdoors and wanting compensated time off to recover from possible side effects.

DISCUSSION

This mixed methods evaluation of a survey completed by HCWs at a large academic medical center sheds light on which HCWs might choose vaccination and why. Compared to vaccinated participants, unvaccinated participants were younger, more likely to be from a non-Asian minority group, and more likely to work in a nonclinical environment. These findings are consistent with prior research regarding the demographics associated with vaccine hesitancy in the general population (Biswas et al., 2021; O’Brien et al., 2021). Consistent with other studies (Ciardi et al., 2021; Harrison et al., 2021), unvaccinated respondents to our survey were predominantly concerned with vaccine safety, citing the need for additional research. Regular messaging that outlines the latest scientific data may ultimately shift the perspectives of these individuals.

A belief in the right to make an individual choice, mistrust, and a desire for more transparent public information about side effects were also major themes from unvaccinated respondents. Mistrust in particular, including specific beliefs that the mRNA vaccine technology has the ability to change DNA, fear of political or financial motivations, and concerns about not wanting to be a guinea pig, may be a more difficult barrier to overcome, and further dialogue may be needed. Finally, strategies endorsed by some participants to address their concerns about safety and access include a communication campaign, personalized medicine approaches (e.g., individual appointments to discuss how the vaccine might interact with personal health conditions), and days off to recover.

Similar to other studies on HCW vaccine hesitancy, our results suggest that workers who had not received the vaccine were more likely to be younger, nonphysicians, and/or Latinx or had concerns surrounding vaccine safety (Biswas et al., 2021; Toth-Manikowski et al., 2022). A surprising proportion of unvaccinated respondents were more likely to report that they or a family member had had COVID-19. While one might conclude that a personal or family experience with COVID-19 would make one more likely to get the vaccination, the opposite appears to be true. Possible explanations include presumed immunity from a prior infection, resulting in reluctance to obtain the vaccination despite recommendations in this population. The reluctance to be vaccinated may also reflect the broader driver of lower socioeconomic status being related to health literacy, even among individuals working in a healthcare system. In our sample, unvaccinated individuals were more likely to attribute their vaccine-related decision to social media than were vaccinated individuals. These individuals may have more difficulty obtaining accurate information and other COVID-19–specific resources (Vilendrer et al., 2021). Further, individuals with fewer resources, such as essential workers who were required to work on-site during lockdown, were more likely to contract COVID-19. Thus, lower socioeconomic status and difficulty accessing health literature may combine to be stronger forces than a personal or family experience with COVID-19 with respect to vaccine-related decisions.

These findings suggest a range of strategies that healthcare organizations can implement to bolster confidence in the vaccine. For example, patient communications could focus on presenting the latest data pertaining to vaccine efficacy and safety, including a transparent discussion of potential side effects. Such communication could incorporate first-person storytelling to affirm the value of vaccines and showcase personal choice, as well as proactive presence in community engagement across all demographic groups. To combat distrust, healthcare organizations could tap several trusted leaders, physicians, and/or researchers to serve as spokespeople; in that capacity, they could present factual information and describe their personal experiences with the vaccine. Consistently relying on experts who are known and respected in the organization can help build trust and begin to shift the perspectives of those who are unvaccinated. Finally, making vaccination appointments more convenient, allowing HCWs to choose the vaccine they wish to receive, and supporting individuals with compensated time off to address potential side effects and/or making medical monitoring available to those who live alone or have specific health concerns (e.g., fear of an autoimmune response) may help overcome barriers to vaccination.

These research findings informed our broader communication strategies and intervention approaches to improve COVID-19 vaccine confidence at our academic medical center. To support management and leaders in discussing the COVID-19 vaccine with their teams, a “vaccine confidence manager toolkit” was created. The toolkit—consisting of talking points, conversation guides, a trusted external resource list, and other resources—empowered managers by providing them with appropriate language and tactics to use when discussing the vaccine. Additional communication tools, such as pocket cards, were developed to aid providers as they navigated vaccine conversations with patients. Other communication campaigns, including our “Voices of COVID” blog series (Stanford Medicine, n.d.) and a video series in which faculty members share their reasons for deciding to get vaccinated, were promoted both internally to our staff and externally to our patient community by way of our social media channels.

The survey results showed significant differences in vaccine uptake and confidence across demographic groups; thus, an important part of our communication efforts targeted specific groups. Social media campaigns—including Facebook ads and Twitter and Instagram posts—and patient messages, created by our patient experience team, were launched to reach our Latinx and Pacific Islander communities. Our inclusion, diversity, and health equity team developed an “I Got the Vaccine” video featuring members of employee resource groups who talked about their experience receiving the COVID-19 vaccine. The team also put on a virtual panel event that was open to the entire Stanford Medicine community. The event featured selected employee resource group members, including those from populations underrepresented in medicine (such as Black/African American and Latinx HCWs) who were found to have lower vaccination rates. The employees candidly shared personal stories and experiences pertaining to the vaccine. As a result of these customized communication strategies, the vaccination acceptance rose steadily, reaching near universal vaccination (> 99.9%) after a vaccine mandate was put into place. We believe our communication strategy—proactive and focused on empathy-building—was an important step prior to mandating the vaccine.

Study Limitations

This study had several limitations that should be considered. The response rate of 20% risks a selection bias, typically toward individuals with strong feelings in either direction about the survey topic. The proportion of individuals in our sample who were vaccinated (93%) was higher than that in the overall HCW population (approximately 85%) at our healthcare system at the time of survey distribution, suggesting a bias toward vaccine confidence in our sample. Despite this difference, we achieved our primary aim of better understanding the perspectives of unvaccinated individuals within the context of a mixed methods approach. The analysis of these qualitative data reached thematic saturation, suggesting that the full range of perspectives was heard. Further, the sex distribution and racial-ethnic diversity of our sample population largely reflect the HCW demographics within the institution. Interviews and focus group discussions would likely provide more in-depth understanding of barriers to and facilitators of COVID-19 vaccination but were less feasible considering our compressed timeline. Finally, our setting is a multisite academic medical center and, therefore, insights are limited to this setting.

CONCLUSION

The results of this study might be useful to other healthcare systems as more hospitals are requiring HCWs to receive COVID-19 vaccines. Surveying HCWs might be a useful first step to understanding and responding to the perspectives of unvaccinated individuals as healthcare systems develop communication and implementation strategies. Vaccination of HCWs is a vital piece of the COVID-19 puzzle, not only in terms of protecting patients and HCWs themselves, but also because these HCWs serve as positive influencers who may be able to persuade the community at large to get vaccinated and overcome the pandemic. The landscape continues to evolve, particularly with the introduction of vaccine mandates, which were not in place when the survey took place. Understanding their impact is another area for future research.

ACKNOWLEDGMENTS

The authors thank Steve Chinn, Emily Harrington, Rachelle Mirkin, Timothy Seay-Morrison, and Joseph Wilson at Stanford Medicine for their help with reviewing and distributing the survey. This project was supported by the Sean N. Parker Center for Allergy and Asthma Research at Stanford University, the Crown Foundation, and the Sunshine Foundation.

REFERENCES

Biswas N., Mustapha T., Khubchandani J., Price J. H. (2021). The nature and extent of COVID-19 vaccination hesitancy in healthcare workers. Journal of Community Health , 46:1244–1251. https://doi.org/10.1007/s10900-021-00984-3
Cavanaugh A. M., Fortier S., Lewis P., Arora V., Johnson M., George K., Tobias J., Lunn S., Miller T., Thoroughman D., Spicer K. B. (2021). COVID-19 outbreak associated with a SARS-CoV-2 R.1 lineage variant in a skilled nursing facility after vaccination program – Kentucky, March 2021. Morbidity and Mortality Weekly Report , 70 (17), 639–643. https://doi.org/10.15585/mmwr.mm7017e2
Ciardi F., Menon V., Jensen J. L., Shariff M. A., Pillai A., Venugopal U., Kasubhai M., Dimitrov V., Kanna B., Poole B. D. (2021). Knowledge, attitudes and perceptions of COVID-19 vaccination among healthcare workers of an inner-city hospital in New York. Vaccines (Basel) , 9 (5), 516. https://doi.org/10.3390/vaccines9050516
Cohen D. J., Crabtree B. F. (2008). Evaluative criteria for qualitative research in health care: Controversies and recommendations. Annals of Family Medicine , 6 (4), 331–339. https://doi.org/10.1370/afm.818
Díaz Crescitelli M. E., Ghirotto L., Sisson H., Sarli L., Artioli G., Bassi M. C., Appicciutoli G., Hayter M. (2020). A meta-synthesis study of the key elements involved in childhood vaccine hesitancy. Public Health , 180, 38–45. https://doi.org/10.1016/j.puhe.2019.10.027
Dzieciolowska S., Hamel D., Gadio S., Dionne M., Gagnon D., Robitaille L., Cook E., Caron I., Talib A., Parkes L., Dubé È., Longtin Y. (2021). Covid-19 vaccine acceptance, hesitancy, and refusal among Canadian healthcare workers: A multicenter survey. American Journal of Infection Control , 49 (9), 1152–1157. https://doi.org/10.1016/j.ajic.2021.04.079
Fetters M. D., Curry L. A., Creswell J. W. (2013). Achieving integration in mixed methods designs—Principles and practices. Health Services Research , 48 (6 Pt 2), 2134–2156. https://doi.org/10.1111/1475-6773.12117
Fisher K. A., Bloomstone S. J., Walder J., Crawford S., Fouayzi H., Mazor K. M. (2020). Attitudes toward a potential SARS-CoV-2 vaccine: A survey of U.S. adults. Annals of Internal Medicine , 173 (12), 964–973. https://doi.org/10.7326/M20-3569
Hardt K., Bonanni P., King S., Santos J. I., El-Hodhod M., Zimet G. D., Preiss S. (2016). Vaccine strategies: Optimising outcomes. Vaccine , 34 (52), 6691–6699. https://doi.org/10.1016/j.vaccine.2016.10.078
Harris P. A., Taylor R., Minor B. L., Elliott V., Fernandez M., O’Neal L., McLeod L., Delacqua G., Delacqua F., Kirby J., Duda S. N. (2019). The REDCap consortium: Building an international community of software partners. Journal of Biomedical Informatics , 95, 103208. https://doi.org/10.1016/j.jbi.2019.103208
Harris P. A., Taylor R., Thielke R., Payne J., Gonzalez N., Conde J. G. (2009). Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics , 42 (2), 377–381. https://doi.org/10.1016/j.jbi.2008.08.010
Harrison J., Berry S., Mor V., Gifford D. (2021). “Somebody like me”: Understanding COVID-19 vaccine hesitancy among staff in skilled nursing facilities. Journal of the American Medical Directors Association , 22 (6), 1133–1137. https://doi.org/10.1016/j.jamda.2021.03.012
Hughes B., Miller-Idriss C., Piltch-Loeb R., Goldberg B., White K., Criezis M., Savoia E. (2021). Development of a codebook of online anti-vaccination rhetoric to manage COVID-19 vaccine misinformation. International Journal of Environmental Research and Public Health , 18 (14), 7556. https://doi.org/10.3390/ijerph18147556
Jarrett C., Wilson R., O’Leary M., Eckersberger E., Larson H. J. (2015). SAGE Working Group on Vaccine Hesitancy. Strategies for addressing vaccine hesitancy: A systematic review. Vaccine , 33 (34), 4180–4190. https://doi.org/10.1016/j.vaccine.2015.04.040
Jennings W., Stoker G., Bunting H., ValgarÐsson V. O., Gaskell J., Devine D., McKay L., Mills M. C. (2021). Lack of trust, conspiracy beliefs, and social media use predict COVID-19 vaccine hesitancy. Vaccines (Basel) , 9 (6), 593. https://doi.org/10.3390/vaccines9060593
Johnson J. L., Adkins D., Chauvin S. (2020). A review of the quality indicators of rigor in qualitative research. American Journal of Pharmaceutical Education , 84 (1), 7120. https://doi.org/10.5688/ajpe7120
Karlsson L. C., Lewandowsky S., Antfolk J., Salo P., Lindfelt M., Oksanen T., Kivimaki M., Soveri A. (2019). The association between vaccination confidence, vaccination behavior, and willingness to recommend vaccines among Finnish healthcare workers. PLoS One , 14 (10), e0224330. https://doi.org/10.1371/journal.pone.0224330
Katzman J. G., Katzman J. W. (2021). Primary care clinicians as COVID-19 vaccine ambassadors. Journal of Primary Care & Community Health , 12, 1–4. https://doi.org/10.1177/21501327211007026
Milkman K. L., Patel M. S., Gandhi L., Graci H., Gromet D., Ho Q. D.H., Kay J., Lee T., Bogard J., Brody I., Chabris C., Chang E., Chapman G. B., Dannals J., Goldstein N. J., Goren A., Hershfield H., Hirsch A., Hmurovic J., … Duckworth A. (2021). A mega-study of text-message nudges encouraging patients to get vaccinated at an upcoming doctor’s appointment. Proceedings of the National Academy of Sciences , 118 (20), e2101165118. https://doi.org/10.1073/pnas.2101165118
Milter, S. M., Phillips, R. A., Schwartz, R. L., Sostman, D., Hackett, C., Bloom, M. L., Schwartz, R. L., Sostman, H. D., Hackett, C., & Boom, M. L. (2021, July 1). How to develop a Covid-19 employee vaccination policy. Harvard Business Review. https://hbr.org/2021/07/ how-to-develop-a-covid-19-employee- vaccination-policy
O’Brien, E., Xu, H., Cohen, L., Shenkman, E., Rothman, R., Forrest, C., & Hernandez, A. (2021, March 3). Recent changes in COVID-19 vaccine hesitancy among healthcare workers [Preprint]. MedRXiv. https://doi.org/10.1101/ 2021.03.01.21252457
O’Cathain A., Murphy E., Nicholl J. (2010). Three techniques for integrating data in mixed methods studies. British Medical Journal , 341, c4587. https://doi.org/10.1136/bmj.c4587
Park G., Deshon R. P. (2010). A multilevel model of minority opinion expression and team decision-making effectiveness. Journal of Applied Psychology , 95 (5), 824–833. https://doi.org/10.1037/a0019939
Polack F. P., Thomas S. J., Kitchin N., Absalon J., Gurtman A., Lockhart S., Perez J. L., Pérez Marc G., Moreira E. D., Zerbini C., Bailey R., Swanson K. A., Roychoudhury S., Koury K., Li P., Kalina W. V., Cooper D., Frenck R. W., Hammitt L. L., … Gruber W. C. (2020). Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. New England Journal of Medicine , 383 (27), 2603–2615. https://doi.org/10.1056/NEJMoa20345775
Puri N., Coomes E. A., Haghbayan H., Gunaratne K. (2020). Social media and vaccine hesitancy: new updates for the era of COVID-19 and globalized infectious diseases. Human Vaccines & Immunotherapeutics , 16 (11), 2586–2593. https://doi.org/10.1080/21645515.2020.1780846
Robinson E., Jones A., Lesser I., Daly M. (2021). International estimates of intended uptake and refusal of COVID-19 vaccines: A rapid systematic review and meta-analysis of large nationally representative samples. Vaccine , 39 (15), 2024–2034. https://doi.org/10.1016/j.vaccine.2021.02.005
Sadoff J., Gray G., Vandebosch A., Cárdenas V., Shukarev G., Grinsztejn B., Goepfert P. A., Truyers C., Fennema H., Spiessens S., Offergeld K., Scheper G., Taylor K. L., Robb M. L., Treanor J., Barouch D. H., Stoddard J., Ryser M. F., Marovich M. A., , … (2021). Safety and efficacy of single-dose Ad26.COV2.S vaccine against COVID-19. New England Journal of Medicine , 384 (23), 2187–2201. https://doi.org/10.1056/NEJMoa2101544
Schuster M., Eskola J., Duclos P. (2015). SAGE Working Group on Vaccine Hesitancy. Review of vaccine hesitancy: Rationale, remit and methods. Vaccine , 33 (34), 4157–4160. https://doi.org/10.1016/j.vaccine.2015.04.035
Smith L. E., Hodson A., Rubin G. J. (2021). Parental attitudes towards mandatory vaccination: A systematic review. Vaccine , 39 (30), 4046–4053. https://doi.org/10.1016/j.vaccine.2021.06.018
Soveri A., Karlsson L. C., Antfolk J., Lindfelt M., Lewandowsky S. (2021). Unwillingness to engage in behaviors that protect against COVID-19: The role of conspiracy beliefs, trust, and endorsement of complementary and alternative medicine. BMC Public Health , 21 (1), 684. https://doi.org/10.1186/s12889-021-10643-w
Stanford Medicine Scope. (n.d.). Voices of COVID. https://scopeblog.stanford.edu/tag/voicesofcovid/
Toner E. (2017). Healthcare preparedness: Saving lives. Health Security , 15 (1), 8–11. https://doi.org/10.1089/hs.2016.0090
Toth-Manikowski S. M., Swirsky E. S., Gandhi R., Piscitello G. (2022). COVID-19 vaccination hesitancy among health care workers, communication, and policy-making. American Journal of Infection Control , 50 (1), 20–25.
Vilendrer S., Amano A., Brown Johnson C. G., Favet M., Safaeinili N., Villasenor J., Shaw J. G., Hertelendy A. J., Asch S. M., Mahoney M. (2021). An app-based intervention to support first responders and essential workers during the COVID-19 pandemic: Needs assessment and mixed methods implementation study. Journal of Medical Internet Research , 23 (5), e26573. https://doi.org/10.2196/26573
Wickham H. (2016). ggplot2: Elegant graphics for data analysis. 2nd Ed. Springer-Verlag.
Wood S., Schulman K. (2021). Beyond politics—Promoting Covid-19 vaccination in the United States. New England Journal of Medicine , 384 (7), e23. https://doi.org/10.1056/NEJMms2033790
Zare S., Dameneh M. M., Esmaeili R., Kazemi R., Naseri S., Panahi D. (2021). Occupational stress assessment of health care workers (HCWs) facing COVID-19 patients in Kerman province hospitals in Iran. Heliyon , 7 (5), e07035. https://doi.org/10.1016/j.heliyon.2021.e07035

Supplemental Digital Content

Copyright © 2022 The Author(s). Published by Wolters Kluwer Health, Inc. on behalf of the Foundation of the American College of Healthcare Executives.