HIV infection in children in sub-Saharan Africa occurs primarily from mother-to-child transmission (MTCT) during pregnancy, childbirth, or breastfeeding. Over the past decade, there have been numerous medical, programmatic, and policy advances to prevent mother-to-child HIV transmission (PMTCT) and reduce pediatric HIV globally.1,2 However, in much of sub-Saharan Africa, these approaches have not been effectively disseminated or implemented1; as a result, MTCT remains a critical public health concern.3
The challenges of implementing evidence-based practices (EBPs) in PMTCT are not unique.4 To address the “knowledge-practice” gap observed in many settings, there has been a growing emphasis on the theoretical processes and mechanisms that underpin effective policy dissemination.5–9 Consistent with this emphasis, there is an increasing focus on implementation science in PMTCT. For example, recent work outlines key implementation considerations including focus on key targets outlined by the United Nations, including primary prevention of HIV among women of child-bearing age and for HIV-infected women, the prevention of unintended pregnancies, prevention of MTCT, and linkage to care for HIV-infected women and children.2 Highlighting the results specific to workforce issues, a recent review focusing on health workers identified a number of barriers including worker motivation, stigma, poor referrals, and capacity of health facilities to deliver care.10 However, multiple stakeholder perspectives should be considered in identifying implementation challenges and developing strategies to accelerate EBP implementation.
Among the greatest challenges in translating evidence-based health care approaches into real-world settings are (1) multiple, at times divergent, stakeholder priorities and perceptions in the implementation process, (2) the diversity of contexts or practice settings, and (3) how adaptations in implementation strategies or EBPs are considered and carried out.11 The experiences and perspectives of different stakeholders may vary greatly depending on their roles (eg, policy-maker, researcher, clinician, and patient).12,13 Although implementation processes are often complex, methods can be used to summarize these experiences and perspectives across projects, phases, and system and organizational levels. This study uses concept mapping (CM), a mixed-methods approach that is well suited for integrating knowledge gained across multiple PMTCT implementation research projects, country and community contexts, and across phases of projects and to garner the perspectives of key stakeholder groups and different levels across service systems.
CM is a powerful and efficient mixed-methods approach to generate an interpretable and useful framework derived from the perspectives of participating stakeholders.14,15 CM moves beyond focus group or survey methodologies to integrate qualitative and quantitative methods in a logical and progressive way that engages stakeholders in the research and theory generation process. It is a useful tool for understanding the implicit dimensions of a phenomenon that may not be readily apparent with typical survey methods. CM has grown from an interest in using theory-driven evaluation,16 to being successfully used mostly in planning and evaluation studies,17 to be generalized for other types of studies, including the implementation of EBPs.12,13,18,19 The resulting products of CM are visual maps that illustrate group ideas and concerns, how the ideas are related to one another in a multidimensional concept space, how the ideas are organized/clustered into general concepts, and how concepts are rated on dimensions such as importance and changeability. The final concept map can be used as a conceptual framework for operationalizing the constructs of interest and/or as the basis for displaying results, developing measures, or action planning.14 In this study, we applied the CM methodology to identify critical issues in PMTCT implementation in the context of sub-Saharan Africa, to determine their relative priority, and to establish areas of consensus and divergence.
Our target population comprised members of the PMTCT Implementation Science Alliance (ISA).1 Supported by the Eunice Kennedy Shriver National Institute of Child Health and Development, the Fogarty International Center, and the President's Emergency Plan for AIDS Relief, the ISA was conceived as a platform for enhanced collaboration and communication among PMTCT researchers, program implementers, policymakers, and donors to extend the reach of implementation science and help overcome implementation challenges. ISA members have multiple perspectives and dimensional vantage points to identify key implementation factors for PMTCT interventions across countries, communities, and stakeholders. The ISA focused on sub-Saharan Africa, with representatives working in Kenya, Mozambique, Nigeria, Zambia, South Africa, and the Democratic Republic of Congo.
This study was approved by the institutional review board at the University of California, San Diego (San Diego, CA), and informed consent was obtained from all participants.
The CM process typically involves 6 phases: (1) preparation—identifying stakeholders and participants and developing focus questions, (2) generation—contributing statements in response to the focus question, (3) structuring—sorting statements based on similarity and rating statements on multiple a priori dimensions (eg, importance and changeability), (4) representation—multidimensional scaling and cluster analysis used to create a “concept map” with statements that were sorted together in closer proximity to each other, (5) interpretation—developing cluster labels and interpretations, and (6) utilization—using results to improve implementation and implementation study design. Below, we describe how this process was applied in this study.
In the preparation phase, the study team collaboratively and iteratively developed a single focus question: “In your experience, what factors have facilitated or hindered implementation of PMTCT intervention?” Members of the ISA were invited to participate via email and in person (at the January 2015 ISA meeting in Johannesburg, South Africa) to submit responses to the study's focus question. Multiple responses from each participant were permitted. Overall, 39 participants provided 161 responses regarding implementation factors. These were carefully reviewed for redundancy or similar meaning by 2 team members (G.A.A. and D.H.S.) and consolidated into 88 distinct statements. ISA members were asked to sort the 88 statements into separate groups (or “clusters”) in a manner they saw fit, using online tools from Concept Systems, Inc.20 Participants were then asked to name the clusters, providing a substantive description what each comprised. Thirty-nine participants completed the online statement sorting.
Participants were asked to rate each statement (using a Likert scale) on multiple dimensions: “How important is this factor for PMTCT program implementation?” (0 = not at all important; 7 = extremely important); “How changeable is this implementation factor?” (0 = not at all changeable; 7 = extremely changeable); and “To what extent has participation in the Alliance enhanced your ability to change/influence this implementation factor?” (0 = not at all; 7 = to a great extent). In this step, 53 ISA members completed at least one set of statement ratings online or using paper instruments.
Statement sorting data were analyzed using multidimensional scaling and hierarchical cluster analysis21 within the CM program. These procedures resulted in the creation of visual representations (ie, concept maps) for how statements were typically clustered across all participants. Multiple concept map solutions were considered based on acceptable overall “stress” fit statistic and interpretability of each potential solution.22 The ideal model would maintain discrimination of key concepts in the fewest number of clusters. This process started by considering a large number of potential thematic clusters (eg, 20) and then, in stepwise fashion, consolidating groups that were most thematically similar based on participant responses. These models were reviewed by the study team, with the final concept map (12 thematic clusters) determined by consensus. Each cluster in the final model was named based on the nature of the statements contained in each cluster. Finally, we examined the overall cluster ratings (as determined by the average rating of the individual statements contained in each cluster) and the correlation between the cluster ratings for perceived importance and enhancement from ISA participation.
Overall, 53 ISA members participated in at least one component of the CM exercise. The self-identified primary PMTCT role of participants were as follows: researcher/academician (45%, n = 24), governmental official/employee (25%, n = 13), nongovernmental organization administrator (9%, n = 5), other experts (9%, n = 5), ISA organizer (4%, n = 2), and nonspecified (8%, n = 4). Participants reported having a median of 10 years of experience [intraquartile range: 5–14] working in the field of PMTCT. The median experience in implementation science was 4 years (intraquartile range: 2–8).
The final model comprised 12 thematic clusters, each representing a key domain of PMTCT implementation (Fig. 1). These clusters were: Logistical and Support Services; Clinic and Provider Services; Personnel Capacity, Training and Support; Leadership-Practice Intersection; Health System Resources, Tracking and Monitoring; Data Measurement and Collection; Funding; Evidence-Based Practice Guidelines; Governmental Commitment; Maternal-Child Clinical Care; Socio-Cultural Issues; and Local Context and Community Engagement. Each dot within a cluster represents a statement that was sorted into similar categories by participants. Lower stress index values indicate a better fit between the concept map and the initial statement sorting data. The stress value of 0.21 for our analyses indicated a good model fit as it was substantially below the 0.29 threshold recommended for CM projects.23
Table 1 lists the mean participant ratings for each cluster and the rank order for the ratings of (1) importance of PMTCT implementation, (2) changeability of the implementation factor, and (3) the extent to which participation in ISA enhanced the ability to change/influence this factor. The “importance” ratings ranged from a low of 5.02 (Local Context & Community Engagement) to a high of 5.64 (Health System Resources, Tracking, and Monitoring). All clusters were ranked substantially above the scale midpoint of 3.5 suggesting moderate to strong importance for PMTCT implementation with some variability across thematic clusters. Similarly, all clusters were perceived as moderately changeable with the lowest changeability cluster rating of 4.33 (Leadership-Practice Intersection) and the highest of 5.32 (Governmental Commitment). The thematic cluster ratings for enhancement through ISA participation ranged from 2.75 (Logistical/Support Services) to 4.14 (Governmental Commitment). The larger variability in the thematic ratings indicated that some areas were minimally affected by ISA participation, whereas others, such as the Governmental Commitment (4.14) and Data Measurement and Collection (3.88), were perceived to be substantially affected by ISA participation.
Overall, there was little correlation between mean cluster importance and cluster changeability (r = 0.15) and a small-to-moderate positive correlation between the mean cluster ratings for importance and enhancement by ISA participation (r = 0.31). The strongest correlation was between cluster changeability and enhancement by ISA participation (r = 0.43). Table 1 provides a detailed cluster-by-cluster comparison of the rating and relative ranking across these 3 dimensions. The results show a relatively strong congruence between the clusters determined to be among the most important for PMTCT implementation, most changeable, and most affected by or enhanced through their participation in ISA. Of note, 2 thematic clusters (Governmental Commitment and Data Measurement and Collection) were ranked within the top 3 clusters for each dimension.
Using a mixed-methods CM approach, we identified 12 primary domains that influence PMTCT implementation. Each domain was generally viewed as both “important” and “changeable” even if the correlation between these dimensions was relatively low. The extent to which participation in ISA was perceived to enhance the ability to change or influence the implementation factor varied substantially across the thematic clusters. This is not surprising as certain factors that might be very important for actual PMTCT program implementation (eg, Logistical/Support Services) were beyond the implementation science scope of the ISA. However, the ability to change or influence other domains such as Governmental Commitment and Data Measurement and Collection were perceived to have been enhanced through ISA participation.
We observed a substantial overlap among the most highly rated thematic clusters across our 3 dimensions of interest. Governmental Commitment and Data Measurement and Collection were rated within the top 3 dimension ratings of Importance, Changeability, and being Enhanced by ISA Participation. This suggests that effective targeting of training and resources which were provided by the ISA are consistent with the 2 key components that were considered to be among the most important and most changeable for PMTCT program implementation. These domains were also perceived as the ones with the greatest enhancement as a result of ISA participation. However, some key differences between the rankings were also evident. In particular, the thematic cluster ranked as the most important, Health System Resources, Tracking, and Monitoring, was not ranked as high on Changeability (seventh) or being Enhanced by ISA Participation (fifth). This variation highlights the need for greater attention to health system factors identified as being very important, amenable to change, and enhanced by alliance support, in order to facilitate successful PMTCT program implementation.
The CM approach was novel in the field of PMTCT. We found the approach to be feasible, allowing data collection and analysis to be completed in a relatively short timeframe using Web-based or in-person approaches or a combination of the 2 approaches. For the in-person approach, it is possible to capitalize on meetings where relevant stakeholders will be present. To assure the greatest efficiency, it is important to have a dedicated time on the meeting agenda (eg, 60 minutes) for participants to complete the different activities. For example, brainstorming can take place in a group format where participants can consider, discuss, and edit potential statements in an interactive way. Likewise, sorting and rating activities can be conducted online or in person. We recognize that statement rating across multiple dimensions can be time-consuming and burdensome. Based on this study—and past experiences—we recommend a maximum of 3 unique ratings for each statement. With the 88 distinct statements in our present study, this still resulted in a large number (ie, 264) of responses required per participant. Separation of the sorting and rating tasks into sequential stages could reduce the burden on participants. Participants would not rate each individual statement, but once the number of clusters has been determined (and named), participants would then rate each cluster on the desired dimensions.
Our findings are based on the experiences of PMTCT ISA stakeholders across multiple countries, implementing different clinical interventions, and using different implementation strategies. They provide a robust and evidence-based foundation that can inform future PMTCT implementation science. Taking stock of the barriers, bottlenecks, and available resources in each of these 12 domains could inform strategies to optimize PMTCT implementation and services in the field. Overlaying relevant implementation frameworks could further guide the process. Although there are many theoretical frameworks,24 those that invoke multiple phases of implementation and a multilevel perspective may be most useful in the settings represented in this study.9,25–29 Such an approach should help implementers to more deliberately and thoughtfully approach the complex task of PMTCT implementation and sustainment to address this dire public health concern.
1. Sturke R, Harmston C, Simonds RJ, et al A multi-disciplinary approach to implementation
science: the NIH-PEPFAR PMTCT implementation
science alliance. J Acquir Immune Defic Syndr. 2014;67:S163–S167.
2. Bhardwaj S, Carter B, Aarons GA, et al Implementation
research for the prevention of mother-to-child HIV
transmission in sub-Saharan Africa: existing evidence, current gaps, and new opportunities. Curr HIV
/AIDS Rep. 2015;12:246–255.
3. Kim MH, Ahmed S, Hosseinipour MC, et al Implementation
and operational research: the impact of option B+ on the antenatal PMTCT cascade in Lilongwe, Malawi. J Acquir Immune Defic Syndr. 2015;68:e77–e83.
4. Brownson RC, Colditz GA, Proctor EK. Dissemination and Implementation
Research in Health: Translating Science to Practice. New York, NY: Oxford University Press; 2012.
5. Aarons GA. Measuring provider attitudes toward evidence-based practice: consideration of organizational context and individual differences. Child Adolesc Psychiatr Clin N Am. 2005;14:255–271.
6. Greenhalgh T, Robert G, Macfarlane F, et al Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82:581–629.
7. Mendel P, Meredith L, Schoenbaum M, et al Interventions in organizational and community context: a framework for building evidence on dissemination and implementation
in health services research. Adm Policy Ment Health. 2008;35:21–37.
8. Raghavan R, Bright CL, Shadoin AL. Toward a policy ecology of implementation
of evidence-based practices in public mental health settings. Implement Sci. 2008;3:3–26.
9. Simpson DD. A conceptual framework for transferring research to practice. J Subst Abuse Treat. 2002;22:171–182.
10. Schuster RC, McMahon DE, Young SL. A comprehensive review of the barriers and promoters health workers experience in delivering prevention of vertical transmission of HIV
services in sub-Saharan Africa. AIDS Care. 2016:28:778–794.
11. Cabassa LJ, Baumann AA. A two-way street: bridging implementation
science and cultural adaptations of mental health treatments. Implement Sci. 2013;8:90.
12. Aarons GA, Wells RS, Zagursky K, et al Implementing evidence-based practice in community mental health agencies: a multiple stakeholder analysis. Am J Public Health. 2009;99:2087–2095.
13. Green AE, Aarons GA. A comparison of policy and direct practice stakeholder perceptions of factors affecting evidence-based practice implementation
using concept mapping. Implement Sci. 2011;6:104.
14. Trochim WM. An introduction to concept mapping for planning and evaluation. Eval Program Plann. 1989;12:1–16.
15. Trochim WMK, Cook JA, Setze RJ. Using concept mapping to develop a conceptual framework of staff's views of a supported employment program for individuals with severe mental illness. J Consult Clin Psychol. 1994;62:766–775.
16. Emrick CD, Aarons GA. Program evaluation and treatment outcome: contemporary developments. Ann Rev Addict Res Treat. 1991;1:215–231.
17. Stokols D, Fuqua J, Gress J, et al Evaluating transdisciplinary science. Nicotine Tob Res. 2003;5:S21–S40.
18. Green AE, Fettes DL, Aarons GA. A concept mapping approach to guide and understand dissemination and implementation
. J Behav Health Serv Res. 2012;39:362–373.
19. Naoom SF, Wallace F, Blase KA, et al Implementation
in the Real World—Taking Programs and Practices to Scale: Concept Mapping Report. Tampa, FL: National Implementation
Research Network, Louis de la Parte Florida Mental Health Institute; 2004. Available at: http://nirn.fpg.unc.edu/resources/implementation
-real-world-taking-programs-and-practices-scale. Accessed June 3, 2016.
20. The Concept System Global MAX (Build 2016.046.12) [Web-based Platform]. Ithaca, NY: Concept Systems; 2016.
21. Davison ML. Multidimensional Scaling. New York, NY: John Wiley & Sons, Inc.; 1983.
22. Rosas SR, Kane M. Quality and rigor of the concept mapping methodology: a pooled study analysis. Eval Program Plann. 2012;35:236–245.
23. Kane M, Trochim W. Concept Mapping for Planning and Evaluation. Thousand Oaks, CA: Sage Publications, Inc.; 2007.
24. Tabak RG, Khoong EC, Chambers DA, et al Bridging research and practice: models for dissemination and implementation
research. Am J Prev Med. 2012;43:337–350.
25. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation
in public service sectors. Adm Policy Ment Health. 2011;38:4–23.
26. Fixsen D, Blase K, Naoom S, et al Core implementation
components. Res Soc Work Pract. 2009;19:531.
27. Damschroder L, Aron D, Keith R, et al Fostering implementation
of health services research findings into practice: a consolidated framework for advancing implementation
science. Implement Sci. 2009;4:50–64.
28. Backer T, David S, Soucy G. Reviewing the Behavioral Science Knowledge Base on Technology Transfer. Vol. 155: National Institute on Drug Abuse; Rockville, MD, 1995.
29. Rogers EM. Diffusions of Innovations. 4th ed. New York, NY: The Free Press; 1995.