Secondary Logo

Journal Logo

Confronting Challenges in Monitoring and Evaluation

Innovation in the Context of the Global Plan Towards the Elimination of New HIV Infections Among Children by 2015 and Keeping Their Mothers Alive

Radin, Anna K., MPH; Abutu, Andrew A., MD, MPH; Okwero, Margaret A., MSc; Adler, Michelle R., MD, MPH; Anyaike, Chukwuma, MD, MCommH, DTM & H, FWACP; Asiimwe, Hilda T., MBChB, MPH; Behumbiize, Prosper, MPH; Efuntoye, Timothy A., MD, AFMCP; King, Rachel L., PhD, MPH; Kisaakye, Linda Nabitaka, MBChB, MPH DPPM; Ogundehin, Dolapo T., MEd; Phelps, Benjamin Ryan, MD, MPH; Watts, Heather, MD; Weissglas, Fitti, BSc, MSc, MBA

JAIDS Journal of Acquired Immune Deficiency Syndromes: May 1st, 2017 - Volume 75 - Issue - p S66–S75
doi: 10.1097/QAI.0000000000001313
Supplement Article
Free

Abstract: The Global Plan Towards the Elimination of New HIV Infections Among Children by 2015 and Keeping Their Mothers Alive (Global Plan), which was launched in 2011, set a series of ambitious targets, including a reduction of new HIV infections among children by 90% by 2015 (from a baseline year of 2009) and AIDS-related maternal mortality by 50% by 2015. To reach these targets, the Global Plan called for unprecedented investments in the prevention of mother-to-child transmission of HIV (PMTCT), innovative new approaches to service delivery, immense collective effort on the programmatic and policy fronts, and importantly, a renewed focus on data collection and use. We provide an overview of major achievements in monitoring and evaluation across Global Plan countries and highlight key challenges and innovative country-driven solutions using PMTCT program data. Specifically, we describe the following: (1) Uganda's development and use of a weekly reporting system for PMTCT using short message service technology that facilitates real-time monitoring and programmatic adjustments throughout the transition to a “treat all” approach for pregnant and breastfeeding women living with HIV (Option B+); (2) Uganda's work to eliminate parallel reporting systems while strengthening the national electronic district health information system; and (3) how routine PMTCT program data in Nigeria can be used to estimate HIV prevalence at the local level and address a critical gap in local descriptive epidemiologic data to better target limited resources. We also identify several ongoing challenges in data collection, analysis, and use, and we suggest potential solutions.

*Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD;

US State Department, Office of the US Global AIDS Coordinator, United States President's Emergency Plan for AIDS Relief (PEPFAR), Washington, DC;

Centers for Disease Control and Prevention (CDC), Atlanta, GA

§Centers for Disease Control and Prevention (CDC), Entebbe, Kampala, Uganda;

National AIDS & STI Control Program, Nigeria Federal Ministry of Health, Abuja, Nigeria;

United States Agency for International Development (USAID), Kampala, Uganda;

#Health Information Systems Program Uganda (HISP Uganda), Kampala, Uganda;

**Centers for Disease Control and Prevention (CDC), Abuja, Nigeria;

††Department of Global Health Sciences, University of California, San Francisco, CA;

‡‡Uganda Ministry of Health, Kampala, Uganda;

§§United States Agency for International Development (USAID), Abuja, Nigeria; and

‖‖United States Agency for International Development (USAID), Washington, DC.

Correspondence to: Anna K. Radin, MPH, Office of the U.S. Global AIDS Coordinator and Health Diplomacy U.S. Department of State SA-22, Room 10300 Washington, DC 20522-2210 (e-mail: annakradin@gmail.com).

A.K.R. receives funding for doctoral studies as part of an NIH Training Grant at Johns Hopkins University (Johns Hopkins HIV Epidemiology and Prevention Sciences Training Program, #1T32AI102623-01A1). In addition, A.K.R., A.A., M.A.O., M.A., H.A., T.E., D.O., B.R.P., and D.H.W. all work for US Government Agencies charged with implementing PEPFAR programs. R.L.K. and F.W. receive PEPFAR funding from CDC through University of California at San Francisco to support and conduct secondary analyses of data collected through Uganda's SMS-based weekly PMTCT reporting system.

The authors have no conflicts of interest to disclose.

The findings and conclusions in this report are those of the authors and do not necessarily represent the official positions of participating federal government Agencies, including the State Department Office of the U.S. Global AIDS Coordinator, United States Agency for International Development and the U.S. Centers for Disease Control and Prevention.

Back to Top | Article Outline

INTRODUCTION

Perhaps the boldest aspect of the Global Plan Towards the Elimination of New HIV Infections Among Children by 2015 and Keeping their Mothers Alive (Global Plan) was the ambitiousness of its goals.1 Launched at the United Nations (UN) General Assembly High Level Meeting on AIDS in 2011 with leadership from UNAIDS and the United States President's Emergency Plan for AIDS Relief (PEPFAR), the 22 countries bearing 90% of the global burden of pediatric HIV to committed to reducing new HIV infections among children by 90% by 2015 and cutting AIDS-related maternal mortality in half by 2015.1

From its inception, the Global Plan required annual monitoring of each country's progress toward these 2 overarching goals and 8 additional, clearly articulated targets (Fig. 1).2 Indicators for several of these targets required disaggregated data elements that were not routinely collected at the time that the Global Plan was launched. The need to report on progress toward the targets outlined in the Global Plan Monitoring Framework drove the routine collection of indicator disaggregations for antiretroviral regimens and HIV test results.1 These disaggregations required a substantial shift in reporting and have enriched the content of routinely collected data on prevention of mother-to-child transmission of HIV (PMTCT). Availability of such data also created abundant opportunities for new analyses to facilitate improvements in both program quality and the strategic expansion of PMTCT services.

FIGURE 1

FIGURE 1

Back to Top | Article Outline

MAJOR ACHIEVEMENTS IN MONITORING AND EVALUATION DURING THE GLOBAL PLAN

A number of key achievements in monitoring and evaluation took place in the context of the Global Plan. Improvements to data collection tools, including the introduction of longitudinal registers tracking mother–baby pairs, helped to optimize ongoing care and follow-up after delivery. The Monitoring and Evaluation Technical Working Group of the Interagency Task Team for the Prevention and Treatment of HIV Infection in Pregnant Women, Mothers, and Children (IATT—The IATT is a group supporting technical collaboration around PMTCT and pediatric HIV among technical experts from ministries of health and other government agencies, multilateral, and bilateral organizations, funders, implementing partners and nongovernmental organizations, academic organizations, faith-based organizations, networks of people living with HIV, and other civil society partners.) and its member organizations drove much of the progress in the area of impact measurement. This was particularly true for development of methods to measure progress (including through cohort analyses) in the expanded use of district health information systems.

Importantly, the IATT also developed a framework—the IATT Option B+ M&E Framework—for enhanced monitoring and evaluation of PMTCT programs as countries shifted to provision of lifelong antiretroviral therapy for all pregnant and breastfeeding women living with HIV.3 This framework provided timely guidance to countries regarding monitoring the rapid and unprecedented decentralization of antiretroviral therapy services. For example, it recommended collection of early warning indicators at a subset of PMTCT sites to monitor stock-outs of HIV test kits and antiretroviral medicines in real time. In addition, under the Global Plan, mother-to-child transmission (MTCT) rates at 6 weeks and 18 months postpartum were directly measured for the first time—rather than modeled—in South Africa and Zimbabwe at the national level. MTCT rates were derived by conducting PMTCT effectiveness surveys with a representative sample of mother–infant pairs followed for up to 18 months to assess vertical transmission rates. Generic protocols were published to support this work in other Global Plan countries.4–6 Assessing final infant HIV status and measuring final MTCT rates represent important ongoing challenges.

Shifts in PMTCT reporting corresponded with a series of achievements in broader HIV/AIDS monitoring, evaluation, and reporting. During this time, national and subnational government leaders joined forces with PEPFAR, UNAIDS, the United Nations Children's Fund (UNICEF), the World Health Organization (WHO), and other partners to begin aggregating, analyzing, and reporting data not only at the national level, but also at subnational levels through the Global AIDS Response Progress Reporting (GARPR).7 Many countries initiated or expanded district health information systems to document progress in subnational units. Moreover, a critical shift occurred as programs moved toward analyzing retention in the HIV care cascade as a way to identify and address programmatic bottlenecks responsible for loss to follow-up and other issues of program quality. PEPFAR began collecting and analyzing site-level data for every supported country on a quarterly basis. The program data were made available for public analysis and use for the first time through a series of online dashboards with downloadable data.8 National, subnational, and health facility-level program leaders were encouraged to track site-level data over time to identify areas for improvement, such as the percentage of pregnant women living with HIV initiating antiretroviral therapy or the proportion of HIV-exposed infants tested for HIV by 2 months of age. The Global Fund to Fight AIDS, Tuberculosis, and Malaria (Global Fund) embraced a system of performance-driven financing that underlines the importance of monitoring programmatic results.9,10

This evolution in global HIV/AIDS monitoring, evaluation, and reporting systems—and the remarkable achievements in data collection and use driven by the Global Planwere mutually reinforcing. However, this progress did not occur without substantial challenges. Below, we profile several of the monitoring and evaluation challenges common across Global Plan countries, and we highlight countries leading the way with innovative solutions using PMTCT program data. We also identify outstanding challenges and potential ways forward.

Back to Top | Article Outline

OPERATIONAL CHALLENGES RELATED TO COLLECTION, ANALYSIS, AND USE OF PMTCT DATA

Paper Registries

There are multiple challenges inherent in the use of paper registries and reporting tools that persist across many Global Plan countries, including their poor completion and updating. The printing and distribution of varying iterations of “updated” registers and reporting tools to the sites often leads to discordant reporting and confusion. To remedy this situation, many Global Plan countries are leveraging existing electronic data systems in antiretroviral therapy programs to serve additional PMTCT sites. For example, South Africa has successfully scaled up electronic data systems to all PMTCT sites.11 Electronic systems with some form of unique identifiers linked between PMTCT sites make tracking of mother–baby pairs possible, even when patients present to alternative sites for care. However, many PMTCT sites are small, low-volume facilities, and many in rural areas lack basic infrastructure. Thus, we expect the use of paper-based registries to remain a challenge in many PMTCT programs.

Back to Top | Article Outline

Follow-Up of Mother–Baby Pairs

Collecting longitudinal data on mother–baby pairs throughout the PMTCT cascade is challenging but necessary to optimize maternal and infant outcomes. Many national registries are not properly formatted to facilitate longitudinal follow-up of HIV-exposed infants or mother–baby pairs. However, 9 Global Plan priority countries—Cameroon, Côte d'Ivoire, Democratic Republic of Congo, Kenya, Mozambique, Namibia, Uganda, Zambia, and Zimbabwe—have successfully designed, piloted, or launched paper-based longitudinal registers that capture data on HIV-exposed infants and mother–baby pairs throughout the breastfeeding period to determine final HIV status for the infant and facilitate retention of the mother on lifelong antiretroviral therapy.11 The development and rollout of these registers has been, and continues to be, a major challenge in monitoring the health of mother–baby pairs.

Back to Top | Article Outline

Coordination and Poor Adherence to Data Collection Protocols

While standard operating procedures or protocols for data collection and analysis often exist, poor coordination between national and subnational government structures and nongovernmental stakeholders is a recurrent issue that may lead to incomplete or duplicate reporting. Burundi is one country that has successfully improved data quality and use through a system of data quality reviews. Burundi's experience with PMTCT data collection and review is described in Box 1.

Back to Top | Article Outline
Routine Data Quality Assurance in Burundi Cited Here...

Burundi has supported a functional system for monthly data quality checks, including regular supervision of data collection and data quality reviews at the site, district, and provincial levels. In health facility sites, health care providers use standard registers and client records to collect raw data. Sites report to health districts on a monthly basis. After collecting and reviewing all reports, the health district organizes monthly data validation meetings, convening representatives of health facilities in the district to review data. This validation session helps minimize errors due to double counting, nonexhaustive reporting, transcription mistakes, misinterpretation of indicator definitions, and lack of consistency in data sources before data are entered in the district health information system. In addition, the Ministry of Health has made significant efforts to harmonize the health data collection tools and ensure that all health facilities use these standardized tools. Most PMTCT sites in Burundi use paper-based registers, whereas the higher volume sites, in addition to using standard registers, record HIV data through a computerized database (called SIDA Info) that facilitates data analysis and patient tracking. SIDA Info also facilitates regular quantification of HIV commodities for stock management.12

Back to Top | Article Outline

Measuring Stigma, Discrimination, and Other Community-Level Factors

Community-level factors, including stigma and discrimination, are major barriers to the uptake of PMTCT services, and they often drive loss to follow-up.13 In addition, lack of respectful maternity care may inhibit pregnant women from seeking care in the first place, depriving them of HIV testing, treatment, and other services that could improve maternal outcomes and the health of their infants.14,15 The Global Plan emphasizes the importance of community engagement in PMTCT programs and the reduction of stigma and discrimination.1 However, it is difficult to assess progress (or lack thereof) in addressing these issues without consistently measuring them. Various stigma and discrimination indexes have been developed, but to our knowledge, none are regularly used in PMTCT programs on a national scale.16–19 Côte d'Ivoire is one country planning to pilot-test community engagement indicators, and this is described in Box 2.

Back to Top | Article Outline
Pilot-Testing New Community Engagement Indicators in Côte d'Ivoire Cited Here...

In 2014–2016, the IATT's Community Engagement Technical Working Group developed indicators to quantitatively measure community engagement in PMTCT programs. “Community” is defined as populations directly affected by the HIV epidemic, including pregnant women and mothers living with HIV. Côte d'Ivoire has demonstrated leadership in this area by planning to pilot 2 innovative community engagement indicators.

  • Community engagement at the national level. Communities participate in annual national reviews of the PMTCT program. Community participation is defined as (1) community and civil society representation at the national PMTCT program review meeting, and (2) systematic analysis of progress at the national level by communities and civil society.
  • Community engagement at the health facility level. This indicator measures the percentage of health facilities providing PMTCT services that had community accountability mechanisms in place in the previous 12 months. Community accountability mechanisms for PMTCT may include (but are not limited to) the following:
    1. Citizen report cards: large-scale surveys of PMTCT client feedback, with results used for advocacy and to increase public accountability.
    2. Partnership-defined quality: working with external facilitation, health workers, and mothers living with HIV define and examine the quality of PMTCT programs, prioritize issues, and develop and implement an action plan.
    3. Integrated supportive supervision: quarterly facility visits for quality assurance by an interdisciplinary supportive supervisory team that includes community representatives and experts in clinical elements, laboratory systems, and monitoring and evaluation.
Back to Top | Article Outline

Measuring Prongs 1, 2, and 4 Indicators

Measuring the number of HIV infections averted in women of childbearing age (prong 1), the unmet need for family planning (prong 2), and maternal and infant mortality (prong 4) remains a substantial challenge. The data for these indicators are not routinely collected through HIV program monitoring, and they are heavily reliant on mathematical modeling or the use of periodic survey or surveillance data. As a result, the accuracy, precision, and timeliness of these indicators tend to lag behind PMTCT indicators measured using routinely collected HIV program data. The lack of timely, reliable data complicates and inhibits efforts to improve these critical aspects of comprehensive PMTCT programs.

One of the primary targets of the Global Plan was to reduce HIV-related maternal mortality by 50%, but this figure proved difficult to measure through program data. In 2013, the Global Plan shifted to reporting modeled data on the rate of HIV-related mortality among women of reproductive age. Some of the successful advances toward measuring progress in prongs 1, 2, and 4 are described in Box 3.

Back to Top | Article Outline
New Efforts to Measure Prongs 1, 2, and 4 Cited Here...
  • Prong 1: Measuring incident HIV infection in young women and adolescent girls. Through the DREAMS initiative, PEPFAR—together with the Bill & Melinda Gates Foundation, Girl Effect, Johnson and Johnson, ViiV Healthcare, and Gilead—is pursuing a 40% reduction in new HIV infections among adolescent girls and young women in districts with the highest HIV incidence in 10 countries.20 HIV incidence among women of childbearing age will be directly measured at the national level by collection of incidence assay data through ongoing Population-based HIV Impact Assessments and modeled at subnational levels for specific age bands using age-disaggregated PMTCT testing data.
  • Prong 2: Measuring unmet need for family planning in antiretroviral therapy clinics. Cameroon and Uganda are currently partnering with PEPFAR to conduct implementation science research aimed at measuring the unmet need for family planning among HIV-positive clients at antiretroviral therapy clinics. The studies provide integrated family planning services in HIV care and treatment clinics and evaluate outcomes related to unmet need for family planning and the contraceptive prevalence rate. Publication of data is expected between late 2016 and 2018.21,22
  • Prong 4: New methods for measuring maternal mortality. The Saving Mothers, Giving Life (SMGL) initiative is a 5-year public–private partnership to accelerate reductions in maternal and newborn mortality in Nigeria, Uganda, and Zambia.23 The SMGL model employs a health systems approach that focuses on the most vulnerable period for women and newborns: labor, delivery, and the first 48 hours after childbirth. The 2015 Saving Mothers Giving Life Mid-Initiative Report indicated that since 2012, the maternal mortality ratio had declined by 45% in target facilities in Uganda and 53% in target facilities in Zambia.24 While the detailed assessments of maternal mortality funded by SMGL were expensive and may prove difficult to implement at scale, a modified indicator-based approach could be included in routine site-level reporting.
Back to Top | Article Outline

Providing Real-Time Data With a Feedback Loop

The individuals with the most potential to improve the programs––the frontline health care workers and data clerks at the site-level, leadership at the district-level, and other leaders at subnational levels––may never see the data they work so hard to collect after they have been compiled and analyzed. Long delays before stakeholders see program data are common, and this may reduce staff motivation to collect high-quality data.25 Below, we describe in detail Uganda's working solution to this problem in case study 1: an innovative system for the weekly reporting of PMTCT program data using uses short message service (SMS) cellular phone technology.

Back to Top | Article Outline

Parallel Reporting Systems

Funders or other stakeholders often require off-cycle reports and have supported duplicate reporting systems. Clinical staff members frequently are busy with patient care and not motivated to complete multiple registers and forms. The result is that collecting and reporting the data that stakeholders require often receives priority over reporting data through the national system.26 Case study 2, below, describes how Uganda is successfully harmonizing and consolidating parallel reporting systems.

Back to Top | Article Outline

Subnational-Level Planning With Inadequate Data

Subnational-level planning can be difficult when subnational data are of poor quality or lacking altogether. Below in case study 3, we highlight a promising example of how Nigeria's routinely collected PMTCT program data are being used to characterize the distribution of HIV cases at the level of the local government area, making it possible to focus PMTCT services in the areas in greatest need.

Back to Top | Article Outline

CASE STUDIES: COUNTRY-DRIVEN SOLUTIONS TO COMPLEX MONITORING AND EVALUATION CHALLENGES

Case Study 1: Addressing the Need for Real-Time Data for Option B+ in Uganda

With an estimated 3500 children newly infected with HIV in 2015 (down from 25,000 in 2009), Uganda has achieved an 86% reduction in the number of new pediatric HIV infections since the launch of the Global Plan.27 Uganda was among the first countries to provide lifelong antiretroviral therapy to all HIV-infected pregnant and breastfeeding women regardless of CD4 count (Option B+).28

Uganda's experience with Option B+ has been carefully documented and monitored since 2013 through an innovative nationwide weekly reporting system that uses SMS technology. In 2015, an average of 79% of the 1687 registered PMTCT sites reported each week on 9 key indicators by texting their data through cellular phone. Results are received in a central database and then analyzed and shared through an electronic dashboard and a report summarizing weekly performance by site, district, and implementing partner (Fig. 2). The dashboard and report are distributed by email to a broad group of stakeholders. The dashboard is open to the public and available online at http://dashboard.mets.or.ug.

FIGURE 2

FIGURE 2

This innovative use of SMS technology to enable real-time reporting and analysis of PMTCT program data on a national scale is unprecedented. It has allowed Uganda to closely monitor and continuously improve the national PMTCT program. Government leaders, PEPFAR, implementing partners, and other stakeholders meet weekly to review and discuss the PMTCT data, and to decide on strategies to improve program areas that require an immediate response.29 Data review meetings are held monthly at the national level and quarterly at the district level to support data quality and continuously improve PMTCT programs.

The weekly reporting system was developed by the Uganda Ministry of Health together with PEPFAR and other development partners as part of its leadership efforts toward the virtual elimination of MTCT of HIV in Uganda.29 The Ministry of Health further prioritized PMTCT by placing the program within the national Emergency Operations Centre, further reinforcing the urgency of action.

The weekly reports provide immediately actionable data, such as summaries of stock-outs of HIV test kits and antiretroviral medication by facility, district, and implementing partner. This facilitates a rapid, targeted response, and district health officers are easily able to compare their district's performance to national performance. At the national level, the Ministry of Health is able to monitor district performance over time to assess which districts may need additional support to optimize programmatic performance. Similarly, implementing partners are easily able to monitor facility-level performance and target technical support to facilities that may be underperforming. The weekly reporting system has built-in measures for quality assurance, as numbers reported are automatically checked against relevant denominators, and over-reporting is flagged where appropriate. In addition, the weekly data support broader data quality assurance efforts as they are triangulated with monthly data submissions in the electronic District Health Information System (DHIS2) to check for consistency.

Back to Top | Article Outline

Potential for Expansion of Weekly PMTCT Reporting

Now that Option B+ has been successfully scaled up, leaders in Uganda are considering new gaps and how the weekly reporting system could be adapted to fill them. Specifically, Uganda has recently pilot-tested new indicators to track retention in care of mothers initiating antiretroviral therapy and new indicators for early infant diagnosis. Discussions also are underway about possible expansion to incorporate key non-HIV indicators, possibly including maternal and infant mortality data. This system can serve as a model for other countries that are interested in enhancing their ability to collect and analyze real-time data. The system can act as an early warning system to alert program managers about acute issues requiring immediate attention, such as the stock-out of rapid test kits, antiretroviral medication, or other key commodities. This capability is particularly important as countries adopt “treat all” guidelines, and antiretroviral therapy is expanded and decentralized to smaller health facilities.

Back to Top | Article Outline

Case Study 2: Addressing Inefficient Parallel Reporting Structures in Uganda

Historically, PEPFAR and other development partners have supported the use of parallel reporting systems that are separate from the national system in many countries; this helps to ensure the timely, accurate, and reliable reporting of data from PMTCT and other HIV programs.30 Maintaining this parallel reporting structure in Uganda resulted in a number of challenges, including such duplications of effort as printing separate tools, holding separate training sessions, and preparing multiple reports for the government and partners.31 Discrepancies in data were common; different reports often presented different numbers.32,33

In an effort to support a sustainable response and strengthen government health reporting systems, PEPFAR—together with implementing partners, UNAIDS, WHO, and other development partners in Uganda—supported the Ministry of Health to revise the data collection tools and reinforce implementation of an electronic Health Management Information System using DHIS2.31 The move from parallel systems to a single streamlined, government-led system was made in a stepwise fashion. First, implementing partners were rationalized in 2011–2012 to avoid overlapping responsibilities and ensure that only one implementing partner was assigned to support PMTCT services in each health facility and district. The Ministry of Health, in collaboration with development partners, revised the HMIS tools to include WHO, PEPFAR, and Global Fund-required PMTCT and other HIV program indicators. The rollout and implementation of electronic systems at facilities, and of the DHIS2 at the district level, were supported through the provision of the hardware, power, and internet connectivity required to ensure timely entry of reports into DHIS2.

This shift toward use of a single system (DHIS2) has improved reporting rates and timeliness and joint data quality processes, and it has made data more readily available for use at the district and health facility levels. In 2015, Uganda reported all WHO, PEPFAR, and Global Fund-required PMTCT indicators through the national system and 65% of all PEPFAR-required indicators through DHIS2. Efforts are underway to include the indicators not covered in the first iteration of the single system.31 In addition, Uganda was able to directly report PMTCT and other HIV program results to the Global Fund and GARPR from DHIS2.34

Given the vast number of PMTCT sites, reporting results for PMTCT programs had been particularly burdensome. Moving toward support for a single, synchronized district health information system has reduced the workload at the facility level, as clinic staff now compile a single report. Critically, this shift has improved PMTCT data quality and the timeliness and use of PMTCT data for programmatic improvements at both the facility and district levels.35 With full support from development partners, DHIS2 has been properly functioning 98% of the time, compared with 65% of the time before partners fully supported this system (P. Behumbiize, MPH, written communication, April 12, 2016). Uganda is now able to use the national DHIS2 to meet most of its reporting needs for PEPFAR, the Global Fund, and GARPR, and the national system has been strengthened. The shift away from parallel reporting structures has taken place or is underway in many other Global Plan countries.11 Uganda's experience offers a model for countries in earlier phases of this transition.

Back to Top | Article Outline

Case Study 3: Addressing the Challenge of Inadequate Subnational Data in Nigeria

UNAIDS estimates that 41,000 infants were newly infected with HIV in Nigeria in 2015.27 This number constitutes 27% of the global burden of new pediatric HIV infections.27,36 In 2015, approximately 30% of HIV-positive pregnant women in Nigeria were diagnosed and received antiretroviral therapy for PMTCT and to protect maternal health.27 Given the low but varied prevalence of HIV across Nigeria, funding must be prioritized to support scale-up of PMTCT services strategically, where the burden of disease is greatest. However, existing epidemiologic data at the state level are inadequate to guide prioritization of sites at the local government area level for the strategic scale-up of PMTCT services.37,38

In the context of the Global Plan, PEPFAR expended over US $160 million in fiscal years 2012–2014 to support a major expansion of the national PMTCT program in Nigeria.8 Roughly 45% (US $75 million) was invested at the site level to support facility improvements, training of clinic staff, provision of HIV test kits and antiretroviral medication, and other recurrent costs; the remaining 55% (US $92 million) was invested above the site level, in health systems strengthening, strategic information (including monitoring and evaluation), and program management activities.8 PEPFAR's implementing partners in Nigeria carefully coordinated this scale-up under the leadership of the Federal Ministry of Health, the National Agency for the Control of AIDS (NACA), State Ministries of Health, and the State Agencies for the Control of AIDS (SACA). Scale-up efforts focused largely on Nigeria's “12 + 1 states,” a subset of states prioritized by NACA based on state HIV prevalence and the estimated number of pregnant women living with HIV.37,38

Under the leadership of state and federal agencies, implementing partners first took a full inventory of every health facility offering antenatal care and maternal and child health services in each priority state, including both public and private sector sites. Intensive statewide health facility assessments were conducted by visiting these facilities and recording the other health services provided, the number and cadres of health care workers assigned to the site, the patient volume, and other key demographic and health service information.39–45 Health facilities in priority states were then ranked by expected number of HIV-positive patients served, using the best available state-level HIV prevalence estimates together with site-level data on patient volume, information on facility staffing, and demographic data.37,38,46,47 Sites were then sorted into batches for staggered scale-up, with the facilities in each state expected to identify the largest number of HIV-positive pregnant women to be the first “activated” to provide lifesaving PMTCT services.

Over the course of scale-up, the PMTCT program expanded from approximately 750 PEPFAR-supported sites in fiscal year 2012 to nearly 6000 PEPFAR-supported sites by fiscal year 2015.48 The Global Fund also supported a major scale-up of PMTCT sites in Nigeria, expanding from 163 supported sites in 2012 to 1130 in fiscal year 2015.49 In 2015, PEPFAR-supported PMTCT sites tested 2.6 million pregnant women for HIV. However, despite focusing the scale-up of new facilities in states that were identified as having a high prevalence of HIV and prioritized by the government of Nigeria, 32% of the sites identified no HIV-positive pregnant women in fiscal year 2015.37,38,48 To maximize impact with limited resources, site-level investments should be directed to areas with the highest burden of disease. In the absence of granular epidemiologic data on the HIV burden within high prevalence states, a carefully implemented massive scale-up of PMTCT services reached far fewer HIV-positive women than expected. This situation illustrates the need for more granular data on local HIV prevalence and the distribution of people living with HIV within states in Nigeria.

Back to Top | Article Outline

Potential for Use of Site-Level PMTCT Program Data in Nigeria

PEPFAR's shift in fiscal year 2015 to the collection and reporting of site-level data, together with the tremendous scale-up of PMTCT services in Nigeria in the context of the Global Plan, created a unique opportunity to characterize the HIV epidemic at a local level. The investment in widespread testing of pregnant women in 2015 was central to establishing site-level positivity rates, which are essential to making further programmatic and policy decisions about resource allocation. The sample size that reflected in the PEPFAR PMTCT program data (2.6 million pregnant women tested for HIV in 2015) is over 62 times the number of women tested in the latest population-representative survey (2012) or antenatal care sentinel surveillance data (2014).37,38,50 The enormity of the sample in PMTCT program data allows for a level of granularity in descriptive epidemiologic data that cannot be achieved using survey or surveillance data. Characterizing the distribution of HIV at local levels is now possible for the first time.

The UNAIDS/WHO Working Group on Global HIV/AIDS and STI Surveillance has highlighted the most important advantages of using routine PMTCT program data for HIV surveillance. This includes the following: (1) the ethical advantage of HIV testing in clinical practice, which ensures that women testing positive can be linked with HIV treatment services; (2) the efficiency in terms of both human and financial resources; (3) the vast expansion of the coverage of HIV surveillance; and (4) the improved sustainability of HIV surveillance, as it is embedded in the national PMTCT program.51 Use of PMTCT program data to estimate local HIV prevalence will enable leaders to refine programmatic targeting of health facilities and local government areas in prioritized states to maximize the impact of the PMTCT program and make most efficient use of limited resources. In 2016 and 2017, Nigeria made use of site-level HIV-positive yield data from PMTCT programs to select priority areas for intensified scale-up of HIV prevention and treatment services in the PEPFAR Country Operational Plan. Areas with the highest HIV prevalence receive additional resources to identify and treat people living with HIV, maintain their health, and prevent further transmission.

This practical application of program data for decision-making would not have been possible without the programmatic scale-up and advancements to monitoring and evaluation that occurred in Nigeria in the context of the Global Plan. Importantly, Nigeria's experience highlights the value and tremendous potential of the collection and use of routine program data at the facility level for epidemiologic and programmatic purposes in all Global Plan countries.51

Back to Top | Article Outline

CONCLUSIONS

The Global Plan had a truly historic impact on preventing new pediatric HIV infections and keeping mothers healthy.52 This was achieved in part by ensuring that routinely collected PMTCT data captured benchmarks along the mother–infant clinical care cascade to inform and improve service delivery and the targeting of resources. As a result, today's PMTCT program data tell a more complete story than ever before, and many epidemiologic analyses—such as those described in the 3 case studies—that have the potential to greatly improve targeting and inform implementation of PMTCT programs are now possible. The availability of real-time PMTCT data has enabled leaders in Uganda to use the data not only to act on urgent programmatic problems such as stock-outs but also for ongoing capacity building in both PMTCT programs and data systems, while bringing national, district, and site-level actors together to improve services for mothers and infants. The transition away from parallel reporting systems in Uganda has measurably strengthened the national district health information system, which further empowers national, district, and site-level leaders to review and use their data to improve services. Similarly, using site-level PMTCT program data in Nigeria to describe HIV prevalence at the local level has greatly improved targeting of HIV programs and thus, maximized the lifesaving potential of available resources.

Despite tremendous progress and many country-driven successes achieved during the Global Plan, operational challenges in data use, monitoring, and evaluation for PMTCT persist: paper registers are still commonly used in many countries, and developing and launching longitudinal registers to monitor mother–baby pairs over time has proven exceptionally complicated. Measuring stigma, discrimination, and other community-level factors is rarely done, and additional effort is needed to measure progress in prongs 1, 2, and 4. And critically, the fundamental task of measuring the final impact of PMTCT programs on mothers and children remains a major hurdle, as countries continue to rely on modeled data to assess programmatic success. These outstanding challenges deserve our collective effort moving forward. Country-driven innovations in monitoring, evaluation, and data use, including the examples highlighted here, have great promise, and together with the momentum-building monitoring framework put forward by the Global Plan, they will continue to play a key role in improving maternal outcomes while hastening the countdown to zero new pediatric HIV infections.

Back to Top | Article Outline

ACKNOWLEDGMENTS

The authors gratefully acknowledge the contributions of Stanislas Ntohbari, Public Health Specialist, HIV Prevention and Treatment, USAID/PEPFAR Burundi; Apollinaire Kavungerwa, Monitoring and Evaluation Specialist, USAID/PEPFAR Burundi; and Clement Niyonsaba, Chief of Party, PMTCT Acceleration Project, FHI360, Burundi for providing a summary of Burundi's data quality assurance processes. They also acknowledge: Surbhi Modi, Team Lead for Prevention of Mother to Child Transmission, Division of Global HIV/TB, Centers for Disease Control & Prevention (CDC) for her technical review and contributions to the manuscript; and Aditi Sharma, advisor with GNP+ and Co-Chair of the IATT Community Engagement Technical Working Group, for contributing a summary of recent work on community engagement indicators. They recognize colleagues in the Republic of Uganda's Ministry of Health, and colleagues in the Federal Republic of Nigeria's Federal Ministry of Health and National Agency for the Control of AIDS, together with other Global Plan country governments, for their steadfast leadership and tireless work toward eliminating new HIV infections in children and keeping mothers healthy. They recognize the many implementing partners and members of the IATT Monitoring and Evaluation and Community Engagement Technical Working Groups, for demonstrating exemplary technical leadership and collaboration. Finally, the authors wish to recognize the millions of mothers living with HIV, whose participation in PMTCT programs has been central to our collective successes in keeping mothers and children healthy.

Back to Top | Article Outline

REFERENCES

1. UNAIDS. Global Plan Towards the Elimination of New HIV Infections in Children and Keeping Their Mothers Alive. Geneva, Switzerland: UNAIDS; 2009.
2. The Interagency Task Team on the Prevention and Treatment of HIV Infection in Pregnant Women, Mothers and Children, World Health Organization, United Nations Children's Fund. Global Monitoring Framework and Strategy for the Global Plan Towards the Elimination of New HIV Infections Among Children by 2015 and Keeping Their Mothers Alive (EMTCT). Geneva, Switzerland: World Health Organization; 2012.
3. The Interagency Task Team on the Prevention and Treatment of HIVInfection in Pregnant Women, Mothers and Children. Monitoring & Evaluation Framework for Antiretroviral Treatment for Pregnant and Breastfeeding Women Living with HIV and Their Infants: IATT Option B+ M&E Framework. New York, NY: Centers for Disease Control and Prevention, World Health Organization, United Nations Children's Fund; 2015.
4. Dinh TH, Delaney KP, Goga A, et al. Impact of maternal HIV seroconversion during pregnancy on early mother to child transmission of HIV (MTCT) measured at 4–8 weeks postpartum in South Africa 2011–2012: a national population-based evaluation. PLoS One. 2015;10:1–17.
5. Buzdugan R, McCoy SI, Watadzaushe C, et al. Evaluating the impact of Zimbabwe's prevention of mother-to-child HIV transmission program: population-level estimates of HIV-free infant survival pre-option A. PLoS One. 2015;10:1–17.
6. World Health Organization. A Short Guide on Methods. Measuring the Impact of National PMTCT Programmes: Towards the Elimination of New HIV Infections Among Children by 2015 and Keeping Their Mothers Alive. Geneva, Switzerland: World Health Organization; 2012.
7. UNAIDS. Global AIDS Response Progress Reporting. Reporting Overview for 2016. Geneva, Switzerland: UNAIDS; 2016. Available at: http://http://www.unaids.org/sites/default/files/media_asset/2016_GARPR_reporting_overview_en.pdf. Accessed January 16, 2017.
8. PEPFAR Dashboards. United States President’s Emergency Plan for AIDS Relief Web site. Available at: https://data.pepfar.net. Accessed January 1, 2015.
9. The Global Fund to Fight AIDS, Tuberculosis and Malaria. The Global Fund's Approach to Monitoring and Evaluation. Geneva, Switzerland: The Global Fund to Fight AIDS, Tuberculosis and Malaria; 2016.
10. The Global Fund to Fight AIDS, Tuberculosis and Malaria. The Global Fund to Fight AIDS, Tuberculosis and Malaria: Governance Handbook. Geneva, Switzerland: The Global Fund to Fight AIDS, Tuberculosis and Malaria; 2014.
11. The Interagency Task Team on the Prevention and Treatment of HIV Infection in Pregnant Women, Mothers and Children (IATT). B+ Monitoring & Evaluation Framework Dissemination and Country Consultation: Technical Synthesis. Kampala, Uganda: IATT; 2015.
12. University Research Co., LLC, United States Agency of International Development. USAID ASSIST Project: Burundi Country Report FY15. USAID ASSIST Project. Bethesda, MD: University Research Co., LLC (URC); 2015.
13. Gourlay A, Birdthistle I, Mburu G, et al. Barriers and facilitating factors to the uptake of antiretroviral drugs for prevention of mother-to-child transmission of HIV in sub-Saharan Africa: a systematic review. J Int AIDS Soc. 2013;16:1–21.
14. Medema-Wijnveen JS, Onono M, Bukusi EA, et al. How perceptions of HIV-related stigma affect decision-making regarding childbirth in rural Kenya. PLoS One. 2012;7:16–18.
15. Laher F, Cescon A, Lazarus E, et al. Conversations with mothers: exploring reasons for prevention of mother-to-child transmission (PMTCT) failures in the era of programmatic scale-up in Soweto, South Africa. AIDS Behav. 2012;16:91–98.
16. Holzemer WL, Makoae LN, Greeff M, et al. Measuring HIV stigma for PLHAs and nurses over time in five African countries. SAHARA J. 2009;6:76–82.
17. Kingori C, Reece M, Obeng S, et al. Psychometric evaluation of a cross-culturally adapted felt sigma questionnaire among people living with HIV in Kenya. AIDS Patient Care STDS. 2013;27:481–488.
18. Rutledge SE, Whyte J, Abell N, et al. Measuring stigma among health care and social service providers: the HIV/AIDS Provider Stigma Inventory. AIDS Patient Care STDS. 2011;25:673–682.
19. Smith E, Miller J, Newsome V, et al. Measuring HIV/AIDS-related stigma across South Africa: a versatile and multidimensional scale. Health Educ Behav. 2013;41:387–391.
20. DREAMS: Working Together for an AIDS-Free Future for Girls and Women. United States President's Emergency Plan for AIDS Relief Web Site. Available at: http://http://www.pepfar.gov/partnerships/ppp/dreams/. Accessed January 16, 2017.
    21. United States President's Emergency Plan for AIDS Relief. PEPFAR Uganda Country Operational Plan, FY 2014. Kampala, Uganda: PEPFAR; 2014.
      22. United States President's Emergency Plan for AIDS Relief. PEPFAR Cameroon Country Operational Plan, FY 2014. Yaounde, Cameroon: PEPFAR; 2014.
        23. Saving Mothers Giving Life. Saving Mothers Giving Life Web site. Available at: http://savingmothersgivinglife.org. Accessed June 6, 2016.
          24. Saving Mothers Giving Life. Saving Mothers Giving Life Mid-initiative Report. 2015. Available at: https://http://www.savingmothersgivinglife.org/docs/SMGL-mid-initiative-report.pdf. Accessed January 16, 2017.
            25. Mate KS, Bennett B, Mphatswe W, et al. Challenges for routine health system data management in a large public programme to prevent mother-to-child HIV transmission in South Africa. PLoS One. 2009;4:1–6.
            26. Ledikwe JH, Grignon J, Lebelonyane R, et al. Improving the quality of health information: a qualitative assessment of data management and reporting systems in Botswana. Health Res Policy Syst. 2014;12:7.
            27. UNAIDS. On the Fast-Track to an AIDS-Free Generation. Geneva, Switzerland: UNAIDS; 2016.
            28. Key Data Points for: Uganda. The Interagency Task Team on the Prevention and Treatment of HIV Infection in Pregnant Women, Mothers and Children Web site. Available at: http://emtct-iatt.org/countries/uganda/. Accessed September 22, 2016.
            29. CDC Innovations: Real-Time Data Strengthens Uganda's Efforts to End Mother-to-Child HIV Transmission. Centers for disease Control & prevention web site; 2015. Available at: http://http://www.cdc.gov/globalaids/success-stories/ugandaeoc.html. Accessed January 16, 2017.
            30. Porter LE, Bouey PD, Curtis S, et al. Beyond indicators: advances in global HIV monitoring and evaluation during the PEPFAR era. J Acquir Immune Defic Syndr. 2012;60(suppl 3):120–126.
            31. Uganda Ministry of Health. National HIV and AIDS Monitoring and Evaluation Plan, 2015/2016-2019/2020; 2016. Available at: http://library.health.go.ug/publications/service-delivery-diseases-control-prevention-communicable-diseases/hivaids/national-h-3. Accessed January 16, 2017.
            32. The Interagency Task Team on the Prevention and Treatment of HIVInfection in Pregnant Women, Mothers and Children. IATT Webinar Summary and Discussion: Monitoring & Evaluation Option B+ Framework. 2015. Available at: http://http://www.emtct-iatt.org/wp-content/uploads/2015/08/IATT-Webinar-Summary-Option-B-M-E-Framework_27-July-20151.pdf. Accessed September 22, 2016.
            33. Uganda Ministry of Health. Integrated National Health Electronic Software Uganda. Kampala, Uganda: Uganda Ministry of Health; 2015.
            34. Uganda AIDS CommissionHIV and AIDS Uganda Country Progress Report; 2013. 2014. Available at: http://http://www.unaids.org/sites/default/files/country/documents/UGA_narrative_report_2015.pdf. Accessed January 16, 2017.
            35. Kiberu VM, Matovu JKB, Makumbi F, et al. Strengthening district-based health reporting through the district health management information software system: the Ugandan experience. BMC Med Inform Decis Mak. 2014;14:1–9.
            36. AIDS Info Online Database [database]. Geneva, Switzerland: UNAIDS; 2016.
            37. Federal Ministry of Health of Nigeria. ANC Sentinel Surveillance for HIV: Technical Report. Abuja, Nigeria: Federal Ministry of Health; 2010.
            38. Federal Ministry of Health of Nigeria. National HIV/AIDS and Reproductive Health Survey, 2012 (NARHS Plus). Abuja, Nigeria: Federal Ministry of Health; 2013. Available at: https://http://www.academia.edu/9469489/NARHS_Plus_2012_Final_18112013. Accessed January 16, 2017.
            39. State Ministry of Health: Anambra State, FHI 360. State-Wide Rapid Health Facility Assessment: Anambra State. 360. Awka, Nigeria: Anambra State Ministry of Health and FHI; 2013.
            40. State Ministry of Health: Abia State, FHI 360. State-Wide Rapid Health Facility Assessment: Abia State. 360. Umuahia, Nigeria: Abia State Ministry of Health and FHI; 2013.
            41. State Ministry of Health: Rivers State, FHI 360. State-Wide Rapid Health Facility Assessment: Rivers State. 360. Port Harcourt, Nigeria: Rivers State Ministry of Health and FHI; 2013.
            42. State Ministry of Health: Kano State, FHI 360. State-Wide Rapid Health Facility Assessment: Kano State. Kano, Nigeria: Kano State Ministry of Health and FHI 360; 2013.
            43. State Ministry of Health: Bayelsa State, FHI 360. State-Wide Rapid Health Facility Assessment: Bayelsa State. 360. Yenagoa, Nigeria: Bayelsa State Ministry of Health and FHI; 2013.
            44. State Ministry of Health: Akwa Ibom State, FHI 360. State-Wide Rapid Health Facility Assessment: Akwa Ibom State. 360. Uyo, Nigeria: Akwa Ibom State Ministry of Health and FHI; 2013.
            45. State Ministry of Health: Cross River State, FHI 360. State-Wide Rapid Health Facility Assessment: Cross River State. 360. Calabar, Nigeria: Cross River State Ministry of Health and FHI; 2013.
            46. Federal Republic of Nigeria National Population Commission. National Census, 2006. Abuja, Nigeria: National Population Commission; 2006.
            47. National Population Commission Federal Republic of Nigeria, ICF International. Nigeria Demographic & Health Survey. Abuja, Nigeria: National Population Commission; 2013.
            48. United States President's Emergency Plan for AIDS Relief (PEPFAR). FY 2015 Annual Progress Report Results. Washington, DC: PEPFAR; 2015.
            49. The Global Fund to Fight AIDS, Tuberculosis and Malaria. Global Fund PMTCT Program ata (2012). Geneva, Switzerland: The Global Fund to Fight AIDS, Tuberculosis and Malaria; 2012.
            50. United States President's Emergency Plan for AIDS Relief (PEPFAR). Nigeria PMTCT Program Data, FY 2015. Washington, DC: PEPFAR; 2015.
            51. UNAIDS/WHO Working Group on Global HIV/AIDS and STI Surveillance. HIV Strategic Information for Impact Conducting HIV Surveillance Based on Routine Programme Data Among Pregnant Women. Geneva, Switzerland: UNAIDS; 2015.
            52. UNAIDS. 2015 Progress Report on the Global Plan towards Elimination of New HIV Infections in Children and Keeping Their Mothers Alive. Geneva, Switzerland: UNAIDS; 2015.
            Keywords:

            prevention of mother-to-child transmission of HIV; global plan towards eliminating new HIV infections among children by 2015 and keeping their mothers alive (global plan); data use; monitoring & evaluation; HIV/AIDS; sub-Saharan Africa

            Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved.