Are the Investments in National HIV Monitoring and Evaluation Systems Paying Off?

Abstract

Background: 

Concerted efforts and substantial financial resources have gone toward strengthening national monitoring and evaluation (M&E) systems for HIV programs. This article explores whether those investments have made a difference in terms of data availability, quality and use for assessing whether national programs are on track to achieve the 2015 Millenium Development Goal (MDG) of halting and reversing the HIV epidemic.

Methods: 

Descriptive analyses, including trends, of the National Composite Policy Index data and M&E expenditures were conducted. Global Fund funding continuation assessments were reviewed for concerns related to M&E. Availability of population-based survey data was assessed.

Results: 

There has been a marked increase in the number of countries where the prerequisites for a national HIV M&E system are in place and in human resources devoted to M&E at the national level. However, crucial gaps remain in M&E capacity, available M&E data, and data quality assurance. The extent to which data are used for program improvement is difficult to ascertain. There is a potential threat to sustaining the current momentum in M&E as governments have not committed long-term funding and current M&E-related expenditures are below the minimum needed to make M&E systems fully functional.

Conclusions: 

There is evidence of rapid scale-up of basic HIV M&E systems, but if M&E is to fulfil its role in guiding optimal use of resources, ensuring effective HIV programs and providing evidence of progress toward the Millenium Development Goal of halting and reversing the HIV epidemic, essential data gaps will need to be filled urgently and those data will need to be used to guide decision making.

INTRODUCTION

Comprehensive and timely monitoring and evaluation (M&E) data are crucial to guide the optimal use of limited resources and to ensure effective HIV programs. To achieve the Millenium Development Goal of halting and beginning to reverse the spread of HIV by 2015 (MDG 6), national AIDS programs require accurate, pertinent and timely data, and the capacity to interpret those data to improve programs. M&E data are also important for demonstrating that investments in HIV responses are averting infections, illness and deaths and therefore warrant continuation and expansion.¹

Data collection efforts need to extend beyond routine monitoring and should include determinants research, situation analysis, operations research and evaluation studies (Fig. 1). Fundamentally, M&E should be guided by the need to answer the questions: “Are we doing the right things? Are we doing them right? Are we doing them on a large enough scale to make a difference?” Effective use of information has been identified as a key element in the success of large-scale efforts that have achieved major health improvements, such as the eradication of small pox and the reduction of guinea worm disease in Africa and Asia.²

FIGURE 1: A public health questions approach to HIV/AIDS M&E. Source: Rugg D, Peersman G, Carael M. Global advances in HIV/AIDS monitoring and evaluation. New Directions for Evaluation. 2004;103:36.

Much of the early data collection efforts focused on AIDS case reporting, HIV sentinel surveillance and surveys of HIV-related risk behaviors. As countries began formulating national AIDS strategies for coordinated multisectoral responses, routine program monitoring became more prominent and focused on assessing whether activities were being implemented effectively and were making reasonable progress toward defined objectives. Those data sources have since been supplemented with more rigorous evaluations that assess the effects of new or innovative interventions on behavioral change,^3,4 evaluations of large-scale programs like the Global Fund to Fight AIDS, Malaria and Tuberculosis (Global Fund) 5-year evaluation⁵ and smaller scale program evaluations and operations research.⁶

In the past decade, several international initiatives, such as the United Nations General Assembly Special Session (UNGASS) Declaration of Commitment on HIV/AIDS (DoC),⁷ the agreement on the “Three Ones Principles” (One agreed AIDS action framework, one inclusive national coordinating authority, one national M&E system),⁸ and the Global Fund have supported or underpinned the strengthening of M&E for HIV programs. The focus on collaboration and partnerships has provided a level of commitment from a range of stakeholders to support a coherent country-led response that is supported by one national M&E system.

This article examines whether the major investments in M&E are paying off in terms of the establishment and strengthening of national M&E systems, data availability and quality, and, most importantly, data use to benefit HIV programs. Specifically, this article seeks to answer the following questions:

1. What is the current status of national HIV M&E systems?
2. What are the investments in M&E yielding in terms of strengthened HIV programs?
3. What should be the focus of M&E system strengthening over the next 5 years to ensure that M&E can play its role in achieving and measuring MDG 6 in 2015?

METHODS

Since 2001, countries have been required to report every 2 years on their progress toward the UNGASS goals, by way of Country Progress Reports. Those reports include data for 25 standardized indicators, 5 of which are also used to monitor progress toward achieving MDG 6. The National Composite Policy Index (NCPI), the UNGASS indicator focusing on political support and national strategies and policies, also addresses national M&E systems. It is an extensive questionnaire that is completed on the basis of a review of relevant documents and interviews with people most knowledgeable about the topics under review.

To examine the status of national HIV M&E systems, descriptive analyses, including trends, were conducted of the following data sources: (1) the M&E section of the UNGASS NCPI (94 countries reported in 2006, 135 in 2008); (2) M&E expenditure data (UNGASS data reported by 59 countries in 2008: 15 countries reported 2005 expenditures, 42 reported 2006 expenditures and 22 reported 2007 expenditures; some countries reported data for multiple years). The percent agreement between overlapping data in the 2007 NCPI and the 2007 Annual Reports submitted by 86 UNAIDS Country Coordinators (referred to here as UCC reports) was calculated as a simple data validation measure.

In addition, we reviewed for which countries and timeframes population-based survey data on HIV-related knowledge and behaviors are available. We reviewed survey data on the general population on most-at-risk populations available on the MEASURE DHS website.⁹ Specifically: the Demographic and Health Survey (DHS) which is a nationally representative household survey that provides data for a wide range of indicators in the areas of population, health and nutrition (including HIV knowledge, HIV behavior, HIV testing); the AIDS Indicator Survey (AIS) which is a nationally representative household survey developed to provide countries with a standardized tool to obtain indicators for effective monitoring of national HIV/AIDS programs (including HIV knowledge, HIV behavior, HIV testing); and, Behavioral Surveillance Surveys (BSS) which track trends in HIV/AIDS knowledge, attitudes and risk behavior in selected populations such as sex workers, injecting drug users and men who have sex with men.

Finally, scorecards were reviewed for 59 Global Fund grants in 37 countries in sub-Saharan Africa.¹⁰ The scorecard includes standardized questions on performance, including on M&E, during the initial 2-year period of a grant. These are used to determine and make transparent specific recommendations for continuation of funding (ie, “conditional go”, “go” or “no go”).

RESULTS

Corroboration of Data Sources

Because NCPI data are collected through a self-completion questionnaire, they may have inherent subjectivity. The inclusion of a range of stakeholder groups (government, civil society and bilateral and multilateral organizations) in the completion of the NCPI is intended to introduce the necessary checks and balances. In all the countries that provided respondent information (125 of the 135 countries that completed the NCPI), representatives from at least 2 stakeholder groups were involved in compiling the NCPI data.

Comparing the responses to identical questions from 2 data collection efforts (the NCPI and the UCC report) in the same country provides a simple validation measure. The percent agreement ranged from 70% to 87% on aspects of the national M&E plan and unit and 60% on data sharing (Fig. 2). Discrepancies are likely due to different interpretation of the question and assessments done at different points in time. Overall, the level of corroboration is considered good.

FIGURE 2: Per cent agreement between data reported through the NCPI and UCC reports for key M&E elements.

Financial Investments in M&E

Country ownership and long-term investment in national M&E are important to ensure sustainable systems; the lack of sustained funding is a major reason why many M&E systems fail in the long run.¹¹

Based on empirical evidence, international agencies recommend allocating 5%-10% of the total HIV program budget to monitoring activities; operations research and evaluation require additional resources. However, only 3% of program funding is typically allocated for M&E in Global Fund grant proposals (with the caveat that salaries for M&E staff are often not considered as M&E costs, hence the M&E budget allocation is likely to be underestimated).¹² Despite this consistently low allocation, only 4 of the 59 grant scorecards (7%) flagged this as a concern.

Seventy-seven percent of countries with a national M&E plan reported that this plan includes budgetary requirements to support its implementation, but just over half of those countries (58%) reported that funding actually had been secured.

The reported median for actual spending on M&E at the national level was 1.8% of overall HIV expenditures and ranged from less than 0.1% of national HIV expenditure in Tanzania to 17% in the Philippines. It should be noted that the M&E expenditure data referred to here do not include expenditures related to surveillance nor operations research; in addition, not all expenses related to M&E may be included such as salaries and training cost of M&E staff. M&E-related expenditures ranged from about US $1200 in the Bahamas (representing less than 0.1% of total reported domestic public and international expenditures on HIV or US $0.2 per person living with HIV) to US $10.9 million in Uganda (representing 5% of total HIV expenditure or US $12 per person living with HIV).

Forty percent of countries indicated that M&E activities are financed exclusively through international sources, whereas 12% reported that HIV M&E was financed exclusively through domestic funding. Expenditures for HIV serosurveillance (essential for tracking the HIV epidemic) were fully funded by external partners in 38% of countries and by domestic sources in 34% of countries.

Status of National HIV M&E Systems

The one national HIV M&E system needs to function across different sectors, service delivery areas and levels of implementation. Developing a fully functional M&E system, therefore, takes time. The technical aspects and organizational structures of the system are equally important for ensuring effective functioning.¹³

Basic M&E Elements

Basic elements that need to be in place in all countries are:

1. One national M&E plan, including budgetary requirements and with funding secured for its implementation;
2. M&E staff in a functional national M&E unit and/or a national M&E technical working group to coordinate M&E activities;
3. Central national HIV database(s) to manage and access data efficiently.

For the 85 countries for which both 2005 and 2007 NCPI data were available, the following positive trends were noted: the number of countries having these basic M&E elements in place increased from 12 to 33 (Fig. 3). The most noticeable improvements occurred in sub-Saharan Africa and Latin America.

FIGURE 3: Basic* M&E elements by country, in 2005 and 2007. *One national M&E plan including budgetary requirements and with funding secured; a functional national M&E unit and/or M&E technical working group; a central national HIV database.

The overall percentage of countries that invested in one national M&E plan increased from 41% to 75%; those with a functional national M&E unit increased from 55% to 66%; those with a national M&E technical working group increased from 58% to 85%; and those with a central HIV database increased from 42% to 67% (Fig. 4).

FIGURE 4: Percentage of countries with specific M&E components, in 2005 and 2007.

Characteristics of National M&E Plans

An important prerequisite for achieving one national M&E system is having in place a comprehensive, multiyear national M&E plan that has been endorsed by all stakeholders involved in HIV M&E. Of the 135 countries reporting, 67% had one national M&E plan in place; 96% of these were endorsed by key stakeholders; and 81% covered a period of at least 3 years. Although the majority (90% or more) of national M&E plans reflect the standard content, a strategy for assessing data quality is missing in 1 in 5 countries. We refer to ‘standard content’ here as having a strategy for data collection and analysis; a well-defined standardized set of indicators; HIV serosurveillance and behavioral surveillance; guidelines on data collection tools; and a strategy for data dissemination and use.

Country Harmonization and Alignment With the National M&E System

One of the major shortcomings in the past has been the fragmentation of M&E efforts across various agencies and sections of government, resulting in duplication of effort, increased data collection burden and ineffective data flows. Those problems have not been overcome yet. Although 96% of national M&E plans were endorsed by key partners, less than one third (30%) of countries reported that all key partners aligned and harmonized their M&E requirements (including indicators) with the national M&E plan. Non-alignment or non-harmonization is particularly problematic in the Caribbean, south and southeast Asia and sub-Saharan Africa; regions with the most serious HIV epidemics (Table 1).

TABLE 1: Percentage of Countries by Region* With Partners' Support for the National M&E Plan and Data Sharing

Two-thirds (66%) of countries have a mechanism in place to ensure data from major implementing partners are shared with the government. Except for south and southeast Asia and sub-Saharan Africa, data-sharing mechanisms have been established in less than two-thirds of countries in each region (Table 1).

Human Resources for M&E

Sufficient numbers of adequately trained staff who have dedicated time for M&E responsibilities is perhaps the most critical factor in the functioning of a national M&E system. Of the 135 countries reporting, 67% indicated that they had a functional national M&E unit and a further 20% were in the process of developing one-up from 55% and 11% of countries, respectively, in 2005 (when 94 countries reported). UCC reporting from 86 countries in 2007 indicated that HIV M&E units included more than 1 staff person in 60% of countries. To assist national M&E staff with coordination and technical oversight of M&E activities, which are typically carried out by an array of organizations, 80% of countries have set up national M&E technical working groups.

Data on M&E training indicate that 69% of countries conducted such training at national level and 44% did so at subnational level, whereas 53% indicated that civil society partners were included in these trainings. However, no standardized information is available on what constitutes a “trained” individual or how the training affected individual job performance.

M&E Concerns in Global Fund Grants

Performance information in the 59 Global Fund scorecards showed that M&E concerns were explicitly listed for remedial action in more than half (56%) of the grants. Systematic weaknesses in M&E systems (such as insufficient human capacity, lack of planning, ill-defined performance frameworks and indicators, weak health information systems and lack of survey data) were indicated in 51% of the grants, and specific data quality concerns were raised for 34% of the grants.

Thirty-nine percent of the grants received a recommendation for funding continuation, 58% received a “conditional go” and 3% a “no go”. These recommendations did not seem to be tied to any M&E concerns nor was remedial action made a condition for continued funding.

Data Availability, Quality, and Use

Timely and good quality data need to be collected for use in improving the HIV response.

A. Data Availability

UNGASS and MDG 6 Data

Although limited to 25 indicators, UNGASS reporting is informative in terms of data availability. Data for 22 of these indicators are derived from program monitoring and standardized surveys. UNGASS data from 147 countries revealed that data availability is highest for indicators pertaining to antiretroviral therapy, prevention of mother-to-child transmission and blood safety (Fig. 5A). It is lowest for most-at-risk populations, especially injecting drug users and men who have sex with men (Fig. 5B). Although 72% of countries indicated that the indicator on coverage of HIV prevention programs for injecting drug users was relevant to their epidemics, only 20% reported such data. For the 5 UNGASS indicators, where sex disaggregation was requested, 62% of the reported data were indeed disaggregated in the 2006 reporting round, increasing to 82% in 2008.

FIGURE 5: Data reported for UNGASS indicators considered relevant to the country context. A, Data reported for specific UNGASS indicators pertaining to HIV programs for the general population. B, Data reported for specific UNGASS indicators pertaining to HIV programs and HIV-related knowledge and behaviors in most-at-risk populations (displayed in order of increasing data availability). ART, antiretroviral therapy; IDU, injecting drug user; MSM, men who have sex with men; OVC, orphans and vulnerable children; PMTCT, prevention of mother-to-child transmission of HIV; SW, sex worker; TB, tuberculosis.

Data availability for the 5 UNGASS indicators which also serve to measure progress toward MDG 6 are provided in Table 2. Around half of the countries (ranging from 43% to 67%) have current data available, with the exception of indicator 12 (which may be due to the fact that it is less relevant in nongeneralized epidemics). Sex-disaggregated data are still a big gap: 48%-69% of countries lack data by sex.

TABLE 2: Availability of Data for Monitoring Progress Toward MDG6 (n = 192)

Survey Data

Although biological and behavioral surveillance and surveys are essential to determine the drivers and the spread of the HIV epidemic in each country, they also provide data for assessing results of HIV responses. The strength of surveillance and surveys depends on a standardized approach and repeated implementation (every 3-5 years) allowing for trend analysis.

Because HIV-related questions were first included in 1988,¹⁴ the DHS has progressively focused on HIV and recent surveys have included over 50 specific HIV-related questions. Such DHS surveys have been conducted in 65 countries. Of the 42 countries where the survey was repeated, 20 were countries with a generalized epidemic (52% of such countries worldwide), and 27 countries had at least 3 survey rounds, including 14 countries (36%) with generalized epidemics. The first AIS, dealing exclusively with HIV and AIDS issues, was implemented in 2003 in Tanzania. The AIS has since been implemented in 6 countries and has been repeated so far only in Tanzania. Since the introduction of HIV testing in DHS/AIS in 2001, 29 countries have included this measurement providing HIV prevalence data in addition to knowledge and behavioral data, and 5 countries have repeated this measurement as part of the survey.

Data from surveys targeting most-at-risk populations are much more difficult to obtain. The MEASURE DHS website includes survey data from 13 countries only (5 in Africa, 4 in Asia, 3 in the Caribbean and 1 in Latin America). This is not a complete list of all countries that have carried out a BSS-type survey; unfortunately, there exists no single public site where such data for multiple countries can be accessed easily. Some national AIDS programs (eg, in India¹⁵ and Brazil¹⁶) disseminate their national survey reports on a dedicated AIDS website.

Routine Program Monitoring Data

Countries need to capture routine monitoring data on facility-based and community-based HIV programs for the purpose of program management. This information is ideally compiled and aggregated in the routine health information system and/or in a specific centralized national database. Table 3 provides an overview of the major information systems with HIV-related data available in country.

TABLE 3: M&E-Related databases (2008 NCPI; n = 135)

AIDS Evaluation/Research

Evaluation is one of the weakest areas in national M&E systems. Yet evaluations provide information which cannot be obtained from routine monitoring because they can answer questions of efficacy/effectiveness, quality of services, equity, efficiency and client satisfaction.

A direct way to ascertain the existence of HIV-related evaluation and research studies is through systematic review efforts such as the Cochrane Collaboration Systematic Review Database. Currently, there are 54 completed HIV-related systematic reviews accessible and 42 under development, each of which includes several evaluation studies of HIV interventions.¹⁷ However, the evidence base is incomplete and it is not clear what studies have led to actionable results for improving the national HIV response and/or specific interventions. A study by the authors in 2004,¹⁸ revealed 142 reports related to evaluations of HIV prevention interventions. Most evaluations took place in formal settings such as a health facility, school or workplace with easy to access populations, and the majority of studies (75%) used a 1-group prepost test design, which does not provide strong evidence. Important information on the evaluation methodology, the study participants and the intervention tested were commonly missing, which hinders application of evaluation findings for program improvement or replication.

The Global Fund reports that 19% of all approved grants (363 grants for HIV, tuberculosis, malaria) in rounds 1-5 and 52% in round 6 (85 grants) included operations research. For round 6, this was the case for 34% of HIV grants, 56% of tuberculosis grants and 74% of malaria grants.¹⁹ The Technical Review Panel for Global Fund grant proposals considered the relatively weak operations research components in round 8 proposals to be a major missed opportunity.²⁰

B. Data Quality

Data have to be valid to avoid misdirecting decision making. Global Fund grant scorecards indicated data quality as an explicit concern in 34% of the 59 grants in sub-Saharan Africa. The key concern is the quality of the data reported to the Global Fund as a basis for performance-based disbursement; only 1 scorecard mentioned the potential negative impact of poor quality data on program decisions. Systematic information on data quality is otherwise not publicly available, although the Global Fund now requires regular data quality assessments, including formal data audits.

C. Data Use

Timely good quality data are wasted if they do not inform decision making: data use is the ultimate purpose for conducting M&E. The 2007 NCPI asked countries to rate themselves on data use in HIV program planning and implementation. About half of the countries (49%) rated their data use as above average. About 1 in 5 countries in the Caribbean, eastern Europe and Central Asia, and south and southeast Asia rated themselves as below average (Table 4). Though evidence of systematic data use for program improvement is not available, the self-ratings suggest that there is considerable room for improvement.

TABLE 4: Percentage of Countries* by Region† Self-Rating Their Data Use in HIV Program Planning and Implementation Using a 5-Point Scale From 0 Low to 5 High

DISCUSSION

UNGASS reporting in 2003 revealed that shortcomings in M&E systems represented 1 of the 4 most pressing challenges facing the achievement of the time-bound targets set out in the DoC.²¹ Agreement on the “Three Ones” principles addressed the prevailing dysfunctions in coordinating national HIV responses and emphasized the integration of various M&E efforts in support of one national M&E system.⁸ Considerable progress in M&E implementation has been made since. There is evidence of rapid scale-up of national M&E systems, even when comparing only the 2005 and 2007 NCPI data for 85 countries. There are also encouraging signs that M&E capacity-building is not just focused on the national level: M&E training often included representatives from subnational levels and civil society partners, and several countries reported having a functional Health Information System at both national and subnational levels. One general marker of increased country commitment to accountability and ownership, especially in low-income and middle-income countries, is the dramatic increase in the number of countries reporting to the United Nations on their progress in implementing the 2001 DoC.

Important improvements in methods to track key aspects of the epidemic have been made in recent years²². Inclusion of HIV testing in representative household surveys and their repeated use have enhanced the accuracy of national estimates of HIV prevalence and increased the capacity to link behavioral and disease outcomes. However, important concerns remain. In a recent review of the quality of serosurveillance systems in 127 low-income and middle-income countries, only 31% were categorized as fully functioning, and as many as 35% were labelled as poorly functioning. Low scores were mainly attributed to lack of data from most-at-risk groups,²³ a finding confirmed in the UNGASS data. Based on the availability of data for the 5 UNGASS indicators to measure MDG 6, serious efforts are needed to address data gaps over the next 5 years. Concerns about data quality also need to be addressed and systematically monitored. For example, the fact that one third of the 59 Global Fund grants had data quality concerns not only seriously compromises the effectiveness of the performance-based disbursements but also may lead to misdirected decision making.

The evidence for data use in support of planning and programming is difficult to find. Being explicit in what data were used and how they were used not only increases transparency and accountability but makes it possible to assemble a clear rationale for why specific programs are supported. A recent analysis of national AIDS strategies by the World Bank indicated that although most strategies contain some analysis of the drivers of the epidemic, the rigor varies and the analyses seldom inform decision making.²⁴ Evaluation of resource flows has found that national allocation of prevention resources is sometimes sharply at odds with the picture of the epidemic generated by national surveillance systems.²² In addition, there are important challenges to data use in the real world because the available information does not necessarily illuminate the optimum courses of action.²⁵ Inertia and political pressures may also work against the continuation of particular programs or strategies, even when evaluation data indicate that they are effective (viz, harm reduction for injecting drug users and sex education for young people).²²

Without oversimplifying the complexities of data use, at least good quality data should be available, properly understood and seriously considered in the process of making policies, drawing up strategies and plans, and improving programs. A better understanding of the barriers and facilitators for data use is needed and effective strategies to facilitate data use and a handful of tried and tested measures that can track progress made over time.

Without the necessary financial resources to support full prioritized implementation, even the best laid-out plans are unlikely to achieve success. Although the Global Fund, the World Bank and the U.S. President's Emergency Plan For AIDS Relief (PEPFAR) permit as much as 10% of any grant to be earmarked for M&E, countries are not yet availing themselves of those provisions. A World Bank report has indicated that if this percentage allocation had been applied to its Multicountry HIV/AIDS Programs costs, some $50-100 million would have been available for M&E in the 2 rounds of Africa Multicountry HIV/AIDS Program projects.²⁶ Countries are urged to make full use of the available monies and donors should use adequate M&E funding allocation as a requirement for sign-off on proposals. The fact that only 7% of the 59 Global Fund scorecards considered in this article explicitly requested an increase in M&E funding allotment, although most grants include less than the minimum budget requirement, points to missed opportunities.

Long-term sustainability of M&E funding is also a concern. The low level of country ownership, expressed by low financial investment of domestic funding, may well be related to the fact that most of the M&E push so far has been donor driven. Donor funding for M&E can be used to catalyze the development of national M&E systems, but governments themselves also need to invest more in M&E. In addition, external funding for M&E does not necessarily support the development of the national M&E system. Data presented in this article indicated endorsement of national M&E plans by most stakeholders, yet these same stakeholders do not necessarily align their M&E requirements with those in the national M&E plans. In addition, data sharing with the national AIDS program is still not routine. These findings indicate that the implementation of the one national HIV M&E approach still has some ways to go. Efficient use of resources is imperative,²⁷ all the more so in the context of the global economic recession.

M&E capacity-building is not a one-off activity. Too often, there has been an exclusive focus on short-term M&E training without the necessary quality assurances and follow-up. Whether the “right” people were trained in the “right” areas, what constitutes a “trained” individual and the impact of training on job performance is not always clear. In addition, training is only 1 approach to improve human capacity. High staff turnover is an additional, common problem. These issues highlight the need for long-term capacity-building planning and an increased focus on mutually reinforcing approaches that can enhance the competencies of individuals.

The Way Forward

There is growing recognition that greater investment in program evaluation and operations research is needed. Budgets for research and analytic work have not necessarily been used to generate data for improving high-priority programs or effective scale-up of effective programs; often funding is allocated to projects demanded by researchers themselves.²⁶ The recent push from the Global Fund, UNAIDS, PEPFAR and the World Health Organization (WHO) to focus more on operations research and evaluation is now putting these issues on the table. Although there has been some positive reaction, Global Fund data show that: HIV proposals lag behind those directed at tuberculosis and malaria in terms of including evaluation studies; and, where evaluation studies are included, they generally do not serve the needs of national AIDS programs or their constituents.²⁰ The development of a coordinated national evaluation agenda is needed to ensure evaluation studies produce actionable results for improving HIV programs, to avoid duplication of efforts and to draw on all available capacity in country to conduct such studies.

In their recent article on behavioral strategies to reduce HIV transmission, Coates et al²⁸ argue that prevention science can and must do better and highlight the urgent need to move beyond evaluating strictly behavioral approaches to include evaluations of integrated behavioral, biomedical and structural HIV prevention strategies. Current debates also acknowledge that there remains a gap in evidence of how HIV resources can be managed best to contribute to building health system capacity, how to integrate HIV interventions into primary healthcare systems and how HIV scale-up is affecting other disease programs.^29,30 There has also been a call for independent population-based assessments to ensure that the scale-up of AIDS control is driven by sound evidence of health effects. It is estimated that around 1% of current external funding commitments to HIV and AIDS programs would be sufficient to conduct rigorous independent evaluations about the population impacts or the collective effectiveness of AIDS control efforts rather than restricted initiative-specific assessments.³¹

CONCLUSIONS

If M&E is to fulfil its role in helping achieve (and assess the achievement of) MDG 6, M&E system strengthening over the next 5 years should focus on:

1. Prioritizing data use;
2. Addressing important data gaps;
3. Fully drawing on available donor funding for M&E and increasing the share of domestic funding sources over time; and,
4. Implementing a systematic evidence-based approach to human capacity-building at all levels.

These recommendations apply generally for all countries. However, to set priorities for M&E system strengthening activities that match the specific contexts of countries, it is important to conduct regular, standardized and transparent assessments of the status of national M&E systems. The results should be seriously reviewed and should guide national M&E planning in a manner that involves all relevant stakeholders, including civil society. Standardized tools and important benchmarks for national M&E systems, have been agreed to and are supported by the global M&E reference group (which includes international and donor agencies, government and civil society representatives).³² As the rapid scale-up in some countries has shown, it is possible to put in place a basic M&E system relatively quickly. Now, countries need to build systematically and strategically on the strengths of existing systems to achieve fully functioning national M&E systems by 2015.

ACKNOWLEDGMENTS

We acknowledge the support of Luisa Frescura and Ali Safarnejad (M&E Division, UNAIDS/Geneva, Switzerland) for their assistance with data analysis and the development of graphs.