Skip Navigation LinksHome > December 2009 - Volume 52 - Issue > Are We on Course for Reporting on the Millennium Development...
Text sizing:
A
A
A
JAIDS Journal of Acquired Immune Deficiency Syndromes:
doi: 10.1097/QAI.0b013e3181baec7c
Supplement Article

Are We on Course for Reporting on the Millennium Development Goals in 2015?

Rugg, Deborah PhD*; Marais, Hein BA Law, Hon. Journalism†; Carael, Michel PhD‡; De Lay, Paul MD*; Warner-Smith, Matthew MPH*

Free Access
Article Outline
Collapse Box

Author Information

From the *Department of Evidence, Monitoring and Policy (EMP), UNAIDS, Geneva, Switzerland; †Independent Writer, Johannesburg, South Africa; and ‡Faculty of Social Sciences, Free University Brussels, Brussels, Belgium.

The authors of this paper declare no conflicts of interest.

Correspondence to: Dr. Deborah Rugg, PhD, Chief, Monitoring & Evaluation (EVA) Division, UNAIDS, 20 Avenue Appia, CH-1211 Geneva 27, Switzerland (e-mail: ruggd@unaids.org).

Collapse Box

Abstract

Objectives: At the 2001 United Nations General Assembly Special Session on HIV/AIDS (UNGASS), Member States agreed to regularly review progress made in national responses to HIV. This article provides (1) a brief overview of how the resultant global UNGASS reporting system was developed; (2) the origins, background, limitations and potential of that system; (3) an overview of the articles in this supplement; and (4) crosscutting institutional and methodological issues.

Methods: United Nations Member States biennially provide The Joint United Nations Programme on HIV/AIDS (UNAIDS) with data on 25 core indicators of national responses to HIV, collected in Country Progress Reports. This article critically reviews and interprets these data in light of international political considerations and overall data needs.

Results: There has been a considerable improvement in response rates, accompanied by an increase in data quality and completeness. Both nationally and internationally, the UNGASS process is viewed as being more substantial and important than a reporting exercise to the United Nations General Assembly. The process has catalyzed the development of national monitoring systems and has created opportunities for civil society to monitor and challenge government commitments and deeds.

Conclusions: Although the UNGASS global reporting system now comprises an unequaled wealth of data on HIV responses, collected from a broad range of countries, it cannot yet answer several critical questions about the progress and effectiveness of those responses. Evaluation studies that go beyond indicator monitoring are needed, but they will take time to design, fund, implement and interpret. In the meantime, this global monitoring system provides a good indication of the overall progress in the global response to HIV and whether Millennium Development Goal (MDG) 6 (to halt and reverse the HIV epidemic) is likely to be reached by 2015.

Back to Top | Article Outline

INTRODUCTION

To be effective, responses to the HIV epidemic need to be based on sufficient evidence and on strategic information that is timely and of good quality. Monitoring and evaluation (M&E) activities provide such information and are essential components of effective national HIV programs.

Monitoring and evaluation activities are closely linked, but they differ in purpose and design. Large-scale monitoring activities are intended to track the epidemic and the response, periodically taking stock to achieve better understandings of the trends, magnitude and changing patterns of the epidemic. Monitoring activities can provide information at global and national levels that can guide the targeting of resources and interventions and the extent of service coverage that is needed or is being achieved. Monitoring can also be useful on a smaller scale (eg, at project level) for tracking services and outputs.

Systematic evaluation activities, on the other hand, are intended to build upon the findings of monitoring activities. They do so by providing additional information from special evaluation studies that are designed to determine the merit of specific interventions, programs, and policies and their scope, quality, intensity, efficiency, effectiveness, and overall impact. Specific evaluation activities can help governments identify and understand factors that facilitate or hinder the achievement of their targets.

National governments are responsible for ensuring that both monitoring and evaluation activities are adequately planned and budgeted, and are systematically implemented. Pressure for public accountability and a need to target limited resources as efficiently as possible and to track progress has placed a great deal of emphasis on the establishment of national and global indicator monitoring and reporting systems. The quality and usefulness of these indicator monitoring systems, however, ultimately depends on how well the monitoring systems are embedded in strategic national and global frameworks that include both monitoring and evaluation activities.

Such quality systems will make it possible to determine the effectiveness of HIV responses (national and global) and to report on the attainment, or not, of United Nations (UN) Millennium Development Goal (MDG) 61 of halting and reversing the HIV epidemic by 2015. Toward that end, the UN Secretary General mandated The Joint United Nations Programme on AIDS (UNAIDS) to lead one of the most ambitious and important global tracking efforts in the history of HIV, namely the monitoring of the UN Declaration of Commitment on HIV/AIDS (DoC) made by 189 Member States at the UN General Assembly Special Session on HIV/AIDS (UNGASS) in 2001.2 UNAIDS established the UNGASS comprehensive global reporting system, which monitors progress toward targets across all key indicators and from all Member States. Those data span 10 years and reside in the UNAIDS Global Response Database,3 where the results are analyzed and biennial reports are produced for the UN and public.4

The articles in this supplement represent a first ever, indepth, thematic technical analysis of the key findings from this global monitoring system, combined with complimentary data from other sources, on the status of both the global HIV epidemic and the response.

This article provides (1) a brief overview of the progress of the global reporting system; (2) origins, background, limitations and potential of the UNGASS reporting; (3) an overview of the articles in this supplement; and (4) cross-cutting institutional and methodological issues.

Back to Top | Article Outline

THE CHALLENGES OF GLOBAL COMMITMENTS

The avowed purpose of the creation of the UN was “to achieve international cooperation in solving international problems of an economic, social, cultural, or humanitarian character and in promoting and encouraging respect for human rights.”5 To that end, UN and other intergovernmental organizations have initiated and facilitated technical, financial and political support around health and development issues, and assisted in building capacity to deal with those issues. However, in addition to other factors, intragovernmental problems relating to interests, policies, agendas and coordinating roles shape Member States' efforts to meet their commitments and their ability (and willingness) to report on those endeavors. In recognition of these potential difficulties, development programming and assistance have undergone significant shifts, and those changes have considerably influenced the manner in which UNAIDS has approached the tracking of national HIV indicators.

Thus, UNAIDS has placed increased emphasis on national coordination and ownership,6 on the harmonization of donor funding and its alignment to national priorities, and on the effectiveness of national programs. The emphasis on harmonization is related to broader efforts to improve aid coordination and to strengthen, rather than bypass or overburden, domestic systems and capacities.7 In doing so, it has sought to build on approaches endorsed, for example, by the Development Assistance Committee of the Organization of Economic Cooperation and Development in 2003 when it proposed guidelines for the harmonization of development assistance. Subsequently, the 2005 Paris Declaration on Aid Effectiveness8 committed donors to aligning aid with national priorities and systems, improving the harmonization of aid procedures and systems, helping countries strengthen their capacities, and enabling them to take the lead within a framework of mutual accountability.8,9

Meanwhile, the increased attention given to evidence of positive change, including outputs, outcomes and impacts of interventions had led to markedly increased attention to M&E systems. It had reinvigorated debates about whether, and how, causal links can be established between specific interventions and outcomes.10,11 It also had underscored concerns about the risks of possible overemphasis on short-term “products”12 and of a bias toward interventions where “outcomes” are easier to measure.

The focus on monitoring and evaluation had been driven by a pragmatic need to strengthen accountability and achieve sound governance. Frequently, though, such accountability was to donors, who (initially at least) also tended to be the arbiters of “good” governance. A driving factor was the perceived need of donors to give account to their domestic political constituencies. By the late 1990s, however, that approach was yielding to a greater emphasis on country ownership and on achieving accountability toward potential beneficiaries.

This emphasis on public accountability became reflected in the emergence of international systems for reporting on progress toward international agreements, particularly those made within the context of the UN.

In the 1998 UN General Assembly Special Session on the World Drug Problem, for example, the UN Office on Drugs and Crime (UNODC) was mandated to publish “comprehensive and balanced information about the world drug problem.” This led to the annual publication, since 1999, of the World Drug Report. The report is based primarily on data obtained from Annual Reports Questionnaires, which UNODC requests national governments to complete and submit. Two of the main limitations identified by UNODC in this reporting process were, (1) limited response rates and (2) “that most countries lack the adequate monitoring systems required to produce reliable, comprehensive, and internationally comparable data.”13

The goal outlined as part of the Education For All initiative, monitored by the United Nations Educational, Scientific and Cultural Organization (UNESCO), is an example of a global commitment which some countries have interpreted as a right or a vision, others as a movement or process, and others as a framework for action. Given this spectrum of interpretation, Education For All has given rise to a variety of strategies, plans, forums, groups, initiatives and research. For some, this diversity was viewed as evidence of richness, but those in charge of implementation and evaluation have raised concerns about the paucity of national strategic frameworks and lack of accountability.14

A contrasting approach was taken in monitoring progress toward the 8 Millennium Development Goals (MDG) adopted by UN Member States in 2000. For those 8 goals, over 20 targets and over 60 indicators are tracked and publicized in the annual MDG Report. Because the broad-ranging and multisectoral nature of the MDG makes the direct collection of data from countries for MDG reporting unfeasible, the report is based on a synthesis of data collected from a variety of (existing) sources. The UN Statistics Division, working through the interagency and expert group on the official MDG indicators (which includes representatives from UN agencies and their partners), identifies the most appropriate sources of data for reporting against each indicator. For example, data for the HIV indicators for Goal 6 (to halt and reverse the HIV epidemic) is provided by UNAIDS, the World Health Organization (WHO) and the United Nations Children's Fund (UNICEF), and is obtained through national reporting, following different data collection streams.

Back to Top | Article Outline

TRACKING INDICATORS OF NATIONAL COMMITMENT

The UNGASS on HIV/AIDS marked the first occasion on which the UN General Assembly deliberated on a specific disease. The resultant DoC15 superseded and extended previous international or regional commitments on HIV. Based on the several commitments made in the DoC, UNAIDS identified a set of 25 “core indicators” that capture specific aspects of HIV responses using a traditional logical framework that ranges from global to national level and from input to impact (Fig. 1). Countries have committed to reporting biennially on those indicators.

Figure 1
Figure 1
Image Tools

In the Declaration of Commitment, countries agreed to:

1. Conduct periodic reviews of progress made and disseminate results widely and to include civil society, people living with HIV, vulnerable groups and caregivers in the process;

2. Include HIV on the agenda of regional meetings at the ministerial and Head of State levels;

3. Support data collection to facilitate periodic reviews by regional groups of progress in implementing regional strategies and disseminate the results widely;

4. Encourage the exchange of information and experiences among countries implementing the DoC and, in particular, South-to-South and triangular cooperation;

5. Devote sufficient time of the annual General Assembly session to review a report by the Secretary General on the progress made toward the DoC, including ensuring that HIV is included in the agenda of all appropriate UN meetings;

6. Explore the feasibility of developing, with partners, systems for voluntary monitoring of global drug prices.

In response to commitment 6, and in explicit support of the MDGs 4, 5 and 6, the governments of Brazil, Chile, France, Norway and the United Kingdom established UNITAID, an international drug purchase facility dedicated to reduce the price of and increase access to essential medicines for HIV, tuberculosis and malaria.

The targets listed in the DoC reflected the key issues, as perceived at the time. Of the indicators subsequently identified for the monitoring of the Declaration of Commitment, 5 are also included in the monitoring framework for the MDG16 and 7 are included in the monitoring framework for the 2006 Political Declaration on Universal Access.

Targets were subsequently set for HIV treatment in the WHO's “3 by 5” campaign to put 3 million people on antiretroviral treatment by 2005.

Back to Top | Article Outline

THE UNGASS REPORTING SYSTEM

The set of core UNGASS indicators has been designed to form a vital basis for national HIV program monitoring and global monitoring. The indicators are linked to and integrated with the monitoring requirements of 3 major global initiatives, namely the UNGASS DoC, the Universal Access (UA) Initiative, and the MDG. This is a unique approach of bringing national and international stakeholders together to reduce reporting burdens, avoid redundancy and to facilitate data reconciliation from different data sources.

The UNGASS indicator definitions meet the international standards set by the UNAIDS-led global M&E Reference Group (MERG) and those developed for the MDG by the UN Statistics Division. Although considerable progress in harmonization has been made, duplication of data collection and reporting still occurs among the UN agencies. The MERG has sought to harmonize the UNGASS indicators with the indicators of the Global Fund to Fight HIV, Tuberculosis and Malaria (the Global Fund); the U.S. President's Emergency Plan for AIDS Relief (PEPFAR); and other major bilateral donors. The Global Fund, WHO, UNICEF and UNAIDS are also harmonizing reporting procedures to enhance access to accurate and timely information.

The national framework for UNGASS reporting has facilitated a process that exerted tangible influence on national decisions.17 It has also tasked UNAIDS with the role of monitoring countries' HIV commitments. The multisectoral character of the UNGASS indicators was intended to encourage different sectors, various development partners and bilateral organizations to collaborate. The parallel emergence of participatory forums and expanded UN theme groups on HIV/AIDS, together with processes to support poverty reduction strategies, has helped strengthen the voice and involvement of civil society organizations.18

Countries have come to regard the UNGASS process as more than a reporting exercise at the UN General Assembly. The process is being seen also as an index of country ownership of national HIV responses and as a tool for exacting accountability. The 3 reporting rounds to date have reminded, as well, that ownership, if it is to be substantive, requires capacity in several respects. Most obvious is a need for the capacity to collect and analyze quality data; and a need to build the capacity that can enable civil society to participate meaningfully in monitoring, evaluating and refining those national responses. Indeed, capacity constraints are one of the reasons why the extent of participation by civil society in the UNGASS reporting process is still mixed.18 Nevertheless, the participatory element of the process (including public availability of national reports on the UNAIDS website) has opened opportunities for civil society to monitor (and challenge claims of) progress toward national commitments. The participatory forums are believed to have strengthened the voice of civil society organizations and to have helped provide them with platforms for airing their concerns and recommendations regarding HIV responses of countries.18 The number of nongovernmental organizations (NGOs) involved in the UNGASS reporting process has steadily increased, with more than 700 NGOs having participated in some capacity in the 2007 reporting round. In instances where civil society was unable to participate or where it believed that its views were adequately reflected in the national progress reports, NGOs have presented “shadow” reports to UNAIDS. About 25 such country “shadow” reports were submitted in 2007, some addressing specific concerns such as reproductive health, young people's needs or gender issues.

Back to Top | Article Outline

MIDWAY TO 2015 COUNTDOWN: TIME FOR CORRECTIONS

The 10-year milestone for UNGASS DoC reporting has almost arrived, as has the 2010 cut-off for UA, and the world is midway to the 2015 deadline for the MDG. This is an opportune juncture for taking stock of the UNGASS reporting system, for making the necessary (although time consuming) adjustments and system enhancements, and for launching complementary evaluation activities that can address questions that a global biennial reporting system cannot reasonably be expected to address.

The global monitoring system developed for HIV cannot yet answer many pertinent questions nor provide many of the data needed for guiding detailed national level program design and management improvements. Currently, it can realistically track and monitor progress only on selected key measures for global and national aggregation and advocacy purposes. As such, it provides a rough sketch of the current situation and, by collecting data over time, of changes and trends.

The UNGASS country response rate has been high from the outset and has increased significantly subsequently. In 2003, 54% of UN Member States submitted progress reports; that percentage rose to 77% in 2007. The increase has been associated with a marked improvement in both data quality and completeness of reporting.19 Core data are now available on the patterns of the HIV epidemic, the behaviors related to it, and on the programmatic and policy responses from a vast range of countries. Some deficiencies remain, but few global reporting systems have succeeded in accumulating such a wealth of data from such a large number of countries across several sectors and aspects of public health and development over so many years.

Among the questions surrounding the data collected in such global reporting systems is the validity and accuracy of the data. Some evidence exists of systematic differences between officially reported coverage measures obtained through administrative statistics and measures independently obtained through surveys. This has led some to question the validity of reported data for measuring changes in coverage.20,21

Systematic overreporting of coverage is particularly problematic in cases where funding is based on performance, such as in the model used by the Global Alliance for Vaccines and Immunisation (GAVI) and the Global Fund to Fight AIDS, Tuberculosis and Malaria, where there is a strong incentive to overreport. In those institutions, considerations of data validity have been deliberately weighed against the need to support strengthening of the capacity of governmental data systems and not creating parallel processes.22 Because data reported through UNGASS are not directly linked to funding, the incentive to systematically overreport coverage would seem to be less. Although political pressures to either underreport or overreport may exist, any underlying reasons for these are specific to particular countries and therefore do not lead to systematic bias in the dataset as a whole.

Critics of nationally reported data argue that the differences observed in the 2 approaches to coverage measurement make independent surveys a more valid measure than administrative statistics. Measures of service coverage used for UNGASS reporting use both methods (service statistics and survey data), depending on what is being monitored. The coverage of prevention services for injecting drug users, for example, is measured through cross-sectional surveys, due to difficulties in aggregating service statistics without double-counting individuals and the challenges of deriving adequate population size estimates. Conversely, coverage of antiretroviral therapy (ART) is based on routinely collected service statistics. Interestingly, within the UNGASS set, data quality issues are more prevalent in the survey-based measures of coverage than in the measures based on service statistics. This suggests that methodological challenges inherent in working with hidden populations are of more immediate concern than reporting bias.

International agencies can (and do) provide tools and other technical assistance to address methodological challenges and to improve transparency in the collection and reporting of service statistics. One example is the vaccination data quality audit tool developed by WHO to help address concerns about the validity of immunization coverage estimates based on administrative statistics.23

The debate about whether differences between survey data and reported administrative statistics result from deliberate overreporting or from methodological issues in measurement is likely to continue.24 However, it is generally agreed that systemic overreporting can be minimized and national ownership can be maintained through improvements in monitoring at the national and subnational levels, improved access to country data, capacity building in data collection, and improvements in survey quality.25

Indeed, an additional distinctive feature of the UNGASS process is its focus on increasing national ownership, including capacity building and the use of reporting as a catalyst for system development. This sets it apart from, for example, the World Development Report or the Human Development Report processes, which are built primarily on statistical analysis of existing indicator data.

Moreover, signing up to a negotiated text does not guarantee the desired outcome. Typically, accountability requires transparency, answerability or justification, and compliance.10 Those 3 characteristics are necessary, although not sufficient, for achieving accountability. The DoC was not legally binding, and accountability, therefore, had to be sought through other means. The inclusion of the core UNGASS indicators in the majority of national M&E frameworks has contributed to their internalization by countries and the UNGASS reporting system has progressively built national ownership.

Current results, however, show limited progress has been made toward the DoC targets:

* Among youth aged 15-24 years, only 38% of females and 40% of males can demonstrate accurate and sufficient knowledge about ways to protect themselves from acquiring HIV; the UNGASS target is 95% by 2010.

* Programs to prevent the transmission of HIV from mother to child currently reach 33% of those in need: the UNGASS target is 80% by 2010.

* By 2008, new infections in infants born to HIV-positive mothers had declined by 25% from 2001 levels in hyperendemic countries; the UNGASS target is a 50% reduction by 2010.

* On the other hand, financial investment in AIDS responses has increased substantially, and the global target of 10 billion US dollars in 2008 has been met. However, the global economic recession calls into the doubt the sustainability of that achievement.

As noted above, no global target was set in 2001 for access to HIV ART because the treatment was not yet available in low-income and middle-income countries. A target was only set in 2003, when the WHO launched the “3 by 5” initiative. That target was eventually met in 2007-2 years late. Meanwhile, UN Member States had agreed to a Political Declaration in 2006 in which they declared their commitment to achieve universal access (UA) to essential prevention, treatment and care services for all in need. The Declaration became known as the UA Initiative. The indicators for measuring the UA targets were based on the core set of UNGASS indicators and are monitored by WHO, UNAIDS and UNICEF. As shown in Figure 2, more than 50% of the countries reporting have established national targets for coverage of the most critical prevention, treatment and care programs-a considerable achievement in less than a decade.

Figure 2
Figure 2
Image Tools

Tracking progress toward UA requires not only the monitoring of output and outcome indicators, but also operations research and special evaluation studies on what does and does not work and why that is the case. A shortage of data and limited understanding about the effectiveness of different interventions continues to hinder the ability of national planners to structure an appropriate mix of interventions in their national HIV responses around those with strong cost benefit ratios.

Back to Top | Article Outline

CONTENT OF THIS SUPPLEMENT

The articles collected in this supplement examine specific aspects of the large body of data generated as part of the UNGASS global reporting process. They also assess whether the reporting system that was established for monitoring the DoC will be adequate for accurately gauging progress toward MDG 6: to halt and reverse the HIV epidemic by 2015. They analyze the strengths and weaknesses of the UNGASS global reporting system and recommend steps for improvements.

The main questions examined are:

* In 2015, will we know if we have achieved MDG 6?

* Are countries on track in terms of HIV programs and data collection efforts?

* And, if not, what needs to be done differently to increase the chances of achieving the goals and accurately assessing achievements?

The articles gathered in the first section chronicle the development of the global reporting system, review progress made in establishing national M&E systems and assess the performance of those systems in monitoring the UNGASS and the MDG targets.

The second set of articles focuses on developments in the creation of enabling environments. These articles examine the extent to which human rights concerns are being incorporated into national HIV responses, summarize data on policies and strategies affecting gender equity in access to antiretroviral treatment and other HIV services, and review national AIDS spending in low-income and middle-income countries. Finally, the role of civil society in holding governments accountable for tracking progress in national responses is assessed.

The final section examines the UNGASS data in terms of programmatic coverage, behavioral outcomes and disease-level impacts. Several recent articles have documented global coverage of HIV treatment and services for the prevention of mother-to-child transmission, and these issues are therefore not addressed here.26-30 The supplement concludes by examining the availability and quality of HIV seroprevalence trend data to assess whether the MDG target of halting and reversing the HIV epidemic by 2015 will indeed be measurable and the extent to which the putative achievement of that target could be attributed causally to the effectiveness of HIV programs.

Back to Top | Article Outline

CROSS-CUTTING METHODOLOGICAL AND INSTITUTIONAL ISSUES

The fact that many actors are involved in the HIV response and in its monitoring and evaluation poses specific challenges for a national and global reporting system. We explore some of these challenges below, whereas others are addressed in more detail in other articles in this supplement. They include:

* Analyzing clusters of indicator data that fall under the responsibility of different sectors;

* Aggregating national level data that are derived from limited geographically specific sampling;

* Comparability of data over time and between countries;19

* Quality assurance and vetting of data;19

* Dealing with outliers in data values, especially with data on hard-to-reach groups such as men who have sex with men and injecting drug users;31,32

* Roles and responsibilities of governments vis-à-vis those of international agencies.

Back to Top | Article Outline
Multisectoral Ownership of the Data

There is mixed evidence of progress in strengthening the involvement of sectors other than health in HIV data collection. Although a majority of countries have multisectoral strategies, the role of those sectors and the link between sectoral objectives and national strategies is not always clear. Multisectoral implementation tends to be slow and its potential has not yet been realized. In many settings, health sector ministries have retained control over both policy and national AIDS budgets. That division of authority and responsibility is reflected in national data reporting systems. Some government sectors do not report on their activities (for example, the military and other uniformed services), or their reports do not reach the National AIDS Commission for analysis alongside other elements of the national strategy. Other sectors have no tradition of data monitoring or are reluctant to report on the outputs of their HIV activities. One of the challenges is to analyze clusters of indicators that fall under the aegis of different ministries and/or organizations.

Back to Top | Article Outline
Aggregation of Data at National Level

For many indicators, subnational values are more relevant than national values (for example, HIV prevalence among most at-risk populations or risk behaviors). The data based on indicators should be useful for decision making at the level at which they are collected. Unfortunately, the data seldom are made available at the local level. In some countries, data are consolidated at the provincial level before being transmitted to national level structures. In addition, the processes used to aggregate data collected from 1 or 2 sites into a national value are not always standardized or transparent, which also can lead to gross overestimation of the national values of some indicators.

Data from routine monitoring systems (such as the number of people receiving ART, the percentage of schools with life skills education, etc.) and data from periodic surveys are not easily reconciled. Instructions for UNGASS indicators define and limit the data collection methods to increase the comparability of data across countries. This is necessary for regional and global data analysis purposes.

Back to Top | Article Outline
Fragmented Data on Various Sectors of the National Response

As a Joint Program, UNAIDS comprises a secretariat and 10 cosponsors, each with a specific division of labor at country and global levels.33 Those entities have a role in compiling UNGASS data at country level and the data are then reported to the UNAIDS Secretariat in Geneva, Switzerland. Because some programs also reflect the corporate priorities of specific UN agencies, HIV support programs can lack coherence-and this is often reflected in the data monitoring frames. Some agencies are more effective than others in performing their mandates and harmonizing their reporting systems. Mechanisms for harmonization at a global level have been attempted, but “turf guarding” remains an impediment. At country level, when it comes to men having sex with men, sex workers and injecting drug users, and other vulnerable populations, such as migrants, there remain major gaps in data and in the coordination of efforts between agencies. In too many countries, surveys remain largely localized and are funded through donor project channels because of weak and fragmented country systems. Ownership of the data and sustainability of national monitoring systems are serious problems and are often disguised with ad hoc indicators of success.

Back to Top | Article Outline
Improve Data Use at National and Regional Levels

Among the priorities is the need to perform better data triangulation (and, possibly, also meta-analyses) and launch a portfolio of strategically designed, targeted and timed evaluation studies that can complement the findings of the global monitoring system and other strategic information. This could be done by involving partners, countries, stakeholders and the MERG in developing and agreeing on evaluation strategies and designing and endorsing methodologies for such studies.

Capacity building is needed to strengthen the use of HIV data by various partners. Technical support initiatives in monitoring and evaluation (as with other development cooperation programs) have to be assessed against criteria of commitment and capacity.

UNAIDS, in coordination with its global and country-level partners, should promote evaluations and operational research at national and regional levels with the aim of generating data to inform national responses. Priority should be given to studies of behavioral change and the role of contextual factors, including gender inequality, stigma and material deprivation. Key indications of where to focus are found in UNGASS indicators, in the National Composite Policy Index and in national spending, resource flow data and coverage data. Other indications are found in the UNAIDS Country Office annual survey reports. Also needed are data reconciliation debates, planning of locally relevant evaluation studies based on the interpretation of the findings from the national monitoring systems, and then strategically using this information in developing and improving policies and programs.

Back to Top | Article Outline

TRACKING PROGRESS USING THE UNGASS FRAMEWORK

More than a quarter century into the HIV epidemic, results from the UNGASS commitment monitoring reports show uneven progress in national responses. The coverage of HIV prevention and treatment programs has to be dramatically scaled up if UA targets are to be reached. Serious deficiencies in monitoring and reporting systems remain. The data collected do not allow for the impact of HIV programs on behaviors and infection rates to be determined. At the same time, however, solid baselines are being created and a wealth of data has been collected and analyzed and is yielding important trend and other information.

Achievement of MDG 6-and the ability to measure such achievement-depends fundamentally on the strength and quality of country capacities. HIV programs rely on periodic population surveys, HIV sentinel and behavioral surveillance and well-functioning health systems to generate the information that can guide potentially effective decisions and achieve objectives. Sound governance and leadership are needed throughout the process not only for the design and implementation of policies but also to assure quality and efficiency of HIV services and to finance social and health services equitably and sufficiently.

UNAIDS and its cosponsors can play important roles supporting countries in prioritizing, harmonizing and expanding effective national responses, with multisectoral partnerships as key components. Efforts to improve coordination among UNAIDS cosponsors to enhance and harmonize that support have been underway for several years. For example, the UNAIDS cosponsors and secretariat have agreed to a division of labor for the provision of technical support.33 Nevertheless, at country level, lack of accountability of UN partners and multiple reporting cycles and procedures still hamper national monitoring and evaluation efforts. If the sometimes divergent accountability and learning needs of UNAIDS partners, donors and beneficiaries can be harmonized, the burden of multiple reporting can be reduced considerably. In turn, this would allow governments to focus more on evaluation evidence and best practices, enabling them to save time and resources necessary to evaluate the issues that are most pertinent to their specific country contexts.34

Greater progress is needed in aligning the efforts of country-level actors with nationally owned and nationally determined strategies. Nearly half of governments (45%) report that not all development partners align their efforts with national HIV strategies.35 The strategic impact of HIV efforts tend to be reduced when stakeholders do not fit their respective agendas to national strategies.36

In conclusion, by 2015, if progress in harmonization among partners and technical support has significantly improved, the national monitoring and evaluation systems should be significantly stronger and the global reporting system thereby considerably strengthened in time for reporting on the achievement of the MDGs. However, the accuracy and completeness of the reporting will depend on the political commitment at country level to keep HIV prominent on the health and social development agenda.

There remains a tension between data collection, use of data and strategic decision making. Using data strategically to inform the design and management of policies and programs remains a major challenge in most countries. The tendency is to deploy selected monitoring data as evidence of success, which reflects the pressure exerted on donors and governments to demonstrate “results”. In such cases, evaluation is reduced to an auditing tool and systems are not built to monitor progress and spending. There is a pressing demand from donors to justify funding in terms of effectiveness, efficiency and impact. In an environment where official assistance funds may become scarcer, funding the HIV response has to compete with other pressing health and development needs. Current projections indicate that funds required for reaching UA to prevention and treatment in 2015 would need to increase far beyond the US $10 billion needed in 2008.37 Such an increase in donor assistance may be feasible, but it will require building a strong evidence base showing whether countries are implementing appropriate programs, are doing so effectively and on a scale that is large enough to achieve significant impact.

The UNGASS DoC is only 1 mechanism for monitoring HIV responses, but it is the only global one. It has a major normative role and a standardization function that harmonizes and coordinates priority data collection methods.

Back to Top | Article Outline

REFERENCES

1. UNDP. Millennium Development Goals. Available at http://www.undp.org/mdg/. Accessed June 18, 2009.

2. UNAIDS. Declaration of Commitment on HIV/AIDS. United Nations General Assembly, June 25-27. New York; 2001. Available at http://data.unaids.org/publications/irc-pub03/aidsdeclaration_en.pdf. Accessed June 18, 2009.


4. UNAIDS. Report on the global AIDS epidemic. Geneva, Switzerland: UNAIDS; 2004, 2006, 2008.

5. United Nations. Charter of the United Nations. New York, NY: United Nations; 1945.

6. UNAIDS. The ‘Three Ones’ in action: Where we are and where we go from here. Geneva, Switzerland: UNAIDS; 2005.

7. Foster M, Leavy J. The choice of financial instruments. London, United Kingdom: Overseas Development Institute; 2001. Available at http://www.odi.org.uk ays Michelccess, untdownet to the AIDS commission.g systems at country level.

8. The Paris Declaration on Aid Effectiveness - Ownership, Harmonisation, Alignment, Results and Mutual Accountability. Available at: http://www.oecd.org/dataoecd/11/41/34428351.pdf.

9. Ebrahim, Alnoor, Edward Weisband, eds. Global Accountabilities: Participation, Pluralism, and Public Ethics. Cambridge, United. Kingdom: Cambridge University Press; 2007.

10. Bourguignon F, Sundberg M. Aid Effectiveness-Opening the Black Box. World Bank; 2007. Working Paper, Washington.

11. Watson D. Monitoring and Evaluation of Capacity and Capacity Development. (ECDPM Discussion Paper, 58B). Maastricht, The Netherlands: European Centre for Development Policy Management; 2005.

12. Watson D. Monitoring and Evaluation of Capacity and Capacity Development. (ECDPM Discussion Paper, 58B). European Centre for Development Policy Management. Maastricht, 2005.

13. UNODC. World Drug Report 2007. Vienna, Austria; UNODC; 2007.

14. Education For All. Report 2006. Paris, France; UNESCO.

15. UNGASS. Declaration of Commitment. New York, NY: UN; 2001.

16. UN. The Millennium Development Goals Report 2008. New York, NY: United Nations Statistics Division; 2008.

17. Peersman G, Rugg D, Erkkola T, et al. Are the investments in national HIV monitoring and evaluation systems paying off? J Acquir Immune Defic Syndr. 2009;52(Suppl 2):S87-S96.

18. Peersman G, Ferguson L, Torres MA, et al. Increasing civil society participation in the national HIV response: The role of UNGASS reporting. J Acquir Immune Defic Syndr. 2009;52(Suppl 2):S97-S103.

19. Warner-Smith M, Rugg D, Frescura L, et al. Monitoring the 2001 declaration of commitment on HIV/AIDS. J Acquir Immune Defic Syndr. 2009;52(Suppl 2):S77-S86.

20. Murray CJL, Shengelia B, Gupta N, et al. Validity of reported vaccination coverage in 45 countries. Lancet. 2003;9389:1022-1027.

21. Lim SS, Stein DB, Charrow A, et al. Tracking progress towards universal childhood immunisation and the impact of global initiatives: a systematic analysis of three-dose diphtheria, tetanus, and pertussis immunisation coverage. Lancet. 2008;9655:2031-2046.

22. Bishai DM. Comment: global initiatives in universal childhood immunization. Lancet. 2008;9655:2004-2005.

23. Lob-Levyta J. Vaccine coverage and the GAVI alliance immunization services support initiative. Lancet. 2009;9659:209.

24. Ronveaux O, Rickert D, Hadler S, et al. The immunization data quality audit: verifying the quality and consistency of immunization monitoring systems. Bull World Health Organ. 2005;7:503-510.

25. Lima S, Steina D, Charrowa A, et al. Correspondence: measurement of immunization coverage-authors' reply. Lancet. 2009;9659:211-212.

26. UNAIDS, WHO, UNICEF. Children and AIDS: Third Stocktaking Report. New York, NY: UNICEF; 2008. Available at: http://data.unaids.org/pub/Report/2008/20081201_3rd_stocktaking_en.pdf. Accessed June 16, 2009.

27. WHO, UNAIDS. Towards universal access: scaling up priority HIV/AIDS interventions in the health sector: progress report. Geneva, Switzerland: WHO; 2008. Available at: http://www.unicef.org/aids/files/towards_universal_access_report_2008.pdf. Accessed June 16, 2009.

28. Bordeaux Working Group. HIV Care & PMTCT in Resource-Limited Settings: Monthly Intelligence Reports. Available at: http://www.who.int/hiv/topics/mtct/newsletters/en/index.html. Accessed June 16, 2009.

29. Volmink J, Siegfried NL, van der Merwe L, et al. Antiretrovirals for reducing the risk of mother-to-child transmission of HIV infection. Cochrane Database Syst Rev. 2007:CD003510.

30. Ginsburg AS, Hoblitzelle CW, Sripipatana TL, et al. Provision of care following prevention of mother-to-child HIV transmission services in resource-limited settings. AIDS. 2007;18:2529-2532.

31. Adam PCG, de Wit JBF, Toskin I, et al. Estimating Levels of HIV Testing, HIV Prevention Coverage, HIV Knowledge, and Condom Use Among Men Who Have Sex With Men (MSM) in Low-Income and Middle-Income Countries. J Acquir Immune Defic Syndr. 2009;52(Suppl 2):S143-S151.

32. Mathers BM, Degenhardt L, Adam P, et al. Estimating The Level of HIV Prevention Coverage, Knowledge and Protective Behavior Among Injecting Drug Users: What Does The 2008 UNGASS Reporting Round Tell Us? J Acquir Immune Defic Syndr. 2009;52(Suppl 2):S132-S142.

33. UNAIDS. UNAIDS Technical Support Division of Labour: Summary and rationale. Geneva, Switzerland: UNAIDS; 2005. Available at: http://data.unaids.org/una-docs/JC1146-Division_of_labour.pdf. Accessed June 16, 2009.

34. Segone M, ed. Country-led Monitoring and Evaluation Systems. Better Evidence, Better Policies, Better Development Results. New York, NY: UNICEF; 2009.


36. Attawell K, Dickinson C. An Independent Assessment of Progress on the Implementation of the Global Task Team Recommendations in Support of National AIDS Responses. London, United Kingdom: HLSP Institute; 2007. Available at http://data.unaids.org/pub/ExternalDocument/2007/hlsp_gtt_assessment_item4_3_en.pdf. Accessed June 16, 2009.

37. UNAIDS. What Countries Need-Investments Needed For 2010 Targets. Geneva, Switzerland: UNAIDS; 2009. Available at: http://data.unaids.org/pub/Report/2009/JC1681_what_countries_need_en.pdf. Accessed June 16, 2009.

Keywords:

HIV; monitoring and evaluation; reporting systems

© 2009 Lippincott Williams & Wilkins, Inc.

Login

Search for Similar Articles
You may search for similar articles that contain these same keywords or you may modify the keyword list to augment your search.