Abstract: Monitoring and evaluation (M&E) is fundamental to global HIV program implementation and has been a cornerstone of the President's Emergency Plan for AIDS Relief (PEPFAR). Rapid results were crucial to demonstrating feasibility and scalability of HIV care and treatment services early in PEPFAR. When national HIV M&E systems were nascent, the rapid influx of funds and the emergency expansion of HIV services contributed to the development of uncoordinated “parallel” information systems to serve donor demands for information. Close collaboration of PEPFAR with multilateral and national partners improved harmonization of indicators, standards, methods, tools, and reports. Concurrent PEPFAR investments in surveillance, surveys, program monitoring, health information systems, and human capacity development began to show signs of progress toward sustainable country-owned systems. Awareness of the need for and usefulness of data increased, far beyond discussions of indicators and reporting. Emphasis has turned toward ensuring the quality of data and using available data to improve the quality of care. Assessing progress toward an AIDS-free generation requires that the global community can measure the reduction of new HIV infections in children and adults and monitor the coverage, quality, and outcomes of highly efficacious interventions in combination. Building national M&E systems requires sustained efforts over long periods of time with effective leadership and coordination. PEPFAR, in close collaboration with its global and national partners, is well positioned to transform the successes and challenges associated with early rapid scale-up into future opportunities for sustainable, cost-effective, country-owned programs and systems.
*Division of Global HIV/AIDS, Centers for Disease Control and Prevention, Atlanta, GA, Maputo, Mozambique, and Bangkok, Thailand
†Office of the US Global AIDS Coordinator, Washington, DC
‡Carolina Population Center, University of North Carolina at Chapel Hill, Chapel Hill, NC
§UNICEF, Division of Policy and Strategy, Statistics and Monitoring Section, New York, NY
‖Futures Group International, Washington, DC, and Dar Es Salaam, Tanzania
¶Center for Global Health Equity, Department of Global Community Health and Behavioral Sciences, School of Public Health and Tropical Medicine, Tulane University, New Orleans, LA
#Global Health Sciences, University of California at San Francisco, San Francisco, CA
**International Center for AIDS Care and Treatment Programs, Mailman School of Public Health, Columbia University, New York, NY
††HIV/AIDS Department, World Health Organization, Geneva, Switzerland.
Correspondence to: Laura E. Porter, PhD, Division of Global HIV/AIDS, Centers for Disease Control and Prevention, 1600 Clifton Road, MS E-30, Corporate Square, Building 1, Room 2306, Atlanta, GA 30333 (e-mail: email@example.com).
Various authors have professional relationships with PEPFAR (either as employees of PEPFAR-supported US Government agencies or as grantees/contractors). as outlined in the Copyright Transfer Agreement forms.
The authors have no other funding or conflicts of interest to disclose.
The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention, the US Government, or the World Health Organization.
Monitoring and evaluation (M&E) is a fundamental aspect of global HIV program planning, implementation, and improvement. Monitoring refers to the routine tracking and reporting of priority information about a program and its intended outputs and outcomes, whereas evaluation is the systematic collection of information about program activities, characteristics, and outcomes to determine the merit or worth of a specific program.1 In global HIV, M&E often refers to the “umbrella” of efforts that are used to inform HIV programs, including surveillance systems and population-based surveys, health information systems, community-based information systems, evaluation, and research.2 Because the President's Emergency Plan for AIDS Relief (PEPFAR) was originally driven by the goals of 2 million people on HIV treatment, 7 million infections averted, and 10 million persons receiving HIV care and support services, the collection and use of “strategic information” has been a cornerstone of PEPFAR.3 This article describes successes and challenges in advancing global capacity to monitor and evaluate HIV/AIDS programs in the era of PEPFAR.
A BRIEF HISTORICAL PERSPECTIVE ON GLOBAL HIV M&E
Prior to PEPFAR's launch in 2003, advances in surveys and surveillance predated important HIV M&E developments. The late 1980s and 1990s saw the development of HIV sentinel surveillance systems and the introduction of survey questions on knowledge and attitudes about HIV/AIDS in population-based surveys, such as the Demographic and Health Survey. There was debate about the feasibility of asking about sensitive sexual behaviors.4–6 The publication of the first Joint United Nations (UN) Program on HIV/AIDS (UNAIDS) M&E guide in 20007 provided the first global HIV/AIDS indicator standards and laid the foundation for standardization of survey instruments. Subsequently, the content on HIV/AIDS in Demographic and Health Survey was expanded significantly.8,9 HIV testing in population-based surveys was also introduced during this period, ultimately revolutionizing the estimation of HIV prevalence.10 Behavioral surveillance survey methodologies were developed with innovative sampling strategies to collect M&E data from key populations most at risk of HIV/AIDS.11
National HIV M&E systems were nascent during this period. Regional workshops held in East and West Africa provided a venue for country teams to share their challenges and plan for strengthening systems.12,13 Several countries had established a national HIV/AIDS M&E unit, but there were few human resources, inadequate physical infrastructure, low organizational capacity, and insufficient funds for M&E. Some countries were developing national HIV/AIDS M&E plans, but often with too many indicators and little harmonization of the indicators within and across countries. This lack of consistent baseline data from this era would later prove challenging to the evaluation of major HIV/AIDS initiatives.14
EARLY ADVANCES IN GLOBAL HIV M&E
Concurrent with the launch of the Global Fund to Fight AIDS, Tuberculosis, and Malaria (“the Global Fund”) in 2002 and PEPFAR in 2003 were several notable events at the global level that emphasized the importance of coordinated, systematic, long-term approaches to supporting the development of national HIV M&E systems. National governments, civil society, and donor partners agreed to the need for “Three Ones”—one national HIV strategic plan, one national HIV coordinating authority, and one national HIV M&E system.15 The “Third One,” as it became known, set an expectation that all HIV M&E efforts would be harmonized under one national HIV M&E system. Rugg et al16 called for a “Public Health Questions Approach to HIV M&E” to systematically approach HIV program monitoring and evaluation. The UNAIDS sponsored an advisory body, known as the Monitoring and Evaluation Reference Group, to coordinate global HIV M&E activities across donor organizations and governments. The UN organizations, the World Health Organization (WHO), the World Bank, the Global Fund, and PEPFAR-affiliated US agencies developed an International M&E Field Officer program, to advance country capacity in HIV M&E through a global network of M&E technical advisors in more than 20 countries.
The strategic use of information was essential in PEPFAR I (2003–2008), when rapid results were crucial to demonstrate feasibility and scalability of HIV care and treatment services. PEPFAR supported strategic information staff at headquarters and in countries to monitor the early scale-up of HIV service delivery programs, respond to planning and reporting requirements, and work with ministries of health and civil society to build capacity in M&E. The enormous influx of funds, the rapid expansion of HIV services through many different implementing partners, and the pressure for information in the context of nascent national M&E systems led to a proliferation of uncoordinated systems to monitor newly funded HIV programs. The urgency to report results contributed to the development of indicator-based systems and insufficient attention to the functionality and sustainability of the underlying information systems.17 Concurrent PEPFAR investments in national M&E systems were vital to future developments, but coordinated systems development takes time and, thus, lagged the rapid expansion of services.
Early efforts at coordination at the global level were notable, but demands from donors remained high. With some exceptions, most countries were not ready to respond to donor demand or enforce alignment of global actors to national priorities and processes.18 Ethiopia was notable for its early leadership, planning, and implementation of national HIV M&E. With international support, the implementation of the first national HIV M&E framework19 was coordinated with the national health information system reform and supported by a 2006 Memorandum of Understanding among PEPFAR, the Global Fund, and the federal Ministry of Health. This memorandum of understanding led to joint support for surveys, national evaluations, national program reviews, human resource development, information technology (IT) infrastructure development, and harmonized national health indicators and system standards. Similarly, Malawi showed strong leadership in the development of a national multisectoral M&E framework and system20 and early development of a national HIV treatment program with a strong standardized monitoring system.21,22 Malawi also demonstrated early use of routine information for evaluation and program improvement and displayed strong leadership in their innovative national consultative processes to gather, synthesize, and use different types of data to assess and inform their HIV programming.23
THE CURRENT STATE OF GLOBAL HIV M&E
During the later years of PEPFAR I, global stakeholders continued to seek ways to coordinate their demands for information, to jointly verify reported results, and to develop global standards, guidelines, and tools to advance HIV M&E.24 Under UNAIDS leadership, global partners, including PEPFAR, committed to further harmonizing indicators, assessing the performance of existing indicators, and defining a process for endorsing new indicators.25 In 2007, PEPFAR committed to an interdisciplinary consultative process to revise reporting requirements toward further harmonization with global standards. The resulting “Next Generation Indicator Guidance” in 2009 reiterated PEPFAR's commitment to monitoring trends in HIV care and treatment, emphasized the need to measure quality and outcomes, and supported alignment with national systems.26
Global organizations including PEPFAR, the Global Fund, UNAIDS, United Nations Children's Fund, and WHO have conducted annual reconciliation of nationally reported data, and the UN organizations have committed to one joint reporting process and tool, similar data collection dates, and a joint report.27 With key contributions from the World Bank, global organizations committed to a common framework for HIV M&E, known as the “12 Components Framework,” which defined M&E systems, formed the basis for M&E system assessments, and guided capacity development for M&E systems.2,28 Aided by substantial support from PEPFAR, global organizations developed, disseminated, and implemented common minimum standards for HIV patient monitoring systems,29 developed standard data quality assessment procedures and tools,30,31 and developed guidance and tools for HIV data synthesis and use approaches.32–34
Global guidelines and coordination, however, do not readily translate into country advancements. Building national M&E systems requires sustained efforts over long periods of time with local leadership, commitment, and extensive stakeholder engagement. Key elements of national M&E systems—including national M&E units, data plans, and central HIV databases—were developed during the first phase of PEPFAR, but performance in measurement across program areas was uneven.35 The 5-year impact evaluation of the Global Fund, for example, noted challenges in collecting uncoordinated information from many partners implementing different interventions. The evaluation also highlighted many types of improved data, attributed largely to PEPFAR investments, including clinic reporting, facility assessments, population-based surveys, antenatal clinic (ANC) surveillance, facility assessments, and national AIDS spending assessments.36
PEPFAR's support also has improved the quality and reach of global HIV surveillance in the general population through population-based surveys, in sentinel populations, like those found at ANCs, and in key populations at highest risk for HIV. For example, PEPFAR's support for global ANC surveillance helped to add more sites per country and a better representation of rural sites in each country's sampling frame, resulting in more accurate estimates of national HIV prevalence over time.37 More recently, PEPFAR is supporting policy, guidelines, tools, pilot testing, and implementation to move away from unlinked anonymous testing toward the use of routine prevention of mother-to-child transmission (PMTCT) program monitoring data for the estimation of HIV prevalence.38 PEPFAR provided some support to mortality surveillance, for example, in collaboration with WHO and others, for the development of the Sample Vital Registration with Verbal Autopsy toolkit and global standards for verbal autopsy.39 Also, PEPFAR added key financial and technical support for the first ever postcensus mortality survey, conducted in Mozambique.40 More recently, PEPFAR is supporting efforts to use the Sample Vital Registration with Verbal Autopsy approach to mortality measurement in Tanzania and Zambia.41
For the biological and behavioral surveillance of key populations, PEPFAR provided support to develop M&E guidelines for key populations42 and innovative methods such as respondent-driven sampling and time–location sampling.43 An example of strategic US collaboration in concentrated epidemics has been with the Royal Thailand Government, where early HIV research improved global understanding of HIV epidemiology and the effectiveness of selected interventions44–46 and later support advanced the surveillance of and services to key populations. Integrated biobehavioral surveys47–49 and sophisticated epidemic modeling50–52 showed the HIV epidemic in Thailand to be concentrated among key populations such as female sex workers, injection drug users, and increasingly men who have sex with men. PEPFAR responded to requests to develop and implement pilot intervention projects in key populations and integrate these strategies into the national response, leveraging funds from other donors. PEPFAR also provided substantial assistance to advance novel methods for HIV incidence measurement, audio computer-assisted self-interview techniques to reduce self-report biases, venue-based and respondent-driven sampling methods, and HIVQUAL for clinical quality improvement.53
As patient and program monitoring systems matured, attention turned to assessing and improving the quality of data and using data to know and improve the quality of care. Data quality assessments are now commonplace, with the standard tools used in dozens of countries by the Global Fund and PEPFAR. Incorporation of data quality assurance steps into routine supervision protocols strengthens the quality of the data54 and seeks to advance the quality of care through the use of higher quality data. PEPFAR-funded partners and projects with multicountry presence and advanced analytic capacity—such as HEALTHQUAL International, Columbia University's ICAP, and the AIDSRelief Consortium—have helped to advance the analysis and use of program data in national, regional, and local stakeholder forums to inform and improve the quality of care.55–59 With maturing HIV care and treatment programs, the measurement of key outcomes on the same patients over time is important for patient management and program monitoring. Analysis of routine antiretroviral pick up data and interdisciplinary clinical review, for example, can improve loss to follow-up among antiretroviral therapy (ART) patients and encourage retention and adherence.60 Despite system advances, however, monitoring HIV program outcomes and tracking clients from HIV testing to care and treatment continue to be difficult,61 requiring special evaluation exercises, such as medical record abstraction protocols, to estimate national HIV treatment outcomes.62,63 Côte d'Ivoire illustrates one country that addressed short-term information needs in the context of longer term system strengthening. Since 2004, PEPFAR has invested in ongoing efforts to strengthen the national facility-based routine health information system and develop a national electronic patient medical record system (SIGDEP), which all partners will adopt to manage HIV patients and drug logistics. Because such system strengthening is a long-term endeavor, complimentary data collection exercises help to meet immediate information needs for HIV program information.64–66
CHALLENGES AND OPPORTUNITIES FOR THE FUTURE
The second phase of PEPFAR (2009–2013) seeks to support global goals to further scale-up–integrated HIV prevention, care, and treatment programs, while accelerating country ownership and capacity development.67 Assessing progress toward the ambitious goal of an AIDS-free generation68 requires that PEPFAR and its partners support countries to better measure the reduction of new HIV infections in children and adults and monitor the coverage, quality, and outcomes of highly efficacious interventions (ie, voluntary medical male circumcision, HIV testing and counseling, PMTCT and ART), delivered in combination. For example, robust program monitoring and adverse events surveillance are essential to the rapid global expansion of male circumcision programs. Although early infant diagnosis systems are evolving rapidly, population-based and facility-based methods may be used to monitor PMTCT program outcomes.69,70 New assay-based methods for detecting recent HIV infections show promise for future incidence surveillance and program impact evaluation.71,72 There are pressures to improve monitoring systems for HIV prevention, care, and treatment services and integrate monitoring of HIV with that of other diseases and health services. For example, enhancements to routine HIV monitoring systems are needed to better track individuals and families across services like testing, care, and treatment and to assess retention and adherence to ART. At the same time, decentralization of HIV services to lower levels of the health system requires simplification of monitoring approaches and integration of monitoring systems across tuberculosis, malaria, and maternal and child health.
Rapidly evolving science requires the ability to translate evidence to practice and to modify guidelines, services, and systems. Evolving PMTCT programs, for example, will improve HIV prevalence estimation derived through routine clinic data and reduce the need for unlinked anonymous testing currently used in the public health surveillance of HIV. But these programs also demand improved longitudinal monitoring of outcomes, such as antiretroviral toxicity and resistance, as pregnant women transition to early uptake of ART. Such changes require revision of monitoring tools and documentation processes, posing challenges to standardized national M&E systems that can be slow to adapt. PEPFAR has provided substantial support for a portfolio of process, outcome, and impact evaluations, now classified under an “implementation science” framework.73 PEPFAR must continue to support local leadership and capacity in the conduct of evaluation and research, including the development of highly skilled cadre of epidemiologists and biostatisticians. PEPFAR must also work closely with partner countries to make data more publicly available. Different types of data collections must be synthesized for improved understanding of epidemics and the impact of programs. Consistent analysis of financial and cost data against output and outcomes is needed in an environment of increasing budget constraints.
Great opportunities lie in the revolution in IT, increasing the “prospects of leapfrogging decades in the development process.”74 Better harnessing of new opportunities in information and communication technology can lead to more accurate and timely communication of information across systems in resource-poor countries.75,76 Examples abound in scanning technology, handheld computerized devices, and the use of cell phones for data entry and data use, and “off-grid” technologies such as solar-powered devices.77 Transitioning from paper-based processes to electronic automated ones has potential to increase the efficiency and quality of patient services. When implemented successfully, for example, electronic medical records have been shown to lead to faster and more informed decision-making processes that benefit patients and costs.78,79 Exciting opportunities also exist to use geospatial data and information systems for analysis, display, and decision making. The full potential of geospatial methods for HIV M&E will require strengthening the underlying spatial data infrastructure and improving coordination of different data systems.80 Advancements in IT for HIV M&E, however, do not reduce the need for leadership and coordination. Inappropriate information systems can waste money if attention is focused on the technology alone or on technology ill suited to reality on the ground. These problems can become magnified if implemented without national coordination and guidance for enterprise architecture and system standards.
Partner nations have made progress to move away from partner-specific or donor-specific systems toward coordinated national ones, as longer term investments in national M&E systems begin to bear fruit. A necessary stage in transition is the assessment of capacity and “readiness,” to ensure that systems can perform upon transition. For example, intensive systematic engagement of PEPFAR and its funded partners with the Government of Rwanda has ensured careful system reviews and transition strategies that balance accountability to external stakeholder funds with the appropriate development of Rwanda national programs and systems.81,82 These processes required engagement, commitment, analysis, review, and change by partners at all levels of the health system, which took time as new data, indicators, tools, systems, and processes were negotiated.
Accelerated transition to country-owned systems and programs also focuses special attention on human resources for M&E. Common challenges to sufficient human resources are low compensation and high turnover of skilled staff. A strong M&E system cannot be built or sustained without a human resource plan. PEPFAR has provided short-term training and support for the skilled workers needed to staff the M&E systems of today, and PEPFAR has invested in the longer term local development of the future M&E workforce. For example, through MEASURE Evaluation, PEPFAR supports regional master's-level programs in M&E in South Africa, Thailand, and Mexico,83 and PEPFAR supports many ministries of health in their Field Epidemiology Training Program, a comprehensive 2-year, postgraduate training and service program through partnerships with Centers for Disease Control and Prevention and local universities.84 These types of programs take years to produce a skilled and decentralized skilled workforce, but countries like Ethiopia are beginning to show the results of these long-term investments. With PEPFAR support and collaboration, the Ethiopian Federal Ministry of Health, Tulane University, Fiocruz-Brazil, and Jimma University have begun to graduate cohorts of master's-level M&E professionals to be placed in the Ethiopian national health system; have launched a biostatistics/informatics graduate program at a local university; and will train and place nearly 9000 health information technicians by 2020.85,86
Measuring and monitoring the development of human and systems capacity has long been a challenge in PEPFAR, especially in countries with relatively smaller PEPFAR budgets strategically focused on technical assistance rather than direct service delivery. A key methodological challenge is demonstrating an association of capacity-building efforts not only with capacity outcomes but also with HIV program outputs, outcomes, and impact. The newly disseminated PEPFAR Capacity Building and Strengthening Framework87 provides guidance to field teams on how to plan, operationalize, and monitor their capacity-building efforts to advance HIV programs. This issue remains of keen interest as PEPFAR moves from funding direct service delivery to more support for system strengthening, where governments and local partners demonstrate greater capacity to deliver and monitor HIV services at facilities.
There has been a significant shift in culture around HIV M&E in the past 10 years, associated with the great infusion of financial support through PEPFAR and the Global Fund. Awareness of the need for and usefulness of data has increased, far beyond discussions of indicators and reporting. Building national M&E systems requires sustained efforts over long periods of time with extensive stakeholder engagement. Although progress has been made, the successes are variable and depend on leadership, baseline capacity, political stability, and other factors integral to any systems strengthening intervention. Demands on country systems remain high as global and national stakeholders seek to achieve universal coverage of multiple HIV interventions, show program impact, and incorporate cost measurement into planning and evaluation. By focusing on the key evidence-based interventions needed to achieve an AIDS-free generation and the capacities needed to monitor and evaluate those interventions, PEPFAR, in close collaboration with its global and national partners, is well positioned to transform the successes and challenges associated with early rapid scale-up into future opportunities for sustainable, cost-effective, country-owned programs and systems.
The authors acknowledge the governments of Côte d'Ivoire, Ethiopia, Malawi, Mozambique, Rwanda, and Thailand for their achievements, which the authors have only been able to highlight briefly herein.
3. Office of the Global AIDS Coordinator. The President's Emergency Plan for AIDS Relief: U.S. Five-Year Global HIV/AIDS Strategy. 2004.
4. Cleland J, Boerma J, Carael M, et al.. Monitoring sexual behaviour in general populations: a synthesis of lessons of the past decade. Sex Transm Infect. 2004;80(suppl II):ii1–ii7.
5. Curtis SL, Sutherland E. Measuring sexual behaviour in the era of HIV/AIDS: the experience of Demographic and Health Surveys and similar enquiries. Sex Transm Infect. 2004;80(suppl II):ii22–ii27.
6. Slaymaker E. A critique of international indicators of sexual risk behaviour. Sex Transm Infect. 2004;80(suppl II):ii13–ii21.
8. Macro International Inc. Demographic and Health Surveys Model “B” Questionnaire: with commentary for low contraceptive prevalence countries. DHS-Core Questionnaire Phase III. Calverton, MD: Macro International Inc; 1995.
9. ORC Macro International. DHS Model Questionnaire-Phase 5. Calverton, MD: ORC Macro International Inc; 2008.
10. Mishra V, Vaessen M, Boerma JT, et al.. HIV testing in national population-based surveys: experience from the Demographic and Health Surveys. Bull World Health Organ. 2006;84:537–545.
11. Mills S, Saidel T, Magnani R, et al.. Surveillance and modelling of HIV, STI, and risk behaviours in concentrated HIV epidemics. Sex Transm Infect. 2004;80(suppl II):ii57–ii62.
12. CDC, USAID, UNAIDS, Centre National de Lutte Contre Sida (Senegal), MEASURE Evaluation. Strengthening monitoring and evaluation of national AIDS programs in the context of the expanded response: workshop report. February 4–6, 2002; Dakar, Senegal.
13. CDC, USAID, UNAIDS, Uganda AIDS Commission, MEASURE Evaluation. Strengthening monitoring and evaluation of national AIDS programmes in East and Southern Africa: workshop report. April 23–26, 2001; Entebbe, Uganda.
14. Ainsworth M, Vaillancourt D, Gaubatz JH. Committing to Results: Improving the Effectiveness of HIV/AIDS Assistance—An OED Evaluation of the World Bank's Assistance for HIV/AIDS Control. Washington, DC: World Bank; 2005.
16. Rugg D, Carael M, Boerma JT, et al.. Global advances in HIV/AIDS monitoring and evaluation: from AIDS case reporting to program improvement. In: Rugg D, Peersman G, Carael M, eds. Global Advances in HIV/AIDS Monitoring and Evaluation: New Directions for Evaluation 103. New York, NY: Jossey-Bass; 2004:33–48.
17. Boerma JT, Stansfield SK. Health statistics now: are we making the right investments? Lancet. 2007;369:779–786.
18. Holzscheiter A, Walt G, Brugha R. Monitoring and evaluation in global HIV/AIDS control: weighing incentives and disincentives for coordination among global and local actors. J Int Dev. 2012;24:61–76.
19. Federal Ministry of Health/HAPCO. National Monitoring and Evaluation Framework for the Multi-Sectorial Response to HIV/AIDS in Ethiopia. 2003.
20. Görgens M, Nkwazi C, Chipeta J, et al.. Malawi: Developing a National Multisector HIV/AIDS Monitoring and Evaluation System: World Bank; 2005.
21. Harries AD, Gomani P, Teck R, et al.. Monitoring the response to antiretroviral therapy in resource-poor settings: the Malawi model. Trans R Soc Trop Med Hyg. 2004;98:695–701.
22. Lowrance DW, Makombe S, Harries AD, et al.. A public health approach to rapid scale-up of antiretroviral treatment in Malawi during 2004-2006. J Acquir Immune Defic Syndr. 2008;49:287–293.
24. Tassie JM, Bertagnolio S, Souteyrand Y. Integrated surveillance of HIV care in low-income and middle-income countries. Curr Opin HIV AIDS. 2011;6:233–238.
28. Görgens M, Kusek J. Making Monitoring and Evaluation Systems Work: A Capacity Development Toolkit. World Bank; 2009.
29. World Health Organization. Three Interlinked Patient Monitoring Systems for HIV Care/ART, MCH/PMTCT (Including Malaria Prevention During Pregnancy), and TB/HIV: Standardized Minimum Data Set and Illustrative Tools. Geneva, Switzerland: World Health Organization; 2009. Available at: http://www.who.int/hiv/pub/imai/three_patient_monitor/en/index.html
, Accessed June 11, 2012.
32. World Health Organization, Joint United Nations Programme on HIV/AIDS, Global Fund to Fight AIDS, Tuberculosis and Malaria. HIV Triangulation Resource Guide: Synthesis of Results From Multiple Data Sources for Evaluation and Decision-Making. Geneva, Switzerland: World Health Organization; 2009. Available at: http://www.who.int/hiv/pub/surveillance/triangulation/en/index.html
. Accessed June 11, 2012.
35. Peersman G, Rugg D, Erkkola T, et al.. Are the investments in national HIV monitoring and evaluation systems paying off? J Acquir Immune Defic Syndr. 2009;52(suppl 2):S87–S96.
36. Macro International Inc. Final report: study area 3—the impact of collective efforts on the reduction of the disease burden of AIDS, tuberculosis, and malaria. Global Fund: The Five-Year Evaluation. Calverton, MD: Macro International Inc; 2009.
37. Diaz T, Garcia-Calleja JM, Ghys PD, et al.. Advances and future directions in HIV surveillance in low- and middle-income countries. Curr Opin HIV AIDS. 2009;4:253–259.
38. Marsh KA, Bolu O, Bodika S, et al.. How can PMTCT program data be used for the purposes of HIV surveillance? J HIV/AID Surveill Epidemiol. 2010;2(1):5.
40. Mozambique National Institute of Statistics, US Census Bureau, MEASURE Evaluation, US Centers for Disease Control and Prevention. Mortality in Mozambique: Results From a 2006-2007 Post-census Mortality Survey. Chapel Hill, NC: MEASURE Evaluation; 2012. Available at: https://www.cpc.unc.edu/measure/publications/tr-11-83/
. Accessed February 24, 2012.
41. Mudenda SS, Kamocha S, Mswia R, et al.. Feasibility of using a World Health Organization-standard methodology for sample vital registration with verbal autopsy (SAVVY) to report leading causes of death in Zambia: results of a pilot in four provinces, 2010. Popul Health Metr. 2011;9:40.
42. Weir S, Sabin K, Butler J, et al.. Operational Guidelines for Monitoring and Evaluation of HIV Programmes for Sex Workers, Men Who Have Sex With Men, and Transgender People. Chapel Hill, NC: MEASURE Evaluation; 2011. Available at: https://www.cpc.unc.edu/measure/publications/ms-11-49a
. Accessed June 11, 2012.
43. Johnston LG, Sabin K, Prybylski D. Update for sampling most-at-risk and hidden populations for HIV biological and behavioral surveillance. J HIV/AIDS Surveill Epidemiol. 2010;2:2.
44. Mastro TD, Satten GA, Nopkesorn T, et al.. Probability of female-to-male transmission of HIV-1 in Thailand. Lancet. 1994;343:204–207.
45. Shaffer N, Chuachoowong R, Mock PA, et al.. Short-course zidovudine for perinatal HIV-1 transmission in Bangkok, Thailand: a randomized controlled trial. Bangkok Collaborative Perinatal HIV Transmission Study Group. Lancet. 1999;353:773–780.
46. Kilmarx PH, Palanuvej T, Limpakarnjanart K, et al.. Seroprevalence of HIV among female sex workers in Bangkok: evidence of ongoing infection risk after the “100% Condom Program” was implemented. J Acquir Immune Defic Syndr. 1999;21:313–316.
47. Wattana W, van Griensven F, Rhucharoenpornpanich O, et al.. Respondent-driven sampling to assess characteristics and estimate the number of injection drug users in Bangkok, Thailand. Drug Alcohol Depend. 2007;90:228–233.
48. Van Griensven F, Varangrat A, Wimonsate W, et al.. Trends in HIV prevalence, estimated HIV incidence and risk behavior among men who have sex with men in Bangkok, Thailand, 2003-2007. J Acquir Immune Defic Syndr. 2010;53:234–239.
49. Manopaiboon C, Whitehead SJ, Subhachaturas W, et al.. Unexpectedly high HIV prevalence among Thai sex workers in a respondent-driven sampling survey. Presented at: 20th Annual Australasian Society for HIV Medicine Conference; September 17–20, 2008; Perth, Australia.
50. Brown T, Peerapatanapokin W. The Asian Epidemic Model: a process model for exploring HIV policy and programme alternatives in Asia. Sex Transm Infect. 2004;80(suppl 1):i19–i24.
51. Thai Working Group on HIV/AIDS Projections. Projections for HIV/AIDS in Thailand: 2005–2025. Bangkok, Thailand: Family Health International and Bureau of AIDS, TB and STIs, Department of Disease Control, Ministry of Public Health; 2008.
52. Park LS, Sirapraspasiri T, Peerapatanapokin W, et al.. HIV transmission rates in Thailand: evidence of HIV prevention and transmission decline. J Acquir Immune Defic Syndr. 2010;54:430–436.
53. Lolekhha R, Chunwimaleung S, Hansudewechakul R, et al.. Pediatric HIVQUAL-T: measuring and improving the quality of pediatric HIV care in Thailand, 2005-2007. Jt Comm J Qual Patient Saf. 2010;36:541–551.
54. Kouacou B, Guede N, Aboua N. Controle qualite des donnees a l'aide de l'outil RDQA dans le region sanitaire des lagunes 2: Rapport de mission. République de Côte d'Ivoire: Ministère de la santé, Et de l'Hygiène publique; Décembre 2009-Janvier 2010.
55. Rowland S, Geoffrey S, Bonita K, et al.. Ministry of Health, The United Republic of Tanzania, National AIDS Control Programme. The Implementation of Care and Treatment Services in Tanzania. 2008. Report Number 1.
56. Rowland S, Geoffrey S, Bonita K, et al.. Ministry of Health,The United Republic of Tanzania, National AIDS Control Programme. The Implementation of Care and Treatment Services in Tanzania. 2011. Report Number 2.
57. Almeida A, Nuwagaba-Biribonwoha H, Idindili B, Ndega J, et al.. Health management information systems (HMIS) challenges as ART programs expand—lessons from ICAP, Tanzania. Presented at: 3rd National Multi-Sectoral AIDS Conference; December 18–21, 2006; Arusha, Tanzania.
58. Lahuerta M, Lima J, Elul B, et al.. Patients enrolled in HIV care in Mozambique: baseline characteristics and follow-up outcomes. J Acquir Immune Defic Syndr. 2011;58:e75–e86.
59. Thanprasertsuk S, Supawitkul S, Rangsima, et al.. HIVQUAL-T: monitoring and improving HIV clinical care in Thailand, 2002–08. Int J Qual Health Care. 2012; Published online ahead of print June 3, 2012. doi: 10.1093/intqhc/mzs008.
60. Byicaza O, Moen M, Umukoza MC, et al.. Reducing lost-to-follow-up by integrating clinical data management in community support system in AIDSRelief-supported rural health facilities in Rwanda. Presented at: 5th IAS Conference on HIV Pathogenesis, Treatment, and Prevention; July 19–22, 2009; Cape Town, South Africa.
61. Nuwagaba-Biribonwoha H, Werq-Semo B, Abdallah A, et al.. Introducing a multi-site program for early diagnosis of HIV infection among HIV-exposed infants in Tanzania. BMC Pediatr. 2010;10:44.
62. Lowrance DW, Ndamage F, Kayirangwa E, et al.. Adult clinical and immunologic outcomes of the national antiretroviral treatment program in Rwanda during 2004-2005. J Acquir Immune Defic Syndr. 2009;52:49–55.
63. Auld AF, Mbofana F, Shiraishi RW, et al.. Four-year treatment outcomes of adult patients enrolled in Mozambique's rapidly expanding antiretroviral therapy program. PLoS One. 2011;6:e18453.
64. Comité National de Lutte Contre le SIDA (CNLS). Rapport national 2005 de lutte contre l'infection par le VIH en CoteD'Ivoire. CNLS. Abidjan, Côte d'Ivoire; 2005.
66. Auld AF, Moise TZ, Shiraishi RW, et al.. Treatment outcomes of HIV-2-infected and HIV-1 and -2 dually reactive adults enrolled in Côte d'Ivoire's antiretroviral therapy program during 2004-2008 (CDB345). Presented at: 6th IAS Conference on HIV Pathogenesis, Treatment and Prevention; July 17–20, 2011; Rome, Italy
69. Rollins N, Mzolo S, Moodley T, et al.. Universal HIV testing of infants at immunization clinics: an acceptable and feasible approach for early infant diagnosis in high HIV prevalence settings. AIDS. 2009;23:1851–1857.
70. Goga A. Impact of the national prevention of mother to child transmission (PMTCT) program on mother-to-child transmission of HIV (MTCT), South Africa, 2010 (MOAC0206). Presented at: 6th IAS Conference on HIV Pathogenesis, Treatment, and Prevention; July 17–20, 2011; Rome, Italy.
71. UNAIDS/WHO Working Group on Global HIV/AIDS and STI Surveillance. When and How to Use Assays for Recent Infection to Estimate HIV Incidence at a Population Level. Geneva, Switzerland: World Health Organization; 2011.
72. Duong YT, Qiu M, De AK, et al.. Detection of recent HIV-1 infection using a new limiting-antigen avidity assay: potential for HIV-1 incidence estimates and avidity maturation studies. PLoS One. 2012;7:e33328.
73. Padian NS, Holmes CB, McCoy SI, et al.. Implementation science for the US President's Emergency Plan for AIDS Relief (PEPFAR). J Acquir Immune Defic Syndr. 2011;56:199–203.
74. Kwankam Y, Ntomambang N. Information Technology in Africa: A Proactive Approach and the Prospects of Leapfrogging Decades in the Development Process. Yaounde, Cameroon: University of Yaounde I. Available at: www.isoc.org/inet97/proceedings/B7/B7_1.HTM
. Accessed July 5, 2012.
75. Nash D, Elul B, Rabkin M, et al.. Strategies for more effective monitoring and evaluation systems in HIV programmatic scale-up in resource-limited settings: implications for health systems strengthening. J Acquir Immune Defic Syndr. 2009;52(suppl 1):S58–S62.
76. Ekouevi D, Karcher S, Coffie P. Strengthening health systems through HIV monitoring and evaluation in Sub-Saharan Africa. Curr Opin HIV AIDS. 2011;6:245–250.
77. Alfredo Vaccaro, et al.. Reliable Electric Power for Developing Countries Working Group. Presented at: IEEE Humanitarian Technology Challenge Conference, June 1–2, 2009, Washington, DC.
78. DesRoches CM, Campbell EG, Rao SR, et al.. Electronic health records in ambulatory care—a national survey of physicians. N Engl J Med. 2008;359:50–60.
79. Hillestad R, Bigelow J, Bower A, et al.. Can electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Aff (Millwood). 2005;24:1103–1117.
82. Raghunathan P. Monitoring the transition of HHS supported Track 1.0 HIV clinical services in Rwanda. Presented at: PEPFAR Annual Meeting; 2011; Johannesburg, South Africa.
85. Lemma W, Bushen O, Azene G. The necessity of task shifting in HMIS: establishing a new cadre of HMIS officers. Presented at: AIDS 2008 XVII International AIDS Conference: Abstract no. CDC0274.
86. Federal Ministry of Health of Ethiopia. Human Resources for Health (HRH) (Draft) strategic plan. 2009 Location : Federal Ministry of Health of Ethiopia Document archive.
87. Office of the Global AIDS Coordinator. PEPFAR Capacity Building and Strengthening Framework. 2011.